Tolerances, Measured and Reported Results

I'm looking for guidance on a recognized, acceptable aerospace-industry approach to recording and evaluating measurement results. Let me explain. The drawing calls for a dimension of 0.050"+/-.005" and I'm measuring it to 4th decimal place (i.e. 0.xxxx"). If I have a result of 0.0552", how do I evaluate this result relative to the stated tolerance? Acceptable or not? And for reporting purposes, what value is reported, 0.055" or 0.0552"? And then the issue of rounding is to be considered? or not? Is there an aero industry standard that defines the proper, acceptable means of evaluation?

 

ANSI Y14.5M clearly states that the limits of tolerance are absolute. The example in the stardard is:

12.2 means 12.20.......0
12.0 12.00......0

There's no rounding, period.

 

So if I intersect two cylinders in a 3D modeling system - say two holes in a hydraulic system coming from different directions and meeting in a block - and call out a point on the intersection back to a face, like the peak of the intersection, I would NOT necessarily expect this distance to be exact. It would be something like 80.235876471 mm. But ... when I actually put the dimension down in the dimension mode, I am going to "show" the distance in whatever precision is default. Let's say 3 place decimal. What I will see on the print is 80.236 mm. Rounding CLEARLY going on. If my tolerance is +/- 0.005 mm then I can be between 80.231 and 80.241. I am already shifted. The question is interesting. Maybe my accurate measuring system kicks back an 80.2414 which would round down to 80.241 and be "in". But it's clearly out. Maybe my somewhat accurate measuring system would only display 80.241 and the part would be "in" on THAT system. What then? If the dimensions are absolute, the accurate gage would reject the part and the so-so gage would accept the SAME part. But make no mistake - rounding IS going on, it is inherent in the CAD.

Perhaps the better question is - if I have randomly picked a part right at the limit, I probably have a bit more of a problem than just this part.

We round in automotive. To the number of digits that the DIMENSION is in. That's how the engineer communicates precision.

 

I did a thought experiment last night.

Let's say I have a 10 +/- 0.05 mm pin.

Certainly if my limit is 10.05 and my accurate device gives me 10.054 normal mathematical rounding rules would say round down to 10.05 (resulting in a good part). But math rules aside, we have a physical world we have to deal with. If the mating part were a bushing and we wanted a slip fit and the bushing was allowed to be 10.05 to 11.00 there could come the day where I got a hole that was 10.051 which would clearly be in by either the accurate gage OR the inaccurate gage. But my “in spec by rounding” 10.054 pin would NOT fit in this hole.

Because of this, one should always “round in the direction of nominal.” (It would be nice if Excel could do this. It can round (conventional), roundup, and rounddown, but you would have to know which way was nominal.

As a corollary, what is interesting is gage selection. It follows the situation logically. If my calipers can tell me to say within 0.02 and my micrometers can tell me within 0.001, think about when you would deploy them. You’d start with the caliper because it’s faster, cheaper and easier. If you measured the pin at 10.01, you’d say to yourself “I am well enough away from the tolerance limit, this is good enough….” But if you measured 10.05 with the caliper and you REALLY had to know, you’d think “better get my mic out.” Which sort of naturally follows…the closer you get to the limit, the more accurate the device you’d need to use to know. I know, it’s risky to base the argument on “what makes sense,” it’s just nice when what makes sense fits ….

 

ncwalker, I would think that using the Gage makers rule, or the rule of 10, would eliminate your issue. It states that measurements should be better than inspections measurements by a factor of 10. So if you are measuring a 10 =/1 .05 mm pin, you would measure to the thousandths. If your measurement tool measured was more accurate than the thousandths, then I personally think you may be able to argue rounding.

In other words, I would round TO the thousandths, but no more.