Auditor added something to NCR that they didn’t observe

We have a NADCAP NCR for test codes G1 and G5 Carbon and Sulfur analysis to ASTM E1019, it’s about the number of reference materials we use to calibrate the instrument. the very last line of the NCR description states
“same problem was identified with test codes G3 and G4”
The thing is auditor didn’t ask to see the calibration curve of that instrument nor ask the operator about it so they are just making an assumption/ guess based on the techniques being similar.

How should I deal with this?

 

The line was added sometime between us trying to convince the auditor that there was no NC in a morning meeting on the Wednesday of last week and the closing meeting on Saturday. The auditor attended our lab on Monday and Tuesday, did not view or discuss the calibration curve for the Oxygen and Nitrogen tester and never returned to our lab. For the rest of the week.

I am going to appeal the whole NCR some of the formatting has been lost so I’ve added Auditor/ me to show who said what


Appeal to Staff engineer on NADCAP NCR 4



The Auditor’s NCR description is reproduced in full below (italics), the grounds for rejection of the NCR are given after each section of the auditors comments followed by a summary in normal type. Quotes from documents are underlined


Auditor:
(Test codes G1/G5)

The tests (in Both Chemistry Laboratory-1 and Chemistry Laboratory -2) were not performed in accordance with ASTM E1019-18, using single point calibration method as defined, but using multi-point calibration method, as instructed by equipment manufacturer LECO engineer. This was allowed, and actually multi points calibration curve works more accurate comparing with single point calibration curve.


Me:
ASTM E1019 does not stipulate that calibration must be single point. From the standard:

ASTM E1019 17.6.2; Follow the calibration procedure recommended by the manufacturer. Use C as the primary calibration reference material and analyse at least three specimens to determine the measurement response to be used in the calibration regression. Treat each specimen, as directed in 18.1.2 and 18.1.3, before proceeding to the next one.



Bold is my emphasis, the meaning of this is that the user should calibrate the instrument according to the manufacturer’s procedure and also include a CRM “C” defined earlier, to serve as a master sample which defines the instrument measurement response at the time the calibration is made, ie a setting up standard generally called a drift standard in C/S analysis. This requirement is not counter to including more reference materials in the calibration curve (instrument manufacturer’s instructions deferred to by the standard recommends multiple). The expectation of multiple standards is further supported by the following clause:



ASTM E1019 19.1;

The calibration function shall yield a linear plot described by Eq 2

Y=mx+b （2）

Where

Y = measurement response

m = slope

X = calibration RM mass fraction

b = intercept on y axis

Calculation of the calibration shall be done using linear least squares regression.



Bold is my emphasis. this mandatory requirement cannot be performed on a single point calibration and it is therefore clear that the intention of the standard is that multiple CRMs are included in the calibration curve.



ASTM E1019 (18) also breaks down the testing of the elements into ranges this is in order to account for the non-linear nature of the instrument response when the range of analysis is very wide. The multiple calibration lines are intended to approximate the response of each range. A single point calibration undermines this intent because all curves will be forced through zero, creating a straight line through the point furthest from the analytical range of interest.



The sections quoted from ASTM E1019 (18) are concerned with carbon but the sections concerned with S (and O and N) are structured in the same way and are equivalent in meaning and intent.


Auditor:
Further verification of the multi points calibration curve in Chemistry Laboratory-1 found that 14 reference materials were used to establish the calibration curve. Some of them with very close concentration, which mean less valuable for linearity. Among them, the highest concentration was not figured out, not defined in the applicable test procedure # YLQI-08-411 Rev.1 (in Chemistry Laboratory-2, was # AYG-MA-SOP0021 Rev. 3), so that the unknown samples to be tested can not exceed this extreme value.


Me
On a Leco instrument a method is set up to analyse both C and S simultaneously also the instrument has both a high and low calibration for each element. To achieve a suitable spread of certificate values to cover the desired range of all four cases may lead to 14 registered standards and some of these standards may have similar C values but have been selected because of their quite different S values or vice versa this is unavoidable and although the Auditor offers their opinion that it adds little to the calibration curve it does not detract from accuracy or traceability.

The image below from YLQI-08-411 Rev.1 was shown to the auditor in response to the question about highest concentration this shows the scope of the analysis based on “actual instrument data” as required by NADCAP. I believe that this objective data should have addressed the question



The applicable scope of all the methods are documented as required.

When an unknown sample gives an out of scope result it will be dealt with as such but it is not possible to say it will never happen, nor can it be prevented by knowing the upper reporting limit the situation will only be revealed after the analysis is completed. AYG-MA-SOP0015 Reporting of results, gives guidance in such circumstances.


Auditor:
Besides, the above procedures didn't define the condition when the calibration shall be performed.

Me:
This was not discussed in the audit of lab 2. If it had been auditor’s attention would have been drawn to AYG-MA-SOP0021 Rev. 3 section 6.2 which covers the requirement for drift calibration of the instrument to be controlled by a procedure as required by AC7101/2 rev E 4.3 see screenshot below.





The subject is also covered in the procedure of Chemistry laboratory 1


Auditor
Same problem was identified with test codes G3 and G4.

Me
The auditor did not actually review the calibration/method on ON836 instrument nor discuss the nature of the calibration with the operator so it is difficult to know how they arrived at this conclusion.

 

The images have not appeared sorry I’m doing this from my phone

 

From what you have provided in your statement of appeal, it seems that you have covered all the bases and know what is required to calibrate your instrument. Looking forward to where your appeal would go. NCs based on assumptions are not acceptable. Let the CB have it.

 

Thanks Andy and Tony for the feedback. I really don’t know what to expect but I will update this thread with the outcome.
the auditor seemed to have a sense of impunity which ties in with Andy’s experience.

 

Here’s the promised update.

The delay is due to the 3 weeks to work on CAs and submit plus the time for PRI’s review, you know how it is.

Anyway the appeal has worked and that NC has been closed off without having to do any work, it’s a relief because as you can see it is a massive word salad so wouldn’t really know where to start with RCA and CA if I’d had to open an NCR.

Just kicking myself for not appealing a couple more, but I didn’t want us to look desperate.

thanks for advice and reviewing the response.