Boundaries of the external auditors.

Last week one my customers had their first ISO9001:2015 stage 1 documentation audit. The audit went pretty well but we did had some tough discussions with the auditors. I honestly don't mind the discussions but it also made me thinks about the boundaries of when they can write an AOC.

The main discussions were about the risk assessment and the stakeholder analysis. They claimed we missed a few stakeholders and some risks. Though we all agreed on the fact they we should add them, the discussion was about wether or not the auditor is allowed to write an AOC because (s)he doesn't agree with the content of it. Especially the risk assessment is a lively and dynamic document which will change due to changes in the surrounding of the company.

With the old 2008 standard we never had any discussion on the content of a procedure because that doesn't affect the management system, though it can have an influence on the effectiveness of it.

Do you have any idea what the boundaries of an auditor are with respect to the content of certain documents?

 

That's the problem with these broad amorphous concepts included in these standards. Who is the auditor to claim they know more than those who run the organization? And where does it stop - you can always find something else which should have been included.

 

You have met the intent of the standard by determining stakeholders and determining risks. I agree with Golfman25 that there is always something else and that this is the domain of the organization (not an external). At most it could be an opportunity for improvement since you found the discussion helpful and are in agreement with the discussions.

 

Thank you all for you feedback. Yes the discussion we had was fruitful and it gave us some important insight and I do understand that the new standard is also new for a lot of auditors. These discussions will not only help the company but also the auditors.

@Andy thanks for pointing out the rules of an AOC. I will dive into that point specifically and figure out what the standard actually says about it.

 

What is AOC?

 

Interesting discussion for a couple of reasons:

1. It covers the approach of at least one CB in assessing organisational context - stakeholder analysis and assessment of risk and opportunity from the OP. I am interested in how they go about it and whether they are covering the topic in sufficient detail and proportionally
2. The role of the AOC (area of concern (from 17021)) in a Stage 1 audit, and
3. When an auditor should record a finding.

I've got some comments on each below:

1. For a CB (and their representative(s) on the audit team) to be competent they have to understand the context of the organization and that includes key groups of stakeholders and what is important to them (risks and opportunities). They have to use this understanding to test the client's system's effectiveness in identifying: stakeholders, risks, opportunities and using this analysis to develop their QMS.
2. The AOC is used at a stage 1 assessment to identify any risk to the organization not meeting standard requirements at the Stage 2 audit. Operating on either side of the fence in certification I wouldn't care if this is an NCR, an Observation or an AOC. The key point is that it is highlighted at the time and the organisation has time to do something about it before Stage 2. It is after Stage 2 assessment that the certification decision is made.
3. This is probably the most difficult one to judge without the facts. The requirement in 9001 is clearly documented, the evidence presented by the organisation was, I assume, also clear; so the only question is whether the evidence satisfies the requirements. Typically how this plays out is the auditor looks at the evidence and identifies what (s)he thinks is a gap and then the two parties discuss. If the organization cannot convince the auditor that the gap does not exist then the auditor raises a finding - an AOC / Observation / NCR.

If the organization does not believe the finding exists they should take this back to the CB with their case. We don't want to get to the situation (as we did in EMS with environmental aspects / legislation) whereby this area becomes a 'battle of the lists' and anyone with a shorter list than the auditor can expect an NCR.

 

A more precise definition of a concrete Stage 1 objective is to review (not audit) the client’ management system documented information (ref ISO 17021:2015 Part 1 cl. 9.3.1.2.2). Review is determination of the suitability, adequacy or effectiveness of an object to achieve established objectives (ISO 9001:2015 cl. 3.11.2).

 

How does the auditor know who your stakeholders are? If you have already defined it in your Context (although the documentation is not a requirement) then so be it. Did the auditor have a canned training of what should constitute as the stakeholders?

 

I have an auditor tell me what should be my process and even gave me a template to identify a value and non-value adding processes. I hope during my stage 1 this will not happen.

 

Auditor should have experience of auditing organizations from the relevant sector. A manufacturing organization may "forget" determine such interested parties relevant to QMS as services providers (repair, calibration, verification, training, attestation), energy suppliers, logistics contractors, banks, NGO, etc. The next step is to determine relevant requirements of these interested parties and make a decision as to which requirements the organization subscribes to. Commitments stated in the Q policy will relate in particular to these requirements. Risks not to fulfill these requirements will be addressed.

 

2011 version of ISO 17021 required to audit MS documentation at Stage 1. Audit is a more stringent check than review; it delivers audit findings: conformity or non-conformity. Audit is now replaced by doc review.

 

The boundaries are set by the organization keeping the standard as the guidance document. It is your call to clarify to the auditor in as much detail as possible what your business context is and therefore who are your stakeholders and how much their needs and expectations get into your risk assessment.
Now in a stage 1 situation where you agree with the auditor, its perfect that a gap exists and an understanding is created, which will help the organization when it goes into the stage 2.
It is certain that you know more and better about your organization context, than that auditor in stage 1., and around this if some stakeholders needs and expectations are unclear, the need here is for clarity of the business context, before an AOC gets decided.
Take this case: If you are in the business of custom made hardware, then the needs and expectations of your customer who is certainly an interested party are with far greater risk levels. Every customer needs and expectations are a project with their own risks as you assess.
Now if you are in the business of catalog based non ferrous hardware, your customer is again a very important interested party, however the risks levels are far less, when it comes to customer needs and expectations. If your customer wants a SS hardware, your plain answer is NO.
Your stage 1 auditor must be sensitive to such risks and these must be discussed and arrived at a common understanding between you and auditor.

 

OK if the organization is already ISO9001:2008 and is transitioning to the 2015 standard, do the auditors need to conduct stage 1 and 2? I was hoping that the auditors will focus on the new requirements of the ISO9001:2015.