1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
Dismiss Notice
You must be a registered member in order to post messages and view/download attached files in this forum.
Click here to register.

Boundaries of the external auditors.

Discussion in 'ISO 9001:2015 - Quality Management Systems' started by Nick1, Feb 14, 2016.

  1. Nick1

    Nick1 Member

    Joined:
    Jan 27, 2016
    Messages:
    49
    Likes Received:
    20
    Trophy Points:
    7
    Last week one my customers had their first ISO9001:2015 stage 1 documentation audit. The audit went pretty well but we did had some tough discussions with the auditors. I honestly don't mind the discussions but it also made me thinks about the boundaries of when they can write an AOC.

    The main discussions were about the risk assessment and the stakeholder analysis. They claimed we missed a few stakeholders and some risks. Though we all agreed on the fact they we should add them, the discussion was about wether or not the auditor is allowed to write an AOC because (s)he doesn't agree with the content of it. Especially the risk assessment is a lively and dynamic document which will change due to changes in the surrounding of the company.

    With the old 2008 standard we never had any discussion on the content of a procedure because that doesn't affect the management system, though it can have an influence on the effectiveness of it.

    Do you have any idea what the boundaries of an auditor are with respect to the content of certain documents?
     
    Somashekar likes this.
  2. Golfman25

    Golfman25 Well-Known Member

    Joined:
    Nov 6, 2015
    Messages:
    816
    Likes Received:
    402
    Trophy Points:
    62
    That's the problem with these broad amorphous concepts included in these standards. Who is the auditor to claim they know more than those who run the organization? And where does it stop - you can always find something else which should have been included.
     
    Nick1 likes this.
  3. Carol Robinson

    Carol Robinson New Member

    Joined:
    Dec 2, 2015
    Messages:
    2
    Likes Received:
    1
    Trophy Points:
    2
    You have met the intent of the standard by determining stakeholders and determining risks. I agree with Golfman25 that there is always something else and that this is the domain of the organization (not an external). At most it could be an opportunity for improvement since you found the discussion helpful and are in agreement with the discussions.
     
    Nick1 likes this.
  4. Andy Nichols

    Andy Nichols Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    5,086
    Likes Received:
    2,553
    Trophy Points:
    112
    Location:
    In the "Rust Belt"
    Well, the fact is, sometimes they do! It may not be your particular experience, but I can assure you that quite a few auditors are also very good management consultants and, as such they frequently know more than some of the organizations they visit. For example, I'd like a $ for every client I've visited who didn't have much of a clue on how to run an effective calibration program, but instead calibrated everything once a year...How much money were they throwing away?
     
  5. Andy Nichols

    Andy Nichols Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    5,086
    Likes Received:
    2,553
    Trophy Points:
    112
    Location:
    In the "Rust Belt"
    I think over the next 3 years or so, there's going to be a lot of these discussions. As long as the outcome was amicable, I see no problem.
    I would imagine the rules for writing an NC (what's an AOC?) haven't changed so that unless there's objective evidence, you might only see a "heads up" written in the visit report.
     
    Nick1 likes this.
  6. Nick1

    Nick1 Member

    Joined:
    Jan 27, 2016
    Messages:
    49
    Likes Received:
    20
    Trophy Points:
    7
    Thank you all for you feedback. Yes the discussion we had was fruitful and it gave us some important insight and I do understand that the new standard is also new for a lot of auditors. These discussions will not only help the company but also the auditors.

    @Andy thanks for pointing out the rules of an AOC. I will dive into that point specifically and figure out what the standard actually says about it.
     
    Andy Nichols likes this.
  7. Jennifer Kirley

    Jennifer Kirley Moderator Staff Member

    Joined:
    Jul 31, 2015
    Messages:
    1,071
    Likes Received:
    722
    Trophy Points:
    112
    Location:
    USA
    First of all, no AOCs (nonconformances?) should be written at a Stage 1, which is meant merely to confirm the client is ready to move to Stage 2. CB auditors should be looking for 4.1, 4.2 etc. conformed to but this is not the time for action requests.

    Secondly, nothing in my training, for even Stage 2, indicates I am to say "You missed one" unless there is a clear reason they should have been included and their lack is problematic, or could be so. Otherwise, an OFI would do.

    There is no doubt going to be variation in how CB auditors perform in this. I am concerned they will ask for things that are not in the standard (like a process map) or over-reach, like this example shows. The Stage 1 procedures for this should be clearly promulgated within the CB, but I have seen mixed messages and so I regret to say this is not surprising though it is dismaying.
     
    Andy Nichols likes this.
  8. MCW8888

    MCW8888 Well-Known Member

    Joined:
    Aug 17, 2015
    Messages:
    642
    Likes Received:
    198
    Trophy Points:
    42
    What is AOC?
     
  9. Paul Simpson

    Paul Simpson Member

    Joined:
    Aug 6, 2015
    Messages:
    41
    Likes Received:
    61
    Trophy Points:
    17
    Interesting discussion for a couple of reasons:
    1. It covers the approach of at least one CB in assessing organisational context - stakeholder analysis and assessment of risk and opportunity from the OP. I am interested in how they go about it and whether they are covering the topic in sufficient detail and proportionally
    2. The role of the AOC (area of concern (from 17021)) in a Stage 1 audit, and
    3. When an auditor should record a finding.
    I've got some comments on each below:
    1. For a CB (and their representative(s) on the audit team) to be competent they have to understand the context of the organization and that includes key groups of stakeholders and what is important to them (risks and opportunities). They have to use this understanding to test the client's system's effectiveness in identifying: stakeholders, risks, opportunities and using this analysis to develop their QMS.
    2. The AOC is used at a stage 1 assessment to identify any risk to the organization not meeting standard requirements at the Stage 2 audit. Operating on either side of the fence in certification I wouldn't care if this is an NCR, an Observation or an AOC. The key point is that it is highlighted at the time and the organisation has time to do something about it before Stage 2. It is after Stage 2 assessment that the certification decision is made.
    3. This is probably the most difficult one to judge without the facts. The requirement in 9001 is clearly documented, the evidence presented by the organisation was, I assume, also clear; so the only question is whether the evidence satisfies the requirements. Typically how this plays out is the auditor looks at the evidence and identifies what (s)he thinks is a gap and then the two parties discuss. If the organization cannot convince the auditor that the gap does not exist then the auditor raises a finding - an AOC / Observation / NCR.
    If the organization does not believe the finding exists they should take this back to the CB with their case. We don't want to get to the situation (as we did in EMS with environmental aspects / legislation) whereby this area becomes a 'battle of the lists' and anyone with a shorter list than the auditor can expect an NCR.
     
    tony s and Tony Wardle like this.
  10. Jennifer Kirley

    Jennifer Kirley Moderator Staff Member

    Joined:
    Jul 31, 2015
    Messages:
    1,071
    Likes Received:
    722
    Trophy Points:
    112
    Location:
    USA
    Oh head thumping moment: AOC = Area of Concern. Yes these go into the Stage 1. Sorry to mis-speak!
     
  11. Andy Nichols

    Andy Nichols Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    5,086
    Likes Received:
    2,553
    Trophy Points:
    112
    Location:
    In the "Rust Belt"
    I think, a bit like Jennifer was head thumping, I was asleep when I answered! In several years of the use of ISO 17021 by CBs I have yet to see a comment from the Stage 1 referred to as an "AOC", although that's exactly what they are and Paul, of course, described them perfectly.

    The issue in my mind still has some relevance to the objectivity of the auditor. I encountered a similar situation where an AOC (I could get used to this terminology) was issued because of a lack of internal auditor training certificate at the stage 1... (there's nothing in any procedure to say they HAD to have gone to a training course or maintain a certificate)
     
  12. Leonid

    Leonid Well-Known Member

    Joined:
    Jan 4, 2016
    Messages:
    164
    Likes Received:
    31
    Trophy Points:
    27
    Location:
    Moscow
    A more precise definition of a concrete Stage 1 objective is to review (not audit) the client’ management system documented information (ref ISO 17021:2015 Part 1 cl. 9.3.1.2.2). Review is determination of the suitability, adequacy or effectiveness of an object to achieve established objectives (ISO 9001:2015 cl. 3.11.2).
     
  13. Andy Nichols

    Andy Nichols Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    5,086
    Likes Received:
    2,553
    Trophy Points:
    112
    Location:
    In the "Rust Belt"
    But it's MORE than this... Also, don't forget that an organization submitting to ISO 9001 (and other) certifications doesn't know about ISO/IEC 17021, nor do they want to purchase it...

    So, frequently, people "think" it's a document review because that's what is commonly taught in auditor course!
     
  14. MCW8888

    MCW8888 Well-Known Member

    Joined:
    Aug 17, 2015
    Messages:
    642
    Likes Received:
    198
    Trophy Points:
    42
    How does the auditor know who your stakeholders are? If you have already defined it in your Context (although the documentation is not a requirement) then so be it. Did the auditor have a canned training of what should constitute as the stakeholders?
     
  15. MCW8888

    MCW8888 Well-Known Member

    Joined:
    Aug 17, 2015
    Messages:
    642
    Likes Received:
    198
    Trophy Points:
    42
    I have an auditor tell me what should be my process and even gave me a template to identify a value and non-value adding processes. I hope during my stage 1 this will not happen.
     
  16. Leonid

    Leonid Well-Known Member

    Joined:
    Jan 4, 2016
    Messages:
    164
    Likes Received:
    31
    Trophy Points:
    27
    Location:
    Moscow
    Auditor should have experience of auditing organizations from the relevant sector. A manufacturing organization may "forget" determine such interested parties relevant to QMS as services providers (repair, calibration, verification, training, attestation), energy suppliers, logistics contractors, banks, NGO, etc. The next step is to determine relevant requirements of these interested parties and make a decision as to which requirements the organization subscribes to. Commitments stated in the Q policy will relate in particular to these requirements. Risks not to fulfill these requirements will be addressed.
     
  17. Leonid

    Leonid Well-Known Member

    Joined:
    Jan 4, 2016
    Messages:
    164
    Likes Received:
    31
    Trophy Points:
    27
    Location:
    Moscow
    2011 version of ISO 17021 required to audit MS documentation at Stage 1. Audit is a more stringent check than review; it delivers audit findings: conformity or non-conformity. Audit is now replaced by doc review.
     
    Somashekar likes this.
  18. Somashekar

    Somashekar Well-Known Member

    Joined:
    Jul 30, 2015
    Messages:
    114
    Likes Received:
    98
    Trophy Points:
    27
    The boundaries are set by the organization keeping the standard as the guidance document. It is your call to clarify to the auditor in as much detail as possible what your business context is and therefore who are your stakeholders and how much their needs and expectations get into your risk assessment.
    Now in a stage 1 situation where you agree with the auditor, its perfect that a gap exists and an understanding is created, which will help the organization when it goes into the stage 2.
    It is certain that you know more and better about your organization context, than that auditor in stage 1., and around this if some stakeholders needs and expectations are unclear, the need here is for clarity of the business context, before an AOC gets decided.
    Take this case: If you are in the business of custom made hardware, then the needs and expectations of your customer who is certainly an interested party are with far greater risk levels. Every customer needs and expectations are a project with their own risks as you assess.
    Now if you are in the business of catalog based non ferrous hardware, your customer is again a very important interested party, however the risks levels are far less, when it comes to customer needs and expectations. If your customer wants a SS hardware, your plain answer is NO.
    Your stage 1 auditor must be sensitive to such risks and these must be discussed and arrived at a common understanding between you and auditor.
     
    Last edited: Feb 18, 2016
  19. Andy Nichols

    Andy Nichols Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    5,086
    Likes Received:
    2,553
    Trophy Points:
    112
    Location:
    In the "Rust Belt"
    I think you missed my point. Organizations which implement ISO 9001 are unaware of ISO/IEC 17021. It's up to the CB to communicate expectations of the stage 1, based upon the requirements THEY have to fulfill as a CB. To simply state what ISO/IEC 17021 requires isn't helpful, when it's incomplete. A stage 1 isn't simply a document review...
     
  20. MCW8888

    MCW8888 Well-Known Member

    Joined:
    Aug 17, 2015
    Messages:
    642
    Likes Received:
    198
    Trophy Points:
    42
    OK if the organization is already ISO9001:2008 and is transitioning to the 2015 standard, do the auditors need to conduct stage 1 and 2? I was hoping that the auditors will focus on the new requirements of the ISO9001:2015.