1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
Dismiss Notice
You must be a registered member in order to post messages and view/download attached files in this forum.
Click here to register.

Virtual Audit possibility

Discussion in 'ISO 9001:2015 - Quality Management Systems' started by Mayor, May 2, 2020.

  1. Mayor

    Mayor Member

    Joined:
    Dec 22, 2019
    Messages:
    13
    Likes Received:
    0
    Trophy Points:
    1
    Good day house.

    My organization has a tradition of conducting internal audit in May of every year and this has been for the past 8 years since the organization's first ISO certification.

    Management have thus considered the option of conducting a virtual audit for this year and the plan is to have it next week. My concern as the QMS Advisor are as follows:

    1. Will the virtual Audit be as effective as the traditional one-on-one audit engagement? Particularly in revealing true nature of operations with specific depts & process owners.

    2. What is the standpoint of ISO Standard regarding virtual Audit? Is it acceptable?

    3. Will virtual Audit not give room for window dressing on the part of site managers? Considering that the organization has over 55 operational sites with a minimum of 4 man-team per site.

    4. Is it possible to request for recertification audit extension from the CB? The 9001 certification cycle will expire in June.

    Thank you as always and I'm standing by for your response.
     
  2. Andy Nichols

    Andy Nichols Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    5,086
    Likes Received:
    2,553
    Trophy Points:
    112
    Location:
    In the "Rust Belt"
    Mayor, I have a number of comments:

    Firstly, with 55 Operational sites doing ONE internal audit? You're worried about being superficial as a remote audit? Your internal audit once a year IS ALREADY superficial. To be very honest, you should never have been allowed to be certified doing this. It's already not compliant with ISO 9001.

    ISO 9001 doesn't care about HOW audits are done. Clearly, your Certification Body don't have a clue either. You won't need an extension. If your CB is an IAF accredited one, they will possibly being doing remote audits for surveillance visits.

    I strongly suggest you re-consider the purpose of internal audits and look at better ways to do them - other than to keep a certificate on the office wall.
     
    Jessica R likes this.
  3. Mayor

    Mayor Member

    Joined:
    Dec 22, 2019
    Messages:
    13
    Likes Received:
    0
    Trophy Points:
    1
    Thank you Andy for your response.

    Please be informed that we do an annual internal audit but it runs through 1 to 2 months. We always ensure that all sites are visited and we don't mind as long as it takes to complete that.

    You said we have not been complying with ISO 9001 because we conduct one internal audit a year. I totally disagree with you on that, and could you be kind to point out the clause in the standard that has been violated?

    Also, need to say that nearly 45% of the site personnel have been trained as auditors so it is assumed they know the expectations on the sites. Currently we have about 15 lead auditors who actively participate in the internal audits, this is aside the 45% ISO trained site personnel.

    So clearly no requirement has been crossed. I think you have assumed wrongly about my organization.

    And yes, the CB is Bureau Veritas, accredited to UKAS and ANAB
     
  4. Andy Nichols

    Andy Nichols Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    5,086
    Likes Received:
    2,553
    Trophy Points:
    112
    Location:
    In the "Rust Belt"
    Mayor:

    The requirements for internal audits are for an audit program to be established which takes into consideration the importance of the process(es) etc. I'm not going to quote it as you know what it says.

    By doing one (massive) internal audit you are not considering importance of processes and changes affecting the QMS. What you are doing is considering EVERYTHING equally. Of 55 sites, that suggests that everyone gets the same results, all the time, every time. Have there been ZERO changes in all this time? I'd also like to know what are the scopes and what criteria you are auditing against?

    I haven't assumed anything. What I have is lots of experience of seeing internal audits done, which - to all intents and purposes - replicate what the CB does and nothing more. Of course, it's also common that, when you - in essence - copy what the CB auditor does, they get lazy because you've done their work. Rubber stamp!

    When one internal audit is done, it is insensitive to the issues the business faces and, worse, non-conformities can be reported many, many months (if the auditors get lucky) after an event. That's of zero use to management.

    In addition to knowing what the scopes and audit criteria are, would you share how much your management team participate in selecting where and what to audit?

    Further, I would also question your management review planning. How often do you do a management review? There's a clue here.
     
  5. Andy Nichols

    Andy Nichols Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    5,086
    Likes Received:
    2,553
    Trophy Points:
    112
    Location:
    In the "Rust Belt"
    Mayor:

    Let's do this: Setting aside the need for your 55 locations to be ISO 9001 certified, what do you understand is the need for internal audits? Why would the writer of ISO 9001 include the need to verify the QMS is being implemented as planned?
     
  6. RoxaneB

    RoxaneB Moderator Staff Member

    Joined:
    Jul 31, 2015
    Messages:
    926
    Likes Received:
    1,081
    Trophy Points:
    92
    Location:
    Ontario, Canada
    Hi, Mayor...while you and Andy debate the reasons behind internal audits, I'll share my own experiences with virtual audits from a few years back.

    First, some context - my first exposure to them was conducting them solely on our Sales processes for an international steel manufacturer. The team that owned these processes was based in the USA, while I sat in Canada. My internal audits, traditionally face-to-face, on the Sales processes over several years was shared with multiple operational locations around Canada and the USA who were pursuing ISO 9001 (my site was already registered), and was successfully used as evidence of internal audits of Sales processes, including how their outputs became the inputs for our operational locations. The decision to go virtual one year was not made lightly, but was political - my site was continually paying for this audit (i.e., my business expenses) while the other sites benefited from it. Part of the decision was, however, based on the historical audit results of the Sales processes - they were usually pretty good. The virtual audit was accepted by our CB

    Fast forward a few years and I had shifted over into not-for-profit healthcare. Tasked with coming up with a method to audit our 13 sites for conformance to our own documented processes, my first attempt was done in a virtual setting (i.e., I was given a tight timeline with no travel budget, so I had to work within the parameters...yes, the constraints were mentioned as a possible risk factor in my reports).

    So, is it as effective as traditional auditing? No. Even with video conferencing capabilities you lose some visibility to people's non-verbal responses - especially in a group setting. If there is a group, the camera is set too far back for you to see much...or if it's zoomed in on one person, you cannot see how others are reacting. It was my experience, as well, that people, both auditors and auditees, were too subject to distractions - more so in a virtual audit than in a face-to-face approach.

    If screen sharing capabilities are lacking or impossible, evidence is sent in after the fact with no context or explanation. This means more time is required by the auditor to process and align with notes, or sometimes, even reaching out to the auditee (again!) for more information.

    Virtual audits become more about checking off boxes (e.g., Do they do Step 1? Check. Do they do Step 2? Check.) and less about value-add discussions (i.e., looking at the connections between processes, how outputs become inputs, and effectiveness of these connections).

    As mentioned, this approach was taken while I was in the steel manufacturing world (full disclosure - that was over 8 years ago) and it was accepted by our CB.

    Yes, it can be treated superficially by site management. Again, because the approach feels more like going through a checklist rather than something that can add value to their results, it would not surprise me to hear that sites are less engaged if a virtual audit is the approach taken.

    More time needed. People less engaged. Results that do little to help the organization. Virtual audits may help you keep your certification, but they will likely not help any more than that.

    Have you reached out to your CB to ask if they can share the internal audit approaches taken by their clients (without naming them) who are similar in size/set-up to your organization?

    Presuming your 55 sites are operating, are there people there on-site who can conduct the internal audits in a face-to-face manner with virtual support from you/your team?
     
  7. Mike S.

    Mike S. Member

    Joined:
    Apr 20, 2020
    Messages:
    37
    Likes Received:
    7
    Trophy Points:
    7
    A company I know of had it right in their quality procedures from day 1 that internal audits would be done once per year, and the registrar accepted it and it was in place for at least 2 years / 2 audits until I pointed out that it wasn't compliant.
     
  8. Andy Nichols

    Andy Nichols Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    5,086
    Likes Received:
    2,553
    Trophy Points:
    112
    Location:
    In the "Rust Belt"
    Indeed, it's all too common. It shouldn't be that way and I wonder what the Accreditation Body auditors know about auditing outside of their narrow viewpoint of the Certification world, that they allow such poor practices...
     
    John C. Abnet likes this.
  9. Andy Nichols

    Andy Nichols Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    5,086
    Likes Received:
    2,553
    Trophy Points:
    112
    Location:
    In the "Rust Belt"
    Which is why (in part) the IAF are not permitting CBs to perform stage 2 or recertification audits without a review of scope and complexity of the client's QMS. ICT audits (virtual audits) are really only applicable to surveillance or stage 1 audits.
     
  10. RoxaneB

    RoxaneB Moderator Staff Member

    Joined:
    Jul 31, 2015
    Messages:
    926
    Likes Received:
    1,081
    Trophy Points:
    92
    Location:
    Ontario, Canada
    Fair enough, however, I thought the scope of @Mayor 's initial question was regarding the efficacy of virtual internal audits, not virtual audits performed by CBs.
     
    Jessica R and Mayor like this.
  11. Andy Nichols

    Andy Nichols Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    5,086
    Likes Received:
    2,553
    Trophy Points:
    112
    Location:
    In the "Rust Belt"
    I didn't make my comment very clear. I agree that the audit's not going to be very effective when conducted over the internet. You do miss a lot of things going on. I was illustrating that the "authorities" in auditing agree it's not the way to go, in that although they are sanctioning "ICT" audits, for the paperwork-oriented types, the "boots on ground" audits are still required because of the need you'd described.
     
  12. Mayor

    Mayor Member

    Joined:
    Dec 22, 2019
    Messages:
    13
    Likes Received:
    0
    Trophy Points:
    1
    Our management review is done once a year.

    For us, audit is not the only way to check our compliance to ISO requirements. We do a number of internal programs to keep us in shape from beginning of the year till the end. We have a robust change management system and procedures, we have a interdepartmental SLA compliance system, and a whole lot of other automated programs to ease the physical/traditional means of documentation. We only have a few KBAs that are not automated.

    Also, the virtual internal audit we are looking at is to consider audit scope of 70%, while remaining 30% is to be completed physically.

    I asked about the reliability of the remote audit because it is new to the organization and this is the first time we are considering it.
     
  13. Andy Nichols

    Andy Nichols Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    5,086
    Likes Received:
    2,553
    Trophy Points:
    112
    Location:
    In the "Rust Belt"
    As with internal audits, management reviews - or anything done once a year (calibration, equipment maintenance, training etc) isn't going to give any value in my experience. Looking at something which happened 11 months ago is a complete waste of time and only being done "because ISO-says-so"
     
  14. RoxaneB

    RoxaneB Moderator Staff Member

    Joined:
    Jul 31, 2015
    Messages:
    926
    Likes Received:
    1,081
    Trophy Points:
    92
    Location:
    Ontario, Canada
    Not quite sure what you mean by this.
     
    Andy Nichols likes this.
  15. Jessica R

    Jessica R Member

    Joined:
    Mar 18, 2020
    Messages:
    9
    Likes Received:
    4
    Trophy Points:
    2
    Location:
    Georgia
    We discussed a virtual external audit method recently, as most organizations likely have considered with respect to the pandemic. The investigation started after our registrar advertised the option, but we decided to change registrars as we were at the end of our assessment schedule and change was needed.
    However, during the discussion we were going to practice internally during our quarterly audit cycle. The primary caveat was resources. There was a mutual understanding that technological resources and their reliability would be key to performing a remote audit in order to best satisfy the need for observation and evidence collection. Bluetooth video and audio glasses, screen sharing software, portable interactive display(s) - everything was considered in order to best mimic being physically present.
    Why not try a sample or mock remote audit and gap analyze the results? It would be worth the validation effort.
    My first choice remains, as an internal auditor, to be in-person.
     
    Andy Nichols likes this.
  16. Andy Nichols

    Andy Nichols Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    5,086
    Likes Received:
    2,553
    Trophy Points:
    112
    Location:
    In the "Rust Belt"
  17. Andy Nichols

    Andy Nichols Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    5,086
    Likes Received:
    2,553
    Trophy Points:
    112
    Location:
    In the "Rust Belt"