1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
Dismiss Notice
You must be a registered member in order to post messages and view/download attached files in this forum.
Click here to register.

A talk, or phone call, objective evidence?

Discussion in 'ISO 9001:2015 - Quality Management Systems' started by Qualmx, Oct 26, 2019.

  1. Qualmx

    Qualmx Well-Known Member

    Joined:
    Oct 7, 2015
    Messages:
    464
    Likes Received:
    59
    Trophy Points:
    27
    Location:
    Mexico
    Hello everybody
    According to 19011, regarding evidences in audits.

    3.8
    objective evidence
    data supporting the existence or verity of something
    Note 1 to entry: Objective evidence can be obtained through observation, measurement, test or by other means.
    Note 2 to entry: Objective evidence for the purpose of the audit (3.1) generally consists of

    When auditing internally the processes, as I have understood, I always look for physical records,
    signed document of a product release, a signed document of Corrective action closeouts,
    a look into a software to identify a check mark, in the approval of a document.
    But I wonder if in 3.8, will fit as options to have as objective evidence of a non compliment,a talk (said declaration), or a phone call?
    one example: in the production area, auditing 7.3 awareness, interviewing to employees face to face , asking about the quality policy, and one employee answers , " I don´t know it" is that an objective evidence? or should I go further and what to ask for?
    Or maybe an interview by phone and getting same response.
    On the other hand, suppose in an area the employees don´t wear safety glasses and clearly is a non conformity, is it possible to raise a NC, without interviewing them? what was seen by auditor
    is an objective evidence?

    Please give some advice

    Thanks






     
  2. Andy Nichols

    Andy Nichols Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    5,086
    Likes Received:
    2,553
    Trophy Points:
    112
    Location:
    In the "Rust Belt"
    This is an incorrect understanding to limit evidence to only documents. What criteria are you using when auditing? ISO 9001?
     
  3. Qualmx

    Qualmx Well-Known Member

    Joined:
    Oct 7, 2015
    Messages:
    464
    Likes Received:
    59
    Trophy Points:
    27
    Location:
    Mexico
    As I said, signed printouts, digital approvals into a software, a list (paper or digital).
    That Is the reason I do post issues here, learning from you, very expert guys.
    Thanks
     
  4. Andy Nichols

    Andy Nichols Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    5,086
    Likes Received:
    2,553
    Trophy Points:
    112
    Location:
    In the "Rust Belt"
    This isn't audit criteria. What audit criteria are you using?
     
  5. Qualmx

    Qualmx Well-Known Member

    Joined:
    Oct 7, 2015
    Messages:
    464
    Likes Received:
    59
    Trophy Points:
    27
    Location:
    Mexico
    Internal documented procedures.
     
  6. Andy Nichols

    Andy Nichols Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    5,086
    Likes Received:
    2,553
    Trophy Points:
    112
    Location:
    In the "Rust Belt"
    Do these state what is required?
     
  7. Golfman25

    Golfman25 Well-Known Member

    Joined:
    Nov 6, 2015
    Messages:
    816
    Likes Received:
    402
    Trophy Points:
    62
    An employee's answer to your question is objective evidence. However, I would be hesitant to leave it at that without inquiring further to make sure there isn't a simple misunderstanding.
     
    Qualmx likes this.
  8. RoxaneB

    RoxaneB Moderator Staff Member

    Joined:
    Jul 31, 2015
    Messages:
    926
    Likes Received:
    1,081
    Trophy Points:
    92
    Location:
    Ontario, Canada
    Documentation and records are nice, but they are not always feasible or practical. Your own excerpt from 19011 3.8 speaks to "...by other means." A well-rounded, robust audit finds the balance between the physical evidence (i.e., records) and the "softer" evidence (i.e., using our ears and eyes for things that are said...or not said).

    This means talking with people. Not one person...PEOPLE. You don't just look at one record and conclude the process is working...you look at a few records. And if they're all good...great. If there are some that are questionable, you look at more records to determine if there is a systemic issue or a one-time hiccough. But getting back to the people, if everyone gives the same answer, you have objective evidence gained through interviews of consistency. That consistency may support conformance to a process...or it may support the identification of a nonconformance. You speak with a few people - if they're all good, great. If there are some questionable responses, you speak to a few more people to determine if the issue is systemic or localized. Same process as records.

    I don't know how you ask someone about the quality policy, but, to Golfman's point, there could be a misunderstanding resulting in the "I don't know" response...or perhaps the auditee is nervous...or perhaps there is some other variable here. Personally, I never ask "What is the quality policy." What if they've never heard the term before. Maybe the company doesn't call it a quality policy. Besides, any parrot can memorize the quality policy and repeat it back. I prefer asking questions about how that individual's role impacts/supports quality...or how they impact the customer's satisfaction with the product/service. These are open-ended questions with responses that should support and be in line with the company's quality policy. And the answers add more value to the audit. Which do you think leadership would gain more insight from - you said "All the employees know the QP" or "The employees really understand their role in quality and take pride in meeting client expectations. In fact, some answers heard were..."

    As for your safety glasses example, I disagree...you don't have a clear understanding of what the situation is, so your N/C would add little value. There is a movie called "Labyrinth" where the female lead is a teenage girl. At one point, she has to go left or right, and decides to go one way. A creature within the labyrinth says "Don't go that way! Never go that way!" The girl says "Thanks!" and runs off the other way. The creature mutters to himself after the girl has run off, "If she had gone that way, it would have taken her to the centre of the labyrinth." Which, ironically, is EXACTLY where the girl was trying to reach. The girl never asked the creature why she shouldn't have gone the way she had originally planned.

    With your safety glasses - maybe the auditee forgot (we ALL have bad days and would you really issue an NC for that?) or maybe the budget has no more funds for safety equipment (that's a planning issue) or maybe he wasn't trained (training and competency issue). So what if you observe something...as a auditor, we should understand the context of what we are observing.
     
    Katrijn, tony s and Qualmx like this.
  9. Andy Nichols

    Andy Nichols Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    5,086
    Likes Received:
    2,553
    Trophy Points:
    112
    Location:
    In the "Rust Belt"
    This, in particular, is an issue which has come from external auditors not understanding a requirement and, taking the requirement too literally as part of their audit. Also, the result, including the reasons you gave, Roxane, stem from organizations who invest nothing in understanding the requirement and articulating it to the workforce. We usually have no difficulty with a number of other policies, like safety, smoking etc but the quality policy usually ends up as meaningless drivel...
     
    tony s likes this.
  10. tony s

    tony s Well-Known Member

    Joined:
    Sep 10, 2015
    Messages:
    1,350
    Likes Received:
    1,054
    Trophy Points:
    112
    Location:
    Laguna Philippines
    As per ISO 19011:2018, objective evidence in the context of audit "consists of records, statements of fact, or other information which are relevant to the audit criteria and verifiable". Therefore, auditors should not just rely on records as a form of evidence. A management review meeting attendance sheet might include the name and signature of the top management, to demonstrate commitment, but was never really there.

    The International Accreditation Forum Auditing Practice Guide on Evidence Collection mentioned this: "Auditors should be aware that objective evidence does not necessarily depend on the existence of documented information, except where specifically mentioned in ISO 9001". There are "other means" to obtain objective evidences depending on the criteria being evaluated. An example is provided by ISO/TS 9002:2016 on auditing against clause 4.4.1b, it clarified that: "when determining the sequence and interaction of these processes... different methods can be used, such as retaining or maintaining documented information (e.g. process maps or flow diagrams), or a more simple approach, such as a verbal explanation of the sequence and interaction of the processes".
     
    Andy Nichols and Qualmx like this.
  11. Qualmx

    Qualmx Well-Known Member

    Joined:
    Oct 7, 2015
    Messages:
    464
    Likes Received:
    59
    Trophy Points:
    27
    Location:
    Mexico
    Thanks Roxane
    I understand your point, the way I see it is that you focus in the process approach, which is to evaluate how the whole process works, and if the "fail" is , isolated (one person) or widespread (several people), is it like this?
    You don´t audit just one person but several persons in order to perform a good and objective job, is it good my asumption?
    Question, if you find the NC in one or several persons, what your write down is just the NC, or you state major or minor? (in case of internal audits).
    or what do your declare in the audit report if it is a nc localized or systematic?

    Because other option could be to just write down the NC , without specifying minor or major, expecting
    that audited process will take the appropriate actions according to your report, applying a corrective action or just a correction.

    A practice, I was following (wrong) at auditing one process, most of the times the only interviewed person, was the owner of the process (manager or supervisor), not all the people which are part of the process, (operators, helpers)
    Please give your comments.

    Thank you so much.
     
  12. Andy Nichols

    Andy Nichols Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    5,086
    Likes Received:
    2,553
    Trophy Points:
    112
    Location:
    In the "Rust Belt"
    Internal auditors shouldn't be doing this. It has no value/meaning to management.

    It depends on who does the work and what you're asking. You haven't told us much about the background to your questions. It's difficult to give clear directions. I asked earlier, what you are auditing against (criteria) It's also useful to know WHY you are auditing. Is there a problem with the process?

    Again, without knowing more, we can't give an authoritative answer. Why are you doing this audit? Your situation may be coming from a lack of clarity when planning what to audit and what criteria you're using. Unless you address these issues here, we are guessing.
     
  13. RoxaneB

    RoxaneB Moderator Staff Member

    Joined:
    Jul 31, 2015
    Messages:
    926
    Likes Received:
    1,081
    Trophy Points:
    92
    Location:
    Ontario, Canada
    That is essentially what I was saying....but each situation is different and an auditor must adapt accordingly.

    See Andy's response.

    Impossible for us to answer. It depends on variables like has it occurred before, what is the risk of the nonconformance occurring, etc.

    What is the value of adding minor or major? Who benefits from that? If it doesn't add value to the solution, why bother doing it all? The appropriate actions should be taken regardless if minor or major is there.

    While agree with Andy that there is much that we don't know so we really are unable to provide a definitive response, I, personally, am in favour of speaking with more than one person where ever possible. I mean, theoretically, one could do an entire audit just by sitting with senior management - presuming they know where to find records and discuss the nonconfomance process, etc. I like starting with management and then working my way down to the floor to see if the two worlds align.

    For example, if management talks about objectives and the metrics that align with the objectives, and then goes on to discuss how the metrics are cascaded down to the shop floor, you better believe I'm then going to ask out the metrics while I'm out there. If the floor folks can speak to the metrics, then all is good. If the floor folks look at me and ask "What metrics?", we have a disconnect.
     
    Qualmx and Andy Nichols like this.
  14. Andy Nichols

    Andy Nichols Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    5,086
    Likes Received:
    2,553
    Trophy Points:
    112
    Location:
    In the "Rust Belt"
    In my world, most organizations don't have many others to talk to! Small - Medium size businesses
     
    Qualmx likes this.
  15. Qualmx

    Qualmx Well-Known Member

    Joined:
    Oct 7, 2015
    Messages:
    464
    Likes Received:
    59
    Trophy Points:
    27
    Location:
    Mexico
    Thanks Andyn and Roxane

    Excelent sharing of experiences in auditing.
    Finally in this point.


    Qualmx said:
    Question, if you find the NC in one or several persons, what your write down is just the NC, or you state major or minor? (in case of internal audits).
    or what do your declare in the audit report if it is a nc localized or systematic?

    Impossible for us to answer. It depends on variables like has it occurred before, what is the risk of the nonconformance occurring, etc.

    Why so impossible?
    in the case of using categories Minor or major, don´t you think is just to apply what is recommended in 19011?
    or in the case of localized or systematic, don´t you think is as simple as: if NC was detected in one or two person, is localized, if found in the whole process,say 5 or 9 persons, then is systematic, why have to be so difficult to define it?

    tThanks
     
  16. Andy Nichols

    Andy Nichols Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    5,086
    Likes Received:
    2,553
    Trophy Points:
    112
    Location:
    In the "Rust Belt"
    Does it recommend this?

    Who are you doing audits for? You? To satisfy a perceived requirement in ISO? Or to be of benefit to management and the organization as a whole?
     
  17. Theodora Rondozai

    Theodora Rondozai New Member

    Joined:
    Nov 4, 2019
    Messages:
    2
    Likes Received:
    1
    Trophy Points:
    2
    NCs are typically raised on the problem manifesting, not it's cause, which as you point out, is not immediately evident. That's an issue for those allocated the responsibility to investigate and establish the cause. So if safety glasses are required, and observed to not be in use, then yes, an NC would be raised (bad day, stretched budget or otherwise). The NC is not apportioning blame to the auditee, it is simply highlighting a (system) failure to comply with a stated requirement, which may compromise the process, and therefore needs action.
     
    Qualmx likes this.
  18. RoxaneB

    RoxaneB Moderator Staff Member

    Joined:
    Jul 31, 2015
    Messages:
    926
    Likes Received:
    1,081
    Trophy Points:
    92
    Location:
    Ontario, Canada
    If you wish to issue an NC for someone not wearing their safety glasses, go for it, however, you run the risk of internal auditors being labelled as cops/police and simply a team that attempts to "catch" people doing the wrong things all while adding little to no value to the organization.

    Context and risk assessment are key. What if someone had just stepped out of an office, on to the shop floor, but not yet in the main area where all the magic of production happens? Would you honestly issue an NC for that? If so, you've done nothing but create extra paperwork for the organization and accomplished nothing in the way of helping the organization. If you see the person without the glasses and not yet in a high-risk area, a quiet word with the individual will suffice. How they respond could then determine the issuance of an NC - if they ignore you and continue on, NC; however, if they slap their forehead and say thanks for the reminder while putting their glasses on, no harm done and you now come across as someone who is more focused on helping than slapping the handcuffs on wrong-doers.
     
  19. Andy Nichols

    Andy Nichols Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    5,086
    Likes Received:
    2,553
    Trophy Points:
    112
    Location:
    In the "Rust Belt"
    I'm with Roxane. If this is a QMS audit and this is the ISO 9001:2015 forum, a PPE issue is best addressed to HR/Safety management NOT as part of a QUALITY audit - out of scope
     
  20. Jennifer Kirley

    Jennifer Kirley Moderator Staff Member

    Joined:
    Jul 31, 2015
    Messages:
    1,071
    Likes Received:
    722
    Trophy Points:
    112
    Location:
    USA
    Roxane and Andy are right. This is not a subject for a QMS audit nonconformance. Continue on your way but take up the issue with the responsible managers outside of the audit.
     
    Qualmx likes this.