1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
Dismiss Notice
You must be a registered member in order to post messages and view/download attached files in this forum.
Click here to register.

Carte Blanche for an Audit Programme

Discussion in 'Process Audits and Layered Process Audits' started by RoxaneB, Jul 25, 2017.

  1. RoxaneB

    RoxaneB Moderator Staff Member

    Joined:
    Jul 31, 2015
    Messages:
    926
    Likes Received:
    1,081
    Trophy Points:
    92
    Location:
    Ontario, Canada
    I'm back to doing what I was originally hired to do (5.5 years ago) with my current organization and that's develop and implement an internal assessment programme. We are not ISO.

    Essentially, I have carte blanche on creating a user-friendly, meaningful, simple yet robust programme.

    If you had to sell the idea of "why" regarding assessment, what would you say? Thinking about Simon Sinek's philosophy of "People don't buy WHAT you do, they buy WHY you do it", how would get people to care about assessments?

    Secondly, forget ISO...if you could create your own audit methodology, what would it entail? I'll be honest, I have an approach that involves a scorecard approach for practices and metrics, this way people have an idea of where they are and where they need to be...but I know that "scores" can intimidate people (especially where I work - in healthcare - with people so concerned about feelings).

    Think about it...you've been invited to create your own audit programme. What do you do? That's essentially the audiing opportunity that is in my lap at the moment...
     
    Atul Khandekar and normzone like this.
  2. Andy Nichols

    Andy Nichols Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    5,086
    Likes Received:
    2,553
    Trophy Points:
    112
    Location:
    In the "Rust Belt"
    Oh boy! Now you've done it, RCB! Only gone and asked me the question I've always wanted to be asked...

    So, fundamentally, internal audits (aka assessments) are about providing independent confirmation/validation of the use of the QMS as a method to achieve process results. If we look at the (ultimate) purpose of internal audits, as an input to management review, we can see that the review is supposed to consider product conformity and process performance (among others) to the stated goals/objectives. Process owners should bring to the review their data/information on this. A simple response from the internal audit process owner should confirm (independently) that the results can be "believed" since the (defined) process was being followed - or not. In the case of good performance, actions may be taken to improve things and, in cases when the performance isn't on target, corrective action will be needed.

    Factoring ISO Certification out (thank heavens), we can then use internal audit to focus on specific parts of the QMS, almost like a sniper might, to determine if the QMS controls which are planned/implemented are actually doing their job. Which takes us to scheduling audits:

    The audit schedule shouldn't be predicted to an annualized calendar, as is common practice. That's what we call in the automotive world a "push" system. Toyota stopped doing that and operate a "demand flow" or "pull" system for parts - a practice which can be applied by management to internal audits. Management "pull" the internal audits when the have a change (or something new/different) going on. New/changed = risk to them. Most new/changed things don't happen to an annual schedule/calendar (unless you want the non-value added task of constantly updating the plan) so a rolling 30-day window could be considered and the scope and criteria of each audit selected based on what is going on within that 30 days.
     
    normzone and RoxaneB like this.
  3. yodon

    yodon Well-Known Member

    Joined:
    Aug 3, 2015
    Messages:
    198
    Likes Received:
    115
    Trophy Points:
    42
    Wow... (cue background music from Beatle's 'Imagine') wouldn't it be nice to talk to the folks and find out what they're doing, why they're doing it, what's effective, what's a barrier, what's valuable, what's useless... And then if they're not doing what the procedures say, find out why (with no fear of retribution) and how we could get alignment.

    But then we still need to do things to show auditors that we did the things we were supposed to so back to reality. Surely, though, there's a sweet spot. Will be interested to hear how this plays out.
     
    normzone likes this.
  4. RoxaneB

    RoxaneB Moderator Staff Member

    Joined:
    Jul 31, 2015
    Messages:
    926
    Likes Received:
    1,081
    Trophy Points:
    92
    Location:
    Ontario, Canada
    As I start to develop things, I'll see about posting them here...sans company info, naturally.

    The last system I developed was too "complicated" and there was not actioning of results afterward...no follow-up. It was super frustrating.

    I need to develop something that is user-friendly, meaningful, logical, applicable across the board...and potentially useful for beyond audits (e.g., due diligence, organizational growth/acquisitions, etc.). The sky's the limit with this opportunity. A management system dream come true...let's just keep it from becoming a nightmare.
     
    normzone likes this.
  5. RoxaneB

    RoxaneB Moderator Staff Member

    Joined:
    Jul 31, 2015
    Messages:
    926
    Likes Received:
    1,081
    Trophy Points:
    92
    Location:
    Ontario, Canada
    What auditors? Remember, there is no ISO standard applicable here. I have no external requirement indicating that our organization must do audits.
     
  6. Andy Nichols

    Andy Nichols Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    5,086
    Likes Received:
    2,553
    Trophy Points:
    112
    Location:
    In the "Rust Belt"
    It occurs to me that one thing that's often missed from considering any internal audit programme is "what do management do with the results?". Since we aren't going to have to consider external audits/auditors, then life becomes a whole lot more challenging! So, a question might become, if you have the most rock and roll internal audit process, can/would management know what to do with what's reported?
     
    normzone and RoxaneB like this.
  7. RoxaneB

    RoxaneB Moderator Staff Member

    Joined:
    Jul 31, 2015
    Messages:
    926
    Likes Received:
    1,081
    Trophy Points:
    92
    Location:
    Ontario, Canada
    This is a sticking point for me, to be honest.

    I've yet to find - within our organization - a transparent, consolidated action showing who is doing what, why, how, resources, alignment to objectives, etc. Perhaps this would not be place for the audit output to be followed-up on, but we do struggle with that whole approach of acknowledging an opportunity/issue, developing a plan, and following up on it.

    That will need to be considered as part of this program - I've actually put it in as a constraint to the project...a red flag, if you will, that will need to be considered and addressed if this initiative is to succeed.
     
    normzone likes this.
  8. yodon

    yodon Well-Known Member

    Joined:
    Aug 3, 2015
    Messages:
    198
    Likes Received:
    115
    Trophy Points:
    42
    Whoops, sorry, missed that.

    So coupled with your response regarding "what does management do with the data" (and I'm just tossing this out for discussion) why do the audit at all; why not just focus on improvement? Digging into issues / inefficiencies (and then assessing effectiveness of changes) will no doubt involve some audit activity but maybe you don't need a defined process for it?
     
  9. Andy Nichols

    Andy Nichols Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    5,086
    Likes Received:
    2,553
    Trophy Points:
    112
    Location:
    In the "Rust Belt"
    Good question! But improve what? Practice? How would you know what needs improvement?
    How will you know to trust the audit process was free from bias and, therefore, subjective? That's part of the reason for having a defined process...
     
  10. yodon

    yodon Well-Known Member

    Joined:
    Aug 3, 2015
    Messages:
    198
    Likes Received:
    115
    Trophy Points:
    42
    Fair question. I'm coming at this mostly with my small-business, I have my finger on the pulse of most everything viewpoint. (Both for my company and most of my clients). In this case, an internal audit has yet to turn up something I didn't know (and I'll grant you it's a "push" system as you described earlier). This is not an indication of poor auditing.

    In anything called an "audit" (and especially as I've seen in large companies - even internal audits), people are reticent to discuss issues fearing some sort of reprisal. If they say something then their boss could get dumped on and that stuff rolls downhill. I realize this is a corporate culture thing but in the bigs I've worked in (granted, not a big sample size), this was certainly the case. I've also seen this in the bigs we've worked with, though. I think your rolling 30-day window approach has merit and with it being continuous, you would probably shake the 'baggage' with "audits."

    The things that need improvement aren't all that hard to find. All sorts of information available on product quality, product reliability, schedule / budget overruns, manufacturing scrap, turnover, etc.
     
    Andy Nichols likes this.
  11. Andy Nichols

    Andy Nichols Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    5,086
    Likes Received:
    2,553
    Trophy Points:
    112
    Location:
    In the "Rust Belt"
    Agreed, Yodon, and I do struggle with the implementation of an effective IA programme in small companies, for exactly the reasons you cite.

    The rolling 30 day window gives an opportunity to look at the landscape and ask what's new/changed/poorly performing which could do with a "rifle shot" at the process/area etc (I don't happen to believe ALL internal audits should or need to be process audits) to determine what's going on with the controls.
     
    yodon likes this.
  12. RoxaneB

    RoxaneB Moderator Staff Member

    Joined:
    Jul 31, 2015
    Messages:
    926
    Likes Received:
    1,081
    Trophy Points:
    92
    Location:
    Ontario, Canada
    Small business does give the leadership a perspective different from that held by leadership in a larger organization. We get the basket weave approach....there is site leadership to consider (akin to a small business) and the process leadership (focusing on one process across sites) . This way, we have the ability to prioritize our actions.

    Basket Wave 01.png

    In the attachment, Site 2 is doing really well, but Site 4 could use a little extra attention. As well, the processes of Operations and Quality appear to be struggling. A logical approach could be to find out why Site 2 doesn't appear to have conformance issues that Site 4 is having and/or focus on improving Operational/Quality conformance at Site 4.

    I do not advocate audits as being punitive...at first. One of my objectives of an audit/assessment is to provide the recognize of best practice. I want people to see how awesome Site 2 is and how well Human Resources has managed to standardize their practices across the organization. Audits provide us with the opportunity to collect evidence which supports the celebration of such benchmark performers.

    You're right...they're not hard to find. But then what?

    • How is this information captured, consolidated, and communicated in a meaningful way?
    • Then what? What happens after the information is captured, consolidated, and communicated?
    • How do we enable our Stakeholders to understand the findings and develop a plan to effectively address opportunities?
    Assessment Quadrant.png

    If the output shows us in Quadrant 1, life is good.

    If we're in Quadrant 2, we're following the defined practices but not achieving results. Why not? Do we have the right practices, technology, metrics? Time to talk.

    If we're in Quadrant 3, we're not following the defined practices but we're achieving the desired results. I want to know what you're doing that's so amazing and determine if it's truly the new best practice to be rolled out across to all.

    If we're in Quadrant 4, well, we need to have a serious talk. Step 1 is to follow the defined practices and see if that will help achieve the desired results.

    I'm at that point where I'm trying to develop the audit tool...call it a checklist if you want. The previous checklist I had was way too cumbersome - if anyone is familiar with ISRS, my checklist was even thicker! I need to develop a tool that will help the auditors be consistent, still allow for an indepth assessment, and provide meaningful results to the stakeholders.

    Haven't even made my way over to frequency of assessments.
     
    normzone and Andy Nichols like this.
  13. Andy Nichols

    Andy Nichols Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    5,086
    Likes Received:
    2,553
    Trophy Points:
    112
    Location:
    In the "Rust Belt"
    It came to me in an epiphany when thinking about my guitar practice and also what I see many audit programmes do - treat everything with the same frequency (which is dull)

    Think "squeaky wheel". High frequency. Gets (management's) attention (grease). Low frequency is the stuff which works...
     
  14. RoxaneB

    RoxaneB Moderator Staff Member

    Joined:
    Jul 31, 2015
    Messages:
    926
    Likes Received:
    1,081
    Trophy Points:
    92
    Location:
    Ontario, Canada
    Contemplated another possible "trigger" for an assessment...our main Funder (I'm in not-for-profit community healthcare for those who don't know) typically issues us a nonconformance after six consecutive months of failing to hit their contractually defined target. Maybe we should do an audit after 3 months of failing to hit the target...still a reactive approach, and somewhat threatening (i.e., "You don't hits the target, you gets the audit!"), but at least no one would be surprised when they receive the phone call saying the audit is next week.
     
  15. normzone

    normzone Well-Known Member

    Joined:
    Aug 3, 2015
    Messages:
    137
    Likes Received:
    77
    Trophy Points:
    27
    Thank you [RoxaneB] for sharing this. I'm following with interest - or bated breath, your choice :)
     
  16. yodon

    yodon Well-Known Member

    Joined:
    Aug 3, 2015
    Messages:
    198
    Likes Received:
    115
    Trophy Points:
    42
    I like your quadrants idea. Indeed, I think you hit the mark: identify and fix what's not working and institutionalize what is working.

    Oh, and I hope you didn't take it that I thought YOU used audit results punitively. I've seen, no matter how good the auditor's intentions were, that issues reported DID end up being used punitively (quite poor management culture).
     
  17. Andy Nichols

    Andy Nichols Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    5,086
    Likes Received:
    2,553
    Trophy Points:
    112
    Location:
    In the "Rust Belt"
    May I throw in here that I'd also put the actual way the internal audits were planned and conducted was part of the issue. I've yet to see an internal audit programme which actually engages management in a manner that they would understand how to use the result correctly...
     
  18. RoxaneB

    RoxaneB Moderator Staff Member

    Joined:
    Jul 31, 2015
    Messages:
    926
    Likes Received:
    1,081
    Trophy Points:
    92
    Location:
    Ontario, Canada
    Which, of course, begs the question...how do we engage management in a manner that they will understand how to use the result in such a way that value is added to the organization? This presumes that the result has the potential to add actual value (i.e., goes beyond "Procedures contain typos and aren't followed.")

    This may require some pens, napkins, and a few glasses of your rhubarb gin, Andy. ;)
     
    Andy Nichols likes this.
  19. RoxaneB

    RoxaneB Moderator Staff Member

    Joined:
    Jul 31, 2015
    Messages:
    926
    Likes Received:
    1,081
    Trophy Points:
    92
    Location:
    Ontario, Canada
    Not to worry, yodon. Such a thought never crossed my mind.

    I'm trying to nudge people out of their comfort zones which contain, like an ISO standard, a lot of WHAT shall be done, but little on the HOW. Many of us here "get" the fundamentals - leadership commitment, planning, audit, share results. But those are all generic, pie-in-the-sky concepts. Let's get down and dirty here...this is the chance to create a system, complete with the tools, that will help an organization flourish!

    Sure, not every HOW will fly, but this is all part of that brainstorming process.