1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
Dismiss Notice
You must be a registered member in order to post messages and view/download attached files in this forum.
Click here to register.

Process Metrics Help

Discussion in 'AS 91XX - Aerospace Quality Standards' started by AdamEllis101, Oct 1, 2015.

  1. AdamEllis101

    AdamEllis101 New Member

    Joined:
    Aug 20, 2015
    Messages:
    1
    Likes Received:
    0
    Trophy Points:
    1
    Hello All,

    I have been tasked with developing and implementing the AS9100 Standards for our small 25 person manufacturing job shop. Unfortunately we are so small, the only quality document that existed prior to myself being hired was a Quality Manual. Basically, I have been creating the entire QMS for our 2 companies from scratch.

    Unfortunately when I was originally assigned this task, Elsmar.com only was online for 1 week, prior to shutting down. So due to my inexperience and naivety, I wasn't able to capture all of that glorious information. Seriously, what an excellent resource, and I surely miss it.

    So the problem I am running into is developing useful metrics for each of the processes. What I am also not certain about is the need for metrics for every single process. Can some be omitted, or at least monitored by management for verification of effectiveness. The majority of the processes have been assigned to Management to control.

    I have created a list of our key processes (46 processes currently) and a process map relating them to one another. However, many of the support resource processes are very challenging to define a metric for. Could you help identify useful metrics for some processes? Maybe some examples of what your company has used.

    Here are support resource processes:
    Control of Monitoring and Measurement (Calibration)
    Approved Supplier Management
    Risk Management
    Control of Outsourced Processes
    Preservation of Product
    Competence, Awareness, and Training
    Control of Work Transfer
    FOD Program
    Inspection and Maintenance of Production Equipment
     
  2. Andy Nichols

    Andy Nichols Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    5,086
    Likes Received:
    2,553
    Trophy Points:
    112
    Location:
    In the "Rust Belt"
    Be careful NOT to try to set objectives and measure too much. Just like the dashboard on a car, there are some "nice" things to know, but ultimately only a few gauges are needed to tell you what you need to know. Simply, what do each of your process owners get measured on currently? As a business, with AS or not, they are being measured on something as part of their responsibilities. Start with that. Of course, you might find they're in conflict in some cases. Purchasing may measure the lowest cost, but not care about quality! So you'll have to get your TOP person to reconcile that stuff...
     
    Brian Vandolah and AdamEllis101 like this.
  3. MCW8888

    MCW8888 Well-Known Member

    Joined:
    Aug 17, 2015
    Messages:
    642
    Likes Received:
    198
    Trophy Points:
    42

    Here's what I remember having seen at the Cove which I have applied to my own QMS (It might have been from Andy):
    "On time delivery - efficiency
    Capacity utilization - efficiency
    Sales related claims - effectiveness
    % off target forecasts - effectiveness

    Ontime delivery can also be considered effectiveness in meeting customer needs"

    In addition we track the following that are related to the Business Plan:
    Cost of Poor Quality
    Tracking Premium Freight
    Equipment unplanned down time
    Effectiveness of Internal Audits
    Open and Closed Correctie Actions
    Supplier OTD
    HSSE - No Harm No Leaks
    CSI
     
  4. Andy Nichols

    Andy Nichols Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    5,086
    Likes Received:
    2,553
    Trophy Points:
    112
    Location:
    In the "Rust Belt"
    Nope, not mine! :)
     
  5. MCW8888

    MCW8888 Well-Known Member

    Joined:
    Aug 17, 2015
    Messages:
    642
    Likes Received:
    198
    Trophy Points:
    42
    OK thanks. I do not remember who posted those as examples.
     
  6. hogheavenfarm

    hogheavenfarm Well-Known Member

    Joined:
    Jul 30, 2015
    Messages:
    220
    Likes Received:
    160
    Trophy Points:
    42
    To add to that, if you have the process in the first place, you must think it is effectively doing what it needs to do, otherwise you would not have that process. Now, how do you know the process is doing what it needs to do? A look at those responses are your process indicators. Another way of looking at it is to ask "why" you have this process, because the answer will pinpoint your objective. You probably already have measurement on these, you just do not realize it or maybe it is not actually measured, just a visual cue that all is working fine.
    My point is - Do not create new measurements and objectives, use the ones that are implicit in the process itself, after all, those are the reasons you have the process in the first place.
     
  7. RoxaneB

    RoxaneB Moderator Staff Member

    Joined:
    Jul 31, 2015
    Messages:
    926
    Likes Received:
    1,081
    Trophy Points:
    92
    Location:
    Ontario, Canada
    Why not let the process owners...or the people who live and breathe the processes on a daily basis...identify what their metrics are?

    Presuming there are some objectives for the organization, then you can develop a matrix that shows which metrics directly impact the objectives. That's a good start for showing the KEY metrics.
     
  8. yodon

    yodon Well-Known Member

    Joined:
    Aug 3, 2015
    Messages:
    198
    Likes Received:
    115
    Trophy Points:
    42
    And a few more thoughts...

    As Andy noted, start slow. Consider taking a risk-based approach; i.e., measure where your risks are the highest first and then expand as needed.

    And have a plan for what to do with the data you gather. As Roxane pointed out, there needs to be objectives to measure against. If you just measure for the sake of measurement, you're wasting time. I come from a software background and have seen time after time where management decides they need to measure (lines of code per day, number of defects per module, etc.) but they never had a plan for using the data.

    Know what the data means. Using my software example, one person wrote very clean, concise code and another wrote sloppy, spaghetti code yet because the latter had better lines of code per day numbers, he was lauded as a great programmer. (Of course when the first woodpecker came along, his civilization collapsed). As a result, others started fudging their code numbers to try to jockey for raise position.

    Finally, always keep the focus on the process. If you focus on people, you will either kill morale or people will start working around the measures (or both).

    Measurements can be quite useful if effectively applied and can be equally harmful if not properly managed.
     
    Zebra418 and AdamEllis101 like this.
  9. Tom Waite

    Tom Waite Member

    Joined:
    Aug 3, 2015
    Messages:
    45
    Likes Received:
    33
    Trophy Points:
    17
    Location:
    Grand Rapids Michigan Area
    Here is a resource I found on a NASA site. Might not be what you are looking for but it does try to help explain the process of developing what you want to measure and the cost of measuring it.
     

    Attached File(s): 1. Scan for viruses before using. 2. Report any 'bad' files by reporting this post. 3. Use at your own Risk.:

    AdamEllis101 likes this.
  10. Nikki

    Nikki Well-Known Member

    Joined:
    Jul 31, 2015
    Messages:
    268
    Likes Received:
    141
    Trophy Points:
    42
    Location:
    Maine
    On time delivery definitely!
     
    AdamEllis101 likes this.
  11. RoxaneB

    RoxaneB Moderator Staff Member

    Joined:
    Jul 31, 2015
    Messages:
    926
    Likes Received:
    1,081
    Trophy Points:
    92
    Location:
    Ontario, Canada
    Whatever metrics are selected, it is important to have a definition for them. Using 'On-Time Delivery' as an example:

    • What constitutes a late delivery? 30 minutes? 3 hours? 1 business day? 2 days?
    • What about an early delivery? From your organization's perspective, you may not consider an early delivery to be an issue, but what does your Customer think? They may have requested a certain date to ensure that there would be sufficient room to receive your delivery.
    • Does delivery mean on-site or when the first/last item is unloaded or when the paperwork is signed?

    Metrics are well and good, but unless everyone in your organization has the same understanding of them, the "story" that the results may tell can be interpreted in several different ways.
     
    AdamEllis101 likes this.
  12. BradM

    BradM Moderator Staff Member

    Joined:
    Jul 31, 2015
    Messages:
    394
    Likes Received:
    314
    Trophy Points:
    62
    Location:
    Arlington, TX
    Allow me to echo what Roxane has here. I think this is an important point.

    You have to be careful with metrics; and equally careful with creating what appears to be performance measures. Management may (and most probably will) develop some metric that, while it may appear to be worthwhile to them, is difficult to measure. Too, they may put stock in a metric, not realizing that the particular metric has a lot of variables that process owners may not have a lot of control over. Thus, if that metric goes "down" or whatever, management "blames" the process personnel for something totally out of their control.

    By starting first with those live within the process, they know much better what metrics are relevant.

    Too, the process owners may have metrics that have already been captured historically.

    So, hopefully you have your processes mapped and identified in your organization.
    1. Explain to the process owners what the metrics are, and how they are going to be used.
    2. Ask the process owners to note on the process flowchart when and where the metrics are to be accumulated.
    3. Then, challenge the status quo with them. See if there may not be better (and more obtainable) metrics.
     
    AdamEllis101 likes this.
  13. Andy Nichols

    Andy Nichols Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    5,086
    Likes Received:
    2,553
    Trophy Points:
    112
    Location:
    In the "Rust Belt"
    One thing you want to avoid is conflict with metrics. Some process owners will, if left alone, decide on their own set of criteria and forget that others need to shape them, too. For example, if the family all share a car, clearly each Process Owner ends up with a different set of performance criteria. The son wants speed, the father wants economy - which is mutually exclusive of speed. The mother wants safety, which is also mutually exclusive of speed but somewhat aligned with economy. The daughter doesn't want to spend time in the car so aligns with speed but wants to be safe too. See? Conflicting measures of "performance"...
     
    AdamEllis101 likes this.
  14. RoxaneB

    RoxaneB Moderator Staff Member

    Joined:
    Jul 31, 2015
    Messages:
    926
    Likes Received:
    1,081
    Trophy Points:
    92
    Location:
    Ontario, Canada
    That's why an exercise in cascading (and reverse cascading) metrics is valuable.

    Top Management, in their ivory tower, may decide on certain objectives. But without knowing what the process owners and the daily users within those process deem as important and meaningful metrics, the objectives may not be SMART.

    At the same time, process owners and daily users of the processes (living in the trenches) need to understand how their metrics (and results) impact the overall objectives and direction.

    This is why I love metrics that flow up and down within an organization. Leaders can identify benchmark performers within the organization and take steps towards standardization (without re-inventing the wheel), while also identifying potential areas that require a bit more attention. And individuals/teams can see how their performance impacts the overall success of the organization. Such an exercise also facilitates a common direction for the organization, as well as common language. Suddenly, it's no longer a quality management system or an environment management system...it's a business management system and it's "what we do".
     
    AdamEllis101 and Brian Vandolah like this.
  15. Che Promubol

    Che Promubol Member

    Joined:
    Jan 30, 2017
    Messages:
    5
    Likes Received:
    4
    Trophy Points:
    2
    Can someone please share with me any suggestions for a Despatch & Delivery measure?

    New to the forum, new to Quality Management, new to AS9100..

    Droppoed in at the deepa end and trying to learn to swim, fast!
     
  16. RoxaneB

    RoxaneB Moderator Staff Member

    Joined:
    Jul 31, 2015
    Messages:
    926
    Likes Received:
    1,081
    Trophy Points:
    92
    Location:
    Ontario, Canada
    Hi, Che...congratulations on your first post and welcome to QFO!

    Without knowing your business, it may be difficult to suggest metrics that are applicable to your organization, however some include:

    • On-Time Delivery
    • Truck Turnaround Time
    • Customer Complaints (associated to delivery)

    What I would encourage you to do is engage the team responsible for this process within your organization (including the guys on the floor). Find out from them what believe is important to measure and how they can positively impact it.
     
    Che Promubol likes this.
  17. David Sanabria

    David Sanabria Active Member

    Joined:
    Jan 2, 2016
    Messages:
    50
    Likes Received:
    4
    Trophy Points:
    7
    The minimum are posted on the standard - but don't let that be the end.

    Measure that in which adds value - makes sense.

    Start with 8.4 and finish with 5.2 requirements on the standard.

    That should keep you busy and out of trouble