1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
Dismiss Notice
You must be a registered member in order to post messages and view/download attached files in this forum.
Click here to register.

Opinion needed with regards to sample size and scoring

Discussion in 'Other Quality and Business Related Topics' started by Thomas Hynes, Aug 6, 2020.

  1. Thomas Hynes

    Thomas Hynes New Member

    Aug 6, 2020
    Likes Received:
    Trophy Points:
    Our company has implemented a Quality Assurance System, and in my own opinion is going through some growing pains. I as an individual who has their work analyzed, and hearing my co-worker's experience with their scoring has led to a total disdain for the method of evaluations.

    I have worked for several companies that evaluate on a monthly basis a certain percentage of work completed. In all cases, and my experience only, the work measured to score an individual was 10%. And for most of my working career, this has been the case, until now.

    All Employees should be closing approximately 5 cases per day, 20 per week. However in most situations in a typical week, it can be up to 25-30 cases. The Average number of cases closed by all individual agents at week end is 26 for a total of 104-128cases per week with an average of total month closures 116 escalations.


    1: Is the evaluation of 3 cases out of a total of 120 cases closed per month a good indicator of the quality of work done by anyone agent?

    2: What percentage of work should be evaluated for a fair and balanced review of anyone agent? if I close 120 cases is 10% of my case closures a better representation then just 0.025% of work closed?

    3: Are three scores every quarter a high enough number out of a 4 month period to be able to reasonably score an employee?

    My proposal -

    If the answer to questions 1, 2, and 3 are NO, then my proposal idea is the following.

    Q&A chose three closed escalations and continue to score then as they have been

    After the score is done, and one of the three scores is below the minimum of 85% then Q&A chooses one more case and scores. If the case scored is higher than the case score that is lower, the new score is used to calculate the final score of the review.


    Case 1 scores at 92%

    Case 2 scores at 89%

    Case 3 scores at 72% --- Q&A reviews the 4th case and that score is 99%

    Case 3 is dropped, thereby 92+89+99=93% for the month - however, case 3 is used as a teachable case

    Knowing the blowback from Q&A I can foresee a HUGE blowback and unwillingness to implement such a procedure. Although they have made changes due to a HUGE blowback from my department, I will say that those that were present at the time the Q&A system was implemented were not, did not participate in any discussions as to what should be reviewed and scored as it relates to the final outcome of solving the problem of being able to understand the problem.

    Our Department has 28 full-time employees that are scored. The downside is that there are only two individuals who work in Q&A which are responsible for scoring all 28 individuals

    I see a HUGE disparage between the ones that score and the ones that actually close cases, it's 14 to 1. This is a monumental task, of which I respect. Add to that the KPI that they are scoring is time-consuming because, in essence, the Q&A staff are basically having to re-work the case themselves.

    Any input from those that work in this field, designed, implemented, administered, or oversee Quality Assurance, your opinion would be of great assistance.

    Thank you.
  2. Jennifer Kirley

    Jennifer Kirley Moderator Staff Member

    Jul 31, 2015
    Likes Received:
    Trophy Points:
    Welcome to the QFO, Thomas.

    I am wondering what type of cases these are, and how much the individual's performance relies on other people or process inputs to succeed.

    Not having such information makes me feel very suspicious of numerical targets such as you described. Is it all about closing the number of cases - is it possible to close a case but later that closed case provides the organization no value? Errors, half-done work, etc.

    What do these cases mean for the organization's success, and would accuracy impact that?

    Quantity vs. quality of work is always a concern. Also, how well does your group's efforts help downstream processes succeed - do any internal stakeholders depend on our work's accuracy, not just quantity?

    I have more questions than answers, I'm afraid.
  3. Andy Nichols

    Andy Nichols Moderator Staff Member

    Jul 30, 2015
    Likes Received:
    Trophy Points:
    In the "Rust Belt"
    I guess I'd need to know is this an evaluation of a process or a person? Unless the process is (statistically) under control, having been base-lined and evaluated for capability, I can't see how people can be evaluated and against what?