AI QA and auto QA
What have we learned from two years of AI-powered QA?
Tom Laird shares the lessons from building OttoQA: keep your old scorecard as a benchmark, drop false-hustle questions, and fix consistency before you trust the scores.
What should you ask before you build or buy an AI auto QA platform?
Tom Laird on why scoring calls is only step one of AI auto QA, what AI still struggles with, and the questions to ask any vendor or internal build team.
How does AI actually score a contact center QA call?
A behind-the-scenes look at how an AI QA platform scores calls: a prompt per form question, accuracy checks, calibration against your own scorers, and analytics questions on top.
What did we learn building an AI QA platform?
Tom Laird's early lessons from building OttoQA: model choice, consistency vs. accuracy, why the LLM shouldn't do the math, and how calibration against a customer's own scores works.
How do you write AI prompts that score QA calls accurately?
A LinkedIn live session on prompting large language models for contact center QA: system prompts, telling the model where to look, handling judgment questions, and asking for a rationale.
Which ChatGPT prompts can help you automate contact center QA?
A 2023 live walkthrough of prompts for automating QA with ChatGPT: auditing what your form really measures, clarifying questions, defining scores, confidence levels, rationale and structured output.
How far can AI go in contact center QA? A friendly debate
Episode 200: Tom Laird and Chris Mounce of Evaluagent debate, in August 2023, how ready AI was for QA, why prompting each form is the hard part, data security, and whether AI can coach.
Can you automate contact center QA with AI using your existing scorecards?
Tom Laird's first look, in mid-2023, at automating QA with ChatGPT inside his own BPO: scoring against clients' existing forms from transcripts, what you need, and what stays human.
QA scoring and sampling
Is the QA score still the number that matters?
Tom Laird on why the single QA score is the least useful thing your QA program produces, and what to look at once AI is scoring hundreds of calls per agent.
How many QA calls do you actually need to score?
Tom Laird on QA sampling: why a well-built, stratified sample scored with a strong model can tell you more than a big pile of lower-accuracy scores, and what to ask a QA vendor.
How do you improve call quality monitoring in a contact center?
Tom Laird's ten steps for call quality monitoring: analytics alongside QA, staffing ratios, weighted scorecards, live vs recorded monitoring, scoring calls with agents and clients, and rotating trainees through QA.
QA forms and scorecards
How do you score empathy on a QA form?
Tom Laird on why "did the agent use an empathy statement" is a checkbox, and how scoring intent and outcome with AI gets closer to real empathy. Plus his view on whether bots can be empathetic.
Is your QA scorecard rewarding false hustle?
Tom Laird on "false hustle" in contact center QA: checkbox questions that reward looking busy over solving problems, and how to rewrite them around intent and outcome.
What belongs on a contact center QA form?
Tom Laird walks through the sections of a solid contact center QA form: opening and verification, etiquette and call control, the core of the call, closing, and a short analytics section.
What makes an effective QA scorecard, and how do you coach from it?
Tom Laird and Chris Mounce of Evaluagent on building QA scorecards that measure what customers care about, avoiding false-hustle questions, and running coaching sessions with a structure like GROW.
How do you set up contact center QA the right way?
One of the earliest episodes of the show: Tom Laird on building a QA program around tone and engagement, training the standard before scoring it, and building a form that breaks the call into sections.
Coaching from QA
How do you turn QA scores into coaching that changes agent behavior?
Tom Laird on moving QA from a policing tool to a development tool: scoring intent and outcome, coaching from patterns, and short micro-coaching sessions.
How should supervisors coach agents after a QA review?
Tom Laird's coaching playbook for supervisors: correct in private and celebrate in public, be specific about the exact moment on the call, coach tone, tie feedback to goals, role play, and re-monitor.
How can supervisors make QA coaching sessions actually work?
Tom Laird's tips for coaching sessions: have a management-wide plan, lead with what's working, know the exact numbers, have agents score their own calls side by side with QA, and role play both ways.
Calibration
Speech analytics and conversation data
Can an auto QA platform replace traditional speech analytics?
Tom Laird on how AI QA platforms changed his view of speech analytics: asking direct questions of your calls and scoring intent, versus keyword trending. And what traditional analytics still does better.
What can speech analytics do for QA beyond word clouds?
Tom Laird's ten practical speech analytics use cases from his own BPO: sentiment-based incentives, finding hidden frustration, tracking silence by agent, flagging possible fraud into QA, competitor mentions and more.