The Calibration report helps you to calibrate evaluation forms. During calibration, the evaluation form is sent to a select number of users for evaluation. The Calibration report automatically tracks variance between evaluators and individual evaluator performance. You can run the report on different versions of the same evaluation form and view the variance change between the versions.
Permissions
You must have these permissions:
-
Dashboard > Dashboard Templates > Calibration Report :On
-
Dashboard > Dashboards: View
-
Dashboard > Dashboards: Edit (optional)
If you cannot access the prebuilt reports, check with your administrator. The administrator can find these permissions in CXone Mpower. Go to Admin > Security Settings > Roles and Permissions and select the role.
Use CaseA Quality Manager wants to ensure consistent scoring across evaluators using different versions of the same evaluation form. By running the Calibration report, the manager can compare evaluator scores, identify high variance, and spot outliers. This helps improve scoring accuracy and maintain evaluation standards. Enhanced filtering and visualization options allow for quicker analysis and more targeted coaching for evaluators.
Data Overview
This prebuilt report includes:
Evaluator Calibration
This widget displays evaluators with the highest variance based on their participation in completed calibrations during the selected time frame. It includes data exclusively from Complete calibrations, which are those where all evaluators finished their assigned calibration evaluations before the deadline.
You can check the completion status in the Completion Rate column in My Zone > Calibrations. For example, if a calibration was sent to 15 participants, all 15 must complete it for the data to appear in this widget. You can access the data for incomplete calibrations within the QM reports.
Permissions
You must have these permissions:
-
QM Tools > My Zone > Calibrations: View
-
Dashboard > Dashboards:View
-
Dashboard > Dashboards: Edit (optional)
If you cannot access the widgets or Dashboard, check with your administrator. The administrator can find these permissions in CXone Mpower. Go to Admin > Security Settings > Roles and Permissions and select the role.
The information displayed on this widget helps you see:
-
The evaluators that score within a ±5% variance or a percentage decided by the quality managers.
-
The evaluator's accuracy in scoring interactions and the evaluators that have trouble scoring.
Click Open in New Page
, to go to calibrations in My Zone, where you can see additional calibration details.
Click Options
and then Settings on the widget to filter the data you want to see in each widget.
Settings
Field descriptions
-
Display Name: Specify a name for the widget.
-
Variance Threshold: Define the threshold percentage of variance in the calibration score for the evaluators. The percentage of variance for an evaluator is shown in orange if it is above the absolute value.
-
Date Range: Define the date range. You can choose from options, like Today, Yesterday, Last 7 days, Last 30 days, Last 90 days, Last Week, Last Month and Custom range.
-
Evaluator: Select the name of the evaluator.
Example
Quality managers can see which evaluators tend to score higher or lower than their peers on related calibrations, to determine whether require quality score coaching is needed for them.
Forms Calibration
This widget helps quality managers to see the completed calibrated forms with the highest variance. It only show data for fully completed calibrations. A calibration is considered complete when all assigned evaluators submit their grades within the given deadline.
You can check the completion status in My Zone > Calibrations under the Completion Rate column. For example, if a calibration was sent to 15 participants, all 15 must complete it for the data to appear in this widget.
On this widget, a high variance indicates that the questions are being scored inconsistently across the evaluators. Quality managers can check if evaluators are having trouble scoring the form and then coach them. The scoring requirements or the questions on the form can also be revised based on this information.
Permissions
You must have these permissions:
-
QM Tools > My Zone > Calibrations: View
-
Dashboard > Dashboards: View
-
Dashboard > Dashboards: Edit (optional)
If you cannot access the widgets or Dashboard, check with your administrator. The administrator can find these permissions in CXone Mpower. Go to Admin > Security Settings > Roles and Permissions and select the role.
Click Open in New Page, to go to calibrations in My Zone, where you can see additional calibration details.
Click Options and then Settings on the widget to filter the data you want to see in each widget.
Settings
Field descriptions
-
Display Name: Specify a name for the widget.
-
Variance Threshold: Define the threshold percentage of variance in the calibration score for a form. The percentage variance for the form is shown in red if it is above the absolute value.
-
Date Range: Define the date range. You can choose from options, like Today, Yesterday, Last 7 days, Last 30 days, Last 90 days, Last Week, Last Month and Custom range.
-
Form Name and Form Version: Select the form name and version from the dropdown, which also displays the form’s status.
Example
With this widget, quality managers can check the need for simple and clear scoring requirements in forms that will help evaluators to know how to score each question.
Form Detail View
| Column | Description |
|---|---|
|
Form Name |
Name of the form. |
|
Form Version |
Version of the Form. |
|
Section |
Section of the form. |
|
Question |
Question in the form. |
|
Calibration Score |
Calibration score for the question.
|
|
Variance |
Variance for the question. |
Evaluator Detail View
| Column | Description |
|---|---|
|
Form Name |
Name of the form. |
|
Form Version |
Version of the Form. |
|
Section |
Section of the form. |
|
Question |
Question in the form. |
|
Calibration Score |
Evaluator's calibration score for the question. If calibration is based off an evaluation, the report displays the evaluation score.If calibration is based off an interaction, the report displays the average calibration score across the evaluators. |
|
Variance |
Evaluator's variance for the question. 0%—The evaluator scored the question the same as other evaluators.Negative percentage: The evaluator gave a lower score for the question than other evaluators.Positive percentage: The evaluator gave a higher score for the question than other evaluators. |
John Middleton is a quality manager for Classics, Inc. He wants to improve evaluator performance. He opens the Calibration report, defines a time period, selects the forms he wants to analyze, and selects his team. He selects Evaluator in the Report By field to filter by evaluator. He notices Lucy Steele, an evaluator, has high variance. He clicks Lucy's name to get more detail on the variance of the forms and questions Lucy has evaluated.