AI Quality Assurance automatically evaluates every chat based on selected Review Profiles, which define both the knowledge sources and the guideline categories to be used for assessment. It generates detailed Accuracy and Guideline Compliance scores with explanations. By reviewing these results, you can identify areas where agents need support and then manually create coaching opportunities for specific agents. This way, you get instant ratings, actionable insights, and a structured process to help agents continuously improve while meeting standards.

Step-by-Step Instructions

1. Log in to your Comm100 Control Panel.
2. From the left navigation pane, go to AI & Automation > AI Quality Assurance. The Reviews tab will open.
   
   
3. Click New Review.
4. In the New Review drawer, set the Time Range, Department, Agent, Category, create additional Filters and select the Review Profile (Required) for the chats you want AI to review. Then click Start. 
   
   
5. Wait for the status to change from In Progress to Finished. Processing time varies depending on the number of review items.
6. Once complete, click the number in the Review Items column to view results.
   
   
7. On the Review Items page, you’ll see the list of filtered chats along with Accuracy Score and Guideline Compliance Score. You can also adjust both score ranges to focus on specific chats. For example, use the score range filter to prioritize low-scoring chats for faster coaching impact.
   
   
8. Click Details in the Operations column to open the Review Item Detail page. Here, you’ll see:
   • Accuracy Score - The agent’s responses to customer questions, AI’s suggested ideal responses, and a breakdown of why the agent received the given score.
   • Guideline Compliance Score - Displays each Category and the Guidelines under that category, showing whether each guideline passed or failed, the rationale behind the result, and the corresponding Category Score.
   
   Note:

   • The Accuracy Score is displayed with at most one decimal place. If rounding is required, the system uses Banker's rounding, rounding half to the nearest even value.
   • The Guideline Compliance Score is calculated by evaluating all relevant guidelines within each selected category. For each category, the score equals the percentage of passed guidelines multiplied by 5 (passes ÷ total relevant guidelines × 5). The overall compliance score for the chat is the average of all Category scores.
   • If any guideline marked as Auto Failure fails, the system automatically sets both the Category Score and the Overall Compliance Score to 0, regardless of other results.
   • Human reviewers can revise the Pass/Fail results for individual guidelines if they determine the AI judgment was incorrect. Once edited, the Guideline Compliance Score and Category Scores automatically recalculate to reflect the manual updates.

Review Profiles

A Review Profile is a reusable setup that defines which knowledge contents (such as webpages, KB articles, cloud files, or uploaded files) and guidelines (categories and rules for compliance) the AI should use when evaluating chats during Quality Assurance. By saving both your knowledge sources and guideline categories in one profile, Review Profiles make it easier to run consistent and repeatable QA reviews across teams.

To set up a Review Profile, follow the steps below:

1. Under AI Quality Assurance, click Review Profiles.
   
   
2. Click New Review Profile.
3. Enter a Name for the Review Profile and click Save & Configure Review Profile.
   
   
4. In the Knowledge tab, click New Content to add the knowledge sources that the AI will reference (e.g., KBs, websites, or uploaded files). You can include multiple content sources, and each source will be listed by type and last updated time.
5. Switch to the Guidelines tab and click Add Guidelines to select one or more Guideline Categories. Guidelines determine the compliance standards (such as Security, Soft Skills, or Compliance) that the AI will test during reviews. 
   
   To learn how to use Guidelines, see this article.
6. Click Save to finish creating your profile.

Coaching Opportunities

If you find a chat that highlights a learning opportunity for your team, you can create a coaching item directly.

1. Click Create Coaching Opportunity.
   
   
2. In the Create Coaching Opportunity drawer, select agents (required), enter a comment, define any tags and click Create Opportunity.
   
   
3. Go to Coaching > Assigned Opportunities. A list of the opportunities with the assigned agents will appear.
   
   
4. You can use filters to narrow the list and click Details in the Operations column to view opportunity details. When the coaching is complete, click Mark as Complete, and the opportunity status will be changed from Open to Completed.
   
   
5. Go to Opportunity Management. You will see the complete rate of each assigned opportunity. You can also delete completed opportunities to keep the list clean and manageable.
   
   

For details on how to manage agent permissions, see this article.