Skip to content

Playground

Playground helps admins test and refine AI Tasks against real conversations before rolling changes out widely.

Playground is especially useful when: - tuning prompts for consistency - validating output schema (JSON) - checking edge cases (angry customers, partial resolution, transfers) - confirming explanations are useful for human review

How to access testing

To test an AI Task, open the task view page and click the Test button.

AI Task with Test button

Figure: AI Task view showing the Test button to access prompt testing.

Testing workflow

Step 1 — Select a call

Choose a conversation to test against:

Test AI Prompt - Select call

Figure: Select a call from the list to use for testing.

Step 2 — Run an experiment

View the transcript and run the prompt against it:

Test AI Prompt - Run experiment

Figure: Run Experiment view showing the transcript and prompt configuration.

Step 3 — View results

Inspect the AI output to verify correctness:

Test AI Prompt - Experiment results

Figure: Experiment results showing the AI-generated output.

If the prompt produces good results, click Save AI Prompt to save your changes.

Step 1 — Choose representative conversations

Pick 5–10 conversations that represent: - typical cases - edge cases - different agents/queues - different customer sentiment levels

Step 2 — Run a single AI Task

  • select the task to test (e.g., CSAT Scoring)
  • run it against one conversation transcript/thread

Step 3 — Inspect output

Verify: - output is valid JSON (if configured as JSON) - values are in the expected range (e.g., CSAT 1–5) - dropdown values match allowed labels - explanation is concise and evidence-based

Step 4 — Iterate safely

  • adjust prompt wording (clarify rubric, define “unknown”, tighten output)
  • adjust filters if too many irrelevant calls qualify
  • re-run on the same conversation and compare results

Step 5 — Validate before rollout

Before enabling a task broadly or changing a prompt: - validate on at least 10 conversations - confirm acceptable consistency for your stakeholders

Tips for faster calibration

  • Add 2–3 “anchor examples” in your validation set (known good/bad calls)
  • Use consistent definitions (write them in a shared document for stakeholders)
  • Keep explanations short (1–3 sentences)