Skip to content

AI Engines (LLM Providers/Models)

AI Engines define which Large Language Model (LLM) provider/model MiaRec uses to run AI Tasks (summaries, sentiment, CSAT, Auto QA, and custom insights).

This chapter describes how platform operators should configure and govern AI Engines in a multi-tenant environment.

What an AI Engine is

An AI Engine is a named configuration that typically includes: - provider (e.g., OpenAI/Azure OpenAI/Anthropic/etc.) - model name/version - authentication (API key, endpoint, deployment name) - safety and policy settings (if applicable) - limits (rate limits, quotas) - default parameters (timeouts, max tokens, temperature) if the platform supports them

AI Tasks reference an AI Engine when executing.

Start with a small set of standard engines

For example: - Standard – balanced cost/quality - High accuracy – for complex scoring/categorization - Low latency – for near-real-time needs (if applicable)

Keep the catalog small to reduce operational complexity.

Decide how engine selection works

Common patterns: - Single global default engine for all tasks - Per-task engine selection (your UI appears to support this) - Per-tenant engine policy (if required for cost or compliance)

Credential and security management

  • Store credentials in a secrets manager (recommended).
  • Rotate credentials regularly and document rotation steps.
  • Limit engine access by environment (dev/stage/prod).
  • Maintain audit logs for engine configuration changes.

Validation / smoke tests

When adding an engine: 1. Run a minimal test prompt (health check) to verify credentials and connectivity. 2. Run a representative AI Task (e.g., summarization) on a test transcript. 3. Verify: - latency - successful JSON responses (if using JSON output) - stable output quality

Operational considerations

Cost management

  • Track usage by tenant and by task (requests, tokens, cost).
  • Use task filters to avoid running tasks on ineligible conversations.
  • Consider “preview mode” for new tasks before broad enablement.

Reliability and failover (if supported)

  • Define a fallback engine if the primary provider is down.
  • Document expected behavior:
  • automatic failover vs manual switch
  • per-task vs global failover

Data handling

  • Document data residency considerations (especially in partner-hosted deployments).
  • Document whether transcripts are sent to external providers and what is included (metadata vs transcript only).

Where to configure

Menu path: Administration > Speech Analytics > AI Assistant > Engines

AI Engine configuration form

Figure: AI Engine configuration showing name, status, visibility settings, and model configuration options.

Engine settings

When configuring an AI Engine, you specify:

  • Name – A descriptive name for the engine
  • Status – Enabled or Disabled
  • Visibility – Global (available to all tenants) or Tenant-specific
  • Model settings – Provider-specific configuration (API endpoint, model name, authentication)

Per-task engine selection

Each AI Task can select which engine to use. This allows you to:

  • Use different models for different task types (e.g., a faster model for simple tasks, a more capable model for complex analysis)
  • Test new models on specific tasks before broader rollout
  • Manage costs by using appropriate models for each use case