AI Call Analytics: Natural-Language Answers to 'Why Do Customers Call?'

Context

A multi-billion-dollar rental company received thousands of customer calls each day and had no systematic answer to the basic question: why are customers calling?

Before this project, employees shared credentials to log into the Nice API and manually download sample recordings. That happened inconsistently. There were no transcripts, no systematic view of customer issues, and no way to measure customer-service performance. Executives wanted answers in natural language rather than static reports. I was the sole architect for what became the client's first production AI system.

The Problem

• Call volume of about 10,000 calls per day made manual sampling insufficient to characterize call drivers.
• The only access path was shared credentials to the recording system, so in practice there was zero systematic insight.
• Executives wanted to ask questions about call drivers, not read static reports.

Architecture

An end-to-end AWS pipeline, from raw call audio to an executive-facing chatbot:

flowchart LR
    N["Nice API<br/>call recordings"] --> T["AWS Transcribe"]
    T -->|"redacted<br/>transcripts"| S3[("S3")]
    S3 -->|"hourly batches<br/>(chunked above 3,000)"| B["Bedrock<br/>Batch Inference"]
    B -->|"JSONL results mapped to<br/>each call's metadata.json"| DL[("Data Lake")]
    DL --> QS["QuickSight<br/>dashboards"]
    DL --> CHAT["Bedrock<br/>executive chatbot"]

AWS Transcribe produces transcripts that are redacted before they land in S3. Each hour, the pipeline sends accumulated transcripts to Bedrock Batch Inference, chunking batches larger than 3,000 transcripts. Batch Inference produces a JSONL output file. The pipeline maps each classification result back to the corresponding recording's metadata.json, preserving a traceable link from an insight to its source call. The data lake feeds QuickSight dashboards and a Bedrock chatbot for plain-language questions.

From Hours to Minutes

The first implementation classified transcripts one at a time. A day's recordings took hours to process, making classification the pipeline bottleneck. I asked our AWS architect about upcoming capabilities, and Bedrock Batch Inference shipped days after I described that need. I redesigned the classification stage to write redacted transcripts to S3 and process them in hourly batches rather than invoking classification per transcript. Classification fell from hours to minutes.

Classification was also version-controlled. Every change to the question set or LLM model was tracked so metrics remained comparable over time.

Key Decisions & Trade-offs

• Bedrock: it allowed model substitution without rewriting the pipeline, provided value-driven token pricing, and integrated with the AWS services already in use.
• Question-set design: executives had limited input on the desired analysis, so I defined the questions and a reusable response schema. Examples included: Was the issue resolved? On a scale of 1–4, how frustrated was the customer? Did the customer attempt self-service? What steps did they take to resolve it on their own?
• XML-structured prompts: I structured the classification prompts with XML to make output parsing reliable.
• Correctness over latency: because this was a batch job, classification accuracy mattered more than response time. I built a React review dashboard that displayed sampled calls beside their classification, then verified the results manually.
• Next on the roadmap: an LLM-as-a-judge harness, in which an independent model scores classification quality from 1 to 5. This would support continuous model evaluation and safe adoption of cheaper models when quality holds.

Closing the Product Loop

Once classified data reached the data lake and QuickSight, a clear pattern emerged: about 30% of inbound calls came from customers canceling reservations or from customers unsure whether a cancellation had completed because of poor cancellation UX.

The analytics pipeline identified its first product fix. I owned product discovery, built the frontend self-service cancellation flow, and specified the backend changes. The work moved from insight to release through one week of discovery, one month for UX to finalize the design, one month to move up the groomed backlog, and one sprint for implementation.

Result: total call volume cut by ~30%.

Outcome

• ~10,000 calls/day transcribed, redacted, and classified automatically with a version-controlled question set
• Classification throughput went from hours to minutes by adopting Batch Inference the week it shipped
• Executives self-serve answers through QuickSight and a Bedrock chatbot, replacing credential sharing and guesswork
• The platform's own insights drove a ~30% reduction in call volume