| Coverage | Pricing | Screening | KPIs |
|---|---|---|---|
| 5 locales (EN-US, HI-IN, EN-IN, KO-KR, JA-JP) | Per hour | Language proficiency + listening test | Screening rigor, guideline iteration cycles |
The Problem
A model can generate a frustrated voice. It cannot yet explain what made a listener believe it.
- Semantic accuracy stops at the words. Transcription tells a research team what was said. It says nothing about how it landed, and how it landed is the entire signal a persona or a guardrail depends on.
- Research teams need more than a mood label. The description must be granular enough to reproduce a specific persona, not just detect a generic mood.
- Safety teams need the opposite problem solved. They need the acoustic facts behind an emotional read, not an annotator's impression of it, or there is nothing to audit when a model's affect skews.
Sigma's answer is high-granularity affective captioning and nuance profiling: annotators describe a voice using specific, named parameters, pitch, projection, texture, pacing, and more, precise enough to generate the prompts required to produce nuanced emotion in text-to-speech (TTS) outputs. In plain terms, we replaced a vague mood label with an acoustic description detailed enough for a model to consistently reproduce.
The Insight
Every recording carries two layers. One is semantic: what the words literally say. The other is acoustic and paralinguistic: pitch, projection, texture, pacing, the parts of a voice that never show up in a transcript. Grading only the first layer misses the reason the recording exists.
- Telling frustrated apart from disappointed, anxious, and betrayed is a psycholinguistics distinction and it is where most annotation pipelines fail.
- A generalist crowd worker hits a ceiling fast. They can log pitch and volume but may not tell a client why a listener heard betrayal instead of disappointment. That takes linguists and acoustic specialists trained against gold-standard examples, not a labeling queue.
- The two layers get annotated together, not separately. The same recording, transcript plus acoustic description, run through both without collapsing one into the other.
- Annotators described only what they heard. Working from audio and transcript side by side, with no interpretation and no filling in what the transcript said or what the speaker probably meant. If the description didn't match what the ear heard on a re-listen, it was rewritten.
The Stakes
- A vague description produces a vague prompt, and a vague prompt produces a flat persona. The model has nothing precise enough to reproduce on demand.
- An emotion label without the acoustic facts behind it isn't auditable. If a fix can't be traced to a specific acoustic parameter, no one can confirm the fix worked.
- Cross-cultural drift compounds the risk. A description that reads as neutral, precise, and safe in EN-US can carry a different emotional weight once it's read against HI-IN code-switching or JA-JP formality conventions.
- There is no second check. Completed tasks route straight to the client with no downstream QA pass on Sigma's side. A screening gap or a calibration miss becomes the client's problem, not something caught and fixed after the fact.
What We Built
- Native-language annotation across five locales. Every annotator cleared a language proficiency check and a TTS test, plus screening for reliability and trustworthiness given the lack of a downstream QA pass.
- Full-spectrum voice description. Gender, age, accent, pitch, projection, texture, persona, style, emotion, tone, intent, speed, and intonation, written in natural language rather than forced into a fixed taxonomy.
- Locale-correct script enforced without exception. Devanagari for Hindi, Latin characters for Indian English, and the correct mix of Kanji, Hiragana, and Katakana for Japanese, with EN-IN and HI-IN code-switching annotated in place rather than flagged and dropped.
- A minimum of five annotators active per locale from day one. The floor needed to meet the client's diversity requirement rather than hope variance would show up on its own.
- Gold-standard examples built before annotation started. The fix for the same failure point that shows up across every subjective-labeling program Sigma has run: annotators confusing technical pitch with perceived tone until they see the difference side by side.
- A translated glossary of acoustic terms. Monotone, tremulous, drawl, and similar, built for annotators who could hear a quality but not name it in their own language.
- A skip-and-report protocol caught what shouldn't be annotated at all. Undecodable audio, ambiguous multi-speaker files, and foreign-language clips were logged by task ID and returned to the client as faulty rather than force-annotated.
- A PM embedded through the iteration, not just at kickoff. A first batch delivered, guidelines and the annotation interface refined against the client research team's read of the results, then a second batch run against the corrected standard.
Why it Matters Now
- Research teams building expressive voice models need a prompt specific enough to reproduce an emotion on demand, not just a label confirming one was present.
- Safety teams programming guardrails auditing an assistant's emotional register for bias require the acoustic parameters that triggered a perception, not the annotator's guess at what the speaker felt.
- Five locales prove the model. The same annotation layer scales to a global voice interface without losing perceived-tone accuracy in translation.
A model can be instructed to sound frustrated. Only a trained human ear can tell it what frustrated really sounds like.
