Checking Automatic Captions Against a Finished Recording
How to measure the errors on your own track and decide whether automation is worth keeping.
Automatic captions have to be checked against the recording itself, not against the lyric sheet you intended the singer to perform. A generated vocal may change a word, repeat a phrase, or blur a line you wrote clearly. The captioning system and the singer can make different mistakes in the same passage.
There is no measured error rate behind this article. ACLTracks did not run a music-video caption benchmark. What is useful here is a way to measure the mistakes on your own finished recording, and then decide whether automation saves enough correction work to be worth keeping.
Begin with the words viewers can hear
Build an audition transcript from the actual final audio. Keep the intended lyrics next to it, and mark every difference between what was written and what was sung. Do not quietly replace an audible wrong word with the intended one and then treat the captions as an accurate transcript.
Listen through the verse, the chorus, the ad-libs, and the ending. Mark unclear passages explicitly instead of guessing from context. If two listeners disagree, keep the disagreement on record until you can resolve it. A caption benchmark built on an invented reference cannot tell you whether the automatic text was wrong.
Decide up front how you will handle backing vocals, repeated syllables, and non-lexical sounds such as a wordless refrain. Accessibility captions and a promotional lyric overlay are related outputs with different purposes. Name the target before you compare results.
Know what the automatic workflow promises
YouTube: Automatic captioning says its machine-generated captions can misrepresent speech because of pronunciation, accents, dialects, or background noise, and it tells creators to review them. That guidance concerns speech recognition. It does not give you a universal accuracy percentage for singing over music.
Adobe: Captions in Premiere documents generating a transcript, editing it, and creating a caption track, with export routes that include burned-in captions and an SRT sidecar. That is a finishing workflow. It is not evidence that a first-pass transcript captures every sung line faithfully.
Record the tool, language, date, settings, and input you used. If captions never appear, or processing fails, log that separately from a completed transcript with wrong words. A missing result and a poor result cost you different amounts of time.
Compare the full mix and the vocal input honestly
For a proposed test, use short original passages: a clear verse, a faster line, a sustained note, and a layered chorus. Keep the raw automatic result before you correct anything. Include the passages in full rather than picking only the line that makes the tool look best or worst.
If the tool supports it, run a vocal-focused version as a second condition. If that audio came from stem separation, label the extraction and keep the source. Separation can reveal words while it changes their sound. Do not report success on a cleaned vocal as success on the untouched full mix.
Track the timing relationship to the finished video. A transcript made from a vocal file with the leading silence trimmed needs its offset accounted for before you place captions on the music video. Otherwise a good text result becomes a poor caption track.
Count word mistakes without hiding the rules
Hugging Face: Speech-recognition evaluation metrics defines word error rate as substitutions plus insertions plus deletions, divided by the number of words in the reference. Use that definition if you report WER, and state the text normalization choices you applied before counting.
As an illustrative calculation, three substitutions, one deletion, and two insertions against a 50-word reference give six errors divided by 50, or 12 percent WER. Those counts are invented for the arithmetic. They are not results from a tool test. WER can exceed 100 percent when the output adds enough extra words, so it is not automatically the percentage of words captioned correctly.
Decide whether case, punctuation, contractions, repeated words, and wordless sounds count toward the score. Apply the same rules to every candidate. Keep both the original text and the normalized comparison so another reader can see how the number was produced.
Score timing and readability separately
Correct words can still arrive too early, linger too long, or straddle a cut awkwardly. Watch the caption track at normal playback speed. Check entrances, phrase endings, overlaps, and whether a long sung word stays readable without covering the next line.
Inspect the line breaks on the actual video. Keep phrases together where practical, and avoid covering the performer’s mouth or an important action. Review it at phone size as well as on the editing monitor. A transcript can be accurate while its displayed captions are hard to use.
Do not fold every defect into the word score. Record text errors, timing repairs, layout changes, and total correction time separately. Those observations are what tell you whether the tool gave you a useful starting point for this particular track.
Finish and verify the real caption track
YouTube: Editing caption text and timing documents editing both the text and the timestamps, including downloading caption files. Keep the raw automatic track and the corrected version distinct. If you supply your own verified transcript, identify it as a different route from automatic recognition.
Export the caption format you intend to deliver, then reopen it against the final video. Confirm the repeated choruses, the last line, special characters, and the timing after any edit. If the music or the picture changes, review the affected captions again instead of treating the earlier approval as permanent.
My recommendation: use automation as a first pass only when correcting its output reaches your required standard faster than writing the captions by hand. An error rate belongs to the specific recording and workflow you tested. Until you have that test, a confident percentage tells you less than an honest correction plan.
Sources and scope
Official sources were checked October 12, 2026. Product and policy statements describe the cited documentation. Proposed experiments and examples are not completed ACLTracks tests.
