A prompt that produced one good song has proven exactly one thing: it can work once. That’s not nothing. But it isn’t a workflow either, and it tells you nothing about the next request or the ninth. If you’re about to spend money on a batch, the question worth answering is whether that prompt is a tool you can call on or a lottery ticket with a subscription attached.
What you want is practical consistency — how often the system hands back material that fits the job. Identical files are a separate question. Bit-for-bit reproducibility and repeatable musical success are not the same claim, and blurring them flatters the tool.
ACLTracks did not run ten generations for this article. What follows is a complete evaluation protocol built from current documentation and proposed editorial criteria. Every number below is illustrative arithmetic, not a test result.
Define what one generation means
Count returned candidates, requests, and charges separately, because they are three different things. Suno’s v6 FAQ documents two songs from a normal generation request, so ten requests need not mean ten recordings. In this protocol, “ten generations” means ten returned candidates, with the request count recorded alongside them.
Retain every output from any request included in the experiment. If the final request returns an extra candidate beyond the planned ten, keep it, flag the overrun, and state how you handled it. Discarding a weak alternate while counting its strong partner is how a dataset quietly becomes a sales pitch.
Set the stopping rule before you start. A fixed ten-candidate run answers a different question than “keep going until I have ten favorites.” Both are legitimate workflows. Only one of them measures the prompt.
Write a brief that can fail clearly
Use original lyrics and specific musical direction. For an illustrative exercise: an electronic-pop song near 120 BPM, an intimate lead vocal, a restrained first verse, a larger chorus, and an instrumental ending. Specify the intended form and duration range before generating — not after you’ve heard what arrived and started rationalizing it.
Separate essentials from preferences. Correct sung lyrics might be essential; a particular synth texture is a preference. A candidate can be genuinely interesting and still fail the deliverable. That distinction lets you keep the discovery without counting it as brief adherence.
| Criterion | How to evaluate it | Decision made before the run |
|---|---|---|
| Lyrics | Compare the sung words with the supplied sheet. | Which omissions or substitutions are disqualifying. |
| Form | Mark audible section boundaries. | Required section order and repetitions. |
| Tempo | Check the pulse over multiple passages. | An acceptable range and whether drift is allowed. |
| Arrangement | Listen for requested musical roles. | Essential instruments and prohibited elements. |
| Vocal and audio quality | Audition the full recording. | Which defects require rejection or repair. |
| Usability | Attempt the intended handoff. | Accepted without repair, accepted after repair, or rejected. |
Thresholds are project choices. A dance loop and a free-tempo cinematic cue should not share timing criteria, and pretending otherwise produces a benchmark nobody can use. Publish the criteria with the results so readers can judge whether your acceptance standard fits their work.
Freeze the inputs and record the environment
Save the exact submitted text, lyrics, references, model selection, settings, date, and output identifiers. Mark which text you wrote and which an assistant expanded. If a workflow rewrites the prompt internally and exposes the revision, retain that too — that revision is an input, whether or not you typed it.
Suno documents a Variety slider that can modify style prompts, and its FAQ recommends zero when you want to retain control of those tags. If that control exists in your workflow, record the setting. Leaving the visible prompt unchanged does not establish that every other generation input stayed fixed.
For an API comparison, retain the actual submitted request with secrets removed. ElevenLabs’s music API reference exposes generation parameters, including a seed. Its February 23, 2026 changelog states that matching seed and parameters helps consistency but does not guarantee exact reproducibility, especially across system updates.
Do not assume a browser app exposes the API’s seed. “Seed not available in this interface” is a valid record. A seeded rerun and an unseeded ten-candidate exploration are different experiments; label them accordingly.
Keep the failures in the dataset
Assign candidates neutral IDs in arrival order. Preserve complete audio, and record empty, interrupted, or unavailable responses separately. A request failure is not an audio-quality failure — don’t file it as one. Its charge and elapsed time still belong in the workflow account.
Listen to every full candidate at comparable levels. Score the brief before choosing your favorite; the other order is how favorites turn into findings. If one person is listening, say so. If several are, keep independent notes before converging on a shared verdict.
Do not repair the first candidate and compare it against the ninth candidate’s raw output without disclosure. Keep a raw-pass score, then a separate repair-pass record. That is what reveals whether the prompt reliably delivers finished material or useful starting points — and those are different products.
Report acceptance without making it a universal benchmark
In the illustrative arithmetic, four accepted recordings out of ten returned candidates would be a 40 percent candidate acceptance rate. That is an example calculation only. It would describe that prompt, model, configuration, brief, and sample — not every song the service can generate.
Keep separate counts for accepted without repair, accepted after repair, and rejected. Then report actual billed charges, request failures, waiting time, and manual repair time. Cost per accepted candidate can be calculated from billed charges divided by accepted candidates; state explicitly whether your calculation also includes labor.
If no candidate passes, report zero acceptance and the failure pattern. Do not divide by zero to produce a cost figure. The useful conclusion may be that the brief or the workflow needs revision before another paid batch.
Identify the failure pattern before rewriting
If the lyrics consistently fail, investigate lyric handling. If the chorus repeatedly misses the intended contrast, revise the arrangement instruction. If the songs meet the brief but fail the export requirement, the problem lives in the delivery workflow. Those findings call for different changes, and fixing the wrong one just burns another batch.
Make the next run a new experiment. Change one meaningful input, retain the earlier criteria where appropriate, and label the new configuration. Combining candidates from several evolving prompts into one “ten-generation test” obscures what improved.
A particularly original rejected candidate can still become a new song. Keep it in the creative folder and count it honestly in the original evaluation. Inspiration and compliance with a fixed brief are both valuable, but they should not be merged into one success statistic.
What ten candidates can actually establish
A small run can expose recurring weaknesses, show the amount of selection required, and estimate practical acceptance for one bounded job. It is too small and too narrow to rank an entire platform reliably. Repeat across different original briefs before making broad claims.
The recommendation is simple: stop judging prompts from their best output alone. Keep the text, retain all candidates, define acceptance before listening, and show the work needed to reach the final result. That turns “this prompt works” into a claim a reader can actually assess.
Sources and scope
Official sources were checked October 12, 2026. The workflow and evaluation examples are proposed work. ACLTracks did not complete the audio experiments described in this article.
