Skip to main content
A shorter prompt can lower input cost when you remove text that does not help the task. The useful question is whether the edited prompt still produces an acceptable result across the requests you serve. This guide uses email routing to show how to test that change and interpret historical results without assuming that brevity always improves quality.

​
The scenario

Suppose your routing app already uses a suitable model, but each email includes a long system prompt. Repeated instructions may account for much of the input, particularly when the email itself is short. Start by separating system instructions, the customer message, and any retrieved context in your usage analysis. That tells you how much of the request a prompt edit can actually affect. Keep requirements that define correct behavior, such as categories, department mappings, output format, and escalation rules. Remove repetition or unnecessary tone instructions only as a testable hypothesis. The cost of a mistaken route or repair call can exceed the token saving.

​
Test configuration

Keep the model, provider, and settings fixed while comparing prompts on the same held-out emails. Save the complete prompt versions and record actual input and output usage for each request. The following concise prompt is an illustrative starting point; it is not claimed to reproduce the historical experiment.
Use your application’s message structure to supply the email separately. Compare this approach with your complete baseline prompt. An abbreviated baseline with omitted categories cannot support a reproducible comparison. Check that the concise version preserves your actual business rules before including it in an evaluation.

​
Price each prompt

For planning, you can compare illustrative total input counts of 443 and 225 tokens while holding output at 25 tokens. Use the comparison script with the same model/provider pair for both runs and adjust the usage for each prompt, or send separate requests to the cost endpoint. If input costs p USD per token and output costs c, the two estimates are 443p + 25c and 225p + 25c. The difference is 218p. Although input count falls by about 49%, total cost falls by less because output cost remains. Additional charges and cache behavior can change the relationship further. Those counts are illustrative whole-request counts, not tokenizer measurements of the prompt printed above. Include the email and other context when recording actual usage. If output length changes, use each run’s own completion count rather than retaining the planning estimate. Check cache usage too: removing cached text does not necessarily save the full standard input rate.

​
Results

The prior guide reported the following values for verbose and concise prompts on gpt-oss-20b. Treat them as historical reported results. The repository does not supply the full baseline prompt, dataset, sample size, run date, provider configuration, or per-request usage needed to reproduce the experiment. The costs are averages scaled to one million requests, not current rates. The reported cost reduction is about 23.3%, and the latency reduction is about 7.9%. This reconciles the earlier inconsistent 24% and 23% descriptions. The table reports equal accuracy on its test cases, but missing sample details prevent a conclusion that there is no quality tradeoff in production. The 443/225-token planning example does not reproduce these averages. It isolates input cost, whereas the historical figures may reflect different whole-request usage. The available evidence does not explain that difference. A shorter prompt reduces input processing work, but it does not by itself establish faster output generation; measure end-to-end latency in your own environment.

​
Example in action

An enterprise pricing inquiry with a request for a demo should usually reach sales. Both prompt versions in the prior guide reportedly handled that example correctly. To decide between prompts, also inspect messages that mention several departments, omit context, or request an unsupported category. Count schema violations, escalations, and wrong departments separately. If the concise prompt sends more ambiguous messages to human review, that may preserve routing accuracy while increasing operating cost. Decide whether that behavior meets the objective before calling the experiment a success.

​
The takeaway

Adopt a shorter prompt when measured usage decreases and your held-out quality checks still pass. Include retries, output changes, and review work in the cost boundary you chose. Roll it out to a limited traffic share using the FinOps evaluation process, then confirm the saving over a complete reporting window.