I shipped a small UI update that immediately changed how the app feels: API cost awareness before and during a run. Nothing fancy on the surface. Just a pre-flight estimate before the user starts, plus a spend indicator in the sidebar while they work. But this little feature touched pricing logic, prompt assembly, token estimates, streaming runs, retries, and a few uncomfortable assumptions I had been avoiding.
This was one of those vibe coding sessions where the idea sounded simple at 9:30 PM and turned into a full pass through the product by midnight. I wanted the app to stop acting like API spend was invisible. If a user is about to run an expensive request, they should know before clicking. If they are exploring, iterating, and sending multiple calls, they should have a visible sense of the meter running.
The problem I wanted to fix
The app already tracked API usage after requests completed. That helped me debug billing and see which models were getting expensive, but it did almost nothing for the person using the product in the moment.
The old flow felt like this:
- Write a prompt.
- Pick a model.
- Click run.
- Wait for the response.
- Find out later what it cost.
That is backwards. Cost needs to show up before the decision, not after the receipt.
So I set a simple goal for this pass: give users a cost estimate before the API call starts, then keep a running spend total visible in the sidebar. The estimate does not need to be perfect. It needs to be honest, close enough, and clearly labeled as an estimate.
What shipped
The shipped version has two visible pieces.
- A pre-flight estimate shown near the run action before sending the request.
- A sidebar spend indicator that tracks estimated and confirmed API spend for the current session.
The pre-flight estimate updates when the user changes the prompt, switches models, adds context, or changes settings that affect output size. The sidebar indicator updates after each completed call using actual provider usage when available.
I also added small status labels so the numbers are not pretending to be more precise than they are. The UI distinguishes between estimated cost, in-progress cost, and confirmed cost. That sounds minor, but it matters. A price shown before a run is a forecast. A price after the provider returns token usage is a record.
The first version was intentionally plain
I resisted the urge to design a whole billing dashboard. This feature needed to live where the user already makes decisions.
The first pass was almost boring:
- Estimated input tokens
- Estimated output tokens
- Estimated cost
- Current session spend
- A tiny note when the number is approximate
That was enough to change behavior. Once I could see that a long context window plus a premium model had a real cost before I clicked run, I started making better choices. Sometimes I trimmed the input. Sometimes I switched models. Sometimes I still ran the expensive call because it was worth it, but at least it was intentional.
The win was not making people spend less. The win was making spend visible at the moment of choice.
How the estimate works
The estimate starts with the prompt payload. I count the user text, attached context, selected template instructions, and hidden system instructions. That last part was easy to miss. The user only sees their prompt, but the API sees the full assembled message stack. If the estimate only counts visible text, it underreports cost every time.
For the first version, I used a practical token estimate instead of trying to perfectly match every provider tokenizer in the browser. Text length gets converted into a token estimate using a model-specific ratio, then I add a small buffer. It is not exact, but it is fast and good enough for pre-flight display.
Then I calculate the expected output cost using the max output setting. This was a design decision I had to think through. If I estimate based on a typical output, the price looks friendlier but can be misleading. If I estimate based on the max possible output, it looks scarier but gives a safer upper bound.
I landed in the middle. The UI shows an estimated cost based on the likely output target, and the detail view can show the possible upper range when the max output is high. That gave me a better balance between useful and alarming.
The pricing table became a real part of the app
Before this, model pricing lived in the code like a quiet helper. This feature forced it to become a proper source of truth. Each supported model now has input pricing, output pricing, currency, provider name, and a last-updated field.
I kept the structure simple:
| Field | Purpose |
|---|---|
| Provider | Groups models by API provider. |
| Model ID | Matches the actual model used in the request. |
| Input price | Cost for prompt tokens. |
| Output price | Cost for generated tokens. |
| Currency | Keeps display formatting consistent. |
| Last updated | Helps catch stale pricing later. |
I am glad I did this now. Cost awareness gets messy fast if pricing data is scattered across random files. Even in a small product, the moment you show money in the UI, stale data becomes a trust problem.
The sidebar spend indicator
The sidebar piece started as a tiny number and turned into one of my favorite parts of the update. It shows the current session spend without making the interface feel like an accounting tool.
I used three states:
- Estimated: shown before a request starts.
- Pending: shown while a request is streaming or waiting for final usage data.
- Confirmed: shown after the provider returns actual token usage.
The sidebar total uses confirmed cost when it has it. During a running request, it can temporarily include the pre-flight estimate so the user does not see the total freeze while work is happening. When the request finishes, the app reconciles the number and swaps estimate for actual cost.
This small reconciliation step made the feature feel much more grounded. Without it, the total could jump around in confusing ways, especially during streaming responses where the real usage arrives only at the end.
The request flow after the update
The request flow now has one extra stop before the API call leaves the app.
- The user writes or edits the prompt.
- The app assembles the full payload, including hidden instructions and context.
- The estimator calculates input tokens, likely output tokens, and estimated cost.
- The UI displays the pre-flight estimate near the run button.
- The user starts the request.
- The sidebar marks the estimate as pending spend.
- The provider returns actual usage after completion.
- The app replaces the pending estimate with confirmed spend.
That sequence sounds obvious now, but writing it out helped me find bugs. Any place where a request could skip one of those steps became a possible accounting issue.
What broke during the build
The first bug was hidden prompt cost. I had the estimator wired to the visible editor text, which made the number look clean but wrong. Once I added system instructions, tool context, and template wrappers, some estimates jumped by a noticeable amount. Annoying, but correct.
The second bug was double counting retries. If a request failed and automatically retried, the UI counted the estimate twice before the final usage came back. That made the sidebar look like the user had spent more than they actually had. I fixed it by giving each run a stable request ID and tracking spend events against that ID instead of blindly adding every state change.
The third bug was decimal formatting. Tiny API calls can cost fractions of a cent, and normal currency formatting can round them down to $0.00. That is technically true for the displayed cents, but it makes the indicator feel broken. I changed the display so very small amounts show as “less than $0.01” instead of zero.
The fourth bug came from streaming. The app starts showing output before the provider sends final usage data, so there is a gap between “the user sees a result” and “the app knows the real cost.” That gap is where the pending state matters. Without it, completed-looking responses had unconfirmed spend, which felt off.
The UI copy mattered more than expected
I spent a surprising amount of time on labels. “Cost” sounded too final before the run. “Estimate” was better. “Projected” sounded a little too finance-y. “Approx.” saved space but felt vague.
The version I shipped uses plain language:
- Estimated API cost
- Session spend
- Pending
- Confirmed
No drama. No scary warning unless the estimate crosses a threshold. Most of the time, the UI should just be quietly useful. If every request feels like a billing alert, users will stop trusting the interface and start ignoring it.
Thresholds and gentle warnings
I added a soft threshold for high-cost requests. When an estimate crosses that line, the UI changes tone slightly. The number gets more visible, and a short note appears telling the user why the estimate is higher. Usually it is because the context is long, the selected model is expensive, or the output limit is set high.
I did not add a blocking confirmation dialog yet. I considered it, but it felt too heavy for this stage. A warning is enough for now. Later, I may add user-defined limits where someone can choose to require confirmation above a certain amount.
The best version of this feature does not nag. It gives users enough information to make a better call.
Testing it with real prompts
I tested the feature with three types of prompts. Short prompts, giant context-heavy prompts, and prompts with high output limits. That covered most of the ways costs can surprise people.
Short prompts were easy. The estimates stayed tiny, and the confirmed costs landed close enough. Long context prompts were more interesting. A user might write one sentence, but if they attach a full document or pull in a big memory block, the cost changes completely. That is exactly the kind of surprise this UI is meant to prevent.
High output limits were the final test. Some models are cheap on input and much more expensive on output. If the user allows a huge response, the estimate needs to reflect that possible spend. This is where showing a range may become the better long-term design.
What I learned
The biggest lesson is that cost awareness is not just a billing feature. It is a product confidence feature. When people can see the shape of spend before they act, the product feels more honest.
I also learned that estimates need humility. If the number is approximate, say so. If actual usage replaces it later, show that clearly. Trying to make an estimate look exact only creates confusion when the final number differs by a bit.
Another lesson: build cost tracking around request IDs early. Any app with retries, streaming, cancellation, background jobs, or tool calls can accidentally count the same thing twice. A clean event trail makes the sidebar total much easier to trust.
What I would improve next
The next pass will probably add user-set budgets. I want someone to be able to say, “Warn me after this session reaches $1,” or “Ask before any single request above $0.25.” That should be optional, not forced.
I also want to support cost ranges for requests with unpredictable output length. A single estimate is nice, but a low-to-high range can be more honest when the output cap is wide open.
Another future improvement is per-feature spend. The sidebar currently shows session spend as one total. That is useful, but I also want to know which workflows are driving cost. Document analysis, chat, image generation, background enrichment, each one should be traceable without turning the app into a spreadsheet.
The small win
After shipping this, I ran a few normal workflows and immediately felt the difference. The app felt more accountable. I could see when a request was cheap enough to fire off without thinking, and I could see when a prompt deserved a quick trim before sending.
That is the whole point. API-powered tools should not hide the meter. They do not need to scare users either. Just show the estimate, keep the running total visible, reconcile it after the call, and be clear about what is known versus guessed.
This shipped as a small UI feature, but it made the product feel more mature right away. Next time I add a new model or workflow, cost display is now part of the build checklist from the start, not something I bolt on later.