Tokenmaxxing And Return-On-Tokens
As AI becomes a crucial part of our daily workflow, Return on Tokens will be the future metric to evaluate the quality of AI-driven outcomes.
TL;DR: With an increase in AI adoption in various industries and workflows, the issue of burning a lot of cash on tokens is rising. Tokenmaxing was introduced so we can use AI to its full potential to learn, explore, build, and share our products. But with the introduction of agentic AI, token consumption increased; it uses 1,000 times more tokens than a normal chat would. And some of these outputs are not satisfactory even though the agents may have used a lot of tokens. So it is important to understand how to effectively use tokens to get the desired output, or, in other words, how to get high return on tokens. This blog covers that, along with the role of product leaders and AI engineers in ensuring tokens yield high returns for any task.
Today, many companies are using AI to build their workflows, whether in product, operations, marketing, or other areas. In all these workflows, one common thing is the tokens: both input and output. As frontier models grow in size and complexity, tokens can become quite expensive. Companies providing frontier AI models base their pricing on the input and output tokens. If the task is large, the number of tokens consumed and generated, along with reasoning or CoT, is enormous.
Now, tokens generated at this size will not always promise that the output is satisfactory, but if the output is satisfactory, the “return on tokens” will be justified.
Return on tokens means the “how many tokens were required to complete a task successfully.”
With AI, as we know, things are shaky. Most of the time, the output isn’t satisfactory, and there are many iterations just to get a single feature released or a single task right. In that case, the return on tokens cannot be justified. Meaning, AI is not capable enough to give you the desired output on the first try. Rather, you have to keep on iterating to get to the desired output. This means a lot of tokens are consumed and generated, increasing cost. Here, the problem might be: the prompt, the instructions, the skill files, the evals, the guardrails, the tool schema, etc.
As such, tokens vary depending on the labs that produced the model. They also depend on the training data and methods used to train them. For instance, GPT-5.6 can successfully complete certain tasks in one shot, whereas Claude Opus 5 cannot, and vice versa.
When it comes to production, the operational metric is cost per successful outcome.
Let’s take a look at two examples that contradict each other:
YC CEO Garry Tan once said (and I am paraphrasing) that founders should spend a lot on AI agents because aggressive inference or generated AI output can buy time and working capability – to explore, learn, adopt, and create better products. This meant spending $50,000-$100,000 annually.
Uber CTO Praveen Neppalli, on the other hand, argued that Uber will stop tokenmaxxing. They will start focusing on prompt caching and lowering the cost to produce desirable outcomes with better model selection, usage visibility, and start incorporating open-source or open-weights models in production.
Now, both the examples are right in their own manner. We'll discuss this later in this blog, but one key point is that in both cases, the outcome matters most. Sometimes you need token maxing to get the desired output, and sometimes you need a very controlled token budget to achieve it.
More Tokens Will Not Produce Valuable Outcomes
Let’s understand why token maxing was introduced in the first place. The idea behind token maxing was to ensure that everyone who uses AI uses it to its full potential. But now we are in a place where agents have been introduced, and they have taken over our workspaces, so every time we spin an agent, it consumes 1,000 times more tokens than a simpler chat would do.
Likewise, the longer the agent runs, the more tokens it consumes, and it tends to lose track. Essentially, the error rate increases as the agent runs longer, as shown by METR data. Furthermore, the accuracy also dips.
This begs a question: should we spend more money on tokens or reduce the use of AI agents to save cost? Well, it depends. Before we move on to the solution, let me share some of the facts OpenAI released in this blog.
The blog that OpenAI released argues that the cheapest AI can also get expensive if the output is not satisfactory. The reason is that the cheapest, or smallest, model is not complex enough to handle extremely difficult tasks. This may eventually lead to more retries, more token consumption and generation, and higher costs. Another issue is that it creates more burden for PMs and AI engineers to review and verify the task.
A valuable outcome is not the function of generating more tokens. It is the function of using intelligence with respect to task requirements.
Who Decides Where To Spend Intelligence
As a product leader, you should decide where to spend intelligence most. You should incorporate token management practices like you would do capital allocation, meaning stop asking how to minimize token spending. Instead, you should ask which task will give you the highest return on tokens.
Another point is that you should connect token value or return on tokens with the business value and also the product value. For instance, ask questions like:
Does this task require a complex model with high or max reasoning, or a small, fast model?
How much time will be saved if I use this model for this task? Fewer tokens means less time consumed.
Does this task involve high risk? If yes, use a complex model with high reasoning.
Given the allocated token budget, does this task really require that much test-time compute, or can we reduce the reasoning or test-time effort and still produce satisfactory results?
Now, the best practice you can follow, apart from asking these questions, is to evaluate your current workflow and rewrite the instructions and architectural decisions. Sometimes, instructions can make a simple task more complex. So start rewriting the instructions to make them simpler and more goal-oriented. This way, every workflow you have will consume and produce fewer tokens while delivering effective, satisfactory results.
In simpler terms, ask whether the workflow produces value. Value is the keyword here. The objective is always to create value. If the workflow does not create value, something is wrong with the workflow.
Who Optimizes The Workflow
Now that we know what product leaders should do, let’s bring our attention to what AI engineers should do.
As product leaders decide where to spend intelligence, AI engineers optimize it to meet or satisfy that need. For instance, AI engineers:
Decide which model to use for which task.
Design a routing system to switch models for appropriate tasks:
Smaller models for simple tasks.
Mid-level models for coding.
Top-tier model for planning and execution.
From an engineering point of view, more context does not mean better reasons. It is the combination of everything. Sometimes more context can confuse the model. I’ve already written a blog on context rot, which you can check out here. The blog essentially argues that too much information can create context rot and confuse the model, leading to sloppy results.
Another thing that engineers should focus on is early stopping. Agents can go rogue and astray. We have already seen this in Mythos research and in the OpenAI cyberattack on HuggingFace. The point I want to make is that engineers should always have a way to stop the agent when it goes rogue or even wanders away from the goal. As it turns out, early stopping can save up to 28-64% of tokens.
AI engineers could also use a mixture of models, blending a closed-source model with an open-weights model. This would give them a huge edge because open-weights models are much cheaper and can do as much as closed-source models. This way, engineers could create valuable outputs using minimal inference.
Explore Like a Tokenmaxxier And Operate Like an Economist
I think token maxing is rational and profitable only if the task is open-ended. This is especially apt for research purposes, like market or product research, where you don’t know what output or results you want. I think it’s very good for that.
Not only that, even if you are exploring the multiple architectures of a certain product, token maxing will be very valuable. It can offer you different architectures that will suit your needs or take you on a path where you might find something valuable. Then you narrow it down.
Essentially, if you don’t have an understanding of what the output should look like, then a token maximizer works very well.
But if the task has a goal or a strict structure, then you should allocate a budget for the task. For example, what is the tech stack you will be using? What are the features that you should incorporate in the product based on the user feedback, market research, and product vision? You should not spend too many tokens on those tasks. Use a larger model to plan, then split the task across different models.
One work framework that I would like to share is:
Explore: Discover and explore various plausible solutions.
Prove: Define what a satisfactory and valuable output should look like. Here you should write your own evals, guardrails, and policies that AI should use.
Measure: Calculate the quality of the output: whether the output is providing return on tokens or it is just wasting away tokens.
Route: Route different tasks across multiple models based on the complexity of the task.
Budget: Set a token limit; essentially budget allocation with respect to time.
Compress: Trim unnecessary instructions and workflows while ensuring the output remains valuable.
Closing Thoughts
Earlier, I gave two examples: one of Y Combinator CEO Gary Tan and Uber CTO Praveen Neppalli. Both examples are correct in their own manner. As I said before, when you want to explore an open-ended question or an open-ended task, you don’t know what the right path is, then you must adopt token maximizing. This is also applicable for tasks where you don’t have enough information about a certain product or a certain feature,
On the other hand, if you are very firm and very structured about what you want, and you have a tight grip on the type of outcome you want, then adopt token budgeting. It is not required for you to spend too many tokens on a task that is already well planned and structured. In both cases, you will get a high return on tokens.
In any case, the right approach matters, and both approaches should eventually give you a valuable outcome if used correctly.