We can tell that the inferencing costs for many of these models are low enough that these models are being sold close to real costs on the basis that many of them are open weight and available from third party providers who have no incentive to subsidize them. I think the frontier labs will need to drop their high per-token prices at least for their low and mid-level models for the reason that several Chinese models (at least Qwen, DeepSeek, Kimi and GLM) are "close enough" that with the right harness they are cost effective alternatives. They won't necessarily need to close the gap - at least not yet -, because these models won't necessarily compete at the same token counts . E.g. at least some of them need to do far more work to solve the same problems. But, yeah, the prices will come down one way or the other. At the same time, even the subscriptions for the cheap Chinese models are probably subsidised, and those subscriptions are likely to get less generous over time.

If you have a good model router, you can route to older, cheaper models that run on older hardware, for simpler tasks. That helps labs extend the economic life of their hardware investments. They will likely fight it at first though as they see it as reducing ASP. This is why I'm building role-model, a routing protocol and a router runtime: https://role-model.dev/

Running cheaper models on newer hardware is always going to beat running them on older hardware.

> more work for the user Model routers allow this to happen automatically without any more work by the user. > a shittier model A ton of tasks don't require the most expensive frontier models, etc. > I’m not sure why anyone does it 1. Faster solutions from the LLM - also reduces employee costs of having the employee waiting on the LLM 2. Avoiding things like the half-billion dollar per month bill for a single company’s LLM use recently reported in Axios

I sometimes let Claude Opus create plans, DeepSeek v4 pro implements and writes tests. Claude reviews and corrects. Saves like $2-3 per session. Same quality code.

Isn't 5.5 a router, though? As in, some prompts get automatically sent to a cheaper model?

> Deepseek has some models in Bedrock. Just looked into it, seems like at most they have just 3.2, not 4: https://aws.amazon.com/bedrock/pricing/ Looking around their catalogue more, most of their models seem quite outdated, aside from the OpenAI and Anthropic ones (but those get more expensive). I wouldn't willingly pick Bedrock and would instead throw money at OpenRouter, that has both a bunch of providers, as well as almost any model for you to try.

Makes me think of how my Claude.md files specifies to use the built in framework code-generators (rails). Those generators are deterministically right every time. I wonder how often the Agent actually follows the guidance. I do see them follow it when I look. But it doesn't seem so every time.

This is tricky since it can and will ignore your md directions. When possible I try to lean on tool call hooks or skills that invoke deterministic scripts. As much as you can remove the "choice" the better though still there's a lot of randomness in how reliably it invokes skills ime.

Hooks are incredibly underused by most people and are the easiest way to establish a first line of defense against bad behavior. Things like blocking tool calls that will read .env file or execute "create or replace table".

How many more months do we need to wait, until big companies realize that flash models work just fine if you: 1) Don't ask LLMs for big changes 2) Review everything and point them in the right direction Large models still suck at big changes, they produce questionable architecture and you still have to review the code, if your project is serious enough. The codebase quickly become a mess, if you don't pay enough attention. Does not matter which model. So why bother with big models, when flash models are 10x cheaper and much faster to iterate under guidance? Large models can be used for security and bug audits. Flash models work almost the same for changes under 300 LOC when you dictate how you want your code to look.

The easy decision is to just go with the biggest SOTA model you can afford. But this overlooks the other critical part of getting the most out of these things: the harness. I run an autonomous plan/design/code/build/test pipeline with agents using my own orchestrator. Different models are better at different stages, and I use LLMs to judge the output between them. Not everything needs Opus 4.8. The harness provides both the scaffolding to get the right things into the model, and the right things out. But it also lets you dictate which model does which work. It's the pipeline, not the model, that gets you quality at a given token budget.

Is your argument that $1500 / mo is too much? Why would the engineering team not be more rigorous in their model selection given a constraint?

I wonder to what extent models should figure out which model to forward a query to. Or perhaps the big models could learn the difference between an easy and a hard question and charge accordingly? Perhaps, if it can measure complexity, even generate a quote? Small models are fine for small coding tasks but I don't see why big ones can't be broken down most of the time.

Many harnesses do this, I've recently dropped all my big subscriptions for using deepseek. Codewhale (formerly deepseek-tui) will use pro for large tasks and route smaller ones to flash. It's pretty good, but I just use pro and everything as the cost is quite low. This one does not have routing, but reasonix is insane, absolutely insane for saving money. I've used 1.3billion tokens at the cost of 4$. (99-100% cache hit)

> I wonder to what extent models should figure out which model to forward a query to. Or perhaps the big models could learn the difference between an easy and a hard question and charge accordingly? This sounds like something a harness could do (and might already be doing), with work delegated to subagents running on lower-cost models.

Yes, they are all already doing this

Every employee doesn't need $1k in token spend per month, either. That kind of spend makes sense for technical workers in r+d. Most other workers are served fine by $20-30 worth of tokens on a budget model. You don't need Opus to help support write emails.

I wonder if you will see app makers begin to open APIs (MCPs) up in ways that replace computer use. Computer use via human interfaces is pretty hacky IME, and if you can use an app that exposes spreadsheets in a way that reduces token costs by 90%. I'm optimistic that the demand for AI accessibility will drive programmatic interfaces in places where companies were previously reluctant to.

I think we're all past the "bet-money-can-buy" stage. The most expensive models are an order of magnitude more expensive than the middle ground ones, so you need to be selective about what you run where. And with a bit of careful routing - there isn't a lot stopping you sending the hard stuff to a cloud model and the average stuff to an on prem model.

How are people using so many tokens? I'm on the $200/month enterprise plan for Claude Code (because it's a better deal than the API pricing) and I don't come close to the limits. If you use stuff like opusplan and /advisor so you use Sonnet for most of the work and only Opus for the really complex stuff then it's quite easy to keep costs low without affecting performance.

A lot of talk about cheaper models here. Just curios, is there any non-Anthropic model that can do UI well? GPT-5.5 is laughably bad, and I'm never restarting my Anthropic subscription after their 6-month sprint of gaslighting, even if opus was really good at UI.

1) This happened because they fundementally misunderstand how to use AI and how AI is priced 2) Most organizations are throwing everything in for analyses and not limiting the answer they want. You need to be specific of about what you analyze and what answers you want 3) People undervalue prompting or templated responses. I will have written. validated and sanity checked a prompt several times and run it across several models before I say its ready for use. But when it is, I know what it will give me and that the scope of its research and answer is as close to what I want as it can be. As little excess as I can. This all saves tokens

I think there's too much variance between what model you're using and how much you turn your brain off. If I just paste a ticket number into 4.8xHigh its going to use a lot more tokens than if I read the ticket, tell Sonnet what it needs to do, make my commit, run unit tests myself, etc.