So companies are quietly mixing Claude, GPT, and cheaper Chinese models depending on the task. Not because they’re exploring options, but because it saves money. That’s the opposite of the “pick one best model” story we’ve been sold.
OpenAI and Anthropic built valuations on being THE model everyone has to use. But if the real move is just combining whatever’s cheapest per task, suddenly being the best at one thing isn’t as valuable as being good enough at most things for half the price.
We did this at my last job actually, routed cheap classification tasks to a smaller model and kept Claude for anything customer facing. Honestly it wasn’t even a hard call once someone actually benchmarked the cost per request. The classification stuff didn’t need Claude’s reasoning, it just needed to not be dumb, and the price difference was stupid at scale. Curious if this holds for anything latency sensitive though, routing adds overhead and if you’re picking a model per request that’s another network hop before you even start inference.
:: Copyright KIRUPA 2024 //--