From $200 to $30: Five Layers of LLM Cost Optimization
The Problem One of the services I’ve been building for an ecommerce app is a product categorizer: given a product name, assign it a 3-level category path from a large taxonomy. The app is in Polish, so both the product names and the category tree are in Polish — which matters less than you’d think, since LLMs handle this well, but it does mean the examples in this post would normally look like “Drzwi sosnowe 80cm” instead of “Wooden door 80cm.” I’ve translated everything to English here for readability. Simple on paper, until you’re classifying ~1M products a month and watching your LLM bill climb past $200. ...