LLMs cost 1,431x more than embedding models for near-identical performance

Tool · Hugging Face · stat: 1,431x

A new Hugging Face paper benchmarks ten LLMs against 26 embedding models across 37 tasks. While the top LLM outperforms the best embedding model by just 0.4 points, the LLM costs up to 1,431x more to run. Open LLMs also process tokens up to 736x slower, costing $154 compared to $0.11 per pass.

Using LLMs for standard embedding pipelines is an expensive mistake Startups can slash vector search costs by reserving LLMs for reasoning-heavy retrieval and using standard embedding models for classification.

Source