Don't classify, hallucinate

	Don't classify, hallucinate(softwaredoug.com)
	243 points by softwaredoug 11 days ago \| 99 comments
	tl;dr: Instead of constraining an LLM's output to a large fixed taxonomy via structured outputs (expensive, hits context limits), have a cheap/dumb LLM hallucinate plausible classifications for the input query. Then embed both the hallucinated labels and your real taxonomy, and use nearest-neighbor lookup to map the fake output to the closest legitimate category. This avoids shipping the full schema on every call and scales better.
	HN Discussion: ↓Absurdity that modern programming involves generating wrong answers then mapping them to correct ones ↓Why not just embed the query directly and skip the LLM hallucination step entirely ↓Question whether hallucinated labels actually match schema better than the raw query would •Sharing related alternative approaches like clustering embeddings or using rerankers/small LLMs on top-k ↑This is essentially HyDE, a known useful technique for retrieval