Building a 93% Accurate Grocery Classifier with LLM Agents

LLM-based grocery classification is the process of using large language models to categorize raw grocery text into a structured product taxonomy. Unlike traditional text classification, it requires multilingual entity resolution — mapping items written in Hindi, Tamil, English, Hinglish, or any of India's 20+ major languages to specific database entities with variant-level specificity.

Our client operates a household management platform serving Indian families. Users add grocery items to shopping carts via WhatsApp messages — and the items arrive as raw text:

Each item needs classification into a 723-entry product taxonomy of Category > SubCategory > Type — from "Dairy, Bread and Eggs > Milk > Toned Milk" to "Masala, Pickle and Papad > Powdered Spices > Turmeric Powder" — then resolved to a database entity with brand, quantity, and unit specificity.

Architecture: Two-Tier Classification

We designed a two-tier system that optimizes for both cost and accuracy. The alias cache handles the 95%+ of items that have been seen before, while the LLM agent classifies genuinely novel items.

Iteration 1: The Naive Approach

Our first working version was a single LLM call per batch of 50 items using LangChain's with_structured_output with the default method (function calling):

structured_llm = llm.with_structured_output(ClassificationResult)
result = await structured_llm.ainvoke([system_msg, user_msg])

Three problems destroyed accuracy:

# One line change worth 19 percentage points
llm.with_structured_output(ClassificationResult, method="json_schema")

Iteration 2: Indian Language Support

19 of 24 Hindi items were already classified correctly. The 5 failures were all generic unbranded items: "दूध" (milk), "आटा" (flour), "चीनी" (sugar). The LLM knew these were dairy/cereal/sweetener items but defaulted to "Others" as the type — it couldn't confidently pick "Toned Milk" vs "Full Cream Milk" for a generic Hindi word.

Default-type rules for unbranded items reflecting Indian household purchasing patterns:

# Indian household defaults
milk / dudh / paal  →  "Toned Milk"       # Most common milk type
atta / flour        →  "Chakki Atta"       # Most common flour
sugar / cheeni      →  "Sulphur-Free Sugar" # Most common sugar

We also added normalized text matching — Unicode NFC, apostrophe variants, case folding — to handle the LLM occasionally modifying Hindi raw text in its output.

Iteration 3: The Agent Approach

At 89-91%, the remaining failures fell into patterns that a single LLM call couldn't fix:

The Classification Agent

We built a LangGraph ReAct agent that orchestrates five steps with a validation loop for invalid items:

What validate_and_resolve Does

The agent LLM then reasons about each invalid item — reading the suggestions and picking the best correction based on item text and brand. This is where the agent approach genuinely outperforms the single-call approach: the fixing step requires contextual reasoning that a structured output call can't do.

Sub-Agent Integration

The classification agent is invoked internally from catalog_utils.classify_items() — the function signature stays identical. All callers benefit transparently:

async def classify_items(client, items) -> dict[str, str | None]:
    set_client(client)  # Inject API client into agent tools
    agent = create_catalog_agent()
    result = await agent.ainvoke({
        "messages": [HumanMessage(content=json.dumps(batch))]
    })
    # Extract variantIds from tool call results
    ...

This follows LangGraph's "invoke subgraph from a node function" pattern. Parent agents never know classification is handled by a sub-agent.

Benchmarking: The Backbone of Iteration

We built a comprehensive benchmarking system early — a 255-item synthetic dataset with ground-truth labels covering:

Results

The agent adds ~50% more cost (extra orchestration LLM calls) but catches and fixes errors that no single-call approach can detect.

6 Lessons from Building LLM Classification Systems

The Path to 95%+

The remaining ~7% of errors break down into addressable categories:

The path to 95%+ likely involves: model upgrade (GPT-5.4 vs Nano), confidence-based retry for uncertain items, and expanding the alias cache to cover more items in Tier 1.

Summary

This system processes real grocery orders daily, classifying items across 12+ Indian languages into a 723-category taxonomy at $0.03 per batch. The agent-based approach turned a brittle single-call pipeline into a self-correcting system that improves with every classification. The three highest-impact changes were: matching results by identity instead of position, switching from function_calling to json_schema structured outputs (+19 percentage points), and adding a ReAct validation agent that catches verifiable errors the LLM makes on its first pass.