Turning 40,000 free-text customer comments a month into a signal the merchandising team acts on

Where they were

A retailer with a healthy data warehouse and a blind spot: everything customers wrote. Product reviews, support tickets, and a post-purchase survey added up to around forty thousand free-text comments a month. A team of two read a sample, tagged it in a spreadsheet, and presented themes quarterly. By the time a theme reached the merchandising team, the product it concerned was often out of season.

What was actually hard

• Volume, obviously — but also consistency. Two people tagging the same comment disagreed about a fifth of the time.
• LLM output is not a column. Asking a model “what is this comment about?” returns prose. Prose doesn’t aggregate.
• Cost anxiety. The first estimate for running every comment through an external API was large enough to end the conversation.

What we did

We kept everything inside Snowflake and treated the model like any other transformation step.

1. Classification with Cortex, in SQL, in a dbt model. Each comment gets a fixed taxonomy (fit, quality, delivery, price, sizing guidance, and so on), a sentiment score, and a short evidence quote. The prompt returns strict JSON; anything that fails to parse is routed to a review queue instead of silently dropped.
2. Parsing into real columns. PARSE_JSON, one row per (comment, topic), with dbt tests on the taxonomy — every topic is an accepted value, every score is between −1 and 1. LLM output became something you can GROUP BY.
3. Dynamic tables refresh the scored set as new comments land, so “this week’s top complaints” is actually this week’s.
4. Three Streamlit apps: a weekly themes view for merchandising, a product-level drill-down for buyers, and a review queue for the two analysts — who now check the model’s uncertain cases instead of reading samples.

What changed

• ~40,000 comments a month classified, every one of them, with an evidence quote you can click through to.
• 3 Streamlit apps in production. The Monday merchandising meeting opens with the themes app.
• Under $200 a month in Cortex spend, because the model runs once per comment inside the warehouse — no data leaves, no per-seat tool.

Within the first month the buyers pulled a product line forward after sizing complaints spiked in week one instead of quarter one. That’s the whole point.

Representative engagement — details anonymized and generalised.