Introduction
Most discussions of classical ML versus AI engineering stay abstract - definitions, principles, high-level contrasts. But the real test of understanding these differences comes when you're staring at an actual product requirement and have to make a concrete architectural call, with real constraints like latency budgets, labeling costs, and a production timeline. This article works through exactly that scenario: an AI engineer at an e-commerce company tasked with building a product category classifier, and the chain of decisions that follow from a single, well-defined requirement.
The feature itself sounds simple: automatically assign each new product listing to one of 75 predefined categories, such as "Electronics > Headphones" or "Home & Garden > Lighting." But "simple" doesn't mean the architectural decision is obvious. Should this be a classical ML model, a foundation model used out of the box, or a foundation model adapted through some combination of prompting, retrieval, or fine-tuning? Working through this question systematically - output type, model approach, data requirements, adaptation technique, and differentiation strategy - illustrates how the concepts from classical ML versus AI engineering actually play out in a concrete decision, not just in theory.
The Scenario: A Product Category Classifier
Before jumping to a solution, it's worth sitting with the problem statement itself, because the answer to "which approach should I use" depends entirely on correctly characterizing the task. The feature requirement states: automatically assign each new product listing to one of 75 predefined categories. Every new listing - whether it's a pair of wireless headphones or a patio lamp - needs to land in exactly one category from a fixed taxonomy that the business has already defined.
This is a deceptively common pattern in production systems. E-commerce platforms, support ticket systems, content moderation pipelines, and document routing systems all share this shape: a large but fixed set of possible outcomes, with new inputs that need to be sorted into one of them. Recognizing this pattern early is valuable, because it immediately narrows the space of reasonable architectural choices - a fixed-category classification task behaves very differently from an open-ended task like drafting marketing copy or answering free-form customer questions, even though both might eventually involve a foundation model somewhere in the pipeline.
Step 1: Characterizing the Output Type
The first question to answer, before anything else, is whether the task is close-ended or open-ended, because this single characterization drives nearly every downstream decision. A close-ended task is one where the model's output can only be among a predefined, enumerable set of values - a classifier choosing between "spam" and "not spam" is the canonical example. An open-ended task, by contrast, has no fixed set of valid outputs; a request to summarize an article or draft a reply has a wide space of acceptable responses that can't be enumerated in advance.
The product category classifier is unambiguously close-ended. The output can only be one of 75 predefined categories - there is a fixed, finite, and already-known set of possible answers, and the task is purely one of selecting the correct member of that set given the product information. This matters because close-ended tasks are exactly the kind of problem classical ML excels at, and it's also the kind of task where annotation is comparatively tractable: a human labeling a product listing just needs to pick the right category from a list, not generate novel open-ended text. Getting this characterization right at the outset prevents a common mistake - reaching for the most powerful, general-purpose tool available simply because it's available, when a more specialized, far cheaper tool is a better structural fit for the problem.
Step 2: Choosing a Model Approach
With the task correctly identified as close-ended, the next question is which kind of model to build it with: a task-specific classical ML model, a foundation model used directly, or a foundation model adapted through some technique. All three are genuinely viable here, and the right choice depends on constraints the requirement doesn't fully specify - expected volume, latency budget, and how much labeled historical data already exists.
A task-specific classical ML model is a strong candidate precisely because classification into a fixed set of predefined categories is exactly the kind of problem traditional supervised ML was built to solve. A model like a gradient-boosted tree or a fine-tuned small transformer classifier trained specifically on this categorization task would likely be considerably smaller, faster at inference, and cheaper to run at scale than a large foundation model - a meaningful consideration when the system may need to classify millions of product listings, potentially in near real-time as sellers add new inventory.
A foundation model adapted through fine-tuning is also viable, and it has one specific advantage: it can leverage the model's pretrained understanding of natural language to handle product descriptions with unusual phrasing, typos, or ambiguous wording more gracefully than a classical model trained from a narrower feature set might. However, for a well-defined, closed classification task like this one, bringing in a full foundation model can be overkill relative to its cost and latency, especially once you're classifying at e-commerce scale.
A pragmatic starting point threads the needle between these options: begin with a foundation model using prompt engineering, because it's by far the fastest way to get a working prototype without any upfront labeling or training investment, and then evaluate - using real production accuracy and cost data - whether a smaller, task-specific model would be more cost-effective once the system is running at scale. This sequencing lets the team validate whether 75-way classification is even difficult enough to need a specialized model before committing engineering time to building one.
Step 3: Working Out Data Requirements
Regardless of which model approach is eventually chosen, this task requires labeled data: product listings paired with their correct category. The good news is that, because this is a close-ended task, labeling is comparatively straightforward - a human labeler just needs to pick one of 75 predefined options for a given listing, rather than making a subjective, open-ended judgment call. Many e-commerce companies already have a substantial head start here, since historical product listings have typically already been manually categorized by merchandising teams, customer service corrections, or prior rule-based systems, which means a labeled dataset may already exist as a byproduct of normal business operations rather than needing to be built from scratch.
The specific data engineering work differs depending on the model approach. If building a classical ML model, the focus shifts toward feature engineering: extracting structured signals from both tabular fields (price range, brand, seller category, shipping weight) and the product's text fields (title, description), potentially using techniques like TF-IDF or simple text embeddings as input features to a traditional classifier. If instead using a foundation model via prompting or fine-tuning, the focus shifts toward data quality control and deduplication rather than feature engineering - ensuring the labeled examples used for few-shot prompting or fine-tuning are accurate, free of near-duplicate listings that could bias the model toward overrepresented categories, and reasonably balanced across all 75 categories so that rare categories aren't systematically underrepresented.
A data engineering challenge worth calling out explicitly is class imbalance. With 75 categories, it's virtually guaranteed that some categories (say, "Electronics > Headphones") will have far more historical examples than others (say, a niche subcategory under "Home & Garden"). Whichever approach is chosen, the team needs a deliberate strategy for rare categories - oversampling, synthetic example generation via the foundation model itself, or simply accepting lower accuracy on the long tail while monitoring it closely - because a model that performs excellently on average but poorly on rare categories can still produce a steady stream of visible, frustrating misclassifications for sellers in those categories.
Step 4: Selecting an Adaptation Technique
Having decided to start with a foundation model for rapid prototyping, the next decision is which adaptation technique to reach for first. The recommended starting point is prompt engineering, specifically using few-shot prompting: crafting detailed instructions that describe the 75-category taxonomy, paired with a handful of concrete examples of product descriptions and their correct categories. This gets a working prototype in front of real listings within days, without any training infrastructure or labeled dataset preparation beyond the handful of examples embedded in the prompt itself.
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic();
const CATEGORIES = [
"Electronics > Headphones",
"Electronics > Laptops",
"Home & Garden > Lighting",
"Home & Garden > Furniture",
// ... remaining categories from the 75-category taxonomy
];
const FEW_SHOT_EXAMPLES = `
Product: "Wireless noise-cancelling over-ear headphones with 30hr battery"
Category: Electronics > Headphones
Product: "Adjustable floor lamp with dimmable LED bulb, modern design"
Category: Home & Garden > Lighting
`;
interface ClassificationResult {
category: string;
confidence: "high" | "medium" | "low";
}
async function classifyProduct(
title: string,
description: string
): Promise<ClassificationResult> {
const response = await client.messages.create({
model: "claude-sonnet-4-5",
max_tokens: 100,
system:
`Classify the product into exactly one of these categories:\n` +
`${CATEGORIES.join("\n")}\n\n` +
`Examples:\n${FEW_SHOT_EXAMPLES}\n\n` +
`Return ONLY valid JSON: {"category": "...", "confidence": "high"|"medium"|"low"}. ` +
`If uncertain between categories, use "low" confidence.`,
messages: [
{
role: "user",
content: `Product: "${title}"\nDescription: "${description}"`,
},
],
});
const block = response.content.find((b) => b.type === "text");
if (!block || block.type !== "text") {
throw new Error("Unexpected response shape");
}
return JSON.parse(block.text) as ClassificationResult;
}
If evaluation against a held-out labeled set shows prompt engineering isn't hitting the required accuracy threshold - which is likely once you account for subtle distinctions between adjacent categories, like "Electronics > Headphones" versus "Electronics > Audio Accessories" - the next step is fine-tuning on the labeled dataset. Fine-tuning can meaningfully improve accuracy on this specific, well-defined classification task because the model no longer needs the full category taxonomy and examples spelled out in every single request; that structure gets partially baked into the model's weights instead. Fine-tuning also opens the door to using a smaller, cheaper base model than the one needed for prompting-only accuracy, since a model doesn't need as much general-purpose reasoning ability once it's been specifically trained for this one task - a direct path toward the cost and latency improvements that matter at e-commerce scale.
Step 5: Thinking About Differentiation
A question that's easy to skip in a scenario like this, but that matters enormously in practice, is where your team's competitive advantage actually comes from. Any competitor building a similar product classifier has access to the same foundation models you do - Claude, GPT, or an open-weight alternative is equally available to everyone. For a close-ended task like this one, differentiation doesn't come from somehow having a "better" model; it comes from three specific areas of engineering discipline.
The first is data quality: a labeled dataset with cleaner, more consistent category assignments, built from a rigorous internal labeling process, directly produces a more accurate classifier regardless of which underlying model or technique is used. The second is evaluation rigor - specifically, measuring accuracy not just in aggregate but broken down across all 75 categories individually, since an aggregate accuracy number can mask systematically poor performance on rare or ambiguous categories that quietly frustrate sellers and buyers in those segments. The third is inference optimization: because this system may need to classify millions of listings, the engineering work of keeping per-classification cost and latency low - through batching, caching repeated or near-duplicate listings, or routing straightforward classifications to a cheaper model while reserving a more capable model for ambiguous cases - becomes a genuine source of operational advantage that a competitor using the same underlying foundation model might not have invested in.
Trade-offs and Pitfalls in This Scenario
The most common mistake in a scenario like this is treating the foundation-model prototype as the final production system rather than as a validation step. A prompt-engineered classifier built in a few days is excellent for confirming the task is tractable and surfacing edge cases in the category taxonomy, but running that same approach at full production volume - potentially millions of listings - can be far more expensive per classification than a purpose-built classical model or a fine-tuned, smaller model would be. Teams that skip the "evaluate whether a smaller model would be more cost-effective" step often end up with a working but needlessly expensive production system.
A second pitfall specific to this scenario is underestimating the difficulty of boundary cases between adjacent categories. With 75 categories, there will inevitably be products that plausibly fit more than one - a pair of headphones marketed as a fashion accessory, for instance, might arguably belong in "Electronics > Headphones" or a fashion-adjacent category. Neither classical ML nor foundation-model approaches eliminate this ambiguity; it needs to be resolved through clear labeling guidelines and, ideally, a documented tie-breaking convention that both human labelers and the model's prompt or fine-tuning data consistently follow. Without this, inter-annotator disagreement on ambiguous products will quietly cap the accuracy ceiling of any model trained or prompted on inconsistent labels.
A third, subtler pitfall is neglecting the long tail of rare categories during evaluation. It's tempting to report a single aggregate accuracy number and call the project successful, but a model that achieves 95% overall accuracy while performing at 60% on the ten rarest categories is hiding a real problem - one that disproportionately affects sellers in those niche categories and can erode trust in the system even while the headline metric looks strong.
Best Practices for Similar Close-Ended Classification Scenarios
When facing a structurally similar problem - fixed taxonomy, high volume, existing historical labels - start by explicitly confirming the task is close-ended, since this single fact should immediately put classical ML back on the table as a serious option rather than defaulting straight to a foundation model. Use a foundation model with prompt engineering as a fast, low-cost way to validate feasibility and surface taxonomy ambiguities before committing to either a classical model build or a fine-tuning investment.
Build your evaluation harness to report per-category accuracy from the very first prototype, not just an aggregate number, so that problems with rare categories surface early rather than after launch. Treat the choice between a classical ML model, a fine-tuned foundation model, and a prompted foundation model as a question to be answered with production cost and accuracy data, not intuition - run the cheapest option (prompting) first precisely because it gives you real evidence to make that later decision well-informed rather than speculative.
Key Takeaways
- Confirm whether your task is close-ended or open-ended before choosing an architecture. This single determination should reopen classical ML as a serious option for well-defined classification problems.
- Start with the cheapest viable approach (prompt engineering) to validate feasibility fast, then invest in fine-tuning or a classical model once you have real accuracy and cost data.
- Labeled data for close-ended tasks is comparatively cheap to produce, and historical business data (manual categorizations, past corrections) is often already available.
- Evaluate accuracy per category, not just in aggregate, since rare categories can hide meaningful performance gaps behind a strong headline number.
- Differentiation for close-ended tasks comes from data quality, evaluation rigor, and inference optimization - not from the underlying foundation model, which your competitors can access just as easily.
Conclusion
The product category classifier scenario is a useful microcosm of the broader classical ML versus AI engineering decision, precisely because it resists a single "right" answer and instead rewards systematic reasoning. Correctly identifying the task as close-ended narrows the decision space meaningfully; from there, starting cheap with prompt engineering, gathering real production evidence, and only escalating to fine-tuning or a dedicated classical model once that evidence justifies it, captures most of the practical value available without over-investing upfront.
What this scenario ultimately illustrates is that the "classical ML vs. foundation model" question isn't really a binary choice made once - it's a sequence of evidence-driven decisions, each one informed by the task's structure, the data already on hand, and what production reality reveals about where the simplest approach's limits actually are. Teams that work through this sequence deliberately, rather than defaulting to whichever approach is most fashionable, consistently end up with systems that are both accurate and economically sound at scale.
References
- Huyen, Chip. AI Engineering: Building Applications with Foundation Models. O'Reilly Media, 2024.
- Huyen, Chip. Designing Machine Learning Systems. O'Reilly Media, 2022.
- Brown, Tom B. et al. "Language Models are Few-Shot Learners." NeurIPS, 2020.
- Hu, Edward J. et al. "LoRA: Low-Rank Adaptation of Large Language Models." arXiv:2106.09685, 2021.
- Anthropic. "Claude API Documentation." docs.anthropic.com.
- scikit-learn Developers. "Supervised Learning." scikit-learn.org documentation.
- Sculley, D. et al. "Hidden Technical Debt in Machine Learning Systems." NeurIPS, 2015.