System Design for ML Interviews: 10 Real Problems Walked Through

ML system design interviews test how well you can think beyond models. In these interviews, choosing an algorithm is only one part of the answer. You also need to explain how data is collected, how features are created, how predictions are served, and how the system improves over time. ML系统设计面试考察的是你在模型之外的思考能力。在这些面试中，选择算法只是答案的一部分。你还需要解释数据如何收集、特征如何创建、预测如何提供以及系统如何随时间改进。

Most real ML systems are built around product decisions. A feed system decides what to show. A fraud system decides what to block. A search system decides what to rank. This article walks through 10 such problems in a practical interview style. 大多数真实的ML系统都是围绕产品决策构建的。Feed系统决定显示什么。欺诈系统决定阻止什么。搜索系统决定排序什么。本文以实用的面试风格介绍了10个这样的问题。

How to Think in an ML System Design Interview如何在ML系统设计面试中思考

Start with the product goal. Every ML system is built to make a decision. A feed system decides which post to show. A fraud system decides whether a payment is risky. A search system decides which products should appear first. 从产品目标开始。每个ML系统都是为了做出决策而构建的。Feed系统决定显示哪篇帖子。欺诈系统决定支付是否有风险。搜索系统决定哪些产品应该首先出现。

Once the goal is clear, define success. Do not only talk about model metrics. A good ML system design answer should cover three types of metrics: 一旦目标明确，定义成功。不要只谈论模型指标。一个好的ML系统设计答案应该涵盖三类指标：

• Model metrics: accuracy, AUC, RMSE, precision, recall, NDCG  模型指标：准确率、AUC、RMSE、精确率、召回率、NDCG
• Product metrics: revenue, retention, conversion, fraud loss, user satisfaction  产品指标：收入、留存率、转化率、欺诈损失、用户满意度
• System metrics: latency, throughput, availability, freshness, cost  系统指标：延迟、吞吐量、可用性、新鲜度、成本

Next, discuss the data. Explain what data is collected, how labels are created, and where bias can enter. Some labels are quick, like clicks. Some labels are delayed, like chargebacks, complaints, or product returns. 接下来，讨论数据。解释收集了什么数据、如何创建标签以及偏差可能在哪里引入。有些标签是快速的，比如点击。有些标签是延迟的，比如拒付、投诉或产品退货。

Then split the system into three views: offline path, online path, and feedback loop. 然后将系统分为三个视图：离线路径、在线路径和反馈循环。

Offline Path离线路径

The offline path is used to prepare data and train the model. It usually runs in batches. It focuses on quality, correctness, and repeatability. 离线路径用于准备数据和训练模型。它通常以批处理方式运行。它侧重于质量、正确性和可重复性。

Online Path 在线路径

The online path is used to serve predictions. It must be fast and reliable because the user is waiting for the result. 在线路径用于提供预测。它必须快速可靠，因为用户正在等待结果。

ML System Feedback Loop ML系统反馈循环

The feedback loop connects online behavior back to training. This is how the system improves over time. 反馈循环将在线行为连接回训练。这是系统随时间改进的方式。

These three diagrams cover the core structure of most ML systems. In an interview, they help you explain the system clearly without jumping directly into algorithms. 这三个图涵盖了大多数ML系统的核心结构。在面试中，它们帮助你清晰地解释系统，而不会直接跳入算法。

1. Feed Ranking System1. Feed排序系统

A feed ranking system decides what a user should see next across social media, short video, news, or networking platforms.Feed排序系统决定用户在社交媒体、短视频、新闻或社交平台上接下来应该看到什么。

While it may seem like a simple ranking problem, production systems deal with millions of possible posts and can show only a few. So instead of scoring every post, the system first narrows the candidate set, then uses a stronger model to rank the best options.虽然这看起来像一个简单的排序问题，但生产系统处理数百万篇可能的帖子，只能显示少数几篇。因此，系统不是对所有帖子进行评分，而是首先缩小候选集，然后使用更强的模型对最佳选项进行排序。

Problem Statement问题陈述

Design a personalized feed ranking system. Given a user and a large pool of posts, return a ranked list of posts that the user is likely to find useful or engaging. 设计一个个性化的Feed排序系统。给定一个用户和大量帖子，返回一个用户可能觉得有用或吸引人的排序列表。

The system should handle freshness, personalization, safety, diversity, and low latency. 系统应处理新鲜度、个性化、安全性、多样性和低延迟。

How the System Works系统工作原理

The system usually works in three stages. 系统通常分三个阶段工作。

1. Candidate generation selects a smaller set of posts. These posts can come from people the user follows, topics the user likes, trending content, similar users, or embedding-based retrieval. 候选生成选择较小的帖子集。这些帖子可以来自用户关注的人、用户喜欢的话题、热门内容、相似用户或基于嵌入的检索。
2. The ranking model scores each candidate. The score can be based on predicted clicks, likes, comments, shares, watch time, skips, or hides. In a real system, the final score is often a weighted mix of many predicted actions. 排序模型对每个候选进行评分。分数可以基于预测的点击、点赞、评论、分享、观看时间、跳过或隐藏。在真实系统中，最终分数通常是多个预测行为的加权组合。
3. A rules layer adjusts the ranked list. It removes unsafe content, avoids duplicates, improves diversity, and prevents the feed from showing too many posts from the same creator. 规则层调整排序列表。它移除不安全内容、避免重复、提高多样性，并防止Feed显示太多来自同一创作者的内容。

Feed Ranking Flow Feed排序流程

Important Signals 重要信号

The model needs signals about the user, the post, and the interaction between them. 模型需要关于用户、帖子以及它们之间交互的信号。

Useful signals include: 有用的信号包括：

• User interests and past behavior  用户兴趣和过去行为
• Creator affinity  创作者亲和力
• Post freshness  帖子新鲜度
• Post engagement rate  帖子互动率
• Content category内容类别

These signals help the model understand both long-term preferences and short-term intent. For example, a user may usually like machine learning content, but in the current session they may be watching more career-related posts. 这些信号帮助模型理解长期偏好和短期意图。例如，用户通常喜欢机器学习内容，但在当前会话中可能正在观看更多与职业相关的帖子。

Model Choice 模型选择

A good first version can use a gradient boosted tree model. It works well with tabular features and is easier to debug than a complex deep model. 一个好的初始版本可以使用梯度提升树模型。它适用于表格特征，并且比复杂的深度模型更容易调试。

As the system grows, candidate generation can use embeddings. The ranking model can also become more advanced. It can use deep learning models, sequence models, or multi-task models that predict multiple actions at once. 随着系统的发展，候选生成可以使用嵌入。排序模型也可以变得更高级。它可以使用深度学习模型、序列模型或多任务模型来同时预测多个行为。

The important point is to start simple. A strong baseline with good logging is more useful than a complex model that is hard to monitor. 重要的是从简单开始。一个具有良好日志记录的强基线比一个难以监控的复杂模型更有用。

Evaluation Metrics 评估指标

Offline evaluation can use AUC, NDCG, precision@K, and recall@K. These metrics show whether the model can rank relevant posts higher. 离线评估可以使用AUC、NDCG、precision@K和recall@K。这些指标显示模型是否能够将相关帖子排得更高。

Online evaluation is more important. The system should track click-through rate, dwell time, session length, hide rate, retention, and content diversity. 在线评估更重要。系统应跟踪点击率、停留时间、会话长度、隐藏率、留存率和内容多样性。

A feed system should not optimize only for clicks. Clickbait content may increase short-term engagement but harm long-term user satisfaction. Feed系统不应仅优化点击。点击诱饵内容可能会增加短期互动，但损害长期用户满意度。

Trade-offs 权衡

The biggest trade-off is relevance versus exploration. If the system only shows content similar to past clicks, the feed becomes repetitive. If it explores too much, the user may see irrelevant posts. 最大的权衡是相关性与探索。如果系统只显示与过去点击相似的内容，Feed会变得重复。如果探索太多，用户可能会看到不相关的帖子。

There is also a trade-off between freshness and quality. New posts may not have enough engagement data yet. But if the system ignores new posts, users may miss timely content. 新鲜度和质量之间也存在权衡。新帖子可能还没有足够的互动数据。但如果系统忽略新帖子，用户可能会错过及时的内容。

Latency is another concern. The system must return the feed quickly. Candidate generation, feature lookup, and ranking should all be optimized for fast response. 延迟是另一个问题。系统必须快速返回Feed。候选生成、特征查找和排序都应针对快速响应进行优化。

Interview Tip 面试提示

In an interview, always mention that the system cannot score every post online. A good feed system first generates candidates, then ranks them, and finally applies business rules. 在面试中，始终提到系统无法在线对所有帖子进行评分。一个好的Feed系统首先生成候选，然后排序，最后应用业务规则。

This shows that you understand both ML and system scalability.这表明你理解ML和系统可扩展性。

2. Ads CTR Prediction System2. 广告CTR预测系统

An ads CTR prediction system estimates how likely a user is to click an ad and uses that score to decide which ad to show.广告CTR预测系统估计用户点击广告的可能性，并使用该分数决定显示哪个广告。

Unlike normal content ranking, it must balance three goals: user relevance, advertiser returns, and platform revenue. So the objective is not just more clicks, but showing ads that are relevant, safe, and useful. 与普通内容排序不同，它必须平衡三个目标：用户相关性、广告主回报和平台收入。因此，目标不仅仅是更多点击，而是显示相关、安全且有用的广告。

Problem Statement 

Design a system that predicts the click-through rate of ads in real time. The system should use this prediction with advertiser bids, budgets, and auction rules to select the best ad for a user. 设计一个实时预测广告点击率的系统。系统应使用此预测以及广告主出价、预算和拍卖规则来选择最佳广告给用户。

It should also respect targeting rules, policy checks, frequency caps, and campaign budgets. 它还应尊重定向规则、策略检查、频次上限和广告活动预算。

How the System Works 

The system starts when an ad request is created. This can happen when a user opens a page, searches for something, or scrolls through a feed. 系统从创建广告请求开始。这可能在用户打开页面、搜索内容或滚动Feed时发生。

1. The system filters ads that are not eligible. It checks campaign status, targeting rules, location, language, device type, budget, and policy constraints. 系统过滤不符合条件的广告。它检查广告活动状态、定向规则、位置、语言、设备类型、预算和策略约束。
2. The CTR model scores the remaining ads. It predicts the probability that the user will click each ad. CTR模型对剩余广告进行评分。它预测用户点击每个广告的概率。
3. The auction layer combines predicted CTR with advertiser bids. The final ad is selected based on expected value, quality, and business rules. 拍卖层结合预测CTR和广告主出价。最终广告基于预期价值、质量和业务规则选择。

Ads CTR Prediction Flow 广告CTR预测流程

Important Signals 

The model should use signals from the user, ad, advertiser, and context. 模型应使用来自用户、广告、广告主和上下文的信号。

Useful signals include: 

• User interests and past ad interactions  用户兴趣和过去广告互动
• Page or search context  页面或搜索上下文
• Ad category and creative type  广告类别和创意类型
• Advertiser quality score  广告主质量得分
• Device type and location  设备类型和位置

These signals help the model understand whether the ad is relevant in the current context. For example, a travel ad may perform better when the user is reading about vacation planning than when they are reading about finance. 这些信号帮助模型理解广告在当前上下文中是否相关。例如，当用户阅读度假计划时，旅行广告可能比阅读金融内容时表现更好。

Model Choice 

A simple baseline can use logistic regression. It is fast, easy to train, and works well with sparse categorical features. 一个简单的基线可以使用逻辑回归。它快速、易于训练，并且适用于稀疏分类特征。

A stronger version can use gradient boosted trees or deep learning models with embeddings. These models can learn better interactions between users, ads, and context. 更强的版本可以使用梯度提升树或带有嵌入的深度学习模型。这些模型可以学习用户、广告和上下文之间更好的交互。

For very large ad systems, deep models are useful because there can be millions of users, ads, keywords, and categories. 对于非常大的广告系统，深度模型很有用，因为可能有数百万用户、广告、关键词和类别。

Evaluation Metrics 

Offline metrics include AUC, log loss, and calibration error. Calibration is very important here. If the model predicts a CTR of 5 percent, the real click rate should be close to 5 percent. 离线指标包括AUC、对数损失和校准误差。校准在这里非常重要。如果模型预测CTR为5%，实际点击率应接近5%。

Online metrics include CTR, conversion rate, revenue per impression, advertiser ROI, budget pacing accuracy, and user complaint rate. 在线指标包括CTR、转化率、每次展示收入、广告主ROI、预算节奏准确性和用户投诉率。

A good system should also track long-term user experience. If users start ignoring or hiding ads, the system may be optimizing the wrong thing.一个好的系统还应跟踪长期用户体验。如果用户开始忽略或隐藏广告，系统可能优化了错误的目标。

Trade-offs 

The main trade-off is revenue versus user experience. Showing high-paying ads may increase revenue, but those ads may not always be relevant. 主要的权衡是收入与用户体验。显示高付费广告可能会增加收入，但这些广告可能并不总是相关。

There is also a trade-off between accuracy and latency. A larger model may predict CTR better, but the ad system must respond very quickly. 准确性和延迟之间也存在权衡。更大的模型可能更好地预测CTR，但广告系统必须非常快速地响应。

Another trade-off is exploration versus exploitation. The system needs to test new ads, but it should not show poor ads too often. 另一个权衡是探索与利用。系统需要测试新广告，但不应过于频繁地显示差广告。

Interview Tip 

In an interview, do not describe ads for CTR prediction as only a classification model. A real ads system also includes eligibility checks, auctions, budgets, frequency caps, policy filters, and logging. 在面试中，不要将CTR预测广告描述为仅是一个分类模型。真实的广告系统还包括资格检查、拍卖、预算、频次上限、策略过滤和日志记录。

This shows that you understand the full production system, not just the ML model. 这表明你理解完整的生产系统，而不仅仅是ML模型。

3. E-commerce Search Ranking System3. 电商搜索排序系统

An e-commerce search ranking system decides which products appear for a user query across shopping apps, marketplaces, food delivery, and travel platforms.电商搜索排序系统决定用户查询时哪些产品出现在购物应用、市场、食品配送和旅行平台上。

The goal is to return useful results, not just keyword matches. The system must understand intent, product type, price, availability, quality, and user preference. For example, a query like “running shoes under 3000” should return affordable running shoes, not formal shoes or expensive products that only match the word “shoes.” 目标是返回有用的结果，而不仅仅是关键词匹配。系统必须理解意图、产品类型、价格、可用性、质量和用户偏好。例如，像“3000元以下的跑鞋”这样的查询应返回价格实惠的跑鞋，而不是正装鞋或仅匹配“鞋”字的昂贵产品。

Problem Statement 

Design a search ranking system for an e-commerce platform. Given a user query, return a ranked list of products that are relevant, available, and likely to satisfy the user. 为电商平台设计一个搜索排序系统。给定用户查询，返回相关、可用且可能满足用户的产品排序列表。

The system should support keyword search, semantic search, spelling correction, filters, personalization, and low-latency ranking. 系统应支持关键词搜索、语义搜索、拼写纠正、筛选、个性化和低延迟排序。

How the System Works 

The system can be broken into three steps:系统可以分为三个步骤：

1. Ranking and Rules: Merge candidates, rank them using relevance, popularity, price, ratings, availability, delivery speed, and user behavior, then apply business rules such as filters, sponsored boosts, and out-of-stock removal.排序和规则：合并候选，使用相关性、流行度、价格、评分、可用性、配送速度和用户行为进行排序，然后应用业务规则，如筛选、赞助提升和缺货移除。
2. Query Understanding: Clean and interpret the query using spelling correction, synonym expansion, category detection, and filter extraction.查询理解：使用拼写纠正、同义词扩展、类别检测和筛选提取来清理和解释查询。
3. Candidate Retrieval: Retrieve products using lexical search for exact matches and semantic search for meaning-based matches.候选检索：使用词汇搜索进行精确匹配，使用语义搜索进行基于含义的匹配来检索产品。

E-commerce Search Ranking Flow 电商搜索排序流程

Important Signals 

The ranking model should use signals from the query, product, user, and context. 排序模型应使用来自查询、产品、用户和上下文的信号。

Useful signals include: 

• Query-product text match  查询-产品文本匹配
• Semantic similarity  语义相似度
• Product category  产品类别
• Price and discount  价格和折扣
• Product rating and reviews  产品评分和评论

These signals help the system avoid shallow keyword matching. A product may match the query text, but if it is out of stock or poorly rated, it should not rank high. 这些信号帮助系统避免浅层的关键词匹配。产品可能匹配查询文本，但如果缺货或评分低，则不应排名靠前。

Model Choice 

A good baseline is BM25 with simple business rules. This is easy to build and gives strong results for exact keyword matching. 一个好的基线是BM25加上简单的业务规则。这易于构建，并且对于精确的关键词匹配给出强结果。

A better system can add vector retrieval for semantic matching. This helps with queries where the words do not exactly match product titles. 更好的系统可以添加向量检索进行语义匹配。这有助于处理词语不完全匹配产品标题的查询。

For final ranking, use a learning-to-rank model. LambdaMART, XGBoost ranker, or a neural re-ranker can be used depending on latency and scale. 对于最终排序，使用学习排序模型。根据延迟和规模，可以使用LambdaMART、XGBoost排序器或神经重排序器。

Start simple. Then improve the system by adding semantic retrieval, personalization, and better ranking features. 从简单开始。然后通过添加语义检索、个性化和更好的排序特征来改进系统。

Evaluation Metrics 

Offline metrics include NDCG, MRR, precision@K, and recall@K. These metrics check whether relevant products appear near the top. 离线指标包括NDCG、MRR、precision@K和recall@K。这些指标检查相关产品是否出现在顶部附近。

Online metrics include CTR, add-to-cart rate, purchase conversion rate, zero-result rate, and query reformulation rate. 在线指标包括CTR、加入购物车率、购买转化率、零结果率和查询重构率。

Zero-result rate is especially important. If many users search and find nothing, the retrieval layer is weak. 零结果率尤其重要。如果许多用户搜索但找不到任何东西，则检索层较弱。

Trade-offs 

The main trade-off is relevance versus business value. The most relevant product may not always be the best result if it is out of stock, expensive, or poorly rated. 主要的权衡是相关性与商业价值。最相关的产品可能并不总是最佳结果，如果它缺货、昂贵或评分低。

There is also a trade-off between lexical and semantic search. Lexical search is fast and precise. Semantic search improves recall but can return unexpected results. 词汇搜索和语义搜索之间也存在权衡。词汇搜索快速且精确。语义搜索提高了召回率，但可能返回意外结果。

Neural re-ranking can improve quality, but it adds latency. So it is usually applied only to the top candidates, not the full product catalog. 神经重排序可以提高质量，但会增加延迟。因此，它通常仅应用于顶部候选，而不是完整的产品目录。

Interview Tip 

In an interview, mention hybrid retrieval. A strong search system should not depend only on keyword search or only on vector search. 在面试中，提到混合检索。一个强大的搜索系统不应仅依赖关键词搜索或仅依赖向量搜索。

Also mention query understanding. Search quality often improves a lot when the system correctly handles spelling mistakes, synonyms, filters, and user intent. 还要提到查询理解。当系统正确处理拼写错误、同义词、筛选和用户意图时，搜索质量通常会大幅提高。

4. Fraud Detection System4. 欺诈检测系统

A real-time fraud detection system checks whether a transaction is risky across payments, banking, e-commerce, insurance, and digital wallets.实时欺诈检测系统检查支付、银行、电商、保险和数字钱包中的交易是否有风险。

The goal is to stop fraud without blocking genuine users. If the system is too strict, good users get declined. If it is too lenient, the company loses money. So the system must make fast, careful risk decisions.目标是阻止欺诈而不阻止真实用户。如果系统过于严格，好用户会被拒绝。如果过于宽松，公司会损失金钱。因此，系统必须快速、谨慎地做出风险决策。

Problem Statement 

Design a fraud detection system that scores payment transactions in real time. For each transaction, the system should decide whether to approve it, decline it, ask for extra verification, or send it for manual review. 设计一个实时对支付交易进行评分的欺诈检测系统。对于每笔交易，系统应决定是批准、拒绝、要求额外验证还是发送人工审核。

The system should use historical behavior, real-time signals, rules, and ML predictions. 系统应使用历史行为、实时信号、规则和ML预测。

How the System Works 

The system can be broken into three steps:

1. Feature Extraction: Fetch transaction signals such as user history, card usage, merchant type, device information, IP location, and recent activity.特征提取：获取交易信号，如用户历史、卡使用情况、商户类型、设备信息、IP位置和近期活动。
2. Rules and ML Scoring: Apply rules for known risky patterns, then use an ML model to predict a fraud risk score.规则和ML评分：对已知风险模式应用规则，然后使用ML模型预测欺诈风险分数。
3. Final Decision: Combine the model score, rules, business limits, and risk policies to approve, decline, request verification, or send the transaction for manual review.最终决策：结合模型分数、规则、业务限制和风险策略，批准、拒绝、请求验证或发送交易进行人工审核。

Fraud Detection Flow 欺诈检测流程

Important Signals 

The model should use signals that capture user behavior, transaction risk, and device patterns. 模型应使用捕捉用户行为、交易风险和设备模式的信号。

Useful signals include: 

• Transaction amount and currency  交易金额和货币
• Merchant category  商户类别
• Account age  账户年龄
• Device fingerprint  设备指纹
• IP location    IP位置

These signals are useful because fraud often appears as unusual behavior. A sudden high-value transaction from a new device or country can be risky. 这些信号很有用，因为欺诈通常表现为异常行为。来自新设备或国家的高额交易可能有风险。

Model Choice 

A good baseline is a gradient boosted tree model. Fraud data is usually tabular, imbalanced, and full of useful hand-crafted features. 一个好的基线是梯度提升树模型。欺诈数据通常是表格形式、不平衡且包含有用的手工特征。

Rules should not be removed. They are useful for hard constraints and known fraud patterns. The model handles patterns that are harder to express as rules. 不应移除规则。它们对于硬约束和已知欺诈模式很有用。模型处理难以表达为规则的模式。

For advanced systems, graph-based features can be added. These can detect groups of accounts connected by shared cards, devices, addresses, or IPs. 对于高级系统，可以添加基于图的特征。这些可以检测由共享卡、设备、地址或IP连接的账户组。

Evaluation Metrics 

Offline metrics include precision, recall, PR-AUC, false positive rate, and cost-weighted loss. 离线指标包括精确率、召回率、PR-AUC、假阳性率和成本加权损失。

PR-AUC is useful because fraud data is highly imbalanced. There are usually far fewer fraud transactions than genuine transactions. PR-AUC很有用，因为欺诈数据高度不平衡。欺诈交易通常远少于真实交易。

Online metrics include fraud loss, approval rate, chargeback rate, manual review rate, and customer friction. 在线指标包括欺诈损失、批准率、拒付率、人工审核率和客户摩擦。

The system should also measure performance by segment. For example, new users, high-value transactions, and cross-border payments may behave differently. 系统还应按细分衡量性能。例如，新用户、高价值交易和跨境支付可能表现不同。

Trade-offs 

The biggest trade-off is fraud loss versus user friction. A strict model catches more fraud, but it may decline genuine users. A lenient model improves approval rate, but it may increase fraud loss. 最大的权衡是欺诈损失与用户摩擦。严格的模型能捕获更多欺诈，但可能拒绝真实用户。宽松的模型提高批准率，但可能增加欺诈损失。

There is also a latency trade-off. The system must score transactions quickly because the user is waiting. Heavy models or slow feature lookups can hurt the payment experience. 延迟权衡也存在。系统必须快速评分交易，因为用户正在等待。重型模型或缓慢的特征查找会损害支付体验。

Another challenge is delayed labels. A transaction may look safe today, but a chargeback may arrive days or weeks later. This makes training and evaluation harder. 另一个挑战是延迟标签。今天的交易可能看起来安全，但拒付可能在几天或几周后出现。这使得训练和评估更加困难。

Interview Tip 

In an interview, mention delayed labels and manual review. These are important in real fraud systems. 在面试中，提到延迟标签和人工审核。这些在真实欺诈系统中很重要。

Also mention that the decision layer should combine rules and ML. Fraud detection is not only a model prediction problem. It is a risk decision system. 还要提到决策层应结合规则和ML。欺诈检测不仅仅是模型预测问题。它是一个风险决策系统。

5. ETA Prediction System5. ETA预测系统

An ETA prediction system estimates when a driver, rider, order, or shipment will arrive. It is widely used in ride-sharing, food delivery, logistics, and mapping platforms.ETA预测系统估计司机、乘客、订单或货物何时到达。它广泛应用于拼车、食品配送、物流和地图平台。

The goal is to provide accurate and reliable arrival times despite changing traffic, route choices, GPS noise, and varying pickup or drop-off delays. A good ETA system should be accurate, stable, and fast.目标是提供准确可靠的到达时间，尽管交通、路线选择、GPS噪声以及不同的接送延迟不断变化。一个好的ETA系统应准确、稳定且快速。

Problem Statement 

Design an ETA prediction system for a ride-sharing or delivery app. Given the origin, destination, route, driver location, and current context, the system should predict the expected arrival or delivery time. 为拼车或配送应用设计一个ETA预测系统。给定起点、目的地、路线、司机位置和当前上下文，系统应预测预计到达或配送时间。

The system should support real-time updates as the trip progresses. 系统应支持行程进行中的实时更新。

How the System Works 

The system can be broken into three steps:

1. Route Generation: Map the origin and destination to the road network and generate candidate routes using distance, road type, speed limits, and traffic data.路线生成：将起点和目的地映射到道路网络，并使用距离、道路类型、限速和交通数据生成候选路线。
2. Base ETA Estimation: Use a routing engine to calculate an initial travel time estimate for the selected route.基础ETA估计：使用路线引擎计算所选路线的初始行程时间估计。
3. ML-Based Adjustment: Refine the base ETA using factors such as live traffic, weather, driver behavior, and historical delays to produce a more accurate prediction.基于ML的调整：使用实时交通、天气、驾驶员行为和历史延迟等因素优化基础ETA，以产生更准确的预测。

ETA Prediction Flow ETA预测流程

Important Signals 

The model should use route, traffic, driver, and context signals. 模型应使用路线、交通、驾驶员和上下文信号。

Useful signals include: 

• Origin and destination  起点和目的地
• Route distance  路线距离
• Road type  道路类型
• Time of day  一天中的时间
• Day of week  一周中的天

These signals help the system adjust for real-world conditions. For example, two routes with the same distance may have very different ETAs during peak traffic. 这些信号帮助系统根据真实世界条件进行调整。例如，两条距离相同的路线在高峰时段可能有非常不同的ETA。

Model Choice 

A good baseline is a gradient boosted tree model. It works well with structured features and is easy to debug. 一个好的基线是梯度提升树模型。它适用于结构化特征，并且易于调试。

The model can predict the final ETA directly, but a better design is to predict the residual error. This means the model learns how much the routing engine is usually wrong in a given context. 模型可以直接预测最终ETA，但更好的设计是预测残差误差。这意味着模型学习路线引擎在给定上下文中通常错误多少。

For advanced systems, sequence models or graph neural networks can be used. These can model traffic patterns across road networks. But they also increase complexity. 对于高级系统，可以使用序列模型或图神经网络。这些可以模拟道路网络上的交通模式。但它们也增加了复杂性。

Evaluation Metrics 

Offline metrics include MAE, RMSE, percentile error, and calibration. MAE is easy to understand because it measures average time error. 离线指标包括MAE、RMSE、百分位误差和校准。MAE易于理解，因为它测量平均时间误差。

Online metrics include late delivery rate, cancellation rate, customer complaints, and ETA stability. 在线指标包括延迟配送率、取消率、客户投诉和ETA稳定性。

ETA stability matters because users do not like estimates that keep changing every few seconds. A slightly less accurate but stable ETA can sometimes feel better than a highly volatile one. ETA稳定性很重要，因为用户不喜欢每隔几秒就变化的估计值。一个稍微不准确但稳定的ETA有时比高度波动的感觉更好。

Trade-offs 

The main trade-off is accuracy versus stability. Updating ETA too often can make the estimate more accurate, but it may also make the user experience worse. 主要的权衡是准确性与稳定性。过于频繁地更新ETA可以使估计更准确，但也可能使用户体验更差。

There is also a trade-off between model complexity and reliability. A complex traffic model may improve accuracy, but it is harder to debug when predictions go wrong. 模型复杂性和可靠性之间也存在权衡。复杂的交通模型可能提高准确性，但当预测出错时更难调试。

Latency is important too. ETA is often shown inside a live user flow, so the system must respond quickly. 延迟也很重要。ETA通常显示在实时用户流程中，因此系统必须快速响应。

Interview Tip 

In an interview, mention that ML should improve the routing engine, not replace it completely. 在面试中，提到ML应改进路线引擎，而不是完全替换它。

Also mention residual prediction. It shows practical thinking because many production ETA systems combine rule-based routing with ML correction. 还要提到残差预测。它展示了实用思维，因为许多生产ETA系统结合了基于规则的路线和ML校正。

6. Spam and Phishing Detection System6. 垃圾邮件和钓鱼检测系统

A spam and phishing detection system decides whether an email is safe, unwanted, suspicious, or harmful.垃圾邮件和钓鱼检测系统决定电子邮件是安全、不需要、可疑还是有害。

The goal is not just text classification. It must also use sender reputation, domain history, links, attachments, and authentication checks to block harmful emails without hiding important ones.目标不仅仅是文本分类。它还必须使用发件人声誉、域名历史、链接、附件和身份验证检查来阻止有害邮件，而不隐藏重要邮件。

Problem Statement 

Design a system that classifies incoming emails as safe, spam, phishing, or suspicious. 设计一个系统，将传入电子邮件分类为安全、垃圾邮件、钓鱼或可疑。

The system should detect malicious links, fake senders, harmful attachments, and suspicious message patterns. It should also learn from user feedback, such as “mark as spam” or “not spam.” 系统应检测恶意链接、虚假发件人、有害附件和可疑消息模式。它还应从用户反馈中学习，例如“标记为垃圾邮件”或“非垃圾邮件”。

How the System Works 

The system can be broken into three steps:

1. Signal Extraction: Parse the email header, sender identity, domain reputation, authentication results, URLs, attachments, subject, and body text.信号提取：解析电子邮件头部、发件人身份、域名声誉、身份验证结果、URL、附件、主题和正文文本。
2. Rules and ML Scoring: Apply rules to catch known threats, then use an ML model to score the email using text, sender, URL, and user behavior signals.规则和ML评分：应用规则捕获已知威胁，然后使用ML模型使用文本、发件人、URL和用户行为信号对电子邮件进行评分。
3. Final Decision: Send the email to inbox, spam, warning, or quarantine based on the final risk score.最终决策：根据最终风险分数将电子邮件发送到收件箱、垃圾邮件、警告或隔离区。

Spam and Phishing Detection Flow 垃圾邮件和钓鱼检测流程

Important Signals 

The system should combine content signals and security signals. Text alone is not enough. 系统应结合内容信号和安全信号。仅文本是不够的。

Useful signals include: 

• Sender domain and sender reputation  发件人域名和发件人声誉
• SPF, DKIM, and DMARC results  SPF、DKIM和DMARC结果
• Subject and body text  主题和正文文本
• URL reputation  URL声誉
• Attachment type  附件类型

These signals help the system catch different types of attacks. A phishing email may look normal in text, but it may contain a suspicious link or come from a newly created domain. 这些信号帮助系统捕获不同类型的攻击。钓鱼邮件在文本上可能看起来正常，但可能包含可疑链接或来自新创建的域名。

Model Choice 

A good baseline is a text classification model with sender and URL features. Logistic regression or gradient boosted trees can work well for the first version. 一个好的基线是带有发件人和URL特征的文本分类模型。逻辑回归或梯度提升树可以很好地用于初始版本。

A more advanced system can use transformer-based models for subject and body understanding. These models can detect subtle phishing patterns better than simple keyword rules. 更高级的系统可以使用基于Transformer的模型来理解主题和正文。这些模型可以比简单的关键词规则更好地检测微妙的钓鱼模式。

Still, the system should not depend only on the ML model. Rules, reputation checks, and authentication results are critical for security. 尽管如此，系统不应仅依赖ML模型。规则、声誉检查和身份验证结果对于安全至关重要。

Evaluation Metrics 

Offline metrics include precision, recall, F1 score, and false positive rate. 离线指标包括精确率、召回率、F1分数和假阳性率。

False positives are very important. If a safe email is moved to spam, the user may miss something important. 假阳性非常重要。如果安全邮件被移到垃圾邮件，用户可能会错过重要内容。

Online metrics include phishing catch rate, user complaint rate, spam folder correction rate, and important-email false positive rate. 在线指标包括钓鱼捕获率、用户投诉率、垃圾邮件文件夹纠正率和重要邮件假阳性率。

The system should also track new attack patterns. Phishing campaigns change quickly, so old test data may not reflect current threats. 系统还应跟踪新的攻击模式。钓鱼活动变化很快，因此旧的测试数据可能无法反映当前威胁。

Trade-offs 

The main trade-off is safety versus user trust. Aggressive filtering catches more harmful emails, but it can also block genuine messages. 主要的权衡是安全性与用户信任。激进的过滤能捕获更多有害邮件，但也可能阻止真实消息。

Conservative filtering reduces false positives, but more spam or phishing may reach the inbox. 保守的过滤减少了假阳性，但更多垃圾邮件或钓鱼可能到达收件箱。

There is also a cost trade-off. Deep content scanning and attachment sandboxing improve safety, but they add latency and infrastructure cost. 成本权衡也存在。深度内容扫描和附件沙箱提高了安全性，但增加了延迟和基础设施成本。

Interview Tip 

In an interview, do not present this as only an NLP problem. A real spam and phishing system combines text classification, sender reputation, URL intelligence, authentication checks, rules, and user feedback. 在面试中，不要将其仅呈现为NLP问题。真实的垃圾邮件和钓鱼系统结合了文本分类、发件人声誉、URL情报、身份验证检查、规则和用户反馈。

This shows that you understand how security-focused ML systems work in production. 这表明你理解以安全为中心的ML系统在生产中如何工作。

7. Visual Defect Detection System7. 视觉缺陷检测系统

A visual defect detection system identifies faulty products on manufacturing lines, warehouses, and quality control pipelines.视觉缺陷检测系统识别生产线、仓库和质量控制管道中的有缺陷产品。

The goal is to catch defects before products reach customers, reducing waste, returns, safety risks, and manual inspection effort. Since products often move continuously, the system must be accurate and fast enough for near real-time decisions.目标是在产品到达客户之前捕获缺陷，减少浪费、退货、安全风险和人工检查工作量。由于产品通常连续移动，系统必须足够准确和快速以进行近实时决策。

Problem Statement 

Design a computer vision system that detects product defects from images. 设计一个从图像中检测产品缺陷的计算机视觉系统。

The system should decide whether a product should pass, fail, or go for human review. If needed, it should also locate the defect in the image. 系统应决定产品应通过、失败还是进行人工审查。如果需要，还应定位图像中的缺陷。

How the System Works 

The system can be broken into three steps:

1. Image Capture and Quality Check: Capture product images on the production line and check for issues like poor lighting, blur, camera movement, or wrong angles.图像捕获和质量检查：在生产线上捕获产品图像，并检查光照不良、模糊、相机移动或角度错误等问题。
2. Vision Model Inference: Preprocess the image and use a vision model to classify defects, detect defect boxes, or segment defect regions.视觉模型推理：预处理图像，并使用视觉模型对缺陷进行分类、检测缺陷框或分割缺陷区域。
3. Final Decision: Mark the product as pass or fail if confidence is high, or send uncertain cases to human reviewers for feedback and future training data.最终决策：如果置信度高，将产品标记为通过或失败，或将不确定的情况发送给人工审查员以获取反馈和未来的训练数据。

Visual Defect Detection Flow 视觉缺陷检测流程

Important Signals 

The image is the main input. But metadata can also help the system understand the production context. 图像是主要输入。但元数据也可以帮助系统理解生产上下文。

Useful signals include: 

• Product type  产品类型
• Camera ID  相机ID
• Production line  生产线
• Batch ID  批次ID
• Timestamp  时间戳

These signals are useful because defects may depend on a specific machine, batch, material, or production condition. 这些信号很有用，因为缺陷可能取决于特定的机器、批次、材料或生产条件。

Model Choice 

The model choice depends on the output needed. 模型选择取决于所需的输出。

If the system only needs pass or fail, image classification is enough. Also it needs to show where the defect is, object detection is better. If it needs exact defect boundaries, segmentation is the better choice. 如果系统只需要判断合格或不合格，图像分类就足够了。如果还需要显示缺陷位置，目标检测更合适。如果需要精确的缺陷边界，分割是更好的选择。

A good baseline is transfer learning with a pretrained CNN or vision transformer. This is practical because defect datasets are often small. 一个好的基线是使用预训练 CNN 或视觉 Transformer 进行迁移学习。这很实用，因为缺陷数据集通常很小。

For object detection, models like YOLO-style detectors or Faster R-CNN can be used. For segmentation, a U-Net-style model is a strong baseline. 对于目标检测，可以使用 YOLO 风格的检测器或 Faster R-CNN 等模型。对于分割，U-Net 风格的模型是一个强大的基线。

Evaluation Metrics 

Offline metrics include precision, recall, F1 score, IoU, and defect-level recall. 离线指标包括精确率、召回率、F1 分数、IoU 以及缺陷级召回率。

Recall is very important when missing a defect is costly. Precision is important when false rejects create waste. 当漏检缺陷代价高昂时，召回率非常重要。当误判拒收造成浪费时，精确率很重要。

Online metrics include false reject rate, false accept rate, review rate, inference latency, and production downtime. 在线指标包括误拒率、误收率、复检率、推理延迟和生产停机时间。

The system should also track model performance by product type, camera, and production line. This helps detect camera drift or process issues. 系统还应按产品类型、摄像头和生产线跟踪模型性能。这有助于检测摄像头漂移或工艺问题。

Trade-offs 

The main trade-off is recall versus waste. High recall catches more defects, but it may reject good products. High precision reduces waste, but it may miss some defects. 主要权衡是召回率与浪费之间的关系。高召回率能发现更多缺陷，但也可能拒收良品。高精确率能减少浪费，但也可能漏掉一些缺陷。

There is also a trade-off between edge inference and cloud inference. Edge inference is faster and works even with weak network connectivity. Cloud inference is easier to update and monitor, but it adds latency and depends on network reliability. 边缘推理与云端推理之间也存在权衡。边缘推理速度更快，即使在网络连接较弱的情况下也能工作。云端推理更易于更新和监控，但会增加延迟并依赖网络可靠性。

Another challenge is data imbalance. Defects are often rare. The system may see thousands of normal products for every defective one. 另一个挑战是数据不平衡。缺陷通常很罕见。系统可能每遇到一个缺陷产品就会看到数千个正常产品。

Interview Tip 

In an interview, mention image quality monitoring. Many real vision systems fail because of lighting changes, camera shifts, blur, or dirty lenses. 在面试中，要提到图像质量监控。许多真实的视觉系统因光照变化、摄像头偏移、模糊或镜头脏污而失效。

Also mention human review. It helps handle uncertain cases and creates new labeled data for retraining. 还要提到人工审核。它有助于处理不确定案例，并为重新训练创建新的标注数据。

8. Demand Forecasting System8. 需求预测系统

A demand forecasting system predicts future product demand for retail, e-commerce, manufacturing, and supply chain operations.需求预测系统预测零售、电子商务、制造和供应链运营中的未来产品需求。

The goal is to maintain the right inventory levels. Underestimating demand can lead to stockouts, while overestimating it can result in excess inventory and higher costs. A good forecasting system should be accurate, stable, and useful for planning.目标是维持适当的库存水平。低估需求可能导致缺货，而高估需求则可能导致库存过剩和成本增加。一个好的预测系统应该准确、稳定，并对规划有用。

Problem Statement 

Design a demand forecasting system for products across stores, regions, or warehouses. 为跨门店、区域或仓库的产品设计一个需求预测系统。

The system should predict future demand for each product and time period. It should also handle holidays, promotions, seasonality, new products, and stockouts. 系统应预测每个产品和时间段的未来需求。它还应该处理节假日、促销活动、季节性、新产品和缺货情况。

How the System Works 

The system can be broken into three steps:

1. Data Preparation: Collect historical sales, inventory, pricing, promotions, holidays, product metadata, and store data, then clean missing values, stockouts, returns, and unusual spikes.数据准备：收集历史销售、库存、定价、促销、节假日、产品元数据和门店数据，然后清洗缺失值、缺货、退货和异常峰值。
2. Feature Engineering and Forecasting: Create time-based features such as day of week, seasonality, holidays, promotions, and recent sales trends, then predict future demand.特征工程与预测：创建基于时间的特征，如星期几、季节性、节假日、促销和近期销售趋势，然后预测未来需求。
3. Planning and Feedback: Send forecasts to inventory or replenishment systems, compare predictions with actual sales, and use the feedback for backtesting and retraining.规划与反馈：将预测发送给库存或补货系统，将预测与实际销售进行比较，并利用反馈进行回溯测试和重新训练。

Demand Forecasting Flow 需求预测流程

Important Signals 

The model should use sales, product, pricing, and calendar signals. 模型应使用销售、产品、定价和日历信号。

Useful signals include: 

• Historical sales  历史销售
• Product category  
• Store or region  门店或区域
• Price and discount  
• Promotion status  促销状态

Stockout information is very important. If a product was out of stock, observed sales do not show true demand. The user may have wanted to buy the product, but could not. 缺货信息非常重要。如果产品缺货，观察到的销售额并不能反映真实需求。用户可能想买但买不到。

Model Choice 

A simple baseline can use moving averages or exponential smoothing. These are easy to explain and work well for stable products. 一个简单的基线可以使用移动平均或指数平滑。它们易于解释，并且对稳定的产品效果很好。

A stronger system can use gradient boosted trees with time-based features. This works well when the model needs to combine sales history with price, promotions, and product metadata. 更强的系统可以使用带有时间特征的梯度提升树。当模型需要结合销售历史与价格、促销和产品元数据时，这效果很好。

For large-scale forecasting, global time-series models can be used. These models learn patterns across many products and stores instead of training one separate model for each item. 对于大规模预测，可以使用全局时间序列模型。这些模型学习跨多个产品和门店的模式，而不是为每个商品训练一个单独的模型。

Probabilistic forecasting is also useful. Instead of giving one number, the system can predict a range. This helps planners prepare for uncertainty. 概率预测也很有用。系统不是给出一个数字，而是可以预测一个区间。这有助于计划人员为不确定性做好准备。

Evaluation Metrics 

Offline metrics include MAE, RMSE, MAPE, WAPE, and pinball loss for probabilistic forecasts. 离线指标包括 MAE、RMSE、MAPE、WAPE 以及概率预测的 pinball loss。

WAPE is often useful in business settings because it measures error relative to total demand. WAPE 在业务场景中通常很有用，因为它衡量的是相对于总需求的误差。

Business metrics include stockout rate, inventory holding cost, waste, service level, and lost sales. 业务指标包括缺货率、库存持有成本、浪费、服务水平和销售损失。

The model should also be evaluated across segments. Fast-moving products, slow-moving products, seasonal products, and new products may behave differently.模型还应按细分品类进行评估。快消品、慢销品、季节性产品和新品的表现可能不同。

Trade-offs 

The main trade-off is granularity versus noise. Forecasting at store-product-day level is useful, but it can be noisy. Forecasting at category-region-week level is more stable, but less detailed. 主要权衡是粒度与噪声。按门店-商品-日级别预测很有用，但可能存在噪声。按品类-区域-周级别预测更稳定，但细节较少。

There is also a trade-off between accuracy and explainability. Simple models are easier for planners to trust. Complex models may be more accurate, but harder to explain. 准确性与可解释性之间也存在权衡。简单的模型更容易让计划人员信任。复杂的模型可能更准确，但更难解释。

Another challenge is new products. They do not have enough history. The system can use similar products, category patterns, or launch plans to create a cold-start forecast. 另一个挑战是新产品。它们没有足够的历史数据。系统可以利用相似产品、品类模式或上市计划来创建冷启动预测。

Interview Tip 

In an interview, mention stockout bias. Sales are not always equal to demand. If inventory was unavailable, the data is censored. 在面试中，要提到缺货偏差。销售额并不总是等于需求。如果库存不可用，数据就是被删失的。

Also mention that business metrics matter. A forecasting model is useful only if it improves inventory decisions. 还要提到业务指标很重要。预测模型只有在能改善库存决策时才有用。

9. Dynamic Pricing System9. 动态定价系统

A dynamic pricing system recommends prices or discounts based on demand, supply, inventory, and business goals.动态定价系统根据需求、供应、库存和业务目标推荐价格或折扣。

The goal is to balance revenue, conversion, margin, inventory, and customer trust. Since pricing affects user experience, fairness, brand value, and legal risk, the system needs strong guardrails. 目标是平衡收入、转化率、利润率、库存和客户信任。由于定价会影响用户体验、公平性、品牌价值和法律风险，系统需要强有力的护栏。

Problem Statement 

Design a system that dynamically recommends prices or discounts for products or services. 设计一个能为产品或服务动态推荐价格或折扣的系统。

The system should use demand, supply, inventory, competitor prices, customer behavior, and business constraints. It should also include guardrails so that prices do not change in unsafe or unfair ways. 系统应利用需求、供应、库存、竞争对手价格、客户行为和业务约束。它还应包含护栏，以防止价格以不安全或不公平的方式变动。

How the System Works 

The system can be broken into three steps:

1. Signal Collection: Collect demand, stock levels, competitor prices, historical conversions, seasonality, and margin data.信号收集：收集需求、库存水平、竞争对手价格、历史转化率、季节性和利润率数据。
2. Price Estimation: Estimate demand at different price points and generate possible prices or discounts.价格估算：估算不同价格点的需求，并生成可能的价格或折扣。
3. Guardrails and Feedback: Apply business, legal, fairness, and margin guardrails, show the final price, and log user actions for future training.护栏与反馈：应用业务、法律、公平性和利润率护栏，展示最终价格，并记录用户行为以供未来训练。

Dynamic Pricing Flow 动态定价流程

Important Signals 

The model should use signals that explain demand and willingness to buy. 模型应使用能够解释需求和购买意愿的信号。

Useful signals include: 

• Current demand  当前需求
• Inventory level  库存水平
• Competitor price  竞争对手价格
• Historical conversion rate  历史转化率
• Price and discount history价格和折扣历史

These signals help the system understand when a price change may help. For example, if inventory is high and demand is low, a discount may improve sell-through. If demand is already high and inventory is limited, a discount may not be needed. 这些信号帮助系统理解何时价格变动可能有效。例如，如果库存高而需求低，折扣可能促进售罄。如果需求已经很高且库存有限，则可能不需要折扣。

Model Choice 

A good baseline is a supervised model that predicts conversion or demand for a given price. This is easier to build and safer than directly letting a model choose prices. 一个好的基线是有监督模型，预测给定价格下的转化率或需求。这比直接让模型选择价格更容易构建，也更安全。

Once the system is stable, contextual bandits can be used for controlled exploration. They help the system learn which price works best in different contexts. 系统稳定后，可以使用上下文赌博机进行可控探索。它们帮助系统学习在不同情境下哪种价格效果最好。

Full reinforcement learning should not be the first choice. It needs strong simulation, enough data, and strict safety controls. Without these, it can make risky pricing decisions. 完整的强化学习不应是首选。它需要强大的模拟、足够的数据和严格的安全控制。没有这些，它可能会做出有风险的定价决策。

Evaluation Metrics 

Offline metrics include demand prediction error, conversion prediction error, and policy simulation performance. 离线指标包括需求预测误差、转化率预测误差和策略模拟表现。

Online metrics include revenue, margin, conversion rate, inventory sell-through, customer complaints, and price volatility. 在线指标包括收入、利润率、转化率、库存售罄率、客户投诉和价格波动。

It is also useful to track fairness and trust-related metrics. If users feel prices are random or unfair, the system may hurt long-term loyalty. 跟踪公平性和信任相关指标也很有用。如果用户觉得价格是随机或不公平的，系统可能会损害长期忠诚度。

Trade-offs 

The main trade-off is short-term revenue versus long-term trust. A high price may increase margin now, but it can reduce repeat purchases if users feel treated unfairly. 主要权衡是短期收入与长期信任。高价可能会立即提高利润率，但如果用户感觉受到不公平对待，可能会减少复购。

There is also a trade-off between exploration and risk. The system needs to test prices to learn, but too much experimentation can harm user experience. 探索与风险之间也存在权衡。系统需要测试价格以学习，但过多的实验可能损害用户体验。

Another trade-off is automation versus control. Fully automated pricing can react quickly, but business teams often need guardrails and approval workflows. 另一个权衡是自动化与控制。完全自动化定价可以快速反应，但业务团队通常需要护栏和审批流程。

Interview Tip 

In an interview, always mention guardrails. Dynamic pricing is not just a prediction problem. It is a decision system with business, legal, and fairness constraints. 在面试中，一定要提到护栏。动态定价不仅仅是一个预测问题。它是一个带有业务、法律和公平性约束的决策系统。

Also mention that the model should start by predicting demand or conversion before moving toward automated price optimization. 还要提到，模型应先从预测需求或转化率开始，然后再走向自动化价格优化。

10. RAG-Based Customer Support Assistant10. 基于 RAG 的客户支持助手

A RAG-based customer support assistant answers user questions using company documents across help centers, SaaS products, banking apps, and e-commerce platforms.基于 RAG 的客户支持助手利用帮助中心、SaaS 产品、银行应用和电商平台的公司文档来回答用户问题。

The goal is to provide accurate, grounded answers rather than relying solely on the LLM’s memory. By retrieving relevant documents before generating a response, the system becomes more reliable and easier to audit.目标是提供准确、有依据的答复，而非仅依赖 LLM 的记忆。通过在生成回复前检索相关文档，系统变得更加可靠且更易于审计。

Problem Statement 

Design a customer support assistant that can answer user questions using product docs, FAQs, policies, manuals, and past support content. 设计一个客户支持助手，能够利用产品文档、常见问题、政策、手册和以往支持内容来回答用户问题。

The system should retrieve relevant information, generate grounded answers, cite sources, and escalate uncertain cases to a human agent. 系统应检索相关信息，生成有依据的答复，引用来源，并将不确定的案例升级给人工客服。

How the System Works 

The system can be broken into three steps:

1. Document Ingestion: Collect, clean, chunk, embed, and store documents with metadata such as source, update date, product name, and access permissions.文档摄取：收集、清洗、分块、嵌入并存储文档，附带来源、更新日期、产品名称和访问权限等元数据。
2. Query and Retrieval: Check access rules, clean the user query, and retrieve relevant chunks using hybrid search with both keyword and vector retrieval.查询与检索：检查访问规则，清洗用户查询，并使用结合关键词和向量检索的混合搜索来检索相关片段。
3. Answer Generation: Pass retrieved chunks to the LLM, generate an answer from the provided context, and ask for clarification or escalate if the context is weak.答案生成：将检索到的片段传给 LLM，基于提供的上下文生成答案，如果上下文不足，则请求澄清或进行升级。

RAG Support Assistant Flow RAG 支持助手流程

Important Signals

The system should use signals from the query, documents, and user context. 系统应使用来自查询、文档和用户上下文的信号。

Useful signals include: 

• User question  用户问题
• Product or account type  产品或账户类型
• Document title  文档标题
• Document freshness  文档新鲜度
• Chunk relevance score  片段相关性分数

Freshness is important. A support assistant can give wrong answers if it retrieves outdated policy documents. 新鲜度很重要。如果支持助手检索到过时的政策文档，可能会给出错误答案。

Model Choice 

The system needs three main model components. 系统需要三个主要的模型组件。

• Embedding model: It converts document chunks and user queries into vectors. 嵌入模型：它将文档片段和用户查询转化为向量。
• Reranker: It improves the order of retrieved chunks before they are sent to the LLM. 重排序器：它在将检索到的片段发送给 LLM 之前改进片段的顺序。
• LLM: It generates the final answer from the retrieved context. LLM：它基于检索到的上下文生成最终答案。

A simple baseline can use keyword search plus an LLM. A stronger system can add vector search, reranking, better chunking, and grounding checks. 一个简单的基线可以使用关键词搜索加 LLM。更强的系统可以添加向量搜索、重排序、更好的分块和依据检查。

Evaluation Metrics 

Evaluation should cover both retrieval and generation. 评估应涵盖检索和生成两个方面。

• Retrieval metrics include recall@K, MRR, and hit rate. These show whether the right document appears in the retrieved results. 检索指标包括 recall@K、MRR 和命中率。这些指标显示正确的文档是否出现在检索结果中。
• Generation metrics include answer correctness, groundedness, citation accuracy, hallucination rate, and refusal quality. 生成指标包括答案正确性、依据充分性、引用准确性、幻觉率和拒答质量。
• Product metrics include resolution rate, escalation rate, average handling time, customer satisfaction, and repeat contact rate. 产品指标包括解决率、升级率、平均处理时间、客户满意度和重复联系率。

Trade-offs 

The main trade-off is answer quality versus cost. More context can improve the answer, but it increases token usage and latency. 主要权衡是答案质量与成本。更多的上下文可以改善答案，但会增加 token 用量和延迟。

There is also a trade-off between strict grounding and helpfulness. If the system is too strict, it may refuse too often. If it is too loose, it may hallucinate. 在严格的依据约束和有帮助性之间也存在权衡。如果系统过于严格，可能会过于频繁地拒绝回答。如果过于松散，可能会产生幻觉。

Another challenge is access control. The assistant should only retrieve and answer from documents the user is allowed to see. 另一个挑战是访问控制。助手只应检索和回答用户有权查看的文档中的内容。

Interview Tip 

In an interview, say clearly that retrieval quality is often more important than the LLM itself. If the wrong chunks are retrieved, even a strong LLM will produce a weak answer. 在面试中，要明确说明检索质量通常比 LLM 本身更重要。如果检索到错误的片段，即使是强大的 LLM 也会给出糟糕的答复。

Also mention source citations, access control, document freshness, and human escalation. These are key parts of a production RAG system. 还要提到来源引用、访问控制、文档新鲜度和人工升级。这些是生产级 RAG 系统的关键部分。

Final Interview Checklist最终面试检查清单

Before you end any ML system design answer, quickly check whether you covered the full system. This helps you avoid giving a model-only answer. 在结束任何机器学习系统设计回答之前，快速检查是否涵盖了完整的系统。这有助于避免只给出模型相关的答案。

1. Define the Goal: Explain what decision the system makes and why it matters.定义目标：解释系统做出什么决策以及为什么重要。
2. Understand the Data: Describe data sources, label creation, and label availability.理解数据：描述数据来源、标签创建和标签可用性。
3. Choose the Model: Start with a simple baseline and discuss possible improvements.选择模型：从简单的基线开始，并讨论可能的改进。
4. Design the Serving Flow: Explain feature lookup, inference, and how predictions are used.设计服务流程：解释特征查找、推理以及预测结果如何使用。
5. Handle Production Concerns: Cover business rules, latency, logging, and fallback mechanisms.处理生产问题：涵盖业务规则、延迟、日志记录和回退机制。

A short checklist can help you structure the answer: 一个简短的检查清单可以帮助你组织回答：

• Product goal  产品目标
• Functional and non-functional requirements  功能性和非功能性需求
• Data sources and labels  数据来源和标签
• Feature engineering  特征工程
• Baseline model  基线模型

This checklist is useful for every problem. It works for ranking, classification, forecasting, computer vision, pricing, and RAG systems. 这个检查清单对每个问题都有用。它适用于排序、分类、预测、计算机视觉、定价和 RAG 系统。

The main idea is simple. Do not stop after choosing a model. Show how the model fits into a complete production system.  主要思路很简单。不要在选择模型后就停下来。要展示模型如何融入完整的生产系统。