PDF Summary:Machine Learning System Design Interview, by Ali Aminian

In Machine Learning System Design Interview, Ali Aminian provides a framework for systematically designing robust end-to-end machine learning systems. He covers formulating the ML task, engineering relevant features, selecting the right model architecture, and evaluating model performance. You'll learn how to tackle challenges like handling imbalanced data, detecting distribution shifts, and leveraging techniques like online learning.

For hands-on guidance, Aminian walks through designing ML systems for diverse use cases like visual search, content moderation, recommendation engines, and ad engagement prediction. He highlights best practices for scalable deployment, monitoring, infrastructure set-up, and system maintenance. Whether you're preparing for an interview or building an ML system from scratch, this guide gives you a structured approach.

(continued)...

• Develop a "metric review ritual" where you regularly assess the relevance of your digital measurements. Set aside a time each month to review your chosen metrics and evaluate whether they still align with your evolving company goals. During this ritual, ask yourself questions like, "Is this metric still providing insight into our objectives?" or "Have our goals changed in a way that requires different metrics?" This practice ensures that your measurement strategy stays current and aligned with your business direction.

Deploy and Serve the Machine Learning Model

Aminian encourages thinking about the overall architecture and how various elements will interact to serve requests.

Improve the Serving Process for Low Latency

Aminian emphasizes the need to optimize for low latency, particularly when real-time forecasts are required. This includes:

• Cloud vs. on-device deployment: Should the model be deployed in the cloud for easier management, or on-device for better privacy and lower latency?

• Model compression: Can you apply techniques like knowledge distillation, pruning, or quantization to reduce model size and speed up inference?

• Batch vs. online prediction: Can you pre-compute predictions for certain tasks, or should you generate predictions online as requests arrive?

• Efficient prediction process: How will the different stages of the pipeline for predictions interact? Can you leverage techniques like caching or approximate nearest-neighbor search to accelerate the process?

Practical Tips

• Opt for tools and services that prioritize speed in their features. When choosing apps or software for everyday use, like web browsers or note-taking apps, look for options that boast fast load times and quick data retrieval. Test different options and monitor which ones help you complete tasks more efficiently, thus lowering your "latency" in day-to-day activities.
• Experiment with different browser settings or extensions that prioritize privacy to understand the trade-offs between on-device and cloud-based data processing.
• You can start by using lightweight apps on your smartphone to familiarize yourself with the concept of model compression. For instance, download an app that optimizes photo storage by compressing images without noticeable loss of quality. Observe how the app reduces file sizes and maintains functionality, which mirrors the principles of model compression in a more accessible context.
• Experiment with a simple email sorting tool that uses batch processing. Set up email filters to sort your incoming messages into categories once a day. This mimics batch processing by accumulating data and processing it in a single batch, helping you understand the efficiency of handling tasks in bulk. Notice if this once-a-day sorting helps you manage your inbox more effectively compared to real-time sorting.
• Enhance your online shopping experience by using browser extensions that compare prices and products in real-time. These tools work similarly to approximate nearest-neighbor search algorithms by quickly scanning through vast amounts of data to find the best deals that match your search criteria, saving you both time and money.

Regularly Check and Retrain the Entire System, Including the Model

Aminian emphasizes designing the architecture for continuous monitoring and retraining:

• Monitoring: Track operation-related metrics like latency, throughput, resource utilization, as well as machine learning metrics like distribution shifts, model accuracy, and output drifts.

• Retraining frequency: How often will you refresh the model? Will you use continuous learning, scheduled retraining, or trigger retraining based on performance degradation?

Practical Tips

• Engage with a community of practice online where members share their experiences with continuous self-improvement. Platforms like Reddit or specialized forums offer spaces where you can post your progress, get feedback, and learn from others' experiences. This social accountability can help you refine your personal monitoring and retraining strategies by incorporating what has worked for others.
• Conduct a monthly digital cleanup on your devices to optimize their performance. Set a reminder to clear cache, delete unnecessary files, and uninstall unused apps. This can help maintain or improve the device's throughput, ensuring that it runs efficiently. Keep a log of how much storage space you free up each time and note any improvements in device speed or battery life.
• Set up a free account on a data visualization platform like Tableau Public to visualize any data you regularly collect, such as monthly expenses or social media engagement. By creating visual representations, you can more easily spot trends and anomalies, akin to output drifts in machine learning. This hands-on approach will give you a feel for how data can change over time and how visual cues can help identify when something needs attention.
• Set calendar reminders to evaluate the relevance of your digital assistants. Digital assistants like Siri, Alexa, or Google Assistant learn from interactions to better serve you. Every few months, set a reminder to consciously engage with your assistant using a variety of requests. If the responses seem off or not as helpful, it might indicate the need for retraining, which can often be done through the assistant's settings or by providing more varied input to help it learn.
• Establish a learning buddy system with a friend or colleague to encourage continuous learning. Pair up with someone interested in similar skills or fields, and commit to sharing one new insight or learning resource each week. This mutual exchange keeps learning on your agenda and can introduce you to new areas for development you might not have considered.

Ensure Efficient Infrastructure and Scaling

Aminian advises considering infrastructure and scalability aspects:

• Storage: How will you keep the data, model, and embeddings? Which data repository will work?

• Compute: What types of machines (CPU, GPU) will you require for training and deploying?

Practical Tips

• Experiment with data organization apps like Trello or Notion to manage and store information related to personal projects or hobbies. You could use Trello to track progress on different tasks with its card system or Notion to create a database of notes, tasks, and files that are easily retrievable and shareable with others.
• Join or create a local machine learning interest group to share resources and access to necessary hardware. By pooling resources, members can share access to high-performance machines, or even negotiate group discounts for cloud services, making it more affordable and practical to work on machine learning projects.

Ongoing Maintenance of the Machine Learning System

Aminian highlights the need to maintain the machine learning system once it's implemented.

Handle Distribution Shifts and Model Drift

Aminian emphasizes designing your solution to be resilient to:

• Data distribution shifts: The data the model sees in production may differ from the training data.

• Model drift: Model performance may deteriorate gradually.

Context

• Ensemble methods, which combine predictions from multiple models, can help mitigate the effects of distribution shifts by leveraging diverse model perspectives.
• Human-in-the-loop systems can be beneficial, where human experts periodically review model outputs to ensure they remain accurate and relevant.
• Changes in how data is collected, such as new sensors or updated software, can alter the characteristics of the data, affecting its distribution.
• Model drift can lead to poor decision-making if not addressed, as the model's outputs may no longer reflect the current reality, affecting business operations and outcomes.

Leverage Techniques Like Online/Continual Learning

Models that continuously learn can adjust to fresh data. Aminian advises considering these factors:

• Online training: Should the model be updated with each new data point, or in batches?

• Personalization: Does the model need to be personalized for each user?

• Retraining triggers: When should the model be retrained (e.g., on a schedule, or based on performance thresholds)?

Practical Tips

• Use apps or tools that incorporate machine learning to get personalized recommendations and then actively apply those suggestions to see how they fit into your life. For example, if a fitness app suggests a new workout based on your past performance and preferences, try it out and track your progress to see if the adaptive model effectively enhances your fitness routine.
• Experiment with a budgeting app that allows for real-time expense tracking versus one that aggregates data weekly or monthly. This will help you determine if immediate updates keep you more accountable and aware of your spending habits or if batch updates reduce the stress of constant monitoring and still keep you on track financially.
• Create a custom reading experience by choosing books based on your reactions to different genres and authors. After finishing a book, write a brief review for yourself, noting what you liked, disliked, and any emotions it evoked. Use this feedback to guide your future selections, ensuring that your reading list is tailored to your personal tastes and emotional responses.
• Create a feedback loop with friends or family where you regularly discuss your progress on various projects or habits. Agree on signs that indicate when it's time to change your approach, such as feeling consistently demotivated or failing to meet your mini-goals, and use these discussions to brainstorm new strategies.

ML Systems for Visual/Multimedia, Recommendation, Predicting/Classifying, and Moderating Content

Aminian showcases the application of the overarching framework to specific ML systems. He provides detailed solutions for frequently asked questions about designing machine learning systems for interviews, walking you through the thought process and highlighting best practices for designing practical and scalable systems.

Visual Search Program

Aminian tackles the design of a system for searching images, where users can find images visually resembling a given query image.

Learn Image Representations Using Contrastive Training

Aminian suggests using representation learning through neural networks, specifically CNN-based architectures or Transformers, to learn image embeddings. He advocates for contrastive training, where the model is taught to distinguish between similar and dissimilar images.

Aminian discusses various strategies for constructing training data: using human judgment, inferring similarity from user clicks, or creating altered copies of the image being queried to represent visually comparable images. He analyzes the compromises of each approach.

Practical Tips

• You can explore image recognition by using online platforms that allow you to train a simple neural network with your own images. Websites like Teachable Machine by Google let you upload images and train a model without any coding. You can start by creating a dataset of various objects around your house, label them, and then see how well the model you trained can classify new images.
• You can enhance your visual learning by creating a personal image sorting game using everyday objects. Start by gathering items that are visually similar but have distinct differences, like various types of leaves, coins from different countries, or patterns on fabrics. Challenge yourself to sort them based on specific characteristics, such as shape, color, or texture. This activity will train your brain to notice subtle differences and improve your ability to distinguish between similar visuals, which is a fundamental skill in fields like art, design, and even quality control.
• You can enhance your online marketplace listings by incorporating customer feedback to show relevance. After selling an item, ask buyers to describe what they liked about the product and how they used it. Use these descriptions to tag similar items in your inventory, making it easier for future customers to find what they're looking for based on real user experiences.
• Experiment with crowdsourcing to understand the trade-offs of scale versus expertise in data labeling. Use a platform like Amazon Mechanical Turk to outsource a small data labeling task, such as categorizing photos. Set up two different tasks: one with a lower pay rate and less stringent requirements for workers, and another with a higher pay rate and requirements for specific expertise. Compare the results in terms of accuracy, time to completion, and overall cost to see how the level of expertise and compensation affects the quality of your training data.

Employ Efficient Search Techniques Using Approximations of Nearest Neighbors

To efficiently retrieve images with comparable embeddings, Aminian advises using methods that approximate nearest-neighbor searches. He outlines common methods like tree, locality-sensitive hashing (LSH), and clustering-based ANN. He recommends becoming familiar with popular ANN libraries like Faiss and Google's ScaNN.

Practical Tips

• Enhance your hobby of birdwatching or nature photography by using an app with image comparison capabilities to identify species or find visually similar scenes. Snap a photo of a bird or landscape, and the app can help you find and learn about similar species or places. This not only enriches your experience but also educates you on the diversity of wildlife or landscapes you encounter.
• You can explore the efficiency of different search methods by organizing your personal digital files using various sorting principles. Start by categorizing your files using a simple folder structure (tree-based approach), then try tagging them with keywords (similar to hashing), and finally, group them by file type or content (akin to clustering). Observe which method helps you retrieve files the quickest and consider applying this insight to other areas where you need to organize information.
• You can start a study group with friends or colleagues to explore and practice using ANN libraries like Faiss and ScaNN. By pooling your collective knowledge and resources, you can create a supportive learning environment. For example, each member could take turns presenting a feature or concept from the libraries, followed by a group discussion or a hands-on coding session to solidify understanding.

Address New Item and User Cold Start Issues

Aminian recognizes the challenge of recommending similar pictures for newly uploaded photos that lack interaction data. He suggests tackling this challenge with heuristics, like using the features of an image that's close in geographic location as a proxy until there's adequate interaction data for the new image. Similar strategies can apply to users with limited interaction history.

Practical Tips

• Start a collaborative photo album with a group of friends or family where each person adds photos based on themes or prompts. By observing the types of photos others add in response to different prompts, you can gain insights into what kinds of images are considered similar or relevant by different individuals, which can inform your own photo uploads and recommendations to others.
• Experiment with personalized follow-ups after brief encounters. If you've had a short interaction with someone, send them a message or an email referencing something specific from your conversation. For instance, if they mentioned a book they enjoy, you could follow up with a link to a related article or podcast. This shows attentiveness and can deepen the connection, making the most of the limited interaction history you have.

Harmful Content Detection System

Aminian guides you through designing an automated mechanism to identify posts with damaging content, like hate speech, nudity, or violence.

Fuse Multimodal Data Early

He emphasizes the need to handle multimodal data, as posts can contain written content, photos, clips, and user reactions. Aminian advocates for early fusion, where the modalities are combined into a single feature vector before they're input into the model. This approach enables the model to identify complex relationships between modalities and detect harmful content that may only emerge from the combination of different elements.

Practical Tips

• Experiment with interactive content elements like polls or quizzes that incorporate multimodal data. For example, create a poll with an image option and a text option, then analyze which one gets more engagement. This hands-on approach will give you direct insight into how your audience interacts with different types of media within the same post.
• Use the principle of early fusion to organize and understand complex information in your personal life. For example, if you're planning a trip, collect data from different modalities such as maps, weather forecasts, travel blogs, and local regulations. Integrate this information into a cohesive travel plan or itinerary, which acts as your 'single feature vector', to ensure a well-rounded understanding and preparation for your journey.
• Improve your ability to detect nuanced relationships by playing "connection games" with friends or family. During a casual gathering, choose a random object and challenge each other to find the most creative and indirect connections to another, seemingly unrelated object or concept. For example, link a teacup to a concept like gravity. This game encourages lateral thinking and can sharpen your capacity to see complex relationships in everyday life, similar to how models discern connections between different modalities.
• Use a buddy system to gain perspective on content that might be harmful when combined. Pair up with a friend or family member and share your experiences with different types of content. Discuss how certain books, shows, articles, or online interactions make you feel, especially when they're combined. For instance, if you both read a stressful news article and then watch a dystopian movie, talk about whether this combination exacerbates feelings of anxiety or hopelessness. This can help you become more aware of how content interacts and affects your well-being.

Use Multiple Tasks to Anticipate Dangerous Material

Aminian frames the task as a multi-task learning problem, where the model learns to anticipate different categories of harm simultaneously. He recommends employing a neural network with multiple tasks that shares layers for efficient learning.

Practical Tips

• Apply the concept of multi-task learning to your fitness routine by combining exercises that work on different but complementary muscle groups. For instance, create a workout that includes squats (for lower body strength) and overhead presses (for upper body strength) in a circuit. This not only saves time but also promotes balanced muscle development and can improve overall functional fitness.
• Develop your conflict resolution skills by role-playing scenarios with friends or family where you must address multiple concerns at once. For instance, if you're planning a family reunion, simulate a discussion where you have to balance the budget, individual preferences, and logistical constraints. This exercise will help you become better at recognizing and managing different interests and finding solutions that address multiple issues simultaneously.
• Start a discussion group with friends or colleagues interested in technology to brainstorm potential multi-task applications. Each member could come up with a scenario where a multi-task neural network might be useful, such as a smartphone app that can both recognize songs and translate spoken language. This collective brainstorming can lead to innovative ideas that might one day be developed into real-world applications.
• Start a hobby project that requires you to learn multiple skills at once, like building a simple piece of furniture, which involves woodworking, design, and possibly sewing for cushions. This hands-on experience will give you a practical understanding of how developing related skills simultaneously can be more efficient, akin to the shared learning process in neural networks.

Use Customer Feedback to Enhance Model Performance

Aminian advises incorporating user input, like reports and appeals, to enhance model performance. For example, harmful posts that are wrongly identified can help fine-tune the model or identify specific categories of harm where the system is underperforming.

Practical Tips

• You can improve your personal blog by asking readers to submit topic suggestions and feedback through a simple online survey. Create a survey using free tools like Google Forms or SurveyMonkey, and share the link at the end of your blog posts. Encourage readers to tell you what they want to read about next or how they think you could improve your content. This direct user input can guide your writing and make your blog more engaging and tailored to your audience's interests.
• Use social media to crowdsource feedback for personal projects or hobbies by posting about your work and asking for constructive criticism. If you're into photography, for instance, post a photo on Instagram and ask your followers to comment on what they like and how you could enhance your technique or composition.
• Create a personal blog or vlog where you discuss and analyze the effectiveness of content moderation on different platforms. Use your experiences to highlight specific cases where content was either wrongfully flagged or missed by automated systems. This not only helps you understand the complexities of content moderation but also educates your audience on the importance of context and human judgment in these systems.
• Start a virtual suggestion box where users can anonymously submit their concerns or problems with your service. Use a platform like Trello or Slack, where you can set up a dedicated channel or board for feedback. Encourage users to post their issues or ideas for improvements, and make it a habit to review these suggestions regularly. If you manage a community website, for instance, this could help you identify recurring complaints about navigation or content accessibility that you weren't aware of.

Video Recommendation System

Aminian tackles the design of a system for recommending videos to personalize what appears on a user's homepage.

Combine Approaches: Content-Based and Collaborative Filters

Aminian proposes employing a hybrid approach that combines content-based and collaborative filtering techniques. He recommends an approach with two phases, where a collaborative filtering model acts as a candidate generator, efficiently narrowing down the selection from billions of videos to thousands based on user-item interactions. A content-focused model then scores and ranks these candidate videos according to user characteristics and video attributes for higher accuracy.

Practical Tips

• Optimize your playlist creation on music streaming services by combining community-driven suggestions with your own analysis. Use the platform's recommended playlists or songs as a starting point (collaborative filtering) and then curate the final playlist by considering the lyrics, genre, and artists you prefer (content-focused model), resulting in a playlist that's both popular and tailored to your taste.
• Create a movie club with friends where each member rates and discusses films watched by the group. This mimics the collaborative filtering process on a smaller scale and can help you discover new movies that align with your group's tastes. As your club grows, you could even develop a shared spreadsheet where everyone enters their ratings, creating a personalized recommendation system based on your collective preferences.
• Enhance your online shopping by scoring products based on your specific needs and past purchases. Make a checklist of attributes that are important to you for a particular type of product, such as price range, brand reputation, and feature set. When you're considering a purchase, score each product against your checklist and rank them to decide which one aligns best with your preferences. This approach applies the principle of matching user characteristics with product attributes to make more informed buying decisions.

Utilize a Dual-Part Architecture for Efficient Serving

Aminian advocates for a dual-layer framework to address the challenge of serving recommendations efficiently when dealing with billions of videos.

Practical Tips

• Enhance your video-sharing with friends by adopting a dual-selection approach. When you come across videos you think your friends might like, don't just send them randomly. Instead, categorize them into 'must-watch' and 'if you have time' before sharing. This helps your friends prioritize which videos to watch first, based on your dual-part recommendation, and can make the sharing experience more valuable for them.

Adapt Model Through Continual Learning

The model must adjust to different videos, new users, and user preference shifts. Aminian emphasizes the need for continuous learning and discusses techniques like online training and fine-tuning to maintain an up-to-date model.

Other Perspectives

• There might be computational and resource constraints that make constant adjustment impractical or too costly for some applications.
• There might be ethical considerations if the model starts to exhibit biases towards new users, potentially at the expense of minority groups within the user base.
• User preferences can be volatile and unpredictable, making it challenging for the model to adapt accurately and may result in a less stable user experience.
• Continuous learning can introduce new vulnerabilities, such as data poisoning or adversarial attacks, which can compromise the model's integrity.
• Depending on the application, online training might raise privacy concerns, as it often involves using real-time user data.
• Fine-tuning requires careful versioning and tracking of model changes, which can become complex and difficult to manage over time.

System to Anticipate Ad Engagement

Aminian explores the design of a system that forecasts click probabilities for ads displayed on users' timelines to increase earnings.

Handle Sparse Data With Feature Crosses

Aminian highlights the challenge of dealing with high-cardinality categorical features, including ad category, advertiser identifier, and user identifier. These features might create an enormous and sparse feature space, where most entries are zero. He suggests using feature crossing to develop new features that capture pairwise interactions, providing more information to the model.

Context

• Techniques such as embedding layers or hashing can be used to reduce the dimensionality of high-cardinality features, making them more manageable for machine learning models.
• These are categorical variables that have a large number of unique values. For example, a user identifier might have millions of unique entries, each representing a different user.
• Sparse data can make it difficult for models to learn effectively because the lack of information (many zeros) can obscure patterns and relationships in the data.
• To prevent overfitting when using feature crosses, regularization techniques like L1 or L2 regularization can be applied to penalize overly complex models and encourage simpler, more generalizable solutions.
• If you have features like "age" and "income," a feature cross might be "age multiplied by income," which could reveal insights about spending behavior across different age groups.

Leverage Social Graph Using Graph Neural Networks

Aminian recommends exploring Deep & Cross Networks (DCN) and factorization machines (FM) for effectively representing feature interactions. These architectures automate learning how features interact, surpassing the limitations of manually crossing features.

Practical Tips

• You can deepen your understanding of feature interactions by playing with interactive machine learning websites. Websites like TensorFlow Playground allow you to visually manipulate neural network parameters and see how changes affect the model's performance. By adjusting the depth and width of the network, you can get a hands-on feel for how deep and cross layers might interact to capture complex relationships in data.
• Use a visual mind-mapping tool to map out your daily tasks and responsibilities, then draw connections between them to identify complex interactions. For instance, you might find that time spent learning a new skill enhances your performance at work, which in turn leads to more leisure time, illustrating a positive interaction effect in your personal productivity.
• Create a simple spreadsheet to track and analyze interactions in your daily life. You might track your exercise, diet, and sleep patterns to see how they interact and affect your overall health. Over time, you can use this data to identify patterns and make informed changes to your lifestyle.
• Use spreadsheet software with predictive analytics features to experiment with your own feature interactions. Start by creating a dataset from an area of interest, like your home energy usage, including variables such as time of day, temperature, and appliance use. Then, apply predictive analytics tools to identify which factors most significantly impact your energy consumption, enabling you to make data-driven decisions to reduce costs and improve efficiency.
• You can enhance your decision-making by using online tools that employ data crossing features similar to DCN and FM. For instance, if you're looking to invest in stocks, use an investment app that utilizes advanced algorithms to analyze market trends and provide personalized recommendations. This mirrors the concept of surpassing manual analysis by leveraging technology that can process and cross-reference vast amounts of data more efficiently.

Retrain to Adjust to Users' Actions

Aminian stresses the need for constant updating in systems that predict ad clicks, as user behavior and ad relevance can change rapidly. The model requires ongoing retraining and adjustment to accommodate fresh data and maintain performance.

Context

• Advertisers frequently update their content to stay relevant, which can alter how appealing an ad is to different user segments.
• Economic conditions, such as recessions or booms, can change consumer spending habits and priorities, influencing ad effectiveness.
• Privacy laws and regulations can impact data availability and usage, requiring models to adapt to new constraints while maintaining performance.
• As the user base grows, the model must be retrained to handle increased data volume and diversity effectively.

Infrastructure and Monitoring for ML Systems

Aminian dedicates a portion of the book to discussing critical aspects of infrastructure and system observation for maintaining and scaling a machine learning setup.

Monitor Model Performance and System Health

Track Both Operation-Related and Model-Specific Metrics

Aminian reminds you to consider monitoring both metrics related to operations and ML-specific metrics:

• Operation-related metrics: Confirm smooth system operation. Track metrics like latency, throughput, request volume, and how resources are utilized.

• Machine learning-specific metrics: Ensure reliable predictions from the model. Track metrics like data and concept drifts, model accuracy, and distributions of outputs.

Practical Tips

• Use a fitness tracker or app to measure and analyze your personal health metrics, such as heart rate (latency), steps taken (throughput), and calories burned (resource utilization). By observing these metrics over time, you can adjust your exercise and diet plans to optimize your health and fitness routine. For instance, if you notice your heart rate is higher than usual during workouts, you might decide to increase your cardiovascular training to improve your heart health.

Detect Drifts to Ensure Reliable Predictions

Aminian emphasizes the importance of detecting drifts, where the distribution of the data or model performance deviates from what was observed during training. He highlights techniques for monitoring these drifts and triggering alerts or retraining when necessary.

Practical Tips

• Improve your diet by tracking your food intake and observing nutritional trends. Use a food diary app to log everything you eat and drink. Analyze the data weekly to spot any drifts in your eating habits, such as a decrease in vegetable intake or an increase in snacking. This insight allows you to make informed adjustments to ensure a balanced diet.
• Create a 'drift detection' dashboard using a simple spreadsheet or a free online tool like Google Sheets. Input key metrics that reflect your progress in areas such as fitness, finance, or learning new skills. Set up conditional formatting to highlight when your numbers are off-target, serving as a visual alert to potential drifts in your performance or behavior. Regularly updating and reviewing this dashboard will help you stay aware of your progress and make necessary corrections.
• Create a visual progress tracker on your wall using colored sticky notes to represent adherence to your learning or behavior modification plan. Each color can represent a different level of adherence, from full compliance to significant deviation. This visual representation can serve as a daily reminder and motivator to stick to your intended path and quickly identify when you're drifting away.
• Implement a habit-tracking app on your smartphone to monitor your daily routines. Choose an app that allows you to set reminders or alerts when you deviate from your desired habits, such as exercising or reading, ensuring you can quickly address any drift from your goals.
• Use a journal to track your daily decisions and outcomes to spot drifts in your personal goals. By regularly reviewing your journal, you can identify patterns where your actions aren't aligning with your intended results, allowing you to make timely adjustments. For example, if you notice that your spending habits are drifting from your budget goals, you can take immediate action to correct your course.

Design Scalable and Efficient Serving Infrastructure

Balance the Compromises of Real-Time and Periodic Prediction

Aminian advises carefully selecting a method of forecasting according to needs and limitations:

• Online inference: Make forecasts upon request. Ideal for systems needing immediate results, but can be computationally intensive and may struggle with high traffic volume.

• Batch predictions: Pre-compute forecasts periodically. Suitable for non-time-sensitive tasks, but may not process fresh data or user preferences immediately.

Other Perspectives

• While careful selection is important, it may not always be feasible due to a lack of expertise or resources to evaluate the different forecasting methods effectively.
• For certain applications, the need for immediate results may not justify the additional complexity and infrastructure required to support online inference.
• Load balancing and distributed computing can help manage the computational demands by spreading the workload across multiple servers or nodes.
• The periodic nature of batch predictions may not be flexible enough to handle unexpected events or anomalies that require immediate attention and re-forecasting.
• With the use of incremental updates and caching strategies, batch systems can be optimized to incorporate new user preferences more quickly than traditionally assumed.

Leverage Caching and Nearby Similarity Search

Aminian suggests employing techniques like:

• Caching: Store recently computed predictions to accelerate the serving process.

• Approximate nearest neighbor search: Efficiently find similar items or embeddings in expansive datasets.

Practical Tips

• Use a recipe recommendation app or website that suggests dishes based on ingredients you already have. Input the ingredients from your pantry or fridge, and the app will use its database to find recipes with similar components, helping you to efficiently plan meals without needing to buy additional groceries.

Make Sure the System Can Be Retrained and Redeployed

Aminian recommends designing the architecture for seamless retraining and redeployment:

• Continual learning: Regularly update the system with additional data.

• Version control: Track model iterations and manage deployments meticulously.

• Evaluate in production: Use methods like A/B trials, shadow implementations, and canary releases to ensure the new model performs well before being fully deployed.

Keep in mind that specific conditions and limitations might lead you to emphasize certain steps more than others. Aminian's main message is that you should demonstrate a thorough understanding of the entire machine learning system lifecycle, from data preparation to model development, evaluation, deployment, and maintenance.

Practical Tips

• Create a personal skills inventory to identify your current abilities and potential areas for retraining. Start by listing all the skills you possess, both professional and personal. Then, research emerging industry trends to determine which skills are becoming more valuable. This will help you pinpoint where you might need retraining and allow you to proactively seek out resources, such as online courses or local workshops, to develop those skills before they become a necessity.
• Volunteer for projects or tasks outside your comfort zone at work or in community organizations. This hands-on approach forces you to acquire new skills and knowledge, effectively updating your personal 'system' with practical data. Document these experiences and reflect on them to solidify the learning and prepare for future challenges.
• Create a simple spreadsheet to log your project changes and outcomes. Even if you're not a tech expert, you can use a tool like Google Sheets or Microsoft Excel to track the versions of any project you're working on, whether it's a personal budget, a workout plan, or a recipe collection. Start by creating columns for the date, version number, changes made, and results observed. This habit will help you see the evolution of your projects and make informed decisions about future changes.
• Implement a personal A/B trial by comparing two methods of completing a common task to determine which is more efficient. For example, if you're trying to optimize your grocery shopping, use a handwritten shopping list one week and a digital list on your phone the next. Track the time spent shopping, the cost, and the quality of the experience to decide which method works best for you.