Book SummaryAdvances in Financial Machine Learning, by Marcos López de Prado

1-Page Summary1-Page Book Summary of Advances in Financial Machine Learning

Techniques for Financial Machine Learning

This book guide is a reorganization of information extracted from "Developments in Machine Learning for Finance" by Marcos López de Prado. It dives into techniques to develop successful machine learning-based investment strategies, specifically tailored to the particular challenges of data from financial markets.

Data Preprocessing For Financial Time Series

Before any analysis can be done, financial data needs to be carefully preprocessed to make it "digestible" by ML models. This will be discussed in the next sections.

Structuring Financial Data for AI

The first step is to transform unstructured, raw financial data into a structured format suitable for Machine Learning algorithms. López de Prado emphasizes that relying on someone else's processed data is likely to lead to discovering what someone else already knows or will figure out soon. The author recommends starting with raw, unstructured data, the kind that will likely irritate your data infrastructure team. It’s that hard-to-process data, perhaps ignored by competitors, that is most promising.

Then, that raw data must be parsed and organized into a format that ML algorithms understand. That usually implies tables, where each row contains information extracted from the raw dataset according to some logic. Practitioners call these rows "bars." López de Prado suggests that while standard bar methods like time, tick, volume, and dollar bars are prevalent, the sampling strategy should be guided by the intended application. Time bars, for example, have the disadvantage that financial markets handle information in varying timeframes. This causes information to be oversampled in times of low activity and undersampled in times of high activity. A better approach might be tick bars, as well as bars based on volume or dollar amount, because these are synchronized with indicators of market activity or value exchanged, producing returns that approximate an independent and identically distributed normal. The author provides insightful code examples, including detailed Python snippets, for implementing these various methods of constructing bars and dealing with common practical situations like the roll process in futures contracts or the "ETF technique," which allows you to represent a complicated portfolio of securities like it's one cash product.

Practical Tips

• Experiment with free online tools that convert bank statements into structured data formats. Many banks offer transaction history downloads in CSV format, which you can then upload to these tools. They'll categorize your expenses and income, making it easier for you to analyze your financial habits and potentially identify areas where you can save money.
• Create a "Fresh Perspectives" club with friends or colleagues where each member brings a dataset they've gathered themselves. This could be anything from a week's worth of commute times to the number of birds seen in the backyard. During meetings, discuss your findings and brainstorm potential discoveries or trends that the data might reveal. This social setting encourages collaborative exploration and can lead to unexpected insights that processed data might overlook.
• Dive into a new hobby without guidance to gather raw experiences. Start a hobby like gardening or cooking without following tutorials or guides. Document your progress, noting what works and what doesn't. This hands-on approach will force you to engage with the raw data of your experiences, leading to unique insights and personal growth.
• You can refine your grocery shopping by creating a meal-based sampling list. Before going shopping, decide on the meals you plan to cook for the week. Then, list the ingredients you need for those specific meals. This targeted approach ensures you buy only what you need, reducing waste and saving money.
• Adjust your time management tools to reflect actual energy levels by setting up alerts or reminders based on your activity log insights. If you notice you're consistently more active in the mornings, schedule reminders for your most important tasks during this time. Conversely, set reminders for breaks or less critical tasks during your identified low activity periods. This way, you're adapting your schedule to your natural productivity rhythms.
• Engage in paper trading using tick, volume, and dollar bars to simulate investment strategies without financial risk. Paper trading platforms often provide real-time market data and the ability to use various bar types. Set up a mock portfolio and make trades based on the patterns you identify with these bars. This will allow you to practice and refine your strategy based on market synchronization before you commit real money.
• Enhance your workout routine by plotting your exercise progress with bar charts. Record your daily or weekly reps, weight lifted, or distance run, and use Python to create a bar chart that shows your improvement. This can serve as a motivational tool, giving you a clear picture of how far you've come and where you might need to focus more effort.
• You can simulate the roll process in futures contracts using a spreadsheet to track hypothetical trades. Start by selecting a futures contract, such as crude oil or wheat, and note its price. When the contract nears expiration, "roll" to the next contract by recording the price difference between the expiring contract and the next one. This exercise will give you a hands-on understanding of the roll yield and its impact on futures trading without risking actual money.

Labeling Financial Observations

Once you have a structured data matrix X, the next step is to produce an array of labels y to be used by your supervised learning algorithm. Standard labeling methods, like using a fixed timeframe, are susceptible to numerous flaws. Their main drawback is that they apply a label to an observation based on a static timeframe,...

Advances in Financial Machine Learning Summary Feature Engineering, Model Interpretation, and Hyperparameter Tuning

This section explores techniques for comprehending feature importance, tuning hyperparameters, and setting up robust performance evaluation methods.

Analyzing Feature Significance for Understanding Finance

One of the key principles advocated by López de Prado is the critical role of feature importance analysis, which he contrasts against the common practice of "researching" through backtests. This approach, so prevalent in academic publications, amounts to brute-force searching for patterns in historical data, often leading to erroneous discoveries because of testing numerous times and selection bias. In fact, the author goes as far as establishing his initial principle for backtesting: "Backtesting isn't meant for research. It's important to know which features matter." This highlights the significance of understanding what information truly drives a model's forecasting capability, rather than merely seeking profitable backtest results. López de Prado emphasizes that by focusing on feature importance, we develop a deeper understanding of the underlying market mechanisms and relationships, leading to strategies that are more resilient and generalizable.

The author defines...

Advances in Financial Machine Learning Summary Applying Machine Learning to Managing Investments

This section explores how machine learning applies to two fundamental aspects of managing investments: bet sizing and backtesting.

Bet Sizing Using Machine Learning Predictions

The author dedicates a chapter to determining bet size, highlighting its importance in achieving consistent profitability. Even if a strategy makes highly accurate predictions, neglecting bet sizing can lead to disastrous outcomes. The author uses the analogy of poker, specifically Texas Hold'em, to illustrate how bet sizing is just as crucial as making the right bet.

The author discusses several techniques for sizing bets. The first approach involves analyzing how concurrent bets are probabilistically distributed. By fitting a blend of Gaussian distributions (using the author's recommended EF3M algorithm) to the observed bet concurrency, we can calibrate how much to wager for specific signal strengths to reserve cash for opportunities when the signal is stronger. Another approach is establishing bet limits using past data. This method sets a cap on how many simultaneous long and short bets you can make, aiming to distribute the bet size to avoid hitting limits too soon. Meta-labeling provides...

Advances in Financial Machine Learning Summary High-Performance Computing for Financial Data Analysis

In this section, the concept of advanced computing capabilities is introduced, which are crucial to analyzing data in finance.

Running Processes Simultaneously and Using Multiprocessing for Speed and Scalability

López de Prado highlights that much of the work involved in developing ML investment strategies require computational brute-force, and that an efficient parallelization of tasks is needed for the analysis be completed within a reasonable time span. Python runs operations one at a time in a lone thread, unless told otherwise, and the author provides examples of how that single-thread execution can be made much more efficient.

When preparing to parallelize, the author notes a key practical distinction: atoms versus molecules. "Atoms" refers to the smallest, indivisible computational tasks. "Molecules" are groups of atoms, where each is allocated to a different processor, and the atoms within are processed sequentially. López de Prado presents two paradigms for grouping atoms as molecules: the simpler case of linear partitions, and the more complex case of two nested-loop partitions, illustrated with helpful plots.

The author provides code snippets for efficiently...