title: “Machine Learning Concepts for Discretionary Traders”
description: “Understand supervised and unsupervised learning, overfitting, feature selection, and cross-validation — and learn to think about trading as a classification problem.”
slug: “learn-trading/machine-learning-trading”
date: 2026-03-15
lastmod: 2026-03-15
draft: false
type: “advanced”
Machine Learning Concepts for Discretionary Traders
Machine learning applied to trading is the use of algorithms that learn patterns from historical data to make predictions about future price movements. This article is not a guide to building ML trading systems — it is a conceptual framework that helps discretionary traders understand ML thinking, recognize its limitations, and apply its core principles to improve their own decision-making. Whether or not you ever write a line of code, understanding how machine learning approaches the prediction problem will sharpen how you evaluate your own setups, avoid overfitting, and think probabilistically about markets.
The concepts covered here — supervised learning, overfitting, feature selection, and cross-validation — are directly applicable to how you evaluate your trading strategies. When you backtest a setup and find it profitable, you are doing manually what a machine learning algorithm does computationally. The same pitfalls that plague ML models — overfitting to noise, data snooping, insufficient out-of-sample testing — plague discretionary traders too. Understanding these concepts gives you a framework for avoiding them.
What Is Machine Learning in Trading and Where It Fits
Machine learning in trading is the application of statistical algorithms that automatically identify patterns in market data and use those patterns to generate trading signals or predictions. It fits at the advanced level because it requires foundational knowledge of probability and expected value, backtesting methodology, and regression analysis.
This article sits at the conceptual level. It prepares you to think in ML terms without requiring you to implement ML systems. For those who do want to build models, the guide on building your first quantitative model provides the practical starting point.
Prerequisites
Before engaging with ML concepts for trading, you should have:
- Solid understanding of backtesting methodology and its limitations
- Experience with statistical evaluation of trading strategies
- Understanding of the difference between in-sample and out-of-sample performance
- Familiarity with basic statistics (mean, standard deviation, correlation)
- Awareness of the overfitting problem in trading strategy development
Technical Foundation: The ML Framework for Markets
Trading as a Classification Problem
Trading as a classification problem reframes the trading question from “where is price going?” to “given this set of market conditions, should I classify the next period as a buying opportunity, selling opportunity, or no-trade?”
In ML terminology:
Features (inputs): The market data you use to make decisions. For a discretionary trader, these are your indicators, price patterns, volume readings, and contextual factors. Examples:
– RSI value
– Distance from 20 EMA (%)
– ADX reading
– Volume relative to 20-day average
– Market regime classification
– Relative strength vs. benchmark
Labels (outputs): The outcome you are trying to predict. Typically:
– Binary: “Trade wins” vs. “Trade loses”
– Multi-class: “Strong buy” vs. “Weak buy” vs. “No trade” vs. “Sell”
– Continuous: The R-multiple outcome of the trade
Model: The function that maps features to labels. For a discretionary trader, your model is the mental decision process you use. For an ML system, it is a mathematical function learned from data.
The key insight is that your discretionary process is already a classification model — just an informal one. You look at RSI, moving averages, volume, and price action (features), and you classify the situation as “trade” or “no trade” (label). ML formalizes this process and measures its accuracy rigorously.
Supervised vs. Unsupervised Learning
Supervised learning uses labeled historical data to learn the relationship between features and outcomes. You provide the algorithm with examples where you know the answer (“this setup led to a 2.3R winner,” “this setup led to a -1R loser”), and the algorithm learns to predict outcomes for new, unseen data.
Trading applications of supervised learning:
– Predicting whether a setup will be profitable based on market conditions at entry
– Classifying market regimes based on indicator readings
– Estimating the expected R-multiple of a trade given specific features
Unsupervised learning finds patterns in data without labeled outcomes. The algorithm groups similar observations together without being told what “success” looks like.
Trading applications of unsupervised learning:
– Clustering market days into regimes based on behavior patterns
– Identifying which of your trades are similar to each other (and which clusters perform best)
– Detecting anomalous market conditions that do not fit historical patterns
For discretionary traders, supervised learning concepts are more immediately applicable because they map directly to the “does this setup work?” question.
The Train/Test Split
The train/test split is the foundational methodology for evaluating any predictive model — including your trading strategy.
Training data (in-sample): The historical data you use to develop and refine your rules. When you backtest a strategy and adjust parameters to improve performance, you are training your model on this data.
Test data (out-of-sample): Data that was not used during development. When you paper trade or forward test a strategy, you are testing on out-of-sample data.
The critical rule: Never evaluate a model on the data used to develop it. In-sample performance is always overly optimistic because the model (or the trader) has adapted to the specific patterns in that data, including patterns that are noise rather than signal.
| Data Split | Purpose | What It Tells You |
|---|---|---|
| Training set (60-70%) | Develop and optimize the strategy | How well the strategy fits historical data |
| Validation set (15-20%) | Tune parameters and select between alternatives | Whether parameter choices generalize |
| Test set (15-20%) | Final evaluation on untouched data | Realistic estimate of future performance |
This three-way split is directly applicable to discretionary strategy development:
– Train: Develop your setup rules by studying the first portion of historical data
– Validate: Test variations of your rules on the second portion
– Test: Evaluate the final version on the third portion, which you have not looked at
If you skip the test set and evaluate only on training data, you have no idea whether your strategy will work in the future. This is the single most common mistake in both ML and discretionary trading.
Core Concepts for Practical Application
Overfitting: The Central Problem
Overfitting is the most important concept in this article. It occurs when a model (or a discretionary strategy) captures noise in the historical data rather than genuine, persistent patterns. An overfit strategy performs brilliantly in backtesting and poorly in live trading.
Signs of overfitting in trading:
| Warning Sign | What It Looks Like | Why It Is Overfitting |
|---|---|---|
| Too many parameters | Strategy has 10+ conditions | Each additional condition adapts more to noise |
| Extreme backtest results | Sharpe ratio > 3.0, win rate > 80% | Results too good for the market’s randomness level |
| Sharp performance drop in forward test | Backtest shows 40% annual return, live shows 5% | Backtest captured noise patterns that did not repeat |
| Sensitivity to small parameter changes | Changing MA from 20 to 21 periods dramatically changes results | True patterns are robust to small parameter changes |
| Strategy only works on specific time periods | Profitable 2020-2022, fails before and after | Captured period-specific dynamics, not universal edge |
The antidote to overfitting:
1. Use fewer parameters (simpler models generalize better)
2. Always evaluate on out-of-sample data
3. Test robustness by varying parameters slightly
4. Require logical explanations for why a pattern should persist
5. Demand larger sample sizes before trusting a result
Feature Selection: Choosing What to Measure
Feature selection is the process of deciding which market data to include in your analysis. In ML, adding too many features increases the risk of overfitting. In discretionary trading, watching too many indicators creates the same problem — you find coincidental relationships that look meaningful but are not.
Principles of good feature selection:
Relevance: Each feature should have a logical connection to the outcome. RSI measures momentum, which logically relates to short-term price direction. The color of your shirt does not. While this seems obvious with silly examples, traders routinely include indicators with no clear causal mechanism.
Independence: Features should provide different information. Using both RSI(14) and stochastic oscillator together adds little value because they measure similar things (momentum). Using RSI and volume provides genuinely different perspectives.
Parsimony: Fewer features are better, all else equal. A model with 3 features that performs 90% as well as a model with 10 features is almost always preferable because it is more likely to generalize.
Stability: Features that produce similar signals across slightly different market conditions are more reliable than features that are sensitive to small data changes.
For your discretionary trading, audit your current indicators and conditions against these four criteria. If you use more than 5-7 independent features in your decision process, you are likely overfit.
Cross-Validation: Robust Testing
Cross-validation is a technique for getting more reliable performance estimates from limited data. Instead of a single train/test split, you rotate through multiple splits and average the results.
The most common form is k-fold cross-validation:
- Divide your data into k equal segments (typically 5)
- Train on segments 1-4, test on segment 5
- Train on segments 1-3 and 5, test on segment 4
- Repeat for all combinations
- Average the test results across all folds
For time series data (which trading data always is), you must use a modified version that preserves the temporal order:
- Walk-forward analysis: Train on months 1-6, test on month 7. Then train on months 1-7, test on month 8. Continue expanding the training window and testing on the next unseen period.
Walk-forward analysis is the gold standard for trading strategy validation because it simulates how you would actually deploy the strategy: always training on past data and testing on future data. If your strategy performs consistently across all walk-forward segments, it provides much stronger evidence of a genuine edge than a single backtest.
Practical Application: Thinking Like an ML Practitioner
Thinking like an ML practitioner means applying four core principles to your discretionary trading without building any actual models. Apply these principles to your discretionary trading:
Step 1: Define Your Feature Set
List every piece of information you use to make a trading decision. These are your features. Evaluate each one: Is it relevant? Is it independent from other features? Is it necessary?
Step 2: Classify Your Training and Testing Data
When did you develop your current strategy? Which historical data did you study? That is your training data. Any subsequent data is your test data. If your live performance is significantly worse than your development-period results, overfitting is the most likely explanation.
Step 3: Test Robustness
Take your primary setup and make small changes to the parameters:
– If you use the 20 EMA, does the setup still work with the 18 or 22 EMA?
– If you require RSI above 50, does it still work with RSI above 45 or 55?
– If you trade in a specific time window, does it still work in the adjacent window?
A robust setup survives small parameter changes. A fragile (overfit) setup breaks.
Step 4: Evaluate Out-of-Sample Honestly
Compare your backtest results (training) with your actual trading results (testing). Calculate the performance degradation percentage. Some degradation is normal (10-20%). Severe degradation (50%+) strongly suggests overfitting.
Measuring Impact on Performance
| Metric | Before ML Thinking | After Applying ML Concepts |
|---|---|---|
| Number of indicators used | Often 8-12 | Reduced to 4-6 (feature selection) |
| Backtest-to-live degradation | Often 50%+ | Target < 25% (reduced overfitting) |
| Strategy robustness | Sensitive to parameters | Stable across reasonable parameter ranges |
| Confidence in strategy viability | Based on backtest alone | Based on out-of-sample validation |
| Time wasted on non-viable strategies | Significant | Reduced through early validation |
Limitations and Edge Cases
Limitation 1: Markets are not stationary. ML models assume that the patterns in training data persist into the future. Markets change. Relationships between features and outcomes shift. No model — human or algorithmic — can guarantee that past patterns will continue.
Limitation 2: Signal-to-noise ratio is low. Financial data is extremely noisy. The genuine predictive signal in most market features is small relative to the random variation. This makes it easy for both ML models and human traders to find patterns that are pure noise.
Limitation 3: Execution differs from prediction. Even a perfect prediction model does not account for slippage, commissions, market impact, and the practical challenges of execution. A model that is profitable in simulation may not be profitable after real-world frictions.
Limitation 4: Black box risk. Complex ML models can make accurate predictions without providing understandable reasons. For discretionary traders, understanding why a setup works is essential for maintaining conviction during drawdowns. Prefer simpler models with interpretable logic.
Limitation 5: Data quality and survivorship bias. Historical market data often contains errors, and the universe of available instruments suffers from survivorship bias (you only see the companies that still exist). Both problems can create false patterns that an ML model or a manual backtest will dutifully “learn.”
Supplementary: Institutional Context and References
Quantitative hedge funds employ teams of PhD researchers to build ML trading models, using techniques far beyond what is described here: deep learning, reinforcement learning, natural language processing of news and filings, and alternative data sources. Despite these resources, the core challenges remain the same — overfitting, non-stationarity, and low signal-to-noise ratios. Many sophisticated ML-based strategies fail in live trading for the same reasons that simple strategies fail: they captured noise rather than signal.
The practical takeaway for discretionary traders is that ML thinking is more valuable than ML tools. Understanding overfitting, feature selection, and out-of-sample validation will improve your trading even if you never write a line of code. The broader Learn Trading curriculum connects these concepts to practical strategy development.
Academic and Professional References
- Hastie, T., Tibshirani, R., and Friedman, J. The Elements of Statistical Learning — the standard reference for ML methodology
- Lopez de Prado, M. (2018) Advances in Financial Machine Learning — ML techniques specifically for trading applications
- Aronson, D. and Masters, T. (2014) Statistically Sound Machine Learning for Algorithmic Trading — practical ML for trading with statistical rigor
- scikit-learn documentation (scikit-learn.org) — accessible implementation reference for Python users