diff --git a/README.md b/README.md index 094e205..dbe4f40 100644 --- a/README.md +++ b/README.md @@ -33,11 +33,11 @@ Notebook numbering: **01** Data & market stats · **02** Factor diagnostics · * | Term | Linear-algebra meaning | Notebooks | |------|------------------------|:---------:| | **Return** | Fractional price change: $r_{t} = p_t / p_{t-1} - 1$ | 01 | -| **Return panel** $\mathbf{R}$ | Matrix $\mathbf{R}\in\mathbb{R}^{T\times N}$: months (rows) × stocks (columns); sparse (NaN where a stock didn't trade yet) | 01–06 | +| **Return panel** $\mathbf{R}$ | Matrix $\mathbf{R}\in\mathbb{R}^{T\times N}$: months (rows) $\times$ stocks (columns); sparse (NaN where a stock didn't trade yet) | 01–06 | | **Cross-section** | A row $r_t \in \mathbb{R}^{N_t}$ — all stock returns at one date $t$ | 01, 02, 03 | | **One stock's history** | A single column of $\mathbf{R}$ | 01 | | **Universe** | The set of columns we're allowed to hold — here the S&P 500 large-caps | 01 | -| **Sector** | A categorical partition of the column index set (Technology, Financials, …); one-hot form → indicator matrix $\mathbf{D}$ | 01, 03 | +| **Sector** | A categorical partition of the column index set (Technology, Financials, …); one-hot form $\rightarrow$ indicator matrix $\mathbf{D}$ | 01, 03 | | **Dispersion** | Cross-sectional std dev — the spread of returns across a row of $\mathbf{R}$ | 01, 06 | | **Equal-weight index** | Mean of a row: $\bar{r}_t = \tfrac{1}{N_t}\mathbf{1}^\top r_t$ (portfolio with $w_i = 1/N_t$) | 01, 04 | | **Survivorship bias** | Columns are a *non-random* sample: delisted/bankrupt names are missing, biasing returns upward | 01, 04 | @@ -49,11 +49,11 @@ Notebook numbering: **01** Data & market stats · **02** Factor diagnostics · * | **Factor / signal** | Vector $f_t \in \mathbb{R}^{N_t}$ — one score per stock; we ask whether $f_t$ predicts $r_{t+1}$ | 02, 03, 04 | | **Momentum (12-1)** | Trailing 12-month return skipping the last month — the headline signal; persistent drift | 02, 03, 04 | | **Value / Quality / Low-vol** | Other candidate factor vectors (proxied by price data here) | 02, 03 | -| **Information coefficient (IC)** | Cosine similarity $\cos\theta = \dfrac{f_t^\top r_{t+1}}{\|f_t\|\,\|r_{t+1}\|}$ (ranks → Spearman IC) | 02, 03 | +| **Information coefficient (IC)** | Cosine similarity $\cos\theta = \dfrac{f_t^\top r_{t+1}}{\|f_t\|\ \|r_{t+1}\|}$ (ranks $\rightarrow$ Spearman IC) | 02, 03 | | **Information ratio (IR)** | Mean IC / std(IC) — a signal-to-noise ratio for the factor | 02, 04 | | **Rank** | A permutation of $\{1,\dots,N\}$; makes factors comparable and robust to outliers | 02 | | **Walk-forward validation** | Split the sample into sub-windows and test stability of IC / performance across regimes | 02, 04 | -| **Winsorize** | Clip extreme entries of $f$ at ±3σ to tame outliers | 03 | +| **Winsorize** | Clip extreme entries of $f$ at $\pm 3\sigma$ to tame outliers | 03 | | **Z-score** | Center each cross-section to mean 0, std 1 | 03 | | **Sector neutralization** | Orthogonal projection $f_\perp = (I - P)f$ with $P = \mathbf{D}(\mathbf{D}^\top\mathbf{D})^{-1}\mathbf{D}^\top$ onto sector dummies | 03, 04 | | **Composite signal** | A linear combination $c_t = \sum_k w_k f_{k,\perp}$ of neutralized factor vectors | 03 | @@ -182,7 +182,7 @@ We pull the current S&P 500 constituents and download adjusted close prices. Thi Key findings: * **Universe breadth** rises from ~385 to ~501 stocks over the sample — but the column set is fixed to *today's* constituents, so this counts how many of today's survivors had price data in month $t$. The matrix isn't truly "getting wider"; its survivor-only columns fill in over time. * Average monthly cross-sectional return dispersion ~7.5% (the spread across a row of $\mathbf{R}$). -* Equal-weight universe Sharpe (rf=0) ≈ 0.98 over the full period. +* Equal-weight universe Sharpe (rf=0) $\approx$ 0.98 over the full period. * Sector composition spans 11 GICS sectors, dominated by Technology, Financials, and Healthcare. ### 2. Factor Analysis and Diagnostics @@ -214,9 +214,9 @@ Notebook 02 established momentum as the only factor with positive IC. This noteb 3. **Sector-neutralizes** via orthogonal projection: $f_\perp = (\mathbf{I} - \mathbf{P})f$ with $\mathbf{P} = \mathbf{D}(\mathbf{D}^\top\mathbf{D})^{-1}\mathbf{D}^\top$ 4. **Compares** momentum-only vs. a 4-factor equal-weight composite -Sector neutralization slightly improves momentum (IR 0.11 raw → 0.15 neutralized). Combining momentum with two negative-IC factors (value, lowvol) and one near-zero factor (quality) **dilutes** the signal — the 4-factor composite has negative IC. This is the linear-algebra intuition made empirical: adding vectors that point in the wrong direction moves the sum away from the target. +Sector neutralization slightly improves momentum (IR 0.11 raw $\rightarrow$ 0.15 neutralized). Combining momentum with two negative-IC factors (value, lowvol) and one near-zero factor (quality) **dilutes** the signal — the 4-factor composite has negative IC. This is the linear-algebra intuition made empirical: adding vectors that point in the wrong direction moves the sum away from the target. -**The headline signal is momentum-only, sector-neutralized** → saved as `momentum_signal.csv`. This is what the backtest trades. +**The headline signal is momentum-only, sector-neutralized** $\rightarrow$ saved as `momentum_signal.csv`. This is what the backtest trades. --- @@ -241,7 +241,7 @@ A portfolio is a weight vector $w$. We form a top-decile long-only portfolio at | EW Universe | 15.9% | 0.95 | -47% | 1.27 | | Long-Short (net) | -0.7% | -0.04 | -70% | — | -| Portfolio | FF 4-factor alpha (ann.) | t-stat | MKT β | MOM β | +| Portfolio | FF 4-factor alpha (ann.) | t-stat | MKT $\beta$ | MOM $\beta$ | |-----------|-------------------------:|-------:|------:|------:| | **Long-Only** | **+5.95%** | **3.95** | 1.19 | 0.25 | | Long-Short | -2.95% | -1.41 | 0.15 | 0.91 | @@ -295,7 +295,6 @@ The momentum long-only portfolio's risk is **~90.7% systematic** and **~9.3% idi * **No intraday execution modeling.** Transaction costs are a flat 5 bps. Real slippage depends on order size, liquidity, and volatility. * **Monthly rebalance only.** Daily/weekly rebalancing might capture different signals but would dramatically increase turnover. * **Momentum crash risk.** The 2006–2011 walk-forward window shows negative active return, driven by the 2008–09 momentum crash. A crash-protection overlay (e.g. volatility scaling) would improve robustness. -* **Sector neutralization only.** A full Barra-style risk model would neutralize against more risk factors (size, beta, liquidity). --- diff --git a/notebooks/01_data_overview_and_market_stats.ipynb b/notebooks/01_data_overview_and_market_stats.ipynb index c0c0e1d..61cd0c4 100644 --- a/notebooks/01_data_overview_and_market_stats.ipynb +++ b/notebooks/01_data_overview_and_market_stats.ipynb @@ -20,7 +20,7 @@ "| Term | Meaning |\n", "|------|---------|\n", "| **Return** | Fractional price change: $r_{t} = p_t / p_{t-1} - 1$ |\n", - "| **Return panel** $\\mathbf{R}$ | The $T \\times N$ matrix of monthly returns — months (rows) × stocks (columns); the project's central object |\n", + "| **Return panel** $\\mathbf{R}$ | The $T \\times N$ matrix of monthly returns — months (rows) $\\times$ stocks (columns); the project's central object |\n", "| **Cross-section** | A row of $\\mathbf{R}$ — all stocks at one date |\n", "| **One stock's history** | A column of $\\mathbf{R}$ |\n", "| **Universe** | The set of stocks (columns) we're allowed to hold — here the S&P 500 |\n", diff --git a/notebooks/02_factor_analysis_and_diagnostics.ipynb b/notebooks/02_factor_analysis_and_diagnostics.ipynb index 13cf6c6..0e616df 100644 --- a/notebooks/02_factor_analysis_and_diagnostics.ipynb +++ b/notebooks/02_factor_analysis_and_diagnostics.ipynb @@ -372,7 +372,7 @@ "id": "5659b941", "metadata": {}, "source": [ - "Note that we added a sample size filter with `mask.sum() < 20`. The `mask` identifies stocks that have both a valid factor score and a valid forward return in a given month, and `mask.sum()` counts how many usable pairs you actually have. The `< 20` threshold prevents the code from computing a correlation on a tiny sample, which is statistically meaningless and numerically unstable (e.g., Spearman correlation on 2 stocks is always exactly ±1). If you don't gate this, early-history months, mass delistings, or data gaps inject garbage ±1.0 values into your IC time series, which then contaminate every downstream statistic like the mean IC, Information Ratio, and decay curves. Setting a floor of 20 filters out those degenerate months while retaining enough valid data to produce a reliable signal.\n", + "Note that we added a sample size filter with `mask.sum() < 20`. The `mask` identifies stocks that have both a valid factor score and a valid forward return in a given month, and `mask.sum()` counts how many usable pairs you actually have. The `< 20` threshold prevents the code from computing a correlation on a tiny sample, which is statistically meaningless and numerically unstable (e.g., Spearman correlation on 2 stocks is always exactly $\\pm$1). If you don't gate this, early-history months, mass delistings, or data gaps inject garbage $\\pm 1.0$ values into your IC time series, which then contaminate every downstream statistic like the mean IC, Information Ratio, and decay curves. Setting a floor of 20 filters out those degenerate months while retaining enough valid data to produce a reliable signal.\n", "\n", "We do this sort of masking throughout." ] diff --git a/notebooks/03_factor_construction_and_composite_signal.ipynb b/notebooks/03_factor_construction_and_composite_signal.ipynb index c67daf7..8c86471 100644 --- a/notebooks/03_factor_construction_and_composite_signal.ipynb +++ b/notebooks/03_factor_construction_and_composite_signal.ipynb @@ -19,7 +19,7 @@ "\n", "| Term | Meaning |\n", "|------|---------|\n", - "| **Winsorization** | Clipping extreme values at ±$k$ standard deviations — a soft truncation to limit outlier influence |\n", + "| **Winsorization** | Clipping extreme values at $\\pm k$ standard deviations — a soft truncation to limit outlier influence |\n", "| **Z-scoring** | Centering to mean 0 and scaling to std 1, so each factor vector lives on the same scale |\n", "| **Neutralization** | Projecting out unwanted components (sector) — an orthogonal projection onto a complement subspace |\n", "| **Composite signal** | A linear combination $c_t = \\sum_k w_k f_{k,\\perp}$ of neutralized factor vectors |\n", @@ -125,7 +125,7 @@ "source": [ "## Winsorization and Z-Scoring\n", "\n", - "**Winsorization** is a statistical technique for limiting the influence of extreme outliers in a dataset by capping them at a specified threshold rather than removing them entirely. Raw factor exposures can have extreme outliers — a stock that returned 500% in the trailing year, for instance. We **Winsorize** at ±3 standard deviations, so that we don't have to drop them.\n", + "**Winsorization** is a statistical technique for limiting the influence of extreme outliers in a dataset by capping them at a specified threshold rather than removing them entirely. Raw factor exposures can have extreme outliers — a stock that returned 500% in the trailing year, for instance. We **Winsorize** at $\\pm 3$ standard deviations, so that we don't have to drop them.\n", "\n", "After winsorization we **z-score** cross-sectionally: for each date $t$, the factor vector $f_t$ is transformed to\n", "$$\\tilde{f}_t = (f_t - \\bar{f}_t) / \\text{std}(f_t),$$\n", @@ -268,7 +268,7 @@ "id": "9740141a", "metadata": {}, "source": [ - "Much like in Notebook 02, we note that we added a sample size filter with `mask.sum() < 30`. The `mask` identifies stocks that have both a valid factor score and a valid forward return in a given month, and `mask.sum()` counts how many usable pairs you actually have. The `< 30` threshold prevents the code from computing a correlation on a tiny sample, which is statistically meaningless and numerically unstable (e.g., Spearman correlation on 2 stocks is always exactly ±1). If you don't gate this, early-history months, mass delistings, or data gaps inject garbage ±1.0 values into your IC time series, which then contaminate every downstream statistic like the mean IC, Information Ratio, and decay curves. Setting a floor of 30 filters out those degenerate months while retaining enough valid data to produce a reliable signal.\n", + "Much like in Notebook 02, we note that we added a sample size filter with `mask.sum() < 30`. The `mask` identifies stocks that have both a valid factor score and a valid forward return in a given month, and `mask.sum()` counts how many usable pairs you actually have. The `< 30` threshold prevents the code from computing a correlation on a tiny sample, which is statistically meaningless and numerically unstable (e.g., Spearman correlation on 2 stocks is always exactly $\\pm$1). If you don't gate this, early-history months, mass delistings, or data gaps inject garbage $\\pm 1.0$ values into your IC time series, which then contaminate every downstream statistic like the mean IC, Information Ratio, and decay curves. Setting a floor of 30 filters out those degenerate months while retaining enough valid data to produce a reliable signal.\n", "\n", "We do this sort of masking throughout." ] diff --git a/notebooks/04_backtest_and_performance.ipynb b/notebooks/04_backtest_and_performance.ipynb index 715aec5..1dc8188 100644 --- a/notebooks/04_backtest_and_performance.ipynb +++ b/notebooks/04_backtest_and_performance.ipynb @@ -787,7 +787,7 @@ "\n", "### When Sortino > Sharpe\n", "\n", - "A strategy with occasional large positive surprises will have Sortino noticeably higher than Sharpe. A strategy with symmetric volatility (gains and losses of similar magnitude) will have Sortino ≈ Sharpe. Big gap between them tells you the return distribution is positively skewed.\n", + "A strategy with occasional large positive surprises will have Sortino noticeably higher than Sharpe. A strategy with symmetric volatility (gains and losses of similar magnitude) will have Sortino $\\approx$ Sharpe. Big gap between them tells you the return distribution is positively skewed.\n", "\n", "---\n", "\n", @@ -808,8 +808,8 @@ "$$V_t = \\prod_{s=1}^{t}(1 + r_{p,s}) = (1+r_1)(1+r_2)\\cdots(1+r_t)$$\n", "\n", "This is **compounding**. Start with $\\$1$. Each month, multiply by $(1 + r_s)$:\n", - "- Return $+5\\%$ → multiply by $1.05$\n", - "- Return $-3\\%$ → multiply by $0.97$\n", + "- Return $+5\\%$ $\\rightarrow$ multiply by $1.05$\n", + "- Return $-3\\%$ $\\rightarrow$ multiply by $0.97$\n", "\n", "$V_t$ is the value of your $\\$1$ at the end of month $t$. If $V_t = 1.50$, your $\\$1$ has grown to $\\$1.50$.\n", "\n",