This commit is contained in:
Pawel Sarkowicz
2026-07-03 08:41:22 -04:00
parent d57af3dec3
commit 09b65787a6
6 changed files with 31 additions and 31 deletions
@@ -9,13 +9,13 @@
"\n",
"## Purpose\n",
"\n",
"Notebook 02 established that momentum was the only factor with positive, persistant IC. The other three have negative or insignicant IC with our price-based proxies.\n",
"Notebook 02 established that momentum was the only factor with positive, persistent IC. The other three have negative or insignificant IC with our price-based proxies.\n",
"\n",
"In this notebook, our goals are the following.\n",
"1. **Build the headline signal**: sector-neutralized momentum. We winsoirze, z-score, and project orthogonal to the sector subspace. The same pipeline that would apply to any factor, but applied to the one that works. \n",
"1. **Build the headline signal**: sector-neutralized momentum. We winsorize, z-score, and project orthogonal to the sector subspace. The same pipeline that would apply to any factor, but applied to the one that works. \n",
"2. **Tests whether combining helps**: we build a 4-factor equal-weight composite and compare its IC to momentum alone. If the composite IC is worse (it is), that confirms the decision to trade based on momentum alone is the right one. \n",
"\n",
"### Terms uesd in this notebook. \n",
"### Terms used in this notebook \n",
"\n",
"| Term | Meaning |\n",
"|------|---------|\n",
@@ -33,7 +33,7 @@
"\n",
"## Outputs\n",
"\n",
"- `momentum_signal.csv` — the **momentum-only** sector-neutralized signal (this is what will backtest trades)\n",
"- `momentum_signal.csv` — the **momentum-only** sector-neutralized signal (this is what the backtest trades)\n",
"- `composite_4factor.csv` — the 4-factor equal weight composite (kept for comparison)\n",
"- Per-factor neutralized exposures\n",
"\n",
@@ -125,7 +125,7 @@
"source": [
"## Winsorization and Z-Scoring\n",
"\n",
"**Winsorization** is a statiscal technique for limiting the influence of extreme outlies in a dataset by capping them at a specified threshold rather than removing them entirely. Raw factor exposures can have extreme outliers — a stock that returned 500% in the trailing year, for instance. We **Winsorize** at ±3 standard deviations, so that we don't have to drop them.\n",
"**Winsorization** is a statistical technique for limiting the influence of extreme outliers in a dataset by capping them at a specified threshold rather than removing them entirely. Raw factor exposures can have extreme outliers — a stock that returned 500% in the trailing year, for instance. We **Winsorize** at ±3 standard deviations, so that we don't have to drop them.\n",
"\n",
"After winsorization we **z-score** cross-sectionally: for each date $t$, the factor vector $f_t$ is transformed to\n",
"$$\\tilde{f}_t = (f_t - \\bar{f}_t) / \\text{std}(f_t),$$\n",
@@ -194,13 +194,13 @@
"\n",
"From the linear algebraists' perspective, the sector matrix $D \\in \\{0,1\\}^{N \\times K}$ spans a subspace $S \\subseteq \\mathbb{R}^N$. The orthogonal projection onto $S$ is:\n",
"$$ P = D(D^TD)^{-1}D^T. $$\n",
"This is the familiar ordinary least-squares projection matrix, which is symmetric and idempotent. Applying to to a factor vector $f$ gives the component of $f$ which lies in the sector subspace:\n",
"This is the familiar ordinary least-squares projection matrix, which is symmetric and idempotent. Applying it to a factor vector $f$ gives the component of $f$ which lies in the sector subspace:\n",
"$$ \\hat{f} = Pf. $$\n",
"The **neutralized factor** is the residual, i.e., the component orthogonal to $S$:\n",
"$$ f_\\perp = (I - P)f = f - \\hat{f}. $$\n",
"This is exactly linear squares regression of $f$ on sectors, returning the residuals. The residual is orthogonal to the sector subspace by construction: $\\langle f_\\perp, \\hat{f} \\rangle = 0$. \n",
"This is exactly linear least squares regression of $f$ on sectors, returning the residuals. The residual is orthogonal to the sector subspace by construction: $\\langle f_\\perp, \\hat{f} \\rangle = 0$. \n",
"\n",
"For size, we use the log of tralining market cap proxied by price $\\times 1$ (best we could do with out data). This is a weak proxy — a real implementation would use actual market cap."
"For size, we use the log of trailing market cap proxied by price $\\times 1$ (best we could do with our data). This is a weak proxy — a real implementation would use actual market cap."
]
},
{
@@ -268,7 +268,7 @@
"id": "9740141a",
"metadata": {},
"source": [
"Much like in Notebook 02, we note that we added a sample size filter with `mask.sum() < 30`. The `mask` identifies stocks that have both a valid factor score and a valid forward return in a given month, and `mask.sum()` counts how many usable pairs you actually have. The `< 30` threshold prevents the code from computing a correlation on a tiny sample, which is statistically meaningless and numerically unstable (e.g., Spearman correlation on 2 stocks is always exactly ±1). If you don't gate this, early-history months, mass delistings, or data gaps inject garbage ±1.0 values into your IC time series, which then contaminate every downstream statistic like the mean IC, Information Ratio, and decay curves. Setting a floor of 20 filters out those degenerate months while retaining enough valid data to produce a reliable signal.\n",
"Much like in Notebook 02, we note that we added a sample size filter with `mask.sum() < 30`. The `mask` identifies stocks that have both a valid factor score and a valid forward return in a given month, and `mask.sum()` counts how many usable pairs you actually have. The `< 30` threshold prevents the code from computing a correlation on a tiny sample, which is statistically meaningless and numerically unstable (e.g., Spearman correlation on 2 stocks is always exactly ±1). If you don't gate this, early-history months, mass delistings, or data gaps inject garbage ±1.0 values into your IC time series, which then contaminate every downstream statistic like the mean IC, Information Ratio, and decay curves. Setting a floor of 30 filters out those degenerate months while retaining enough valid data to produce a reliable signal.\n",
"\n",
"We do this sort of masking throughout."
]