{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "a36f5610",
   "metadata": {},
   "source": [
    "# Factor Analysis and Diagnostics"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d09530ec",
   "metadata": {},
   "source": [
    "## Purpose\n",
    "\n",
    "A **factor** or **signal** is a vector of scores for the stocks available at a date:\n",
    "\n",
    "$$f_t \\in \\mathbb{R}^{N_t}.$$\n",
    "\n",
    "The question in this notebook is simple: when we rank stocks by $f_t$, do the higher-ranked stocks tend to have better subsequent returns?\n",
    "\n",
    "In practice we measure this with the **information coefficient (IC)**, a Spearman rank correlation. After ranks are centered, a correlation is also a cosine similarity:\n",
    "\n",
    "$$\\rho(u,v)=\\frac{\\langle u,v\\rangle}{\\|u\\|\\|v\\|}=\\cos\\theta.$$\n",
    "\n",
    "So the IC is a geometric question: does the signal vector point roughly in the same direction as the return vector?\n",
    "\n",
    "We test four common factor ideas using data we can build from prices alone:\n",
    "1. Momentum\n",
    "2. A crude value proxy\n",
    "3. A crude quality proxy\n",
    "4. Low volatility\n",
    "\n",
    "Momentum is the only one that looks useful in this dataset. That is not a universal statement about factor investing; it mostly tells us that the other three proxies are too crude for this public-price-only setup.\n",
    "\n",
    "### Timing convention\n",
    "\n",
    "The raw factor formulas use `.shift(1)`, so the score at a given return date is based only on information available before that return was realized. That is why the IC code can pair `factor.loc[date]` with `returns.loc[date]` without looking ahead.\n",
    "\n",
    "## Terms used\n",
    "\n",
    "| Term | Meaning |\n",
    "|------|---------|\n",
    "| **Factor / signal** | A vector $f_t \\in \\mathbb{R}^{N_t}$ assigning one score per stock |\n",
    "| **Momentum** | Trailing 12-1 return: recent winners may keep winning |\n",
    "| **Value** | Cheap stocks may outperform; here proxied crudely by inverse long-term return |\n",
    "| **Quality** | Profitable/stable firms may outperform; here proxied by a rolling Sharpe-like ratio |\n",
    "| **Low volatility** | Lower-risk stocks may outperform on a risk-adjusted basis |\n",
    "| **Information coefficient (IC)** | Spearman rank correlation between signal ranks and subsequent return ranks |\n",
    "| **Information ratio (IR)** | Mean IC / std(IC), annualized here by $\\sqrt{12}$ |\n",
    "| **Rank** | Cross-sectional ordering of stocks; useful because it is robust to outliers |\n",
    "| **Turnover proxy** | How much the signal changes; here approximated with rank autocorrelation |\n",
    "| **Walk-forward** | Split into subperiods to see whether a result is stable across regimes |\n",
    "| **Return panel** $R$ ↻ | The monthly return matrix from notebook 01 |\n",
    "\n",
    "## Outputs\n",
    "\n",
    "This notebook writes factor exposure CSVs, monthly IC series, IC decay, subperiod diagnostics, and a cross-factor correlation matrix.\n",
    "\n",
    "## Notebook Structure\n",
    "1. [Setup and Imports](#setup-and-imports)\n",
    "2. [Load Data](#load-data)\n",
    "3. [Factor Definitions](#factor-definitions)\n",
    "4. [Information Coefficient Analysis](#information-coefficient-analysis)\n",
    "5. [Subperiod IC Stability (Walk-Forward)](#subperiod-ic-stability-walk-forward)\n",
    "6. [Factor Decay](#factor-decay)\n",
    "7. [Turnover via Rank Autocorrelation](#turnover-via-rank-autocorrelation)\n",
    "8. [Cross-Factor Correlations](#cross-factor-correlations)\n",
    "9. [Conclusion](#conclusion)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff97ddb4",
   "metadata": {},
   "source": [
    "## Setup and Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "b6109fc9",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-07-31T11:08:33.898987Z",
     "iopub.status.busy": "2026-07-31T11:08:33.898337Z",
     "iopub.status.idle": "2026-07-31T11:08:34.945114Z",
     "shell.execute_reply": "2026-07-31T11:08:34.944563Z"
    }
   },
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "==================================\n",
    "Setup and imports\n",
    "==================================\n",
    "\"\"\"\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "import os\n",
    "\n",
    "pd.set_option('display.max_columns', None)\n",
    "pd.set_option('display.max_rows', 100)\n",
    "\n",
    "palette = ['steelblue', 'coral', 'seagreen']\n",
    "\n",
    "os.makedirs('../data/processed', exist_ok=True)\n",
    "os.makedirs('../images/02_factor_diagnostics', exist_ok=True)\n",
    "\n",
    "RANDOM_STATE = 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "aebef262",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-07-31T11:08:34.947030Z",
     "iopub.status.busy": "2026-07-31T11:08:34.946720Z",
     "iopub.status.idle": "2026-07-31T11:08:35.015122Z",
     "shell.execute_reply": "2026-07-31T11:08:35.014620Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Returns: (251, 501)\n",
      "Prices: (252, 503)\n",
      "Sectors: 11\n"
     ]
    }
   ],
   "source": [
    "\"\"\"\n",
    "==================================\n",
    "Load return panel and sector mapping\n",
    "==================================\n",
    "\"\"\"\n",
    "df_returns = pd.read_csv('../data/processed/returns_monthly.csv', index_col=0, parse_dates=True)\n",
    "df_prices = pd.read_csv('../data/processed/prices_monthly.csv', index_col=0, parse_dates=True)\n",
    "df_sector = pd.read_csv('../data/processed/sector_mapping.csv')\n",
    "\n",
    "print(f\"Returns: {df_returns.shape}\")\n",
    "print(f\"Prices: {df_prices.shape}\")\n",
    "print(f\"Sectors: {df_sector['sector'].nunique()}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac67fd55",
   "metadata": {},
   "source": [
    "## Factor Definitions\n",
    "\n",
    "We compute four cross-sectional factor matrices. Each matrix has the same shape as `df_returns`: one row per month and one column per stock. Each entry is a percentile rank centered around zero, so the scores live on roughly the same scale.\n",
    "\n",
    "The factors are:\n",
    "\n",
    "- **Momentum:** trailing 12-1 return. In code this is an 11-month rolling sum shifted by one month, so the most recent month is skipped.\n",
    "- **Value proxy:** negative 60-month trailing return. This is not true book-to-market value; it is a rough price-only mean-reversion proxy.\n",
    "- **Quality proxy:** 12-month mean return divided by 12-month volatility. This is closer to a recent-return quality proxy than a true profitability or balance-sheet quality measure.\n",
    "- **Low-volatility:** negative 60-month trailing volatility, ranked so lower-vol names score higher.\n",
    "\n",
    "The caveat matters: momentum is a fairly standard price signal, but value and quality are usually built from fundamentals. Here they are learning proxies, not production-grade definitions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "dbc1fa9f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-07-31T11:08:35.016900Z",
     "iopub.status.busy": "2026-07-31T11:08:35.016702Z",
     "iopub.status.idle": "2026-07-31T11:08:35.316517Z",
     "shell.execute_reply": "2026-07-31T11:08:35.315893Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "momentum: (251, 501), non-null months: 240\n",
      "value: (251, 501), non-null months: 191\n",
      "quality: (251, 501), non-null months: 239\n",
      "lowvol: (251, 501), non-null months: 191\n"
     ]
    }
   ],
   "source": [
    "\"\"\"\n",
    "==================================\n",
    "Compute factor exposures\n",
    "==================================\n",
    "\"\"\"\n",
    "def cross_sectional_rank(series):\n",
    "    \"\"\"Rank cross-sectionally, scaled to [-0.5,0.5]\"\"\"\n",
    "    return series.rank(pct=True) - 0.5\n",
    "\n",
    "# Momentum (12-1): sum of returns from t-12 to t-2\n",
    "mom_12_1 = df_returns.rolling(11).sum().shift(1)\n",
    "df_momentum = mom_12_1.apply(cross_sectional_rank, axis=1)\n",
    "\n",
    "# Value (proxy): inverse 60-month momentum\n",
    "mom_60 = df_returns.rolling(60).sum().shift(1)\n",
    "df_value = (-mom_60).apply(cross_sectional_rank, axis=1)\n",
    "\n",
    "# Quality (proxy): 12m sharpe-like\n",
    "mean_12 = df_returns.rolling(12).mean().shift(1)\n",
    "std_12 = df_returns.rolling(12).std().shift(1)\n",
    "quality_raw = mean_12 / std_12.replace(0, np.nan)\n",
    "df_quality = quality_raw.apply(cross_sectional_rank, axis=1)\n",
    "\n",
    "# Low volatility: inverse 60m vol\n",
    "vol_60 = df_returns.rolling(60).std().shift(1)\n",
    "df_lowvol = (-vol_60).apply(cross_sectional_rank, axis=1)\n",
    "\n",
    "\n",
    "factor_dict = {\n",
    "    'momentum': df_momentum,\n",
    "    'value': df_value,\n",
    "    'quality': df_quality,\n",
    "    'lowvol': df_lowvol,\n",
    "}\n",
    "\n",
    "for name, df_f in factor_dict.items():\n",
    "    print(f\"{name}: {df_f.shape}, non-null months: {df_f.notna().any(axis=1).sum()}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9ea22e1",
   "metadata": {},
   "source": [
    "## Information Coefficient Analysis\n",
    "\n",
    "The **information coefficient (IC)** is the Spearman rank correlation between a factor vector and the return vector it is meant to predict. Since Spearman correlation is Pearson correlation applied to ranks, the IC can be read as a cosine similarity between centered rank vectors:\n",
    "\n",
    "$$\\rho(u,v)=\\frac{\\langle u,v\\rangle}{\\|u\\|\\|v\\|}=\\cos\\theta.$$\n",
    "\n",
    "- IC near $+1$: the factor ranking and return ranking are almost perfectly aligned.\n",
    "- IC near $0$: the factor is not directionally useful in that month.\n",
    "- IC near $-1$: the factor points the wrong way.\n",
    "\n",
    "The **IC information ratio** annualizes the signal-to-noise ratio of the monthly IC series:\n",
    "\n",
    "$$\\text{IC IR}=\\frac{\\text{mean monthly IC}}{\\text{std monthly IC}}\\sqrt{12}.$$\n",
    "\n",
    "A small positive IC can still matter if it is stable, but a tiny IC with a noisy sign should be treated as weak evidence, not a discovery."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "a7146b0c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-07-31T11:08:35.318513Z",
     "iopub.status.busy": "2026-07-31T11:08:35.318301Z",
     "iopub.status.idle": "2026-07-31T11:08:36.607209Z",
     "shell.execute_reply": "2026-07-31T11:08:36.606630Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Computing IC for momentum... \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Computing IC for value... \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Computing IC for quality... \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Computing IC for lowvol... \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "IC Summary\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>min</th>\n",
       "      <th>max</th>\n",
       "      <th>ic_ir</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>momentum</th>\n",
       "      <td>0.0060</td>\n",
       "      <td>0.1905</td>\n",
       "      <td>-0.6010</td>\n",
       "      <td>0.4032</td>\n",
       "      <td>0.1091</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>value</th>\n",
       "      <td>-0.0216</td>\n",
       "      <td>0.1576</td>\n",
       "      <td>-0.3878</td>\n",
       "      <td>0.6297</td>\n",
       "      <td>-0.4750</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>quality</th>\n",
       "      <td>-0.0025</td>\n",
       "      <td>0.1945</td>\n",
       "      <td>-0.5120</td>\n",
       "      <td>0.4798</td>\n",
       "      <td>-0.0448</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>lowvol</th>\n",
       "      <td>-0.0259</td>\n",
       "      <td>0.2360</td>\n",
       "      <td>-0.5288</td>\n",
       "      <td>0.5478</td>\n",
       "      <td>-0.3808</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            mean     std     min     max   ic_ir\n",
       "momentum  0.0060  0.1905 -0.6010  0.4032  0.1091\n",
       "value    -0.0216  0.1576 -0.3878  0.6297 -0.4750\n",
       "quality  -0.0025  0.1945 -0.5120  0.4798 -0.0448\n",
       "lowvol   -0.0259  0.2360 -0.5288  0.5478 -0.3808"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "\"\"\"\n",
    "==================================\n",
    "Compute monthly IC (Spearman rank correlation)\n",
    "==================================\n",
    "\"\"\"\n",
    "from scipy.stats import spearmanr\n",
    "\n",
    "def compute_monthly_ic(factor_df, return_df):\n",
    "    \"\"\"Compute Spearman IC using shifted signals and aligned return rows.\"\"\"\n",
    "    common_dates = factor_df.index.intersection(return_df.index)\n",
    "    common_tickers = factor_df.columns.intersection(return_df.columns)\n",
    "\n",
    "    ic_series = []\n",
    "    for date in common_dates:\n",
    "        f = factor_df.loc[date, common_tickers]\n",
    "        r = return_df.loc[date, common_tickers]\n",
    "\n",
    "        mask = f.notna() & r.notna()\n",
    "        if mask.sum() < 20: # sample size filter\n",
    "            ic_series.append(np.nan)\n",
    "            continue\n",
    "        \n",
    "        ic, _ = spearmanr(f[mask], r[mask])\n",
    "        ic_series.append(ic)\n",
    "    return pd.Series(ic_series, index=common_dates)\n",
    "\n",
    "ic_results = {}\n",
    "for name, df_f in factor_dict.items():\n",
    "    print(f\"Computing IC for {name}... \")\n",
    "    ic_results[name] = compute_monthly_ic(df_f,df_returns)\n",
    "\n",
    "df_ic = pd.DataFrame(ic_results)\n",
    "df_ic = df_ic.dropna(how='all')\n",
    "\n",
    "print(\"\\nIC Summary\\n\")\n",
    "ic_summary = df_ic.describe().T[['mean', 'std', 'min', 'max']]\n",
    "ic_summary['ic_ir'] = ic_summary['mean']/ic_summary['std'] * np.sqrt(12)\n",
    "display(ic_summary.round(4))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5659b941",
   "metadata": {},
   "source": [
    "The `mask.sum() < 20` rule is a sample-size guardrail. The `mask` keeps only stocks with both a valid factor score and a valid return for that date. If only a handful of stocks are available, a rank correlation can become mechanically extreme; with two stocks, Spearman correlation is always $+1$ or $-1$.\n",
    "\n",
    "The threshold drops those degenerate months before they leak into mean IC, IC IR, and decay statistics. We use the same kind of guardrail throughout the project."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "d0fe3ee3",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-07-31T11:08:36.609261Z",
     "iopub.status.busy": "2026-07-31T11:08:36.609030Z",
     "iopub.status.idle": "2026-07-31T11:08:39.039239Z",
     "shell.execute_reply": "2026-07-31T11:08:39.038702Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 178168 characters]",
      "text/plain": [
       "<Figure size 1400x1200 with 4 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "\"\"\"\n",
    "==================================\n",
    "Plot IC time series\n",
    "==================================\n",
    "\"\"\"\n",
    "fig, axes = plt.subplots(4,1, figsize=(14,12), sharex = True)\n",
    "\n",
    "for ax, (name, ic_series) in zip(axes, ic_results.items()):\n",
    "    ax.bar(ic_series.index, ic_series.values, color='steelblue', alpha=0.6, width=20)\n",
    "    ax.plot(ic_series.index, ic_series.rolling(12).mean(), color='coral', linewidth=2, label='12m  MA')\n",
    "    ax.axhline(y=0, color='black', linestyle='-', linewidth=0.5)\n",
    "    ax.set_ylabel('IC')\n",
    "    ax.set_title(f'{name.capitalize()} Factor IC')\n",
    "    ax.legend(loc='upper right')\n",
    "    ax.grid(alpha=0.3)\n",
    "\n",
    "axes[-1].set_xlabel('Date')\n",
    "plt.tight_layout()\n",
    "plt.savefig('../images/02_factor_diagnostics/ic_time_series.png', dpi=150, bbox_inches='tight')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2cce2714",
   "metadata": {},
   "source": [
    "## Subperiod IC Stability (Walk-Forward)\n",
    "\n",
    "A positive full-sample IC can hide a fragile result. Maybe the signal worked in one window and did nothing elsewhere. Walk-forward analysis splits the sample into non-overlapping windows and recomputes the diagnostics in each one.\n",
    "\n",
    "This is not parameter tuning. It is just a stability check: does the sign and size of the signal look similar across regimes, or is the full-sample average doing too much storytelling?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "fba556cd",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-07-31T11:08:39.040999Z",
     "iopub.status.busy": "2026-07-31T11:08:39.040823Z",
     "iopub.status.idle": "2026-07-31T11:08:39.075634Z",
     "shell.execute_reply": "2026-07-31T11:08:39.075169Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mean IC by Subperiod\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>period</th>\n",
       "      <th>2006-2011</th>\n",
       "      <th>2011-2016</th>\n",
       "      <th>2016-2021</th>\n",
       "      <th>2021-2026</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>factor</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>lowvol</th>\n",
       "      <td>NaN</td>\n",
       "      <td>-0.0029</td>\n",
       "      <td>-0.0361</td>\n",
       "      <td>-0.0295</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>momentum</th>\n",
       "      <td>-0.0131</td>\n",
       "      <td>0.0276</td>\n",
       "      <td>-0.0079</td>\n",
       "      <td>0.0174</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>quality</th>\n",
       "      <td>-0.0287</td>\n",
       "      <td>0.0258</td>\n",
       "      <td>-0.0258</td>\n",
       "      <td>0.0183</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>value</th>\n",
       "      <td>NaN</td>\n",
       "      <td>-0.0179</td>\n",
       "      <td>-0.0331</td>\n",
       "      <td>-0.0095</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "period    2006-2011  2011-2016  2016-2021  2021-2026\n",
       "factor                                              \n",
       "lowvol          NaN    -0.0029    -0.0361    -0.0295\n",
       "momentum    -0.0131     0.0276    -0.0079     0.0174\n",
       "quality     -0.0287     0.0258    -0.0258     0.0183\n",
       "value           NaN    -0.0179    -0.0331    -0.0095"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " Information Ratio by Subperiod\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>period</th>\n",
       "      <th>2006-2011</th>\n",
       "      <th>2011-2016</th>\n",
       "      <th>2016-2021</th>\n",
       "      <th>2021-2026</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>factor</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>lowvol</th>\n",
       "      <td>NaN</td>\n",
       "      <td>-0.043</td>\n",
       "      <td>-0.560</td>\n",
       "      <td>-0.420</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>momentum</th>\n",
       "      <td>-0.237</td>\n",
       "      <td>0.547</td>\n",
       "      <td>-0.130</td>\n",
       "      <td>0.325</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>quality</th>\n",
       "      <td>-0.593</td>\n",
       "      <td>0.461</td>\n",
       "      <td>-0.390</td>\n",
       "      <td>0.353</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>value</th>\n",
       "      <td>NaN</td>\n",
       "      <td>-0.576</td>\n",
       "      <td>-0.676</td>\n",
       "      <td>-0.170</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "period    2006-2011  2011-2016  2016-2021  2021-2026\n",
       "factor                                              \n",
       "lowvol          NaN     -0.043     -0.560     -0.420\n",
       "momentum     -0.237      0.547     -0.130      0.325\n",
       "quality      -0.593      0.461     -0.390      0.353\n",
       "value           NaN     -0.576     -0.676     -0.170"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " % Months with Positive IC\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>period</th>\n",
       "      <th>2006-2011</th>\n",
       "      <th>2011-2016</th>\n",
       "      <th>2016-2021</th>\n",
       "      <th>2021-2026</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>factor</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>lowvol</th>\n",
       "      <td>NaN</td>\n",
       "      <td>51.7</td>\n",
       "      <td>43.3</td>\n",
       "      <td>41.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>momentum</th>\n",
       "      <td>48.3</td>\n",
       "      <td>61.7</td>\n",
       "      <td>45.0</td>\n",
       "      <td>58.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>quality</th>\n",
       "      <td>47.5</td>\n",
       "      <td>63.3</td>\n",
       "      <td>43.3</td>\n",
       "      <td>60.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>value</th>\n",
       "      <td>NaN</td>\n",
       "      <td>40.0</td>\n",
       "      <td>46.7</td>\n",
       "      <td>48.3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "period    2006-2011  2011-2016  2016-2021  2021-2026\n",
       "factor                                              \n",
       "lowvol          NaN       51.7       43.3       41.7\n",
       "momentum       48.3       61.7       45.0       58.3\n",
       "quality        47.5       63.3       43.3       60.0\n",
       "value           NaN       40.0       46.7       48.3"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "\"\"\"\n",
    "==================================\n",
    "Subperiod IC stability (walk-forward)\n",
    "==================================\n",
    "\"\"\"\n",
    "windows = [(2006,2011), (2011,2016),(2016,2021),(2021,2026)]\n",
    "subperiod_results = []\n",
    "\n",
    "for name in factor_dict:\n",
    "    ic = ic_results[name]\n",
    "    for start, end in windows:\n",
    "        mask = (ic.index.year >= start) & (ic.index.year < end)\n",
    "        sub = ic[mask].dropna()\n",
    "        if len(sub) < 12:\n",
    "            continue\n",
    "        subperiod_results.append({\n",
    "            'factor': name,\n",
    "            'period': f'{start}-{end}',\n",
    "            'mean_ic': sub.mean(),\n",
    "            'std_ic': sub.std(),\n",
    "            'ir': sub.mean() / sub.std() * np.sqrt(12) if sub.std() > 0 else np.nan,\n",
    "            'n': len(sub),\n",
    "            'pct_positive': (sub > 0).mean()\n",
    "        })\n",
    "\n",
    "df_subperiod = pd.DataFrame(subperiod_results)\n",
    "\n",
    "# Pivot for display\n",
    "pivot_ic = df_subperiod.pivot(index='factor', columns='period', values='mean_ic')\n",
    "pivot_ir = df_subperiod.pivot(index='factor', columns='period', values='ir')\n",
    "pivot_pct = df_subperiod.pivot(index='factor', columns='period', values='pct_positive')\n",
    "\n",
    "print(\"Mean IC by Subperiod\\n\")\n",
    "display(pivot_ic.round(4))\n",
    "\n",
    "print(\"\\n Information Ratio by Subperiod\\n\")\n",
    "display(pivot_ir.round(3))\n",
    "\n",
    "print(\"\\n % Months with Positive IC\\n\")\n",
    "display((pivot_pct * 100).round(1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "7b0bc01a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-07-31T11:08:39.077211Z",
     "iopub.status.busy": "2026-07-31T11:08:39.077036Z",
     "iopub.status.idle": "2026-07-31T11:08:39.399850Z",
     "shell.execute_reply": "2026-07-31T11:08:39.399204Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 33204 characters]",
      "text/plain": [
       "<Figure size 1200x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot subperiod IC\n",
    "fig,ax = plt.subplots(figsize=(12,6))\n",
    "x = np.arange(len(windows))\n",
    "width = 0.2\n",
    "colors = ['steelblue', 'coral', 'seagreen', 'goldenrod']\n",
    "for i, name in enumerate(factor_dict):\n",
    "    vals = [pivot_ic.loc[name, f'{s}-{e}'] if f'{s}-{e}' in pivot_ic.columns else 0 for s, e in windows]\n",
    "    ax.bar(x + i * width, vals, width, label=name, color=colors[i], alpha=0.8)\n",
    "\n",
    "ax.set_xticks(x + width * 1.5)\n",
    "ax.set_xticklabels([f'{s} - {e}' for s,e in windows])\n",
    "ax.set_ylabel('Mean monthly IC')\n",
    "ax.set_title('Factor IC by Subperiod')\n",
    "ax.axhline(y=0, color='black', linewidth=0.5)\n",
    "ax.legend()\n",
    "ax.grid(alpha=0.3, axis='y')\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.savefig('../images/02_factor_diagnostics/ic_subperiod.png', dpi=150, bbox_inches='tight')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c6cc2c9",
   "metadata": {},
   "source": [
    "## Factor Decay\n",
    "\n",
    "A one-month IC tells us whether a signal is useful right after it is measured. IC decay asks how long that usefulness lasts.\n",
    "\n",
    "For a horizon $h$, we compare today's factor ranks to cumulative returns over the next $h$ months. If the IC stays positive as $h$ grows, the signal is slow-moving and may tolerate less frequent rebalancing. If it drops quickly, the signal needs faster trading and is more exposed to transaction costs.\n",
    "\n",
    "In vector language, a slow signal keeps a small angle between $\\mathrm{rank}(f_t)$ and $\\mathrm{rank}(r_{t:t+h})$ for several horizons. A fast signal loses that alignment quickly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "29d1c1d8",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-07-31T11:08:39.401596Z",
     "iopub.status.busy": "2026-07-31T11:08:39.401424Z",
     "iopub.status.idle": "2026-07-31T11:08:45.405229Z",
     "shell.execute_reply": "2026-07-31T11:08:45.404664Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 73840 characters]",
      "text/plain": [
       "<Figure size 1000x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>1m</th>\n",
       "      <th>3m</th>\n",
       "      <th>6m</th>\n",
       "      <th>12m</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>momentum</th>\n",
       "      <td>0.0015</td>\n",
       "      <td>-0.0023</td>\n",
       "      <td>-0.0032</td>\n",
       "      <td>-0.0048</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>value</th>\n",
       "      <td>-0.0229</td>\n",
       "      <td>-0.0400</td>\n",
       "      <td>-0.0463</td>\n",
       "      <td>-0.0458</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>quality</th>\n",
       "      <td>-0.0041</td>\n",
       "      <td>-0.0128</td>\n",
       "      <td>-0.0167</td>\n",
       "      <td>-0.0251</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>lowvol</th>\n",
       "      <td>-0.0251</td>\n",
       "      <td>-0.0621</td>\n",
       "      <td>-0.0873</td>\n",
       "      <td>-0.1108</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              1m      3m      6m     12m\n",
       "momentum  0.0015 -0.0023 -0.0032 -0.0048\n",
       "value    -0.0229 -0.0400 -0.0463 -0.0458\n",
       "quality  -0.0041 -0.0128 -0.0167 -0.0251\n",
       "lowvol   -0.0251 -0.0621 -0.0873 -0.1108"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "\"\"\"\n",
    "==================================\n",
    "IC decay at horizons 1, 3, 6, 12 months\n",
    "==================================\n",
    "\"\"\"\n",
    "horizons = [1,3,6,12]\n",
    "decay_results = {}\n",
    "\n",
    "for name, df_f in factor_dict.items():\n",
    "    decay_results[name] = []\n",
    "    for h in horizons:\n",
    "        # Forward returns at horizon h\n",
    "        fwd_returns = df_returns.rolling(h).sum().shift(-h)\n",
    "        ic_series = compute_monthly_ic(df_f, fwd_returns)\n",
    "        decay_results[name].append(ic_series.mean())\n",
    "\n",
    "df_decay = pd.DataFrame(decay_results, index=[f'{h}m' for h in horizons]).T\n",
    "\n",
    "fig,ax = plt.subplots(figsize=(10,6))\n",
    "df_decay.T.plot(ax=ax, marker='o')\n",
    "ax.set_xlabel('Horizon')\n",
    "ax.set_ylabel('Mean IC')\n",
    "ax.set_title('Factor IC Decay by Horizon')\n",
    "ax.axhline(y=0, color='black', linestyle='--', linewidth=0.5)\n",
    "ax.grid(alpha=0.3)\n",
    "ax.legend(title='Factor')\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.savefig('../images/02_factor_diagnostics/ic_decay.png', dpi=150, bbox_inches='tight')\n",
    "plt.show()\n",
    "\n",
    "display(df_decay.round(4))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d0205e12",
   "metadata": {},
   "source": [
    "## Turnover via Rank Autocorrelation\n",
    "\n",
    "A factor that re-ranks the universe every month will produce high turnover, and this contributes to lowering net returns after transaction costs.\n",
    "\n",
    "**Rank autocorrelation** is the correlation between this month's factor rank vector and last month's. If the rank vector barely changes (autocorrelation $\\approx$ 1), the portfolio barely trades. If it changes a lot (autocorrelation $\\approx$ 0), turnover is high. In linear algebraic terms, the rank vector at $t$ is nearly parallel to the rank vector at $t-1$ for stable factors, and nearly orthogonal for unstable ones."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "65a97db6",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-07-31T11:08:45.407354Z",
     "iopub.status.busy": "2026-07-31T11:08:45.407075Z",
     "iopub.status.idle": "2026-07-31T11:08:46.547766Z",
     "shell.execute_reply": "2026-07-31T11:08:46.547146Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "momentum: rank autocorr = 0.888, turnover proxy = 0.112\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "value: rank autocorr = 0.977, turnover proxy = 0.023\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "quality: rank autocorr = 0.898, turnover proxy = 0.102\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "lowvol: rank autocorr = 0.997, turnover proxy = 0.003\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 26684 characters]",
      "text/plain": [
       "<Figure size 800x500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "\"\"\"\n",
    "==================================\n",
    "Rank autocorrelation as turnover proxy\n",
    "==================================\n",
    "\"\"\"\n",
    "def rank_autocorrelation(factor_df, lag=1):\n",
    "    \"\"\"Averages cross-sectional rank correlation between t and t-lag.\"\"\"\n",
    "    common_dates = factor_df.index[:-lag]\n",
    "    autocorr = []\n",
    "    for date in common_dates:\n",
    "        f_now = factor_df.loc[date]\n",
    "        f_lag = factor_df.loc[factor_df.index[factor_df.index.get_loc(date) + lag]]\n",
    "        mask = f_now.notna() & f_lag.notna()\n",
    "        if mask.sum() < 20:\n",
    "            continue\n",
    "        rho,_ = spearmanr(f_now[mask], f_lag[mask])\n",
    "        autocorr.append(rho)\n",
    "    return np.mean(autocorr)\n",
    "\n",
    "turnover_proxy = {}\n",
    "for name, df_f in factor_dict.items():\n",
    "    rho = rank_autocorrelation(df_f, lag=1)\n",
    "    turnover_proxy[name] = rho\n",
    "    # Rough turnover estimate: 1 - rho\n",
    "    print(f\"{name}: rank autocorr = {rho:.3f}, turnover proxy = {1 - rho:.3f}\")\n",
    "\n",
    "fig,ax = plt.subplots(figsize=(8,5))\n",
    "ax.bar(turnover_proxy.keys(), [1-v for v in turnover_proxy.values()], color='coral')\n",
    "ax.set_ylabel('Turnover proxy (1 - Rank Autocorr)')\n",
    "ax.set_title('Factor Turnover Proxy')\n",
    "ax.grid(alpha=0.3, axis='y')\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.savefig('../images/02_factor_diagnostics/turnover_proxy.png', dpi=150, bbox_inches='tight')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ac614b4",
   "metadata": {},
   "source": [
    "## Cross-Factor Correlations\n",
    "\n",
    "The cross-factor correlation matrix asks whether the factors are actually different from each other.\n",
    "\n",
    "If we stack factor vectors into a matrix\n",
    "\n",
    "$$F_t = [f_{1,t}, f_{2,t}, f_{3,t}, f_{4,t}],$$\n",
    "\n",
    "then a Gram matrix has entries\n",
    "\n",
    "$$G_{ij}=\\langle f_{i,t}, f_{j,t}\\rangle.$$\n",
    "\n",
    "Because our factors are centered ranks, these inner products are proportional to correlations. A high off-diagonal entry means two factors are nearly collinear and probably redundant. A value near zero means the factors are closer to orthogonal and may diversify each other.\n",
    "\n",
    "This matters because combining four factor names is not the same as combining four independent sources of information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "2d488e92",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-07-31T11:08:46.549585Z",
     "iopub.status.busy": "2026-07-31T11:08:46.549416Z",
     "iopub.status.idle": "2026-07-31T11:08:49.782467Z",
     "shell.execute_reply": "2026-07-31T11:08:49.781755Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 45292 characters]",
      "text/plain": [
       "<Figure size 800x600 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>momentum</th>\n",
       "      <th>value</th>\n",
       "      <th>quality</th>\n",
       "      <th>lowvol</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>momentum</th>\n",
       "      <td>1.000</td>\n",
       "      <td>-0.444</td>\n",
       "      <td>0.860</td>\n",
       "      <td>-0.104</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>value</th>\n",
       "      <td>-0.444</td>\n",
       "      <td>1.000</td>\n",
       "      <td>-0.412</td>\n",
       "      <td>0.210</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>quality</th>\n",
       "      <td>0.860</td>\n",
       "      <td>-0.412</td>\n",
       "      <td>1.000</td>\n",
       "      <td>0.109</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>lowvol</th>\n",
       "      <td>-0.104</td>\n",
       "      <td>0.210</td>\n",
       "      <td>0.109</td>\n",
       "      <td>1.000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          momentum  value  quality  lowvol\n",
       "momentum     1.000 -0.444    0.860  -0.104\n",
       "value       -0.444  1.000   -0.412   0.210\n",
       "quality      0.860 -0.412    1.000   0.109\n",
       "lowvol      -0.104  0.210    0.109   1.000"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "\"\"\"\n",
    "==================================\n",
    "Cross-factor rank correlation (time-averaged)\n",
    "==================================\n",
    "\"\"\"\n",
    "factor_names = list(factor_dict.keys())\n",
    "corr_matrix = np.zeros((len(factor_names), len(factor_names)))\n",
    "\n",
    "for i, name_i in enumerate(factor_names):\n",
    "    for j, name_j in enumerate(factor_names):\n",
    "        if i == j:\n",
    "            corr_matrix[i,j] = 1.0\n",
    "            continue\n",
    "        df_i = factor_dict[name_i]\n",
    "        df_j = factor_dict[name_j]\n",
    "        common_dates = df_i.index.intersection(df_j.index)\n",
    "\n",
    "        corrs = []\n",
    "        for date in common_dates:\n",
    "            a = df_i.loc[date]\n",
    "            b = df_j.loc[date]\n",
    "            mask = a.notna() & b.notna()\n",
    "            if mask.sum() < 20:\n",
    "                continue\n",
    "            rho, _ = spearmanr(a[mask], b[mask])\n",
    "            corrs.append(rho)\n",
    "        corr_matrix[i,j] = np.mean(corrs)\n",
    "\n",
    "df_corr = pd.DataFrame(corr_matrix, index=factor_names, columns=factor_names)\n",
    "\n",
    "fig,ax = plt.subplots(figsize=(8,6))\n",
    "sns.heatmap(df_corr, annot=True, fmt='.2f', cmap='RdBu_r', center=0, ax=ax, square=True)\n",
    "ax.set_title('Cross-Factor Rank Correlation (Time-Averaged)')\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.savefig('../images/02_factor_diagnostics/factor_correlation.png', dpi=150, bbox_inches='tight')\n",
    "plt.show()\n",
    "\n",
    "display(df_corr.round(3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "f5fea3fa",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-07-31T11:08:49.784322Z",
     "iopub.status.busy": "2026-07-31T11:08:49.784108Z",
     "iopub.status.idle": "2026-07-31T11:08:50.390890Z",
     "shell.execute_reply": "2026-07-31T11:08:50.390284Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Saved factor exposures, IC series, decay, and correlation matrix.\n"
     ]
    }
   ],
   "source": [
    "\"\"\"\n",
    "==================================\n",
    "Save factor exposures and IC results\n",
    "==================================\n",
    "\"\"\"\n",
    "for name, df_f in factor_dict.items():\n",
    "    df_f.to_csv(f'../data/processed/factor_{name}.csv')\n",
    "\n",
    "df_ic.to_csv('../data/processed/ic_monthly.csv')\n",
    "df_decay.to_csv('../data/processed/ic_decay.csv')\n",
    "df_corr.to_csv('../data/processed/factor_correlation.csv')\n",
    "\n",
    "print(\"Saved factor exposures, IC series, decay, and correlation matrix.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a76a4424",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "\n",
    "The diagnostics point to one usable signal in this price-only setup: **momentum**. Its full-sample IC is small but positive, and it is more stable than the other proxies. That is enough to justify carrying it forward, but not enough to pretend we have found a large or universal edge.\n",
    "\n",
    "The other three proxies are mostly useful as negative examples:\n",
    "\n",
    "- **Value:** the inverse 60-month return proxy behaves more like a bet against long-term winners than a true value measure. Without fundamentals, it cannot distinguish cheapness from price drift.\n",
    "- **Quality:** the rolling Sharpe proxy is highly correlated with momentum, so it is not adding much independent information.\n",
    "- **Low-volatility:** the signal is stable, but in this sample it points the wrong way.\n",
    "\n",
    "The factor correlation matrix explains why a naive composite is unlikely to help. Momentum and quality are close to the same direction, value is partly the opposite direction, and low-vol is more independent but has negative IC. The next notebook builds the signal we will actually trade: sector-neutralized momentum."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.14.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
