434 lines
99 KiB
Plaintext
434 lines
99 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "f385cc40",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n",
|
|
"# A note on other norms\n",
|
|
"\n",
|
|
"There are other canonical choices of norms for vectors and matrices. While $L^2$ leads naturally to least-squares problems with closed-form solutions, other norms induce different geometries and different optimal solutions. From the linear algebra perspective, changing the norm affects:\n",
|
|
"- the shape of the unit ball,\n",
|
|
"- the geometry of approximation,\n",
|
|
"- the numerical behaviour of optimization problems. "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ca50a202",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n",
|
|
"## $L^1$ norm (Manhattan distance)\n",
|
|
"The $L^1$ norm of a vector $x = (x_1,\\dots,x_p) \\in \\mathbb{R}^p$ is defined as\n",
|
|
"$$ \\|x\\|_1 = \\sum |x_i|. $$\n",
|
|
"Minimizing the $L^1$ norm is less sensitive to outliers. Geometrically, the $L^1$ unit ball in $\\mathbb{R}^2$ is a diamond (a rotated square), rather than a circle.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"id": "77e7c0b3",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAkgAAAJNCAYAAADK04ocAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQAANz1JREFUeJzt3X90VPWd//HXQH4ASkZLJAkaA24lqPgDYYHQ8qNVo1GpovUn5YetnoJrWaGIIqsGW0VdpeBRtAqoXXH9BdhuQZRtCXhKUNCkIiDQUyRZTQzw1QREAyH3+wfOmLkzJDPJ3Lm/no9zck5zuZN84jT49N73vTdgGIYhAAAAhHWyewEAAABOQyABAACYEEgAAAAmBBIAAIAJgQQAAGBCIAEAAJgQSAAAACYEEgAAgAmBBAAAYEIgAQAAmBBIAAAAJgQS4CBPPfWUzj//fKWnp6u0tNTu5QCAbxFIgIPk5eVp9uzZuvLKK+1eCgD4WprdCwDwnVAY/fGPf7R7KQDgaxxBAlLkT3/6kwKBgF566SW7l9Km559/XoFAQJ988kl42/r161VaWqovv/wyrq9RWlqqQCCgvXv3WramWNsS1bt373afzuzIa1P5fVKxzv3792vGjBkqLi7WSSedpEAgwGliuBqBBKTIBx98IEkaMGCA3Utp02WXXaby8nLl5eWFt61fv16zZ8+OO5DgL/v27dMzzzyjxsZGThHDEzjFBqTIBx98oOOOO06FhYV2L6VNJ510kk466SS7l+FJo0aNUu/evfX888/bvZS4NTc367333tObb76pm266Sb17947ap6CgQF988UX4qOHChQttWSuQLBxBAlLkgw8+0LnnnqtOnZL7azdx4sSY/8IKneKKtW3Lli264YYbFAwGlZOTo5///Oeqr68P72c+dVVaWqo77rhDktSnTx8FAgEFAgGVlZW1ub7q6mpdddVVysrKUjAY1M9+9jPt2bMn/Of/+Mc/dNNNN+n0009Xt27ddPLJJ2v06NHavHlzh/65tFdNTY2OP/54XX/99RHb//znPys9PV2zZs2yZV1mV111lU455ZSo7U1NTTrvvPN00UUXdejr79mzRy+++KLGjh2rnj17qqioSE8++aSamppi7h/6/wTgFQQSkAJ79uzRp59+qvPPP7/V/ZqamvTNN9/oyJEjEf872a6++mr17dtXS5cu1V133aWXXnpJU6dOPeb+N998s371q19JkpYtW6by8nKVl5e3+fNI0pgxY/T9739fr7/+ukpLS/XGG2/o4osv1uHDhyVJn332mXr06KGHHnpIq1at0pNPPqm0tDQNGTJE27dvb9fPFwgENGrUqHa9Ni8vTzNmzNCrr76q999/X5JUVlama665RpMnT9YDDzzQrq+bbCNGjNCnn36q3bt3R2yfO3euPv74Yy1YsCChr9fc3Kx3331XpaWlGjx4sHJycjR+/Hht375dkydP1vr161VXV6fvf//7Sf5JAGfiFBuQAqF/0bY1f/Tb3/5Ws2fPDn/+wAMP6LnnntPEiROTup5f/OIX4SNCF154of7xj39o8eLFWrRoUcyjAKeccopOPfXU8M8Q64jVsVx11VV65JFHJEnFxcXKycnR2LFj9eqrr2rs2LEaMWKERowYEd7/yJEjuuyyy3TWWWfp97//vebOnZvwz9e5c2d17tw54deFTJ8+Xb///e915513as6cOfrJT36iG264QfPnz0/o6xiGERW4hmHIMIyoIzFpaYn9dRz6Z7Z+/XoVFBRIknbt2qXZs2dr1qxZOv300+P+WpWVlbrwwgu1b98+nXDCCSouLtatt96qkpIS5eTkJLQuwCs4ggSkQGhAu60jLqWlpeF/gYY+kh1HkvSTn/wk4vNzzjlH33zzjerq6pL+vcaOHRvx+bXXXqu0tDStWbNG+vao2YMPPqgzzzxTGRkZSktLU0ZGhnbu3Klt27a163s2NTXpL3/5S7vX3K1bN/32t7/VX/7yF/3oRz9SSUmJnn322YRPIa1du1bp6ekRH+vWrdMf/vCHqO2JXol33nnnKSsrS3/729/C2yZPnqz8/HzdeeedCX2tzp07q1u3blKLo5ihD8CvOIIEpMAHH3ygzMxMnXXWWXYvRZLUo0ePiM8zMzMlSV9//XXSv1dubm7E52lpaerRo4f27dsnSZo2bZqefPJJ3XnnnRo5cqROPPFEderUSTfffLMl64lX3759pW9P1z3//PPtOiI1cOBAbdy4MWLbL3/5S/Xq1Uv33XdfxPZevXol9LU7deqkYcOGaf369ZKkJUuW6K233tKaNWuUkZGR0Nc6++yzVVVVpY8++khvvvmmVq5cqSlTpujw4cM688wzVVJSopKSEg0fPjzhrw24FYEEpMAHH3yg/v37Kz09Pelfu0uXLmpsbIzanqz7D3VUbW2tTj755PDnTU1N2rdvXzjSXnzxRY0fP14PPvhgxOv27t2rE044IeXr1bennC6//HL94Ac/0N/+9jctXrxY//Zv/5bw1+nevbsGDRoUta1Hjx5R29tjxIgRuueee1RVVaVp06ZpwoQJ7Z69kqT+/furf//+uuOOO7R//3797//+r95880298soreuyxx3T88cfr/fffD8cj4GWcYgMs9uWXX2rXrl1xDTS3R+/evVVXV6fPP/88vO3QoUN66623kvp92nuUacmSJRGfv/rqq2pqagr/izwQCIS/dsiKFSv06aefdnjN7bF9+3ZdfPHFKioq0po1a3TFFVeotLQ04io/pxgxYoSOHDmiyy+/XEeOHNGjjz6atK/dvXt3jRkzRs8884yqq6v14Ycf6p577lGXLl2S9j0AJ+MIEmCx0PzRkSNH9MYbb0T9+fDhw6NOeSXiuuuu07333qvrr79ed9xxh7755hs9/vjjSb/67eyzz5YkzZ8/XxMmTFB6eroKCwvVvXv3Vl+3bNkypaWl6aKLLtKWLVt0zz336Nxzz9W1114rSbr88sv1/PPPq1+/fjrnnHP0/vvv6z//8z9jXsIer7S0NI0cOTLhOaRPPvlEF154oQoLC7V06VKlp6froYceUv/+/fXggw/q4YcfbvearPCv//qv6tq1qzZv3qzFixcrOzu7XV+npqZGK1asaHO/733vezr++OOP+edvvvmmvvrqK+3fv1+StHXrVr3++uuSpEsvvTQ85wS4AYEEWKyiokKStHjxYi1evDjqz6uqqjoUSH369NEf//hH3X333frpT3+qvLw8TZs2TXv27Im4Iq6jRo0apZkzZ+qFF17Qs88+q+bmZq1Zs6bNUzrLli1TaWmpnnrqKQUCAY0ePVrz5s0Lz7LMnz9f6enpmjNnjg4cOKDzzz9fy5Yt03/8x3+0e61HjhxJOBBramp04YUXqmfPnvrzn/+srl27SpL69eunn//855o/f74mT56c0BV8VuvUqZNOPPFEDRo0qEPD/Nu3b9ctt9wS176DBg3S9773vZh/Nnny5IjbDrz22mt67bXXpG+vsHPSPzugLQHDMAy7FwEAdundu7cmTpzYrueGdeS1yfg+jz76qGbNmqXKykqdccYZtq8T8BKOIAGAixw8eFB///vftXHjRs2aNUsPPPBAq3EEoH0IJABwkbfffltjxoxRbm6u7r77bk2fPt3uJQGeRCABgItceeWVYjICsB4zSAAAACbcBwkAAMCEQAIAADDx3AxSc3OzPvvsM3Xv3j3hB0sCAADvMgxD+/fvV69evdSpU+vHiDwXSJ999pny8/PtXgYAAHCo6urqNu/W77lACj32oLq6WllZWXYvB4CNDh5q0uAHjj5u5L1ZF6hbhuf+ygOQgIaGBuXn57f5iCR5MZBCp9WysrIIJMDn0g41qVPm0ed/ZWVlEUgApBat0BqGtAEAAEwIJAAAABMCCQAAwIRAAgAAMCGQAAAATAgkAAAAEwIJAADAhEACAAAwIZAAAABMCCQAAAATAgkAAMCEQAIAADAhkAAAAEwIJAAAABMCCQAAwIRAAgAAMCGQAAAATAgkAAAAEwIJAADAhEACAAAwIZAAAABMCCQAAAATAgkAAMCEQAIAADAhkAAAAEwIJAAAABMCCQAAwIRAAgAAMCGQAAAATAgkAAAAEwIJAADAhEACAAAwIZAAAABMCCQAAAATAgkAAMCEQAIAADAhkAAAAEwsDaR169Zp9OjR6tWrlwKBgN544402X7N27VoNHDhQXbp00Wmnnaann37ayiUCAABEsTSQvvrqK5177rl64okn4tp/165duvTSSzV8+HBVVFTo7rvv1pQpU7R06VIrlwkAABAhzcovXlJSopKSkrj3f/rpp3Xqqadq3rx5kqQzzjhDmzZt0qOPPqqrr7465msaGxvV2NgY/ryhoSEJKwfgBXuq94b/9+FDh6UMS//KA+AhjppBKi8vV3FxccS2iy++WJs2bdLhw4djvmbOnDkKBoPhj/z8/BStFoCTbd2wQ5MG3Rn+/LfX/U6Hvjlk65oAuIejAqm2tlY5OTkR23JyctTU1KS9e/fGfM3MmTNVX18f/qiurk7RagE41dYNO3THRffr0MHvguiD/92s+8Y8QiQBiIujAkmSAoFAxOeGYcTcHpKZmamsrKyIDwD+FY6jrxqVP6h3eHt6lzRteuvvRBKAuDgqkHJzc1VbWxuxra6uTmlpaerRo4dt6wLgDls37NCM4qNxdOqQ03Tlkz8L/9mYJ8cpvWs6kQQgLo4KpKKiIq1evTpi29tvv61BgwYpPT3dtnUBcL7QkaPGA0fj6KfPTlRG14zwn+f/ax9ds/AmIglAXCwNpAMHDqiyslKVlZXSt5fxV1ZWqqqqSvp2fmj8+PHh/SdNmqTdu3dr2rRp2rZtmxYvXqxFixZp+vTpVi4TgMuZjxz99NmJyuiWEbXfqUNOI5IAxMXSQNq0aZMGDBigAQMGSJKmTZumAQMG6N5775Uk1dTUhGNJkvr06aOVK1eqrKxM5513nn7zm9/o8ccfP+Yl/gAQiqPGA43KH9znmHEUQiQBiEfACE1Be0RDQ4OCwaDq6+sZ2AY8LmIge/DRU2gt46jxcLNufW6HJGnBTX2Vmf7dfxNWvftPvXbzczr89WENuvhczV4+Qxldjh1WANwvkUZw1AwSAMSrrThqC0eSALSGQALgOh2NoxAiCcCxEEgAXKXlQHZH4iiESAIQC4EEwDVaXsqfjDgKIZIAmBFIAFyh5Wm1UNAkI45CiCQALRFIABzPHEdtXcrfXkQSgBACCYCjpSqOQogkACKQADhZquMohEgCQCABcCS74iiESAL8jUAC4Dh2x1EIkQT4F4EEwFHMN4G0K45CiCTAnwgkAI6RrDtkJxuRBPgPgQTAEZwaRyFEEuAvBBIA2zk9jkKIJMA/CCQAtnJLHIUQSYA/EEgAbOO2OAohkgDvI5AA2MKtcRRCJAHeRiABSDm3x1EIkQR4F4EEIKW8EkchRBLgTQQSgJQx3yHb7XEUQiQB3kMgAUgJpzw+xCpEEuAtBBIAy3k9jkKIJMA7CCQAlvJLHIUQSYA3EEgALOO3OAohkgD3I5AAWGLrhh2aUey/OAohkgB3I5AAJF3oyFHjAX/GUQiRBLgXgQQgqcz3OfJrHIUQSYA7EUgAksZrN4FMFiIJcB8CCUBSEEetI5IAdyGQAHQYcRQfIglwDwIJQId49fEhViGSAHcgkAC0m1/vc9RRRBLgfAQSgHYhjjqGSAKcjUACkDDiKDmIJMC5CCQACSGOkotIApyJQAIQN+LIGkQS4DwEEoC4EEfWIpIAZyGQALSJx4ekBpEEOAeBBKBV3OcotYgkwBkIJADHxJEjexBJgP0IJAAx8fgQexFJgL0IJABRiCNnIJIA+xBIACIQR85CJAH2IJAAhG3dsEMziokjpyGSgNQjkABILeKo8QBx5EREEpBaBBKA8Gk14sjZiCQgdQgkwOeYOXIXIglIDQIJ8DFuAulORBJgPQIJ8CluAuluRBJgLQIJ8CGOHHkDkQRYh0ACfMYcRxw5cjciCbAGgQT4CHHkTUQSkHwEEuATxJG3EUlAchFIgA8wkO0PRBKQPAQS4HHc58hfiCQgOQgkwMOII38ikoCOI5AAj+LBs/5GJAEdQyABHsSDZyEiCegQAgnwGB48i5aIJKB9CCTAQ5g5QixEEpA4AgnwCOIIrSGSgMQQSIAHEEeIB5EExI9AAlyOB88iEUQSEB8CCXAx7pCN9iCSgLYRSIBLceQIHUEkAa0jkAAX4sGzSAYiCTg2AglwGU6rIZmIJCA2AglwEa5WgxWIJCAagQS4BHEEKxFJQCQCCXABHjyLVCCSgO8QSIDD8eBZpBKRBBxFIAEORhzBDkQSQCABjhWaOSKOYAciCX5HIAEOxE0g4QREEvyMQAIchptAwkmIJPgVgQQ4CHEEJyKS4EcEEuAQxBGcjEiC3xBIgAMQR3ADIgl+QiABNmt5E0jiCE5HJMEvCCTARi3vc0QcwS2IJPgBgQTYpOV9jogjuA2RBK8jkAAbMHMELzh1yGm6ZhGRBG8ikIAUI47gJacO5kgSvIlAAlKoZRzlD+5DHMETON0GLyKQgBQxxxGPD4GXEEnwGgIJSAHiCH5AJMFLCCTAYjx4Fn5CJMErCCTAQgxkw4+IJHgBgQRYhDiCnxFJcDsCCbAAcQQQSXA3AglIMp6tBnyHSIJbEUhAEvH4ECAakQQ3IpCAJOG0GnBsRBLchkACkoA7ZANtI5LgJgQS0EHcBBKIH5EEtyCQgA4gjoDEEUlwAwIJaCfiCGg/IglOZ3kgLViwQH369FGXLl00cOBAvfPOO8fct6ysTIFAIOrj448/tnqZQEKII6DjiCQ4maWB9Morr+j222/XrFmzVFFRoeHDh6ukpERVVVWtvm779u2qqakJf5x++ulWLhNICHEEJA+RBKeyNJDmzp2rX/ziF7r55pt1xhlnaN68ecrPz9dTTz3V6ut69uyp3Nzc8Efnzp2PuW9jY6MaGhoiPgCrEEdA8hFJcCLLAunQoUN6//33VVxcHLG9uLhY69evb/W1AwYMUF5eni644AKtWbOm1X3nzJmjYDAY/sjPz0/K+gEz4giwDpEEp7EskPbu3asjR44oJycnYntOTo5qa2tjviYvL0/PPPOMli5dqmXLlqmwsFAXXHCB1q1bd8zvM3PmTNXX14c/qqurk/6zAMQRYD0iCU6SZvU3CAQCEZ8bhhG1LaSwsFCFhYXhz4uKilRdXa1HH31UI0aMiPmazMxMZWZmJnnVwHe4QzaQOqFIeu3m58KRNHv5DGV04XcOqWXZEaTs7Gx17tw56mhRXV1d1FGl1gwdOlQ7d+60YIVA24gjIPU4kgQnsCyQMjIyNHDgQK1evTpi++rVqzVs2LC4v05FRYXy8vIsWCHQuq0bdmhGMXEE2IFIgt0svYpt2rRpWrhwoRYvXqxt27Zp6tSpqqqq0qRJk6Rv54fGjx8f3n/evHl64403tHPnTm3ZskUzZ87U0qVLddttt1m5TCBKKI4aDxBHgF2IJNjJ0hmk6667Tvv27dP999+vmpoa9e/fXytXrlRBQYEkqaamJuKeSIcOHdL06dP16aefqmvXrjrrrLO0YsUKXXrppVYuE4jAaTXAOZhJgl0ChmEYdi8imRoaGhQMBlVfX6+srCy7lwOXIY68pfFws259bockacFNfZWZztOV3Krq3X/qtZuf0+GvD2vQxecSSWiXRBqBvy2AbxFHgHOdOuQ0XbOI021IHQIJiHGfI+IIcJ5TBzOThNQhkOB73AQScA8Gt5EqBBJ8jTgC3IdIQioQSPAt4ghwLyIJViOQ4EvEEeB+RBKsRCDBd8xXqxFHgHsRSbAKgQRf4VJ+wHuIJFiBQIJvEEeAdxFJSDYCCb5AHAHeRyQhmQgkeB5xBPgHkYRkIZDgacQR4D9EEpKBQIJnEUeAfxFJ6CgCCZ7Es9UAEEnoCAIJnsN9jgCEEEloLwIJnsJpNQBmRBLag0CCZxBHAI6FSEKiCCR4AjNHANpCJCERBBJcjwfPAogXkYR4EUhwNeIIQKKIJMSDQIJrEUcA2otIQlsIJLgSl/ID6CgiCa0hkOA6XK0GIFmIJBwLgQRX2bphh2YUE0cAkodIQiwEElwjdOSo8QBxBCC5iCSYEUhwBe5zBMBqRBJaIpDgeAxkA0gVIgkhBBIcjYFsAKlGJEEEEpyMOAJgFyIJBBIciTgCYDciyd8IJDgOcQTAKYgk/yKQ4ChcrQbAaYgkfyKQ4BgtbwLJs9UAOAmR5D8EEhzBfBNI4giA0xBJ/kIgwXacVgPgFkSSfxBIsJU5jjhyBMDpiCR/IJBgG+6QDcCtiCTvI5BgCy7lB+B2RJK3EUhIOeIIgFcQSd5FICGliCMAXkMkeROBhJQhjgB4FZHkPQQSUoJL+QF4HZHkLQQSLMel/AD8gkjyDgIJliKOAPgNkeQNBBIsQxwB8Csiyf0IJFiCOALgd0SSuxFISDriCACOIpLci0BCUvH4EACIRCS5E4GEpOE+RwAQG5HkPgQSkoI4AoDWEUnuQiChw4gjAIgPkeQeBBI6hDgCgMQQSe5AIKHdiCMAaB8iyfkIJLQLcQQAHUMkORuBhIRt3bBDM4qJIwDoKCLJuQgkJCQUR40Hjt4EkjgCgI4hkpyJQELczHHETSABIDmIJOchkBCX0MwRcQQA1iCSnIVAQpt4thoApAaR5BwEElpFHAFAahFJzkAg4ZiIIwCwB5FkPwIJMZkv5SeOACC1iCR7EUiI0nIgm/scAYB9iCT7EEiIwB2yAcBZiCR7EEgII44AwJmIpNQjkCARRwDgeERSahFIII4AwCWIpNQhkHyOOAIAdyGSUoNA8rGWl/Lz4FkAcA8iyXoEkk/x4FkAcDciyVoEkg/x4FkA8AYiyToEks/w+BAA8BYiyRoEko8QRwDgTURS8hFIPmEeyCaOAMBbiKTkIpB8gIFsAPAHIil5CCSPYyAbAPyFSEoOAsnDzDeBJI4AwB+IpI4jkDzKPJDNTSABwF+IpI4hkDyII0cAABFJHUIgeQzPVgMAtEQktQ+B5CEtL+UnjgAAIURS4ggkjzBfyk8cAQBaIpISQyB5QMtL+Zk5AgAcC5EUPwLJ5bhaDQCQCCIpPgSSi/FsNQBAexBJbSOQXIo4AgB0BJHUOgLJhbjPEQAgGYikYyOQXIaZIwBAMhFJsRFILsJpNQCAFYikaASSS3BaDQBgJSIpEoHkAjw+BACQCkTSdwgkhyOOAACpRCQdRSA5GHEEALADkUQgORYPngUA2MnvkUQgOVDLB88SRwAAu/g5kggkhzE/eJY4AgDYya+RRCA5CDNHAAAn8mMkWR5ICxYsUJ8+fdSlSxcNHDhQ77zzTqv7r127VgMHDlSXLl102mmn6emnn7Z6iY7QcuaIO2QDAJzGb5FkaSC98soruv322zVr1ixVVFRo+PDhKikpUVVVVcz9d+3apUsvvVTDhw9XRUWF7r77bk2ZMkVLly61cpm2a3lajTtkAwCcyk+RFDAMw7Dqiw8ZMkTnn3++nnrqqfC2M844Q1deeaXmzJkTtf+dd96pP/3pT9q2bVt426RJk/T3v/9d5eXlcX3PhoYGBYNB1ezZp6ysrCT9JNap2vappo66V4cOHtIpAws0ZsE4ZXQljoBkaDzcrKkv/kOS9LuffV+Z6UwVAMlQvXGXlv/bf+nwN00afOkA3fPKNLuXFJeGhgblndRD9fX1bTaCZYF06NAhdevWTa+99prGjBkT3v7v//7vqqys1Nq1a6NeM2LECA0YMEDz588Pb1u+fLmuvfZaHTx4UOnp6VGvaWxsVGNjY/jzhoYG5efnK//2V9Ups5sVPxoAAHCh5saDqp53bVyBZNl/Tu3du1dHjhxRTk5OxPacnBzV1tbGfE1tbW3M/ZuamrR3796Yr5kzZ46CwWD4Iz8/P4k/BQAA8KM0q79BIBCI+NwwjKhtbe0fa3vIzJkzNW3ad4f2QkeQ3pt1gStOsdV+UqcpI+/V1//vK/Xsl6efPjtBXYMc+QKSgVNsgDV2/nWb/vzrV9R8pFk/vvGHmvr7X9q9pLg0NDQob158+1oWSNnZ2ercuXPU0aK6urqoo0Qhubm5MfdPS0tTjx49Yr4mMzNTmZmZUdu7ZaSpW4bl/ddhp/XtpSf/cp+mjLxXezdXa9lNi3TDH25W1xOPs3tpgKdkpncikIAk2LF6i1b++xKpqVkX3PAD3fnsJHVO62z3suLSlEAXWPa3RUZGhgYOHKjVq1dHbF+9erWGDRsW8zVFRUVR+7/99tsaNGhQzPkjryg4M1+Pr71fx2Ufr7ptNXp5wiJ9/eVBu5cFAECEHau36I1fLVFzU7N+dMMPdOcLv3JNHCXK0v+cmjZtmhYuXKjFixdr27Ztmjp1qqqqqjRp0iTp29Nj48ePD+8/adIk7d69W9OmTdO2bdu0ePFiLVq0SNOnT7dymY5QcGa+5pfdr249jtfnWz/Ty+MXEkkAAMfwUxzJ6kC67rrrNG/ePN1///0677zztG7dOq1cuVIFBQWSpJqamoh7IvXp00crV65UWVmZzjvvPP3mN7/R448/rquvvtrKZTpG6EgSkQQAcBK/xZGsvg+SHUL3QYrnEj6n2r21WlNG3quD+w6o5xl5zCQB7dR4uFm3PrdDkrTgpr7MIAHt4KU4SqQR+NvCgVoeSarbVqP/Hr9QX3/xld3LAgD4jJfiKFEEkkOZI4nBbQBAKvk5jkQgOZt5Jum/xz3LkSQAgOX8HkcikJyPI0kAgFQijo4ikFyAq9sAAKlAHH2HQHIJIgkAYCXiKBKB5CLMJAEArNAyjn584w99H0cikNyHWwAAAJLJfORoxvO3+T6ORCC5E5EEAEgGTqsdG4HkUlzdBgDoCE6rtY5AcjFmkgAA7WGOI06rRSOQXI4jSQCARDBzFB8CyQNi3gKAI0kAABNmjuJHIHlEVCRxJAkA0AJxlBgCyUOYSQIAxMJAduIIJI9hJgkA0BIzR+1DIHkQR5IAAOK0WocQSB7FkSQA8DfiqGMIJA/jSBIA+BNx1HEEksfxWBIA8BfiKDkIJB/gdBsA+ANxlDwEkk9wug0AvI1L+ZOLQPIRjiQBgDfxbLXkI5B8JuaRJCIJAFyL+xxZg0DyoagjSeMXEkkA4ELMHFmHQPIpZpIAwN2II2sRSD4WiqTjsrkFAAC4CQPZ1iOQfK7gzHzNL2NwGwDcgpmj1CCQwOA2ALgEp9VSh0CCFOuO28wkAYCjEEepRSAhjMeSAIAzMXOUegQSInAzSQBwFmaO7EEgIQozSQDgDJxWsw+BhJiYSQIAe3FazV4EEo4p5uk2IgkALMdpNfsRSGiV+XQbM0kAYC2OHDkDgYQ28VgSAEgNcxxx5Mg+BBLiwtVtAGAtTqs5C4GEuHEkCQCswdVqzkMgISEcSQKA5GLmyJkIJCSMI0kAkBycVnMuAgntwmNJAKBjOK3mbAQS2i0UScdlE0kAkAjiyPkIJHRIwZn5ml/GTBIAxIs4cgcCCR3GTBIAxIc4cg8CCUnBTBIAtI6r1dyFQELScAsAAIiNq9Xch0BCUnG6DQAiceTInQgkJB1HkgDgKJ6t5l4EEiwR80gSkQTARzit5m4EEiwTdSRp/EIiCYAvcLWa+xFIsBQzSQD8hpkjbyCQYDluAQDAL5g58g4CCSlhfiwJg9sAvIaZI28hkJAyoceSHJd99HQbM0kAvILTat5DICGlWj67jZkkAF7AkSNvIpCQcubTbcwkAXArjhx5F4EEW7Q8ksRMEgA34siRtxFIsA03kwTgVtznyPsIJNiKm0kCcBviyB8IJNiOm0kCcAtmjvyDQIIjxHzALZEEwEG4CaS/EEhwDPORJAa3ATgFA9n+QyDBUaIiiZkkADZj5sifCCQ4DjNJAJyCOPIvAgmOxANuAdiNgWx/I5DgWEQSALswkA0CCY4W8+o2ZpIAWIiBbIhAghu0fHYbM0kArMTMEUIIJLgCz24DYDVmjtASgQTX4Oo2AFZh5ghmBBJchcFtAMkWFUcvEEcgkOBCRBKAZIl55KgzcQQCCS5FJAHoKE6roTUEElyLWwAAaC8u5UdbCCS4GrcAAJAoLuVHPAgkuB63AAAQL+II8SKQ4AnmWwC8PH4hkQQgAvc5QiIIJHgGkQTgWJg5QqIIJHgKkQTAjCNHaA8CCZ5DJAEI4cgR2otAgifxWBIADGSjIwgkeBY3kwT8i9Nq6CgCCZ7GzSQB/+G0GpKBQILnxTzdRiQBnsRpNSQLgQRfiDrdxkwS4DnEEZKJQIJvxDzdRiQBnkAcIdkIJPhK1C0AmEkCXI+BbFiBQILvcAsAwDsYyIZVCCT4ErcAANyP02qwEoEE3+IWAIB7EUewGoEEX+N0G+A+zBwhFQgk+B6n2wD3YOYIqUIgAUQS4AqcVkMqEUjAt7hPEuBcxBFSjUACWuA+SYDzMHMEOxBIgAmD24BzmOOImSOkCoEExMAtAAD7MZANO1kaSF988YXGjRunYDCoYDCocePG6csvv2z1NRMnTlQgEIj4GDp0qJXLBGLiSBJgH2aOYDdLA+nGG29UZWWlVq1apVWrVqmyslLjxo1r83WXXHKJampqwh8rV660cpnAMXEkCUg94ghOkGbVF962bZtWrVqlDRs2aMiQIZKkZ599VkVFRdq+fbsKCwuP+drMzEzl5ubG9X0aGxvV2NgY/ryhoSEJqwe+E4qkKSPvDR9JuuEPN6vricfZvTTAc4gjOIVlR5DKy8sVDAbDcSRJQ4cOVTAY1Pr161t9bVlZmXr27Km+ffvqlltuUV1d3TH3nTNnTvgUXjAYVH5+flJ/DkDcJwlICeIITmJZINXW1qpnz55R23v27Kna2tpjvq6kpERLlizRX//6Vz322GPauHGjfvzjH0ccJWpp5syZqq+vD39UV1cn9ecAQogkwDrEEZwm4UAqLS2NGqI2f2zatEmSFAgEol5vGEbM7SHXXXedLrvsMvXv31+jR4/Wm2++qR07dmjFihUx98/MzFRWVlbEB2AVIglIPu5zBCdKeAbptttu0/XXX9/qPr1799aHH36ozz//POrP9uzZo5ycnLi/X15engoKCrRz585ElwpYouVMUmhw+/o/3KyuJ3Sze2mA63ApP5wq4UDKzs5WdnZ2m/sVFRWpvr5e7733ngYPHixJevfdd1VfX69hw4bF/f327dun6upq5eXlJbpUwDIMbgMdx2k1OJllM0hnnHGGLrnkEt1yyy3asGGDNmzYoFtuuUWXX355xBVs/fr10/LlyyVJBw4c0PTp01VeXq5PPvlEZWVlGj16tLKzszVmzBirlgq0C6fbgPYjjuB0lt4HacmSJTr77LNVXFys4uJinXPOOfqv//qviH22b9+u+vp6SVLnzp21efNmXXHFFerbt68mTJigvn37qry8XN27d7dyqUC7EElA4pg5ghsEDMMw7F5EMjU0NCgYDKq+vp6BbaTM7q3VmjLyXh3cd0A5Z/ZiJskhGg8369bndkiSFtzUV5npPF3Jbhw5gp0SaQT+tgCSIOZjSbjjNhCBOIKbEEhAkkSdbuPZbUAYcQS3IZCAJGImCYjGzBHciEACkoxIAr7DfY7gVgQSYAFzJL08YREzSfAdjhzBzQgkwCLmwe2Xxy8kkuAbHDmC2xFIgIWIJPgRA9nwAgIJsBiRBD/htBq8gkACUiDmfZIY3IbHmOOI02pwMwIJSBGuboOXMXMEryGQgBQikuBFzBzBiwgkIMVCkXRcNrcAgPsRR/AqAgmwQcGZ+ZpfxkwS3I2BbHgZgQTYhNNtcDNmjuB1BBJgIyIJbsRpNfgBgQTYjEiCmxBH8AsCCXAAIgluwMwR/IRAAhyCSIKTMXMEvyGQAAchkuBEnFaDHxFIgMMQSXASTqvBrwgkwIHMkcTNJGEHnq0GPyOQAIfiAbewEzNH8DsCCXAwjiTBDswcAQQS4HgcSUIqEUfAUQQS4AIcSUIqMJANfIdAAlzCfCTp5fELiSQkDTNHQCQCCXARIglW4LQaEI1AAlyGmSQkE3EExEYgAS7ETBKSgZkj4NgIJMClON2GjuAmkEDrCCTAxWKebiOS0AbiCGgbgQS4XNSz25hJQiu4Wg2ID4EEeAAPuEU8GMgG4kcgAR7B4DZaw0A2kBgCCfAQbgGAWJg5AhJHIAEew5EktEQcAe1DIAEexJEkiIFsoEMIJMCjOJLkb8wcAR1DIAEexpEkf+K0GtBxBBLgcdwCwF84rQYkB4EE+EAoko7L5nSbl3GfIyB5CCTAJwrOzNf8Mk63eRVxBCQXgQT4CIPb3sRANpB8BBLgMwxuewszR4A1CCTAhxjc9gZOqwHWIZAAn+J0m7txWg2wFoEE+Bin29yJ02qA9QgkwOc4kuQunFYDUoNAAhD7SBKR5DjEEZA6BBIAKdbgNqfbHIWZIyC1CCQAYVzd5kw8Ww1IPQIJQARmkpyFgWzAHgQSgChc3eYMzBwB9iGQAMTE6TZ7EUeAvQgkAMdEJNmDgWzAfgQSgFYxk5RaDGQDzkAgAWgT90lKDQayAecgkADEhfskWYuZI8BZCCQAcWMmyRrEEeA8BBKAhDCTlFwMZAPORCABSJh5Junl8QuJpHZg5ghwLgIJQLtwM8mO4bQa4GwEEoB2i3m6jUhqE3EEOB+BBKBDoo4kMbjdKmaOAHcgkAB0GFe3xYebQALuQSABSAoiqXUMZAPuQiABSBoiKTZmjgD3IZAAJBX3SYrEzBHgTgQSgKTjFgBHMXMEuBeBBMASfj/dxswR4G4EEgDL+DWSmDkC3I9AAmApv80kEUeANxBIACznl5kk4gjwDgIJQEp4/UgScQR4C4EEIGXMR5Je9shMEpfyA95DIAFIKa89u41L+QFvIpAApJxXTrdxKT/gXQQSAFvEPN3mokhi5gjwNgIJgG3cGknEEeB9BBIAW7ntFgAMZAP+QCABsJ1b7rjNQDbgHwQSAEdw+uA2A9mAvxBIABwj5uk2B0QSM0eA/xBIABwl6nSbzTNJzBwB/kQgAXCcUCQdl23vTBIzR4B/EUgAHKngzHzNL7NvJomZI8DfCCQAjmXXfZKYOQJAIAFwtFRHEnEEQAQSADdIVSQxkA0ghEAC4ApW33GbgWwALRFIAFzDqjtuE0cAzAgkAK6S7EjiajUAsRBIAFwn5mNJ2hFJDGQDOBYCCYArRc0kJXgkiYFsAK0hkAC4VnsfcMvMEYC2EEgAXC3Rq9uYOQIQDwIJgOvFeySJmSMA8bI0kB544AENGzZM3bp10wknnBDXawzDUGlpqXr16qWuXbtq1KhR2rJli5XLBOABMW8mWf9dJO386zZmjgDEzdJAOnTokK655hpNnjw57tc88sgjmjt3rp544glt3LhRubm5uuiii7R//34rlwrAA8yR9NrNz4f/7M+/foWZIwBxszSQZs+eralTp+rss8+Oa3/DMDRv3jzNmjVLV111lfr3768XXnhBBw8e1EsvvWTlUgF4RMtI2rO9Nry9+QgzRwDi56gZpF27dqm2tlbFxcXhbZmZmRo5cqTWr18f8zWNjY1qaGiI+ADgb6FIKhj2/fC2kdcWcVoNQNwcFUi1tUf/ay8nJydie05OTvjPzObMmaNgMBj+yM/PT8laAThbwZn5euKt+8Kf/3rhZOIIQNwSDqTS0lIFAoFWPzZt2tShRQUCgYjPDcOI2hYyc+ZM1dfXhz+qq6s79L0BeNOx/g4BgFjSEn3Bbbfdpuuvv77VfXr37t2uxeTm5krfHknKy8sLb6+rq4s6qhSSmZmpzMzMdn0/AACAWBIOpOzsbGVnZ1uymD59+ig3N1erV6/WgAEDpG+vhFu7dq0efvhhS74nAACAmaUzSFVVVaqsrFRVVZWOHDmiyspKVVZW6sCBA+F9+vXrp+XLl0vfHgK//fbb9eCDD2r58uX66KOPNHHiRHXr1k033nijlUsFAAAIS/gIUiLuvfdevfDCC+HPQ0eF1qxZo1GjRkmStm/frvr6+vA+M2bM0Ndff61bb71VX3zxhYYMGaK3335b3bt3t3KpAAAAYQHDMAy7F5FMDQ0NCgaDqq+vV1ZWlt3LAWCjg4eadOa9b0mStt5/sbplWPrfhAAcLpFGcNRl/gAAAE5AIAEAAJgQSAAAACYEEgAAgAmBBAAAYEIgAQAAmBBIAAAAJgQSAACACYEEAABgQiABAACYEEgAAAAmBBIAAIAJgQQAAGBCIAEAAJgQSAAAACYEEgAAgAmBBAAAYEIgAQAAmBBIAAAAJgQSAACACYEEAABgQiABAACYEEgAAAAmBBIAAIAJgQQAAGBCIAEAAJgQSAAAACYEEgAAgAmBBAAAYEIgAQAAmBBIAAAAJgQSAACACYEEAABgQiABAACYEEgAAAAmaXYvINkMw5AkNTQ02L0UADY7eKhJzY0HpW//TmjK8NxfeQASEGqDUCu0JmDEs5eL/N///Z/y8/PtXgYAAHCo6upqnXLKKa3u47lAam5u1meffabu3bsrEAjYvZw2NTQ0KD8/X9XV1crKyrJ7OWgD75f78J65C++X+7jpPTMMQ/v371evXr3UqVPrU0aeO97cqVOnNqvQibKyshz/fyx8h/fLfXjP3IX3y33c8p4Fg8G49mNIGwAAwIRAAgAAMCGQbJaZman77rtPmZmZdi8FceD9ch/eM3fh/XIfr75nnhvSBgAA6CiOIAEAAJgQSAAAACYEEgAAgAmBBAAAYEIgAQAAmBBIKfbAAw9o2LBh6tatm0444YS4XmMYhkpLS9WrVy917dpVo0aN0pYtWyxfK4764osvNG7cOAWDQQWDQY0bN05ffvllq6+ZOHGiAoFAxMfQoUNTtma/WbBggfr06aMuXbpo4MCBeuedd1rdf+3atRo4cKC6dOmi0047TU8//XTK1orE3q+ysrKo36VAIKCPP/44pWv2q3Xr1mn06NHq1auXAoGA3njjjTZf45XfLwIpxQ4dOqRrrrlGkydPjvs1jzzyiObOnasnnnhCGzduVG5uri666CLt37/f0rXiqBtvvFGVlZVatWqVVq1apcrKSo0bN67N111yySWqqakJf6xcuTIl6/WbV155RbfffrtmzZqliooKDR8+XCUlJaqqqoq5/65du3TppZdq+PDhqqio0N13360pU6Zo6dKlKV+7HyX6foVs37494vfp9NNPT9ma/eyrr77SueeeqyeeeCKu/T31+2XAFs8995wRDAbb3K+5udnIzc01HnroofC2b775xggGg8bTTz9t8SqxdetWQ5KxYcOG8Lby8nJDkvHxxx8f83UTJkwwrrjiihSt0t8GDx5sTJo0KWJbv379jLvuuivm/jNmzDD69esXse2Xv/ylMXToUEvXiaMSfb/WrFljSDK++OKLFK0QxyLJWL58eav7eOn3iyNIDrdr1y7V1taquLg4vC0zM1MjR47U+vXrbV2bH5SXlysYDGrIkCHhbUOHDlUwGGzzn39ZWZl69uypvn376pZbblFdXV0KVuwvhw4d0vvvvx/x+yFJxcXFx3x/ysvLo/a/+OKLtWnTJh0+fNjS9fpde96vkAEDBigvL08XXHCB1qxZY/FK0V5e+v0ikByutrZWkpSTkxOxPScnJ/xnsE5tba169uwZtb1nz56t/vMvKSnRkiVL9Ne//lWPPfaYNm7cqB//+MdqbGy0eMX+snfvXh05ciSh34/a2tqY+zc1NWnv3r2Wrtfv2vN+5eXl6ZlnntHSpUu1bNkyFRYW6oILLtC6detStGokwku/X2l2L8ALSktLNXv27Fb32bhxowYNGtTu7xEIBCI+NwwjahviF+97phj/7BXHP//rrrsu/L/79++vQYMGqaCgQCtWrNBVV13VobUjWqK/H7H2j7Ud1kjk/SosLFRhYWH486KiIlVXV+vRRx/ViBEjLF8rEueV3y8CKQluu+02XX/99a3u07t373Z97dzcXOnbKs/Lywtvr6uri6p0xC/e9+zDDz/U559/HvVne/bsSeiff15engoKCrRz5852rRexZWdnq3PnzlFHH1r7/cjNzY25f1pamnr06GHpev2uPe9XLEOHDtWLL75owQrRUV76/SKQkiA7O1vZ2dmWfO0+ffooNzdXq1ev1oABA6Rvz+OvXbtWDz/8sCXf0w/ifc+KiopUX1+v9957T4MHD5Ykvfvuu6qvr9ewYcPi/n779u1TdXV1ROSi4zIyMjRw4ECtXr1aY8aMCW9fvXq1rrjiipivKSoq0v/8z/9EbHv77bc1aNAgpaenW75mP2vP+xVLRUUFv0sO5anfL7unxP1m9+7dRkVFhTF79mzj+OOPNyoqKoyKigpj//794X0KCwuNZcuWhT9/6KGHjGAwaCxbtszYvHmzccMNNxh5eXlGQ0ODTT+Fv1xyySXGOeecY5SXlxvl5eXG2WefbVx++eUR+7R8z/bv32/8+te/NtavX2/s2rXLWLNmjVFUVGScfPLJvGcWePnll4309HRj0aJFxtatW43bb7/dOO6444xPPvnEMAzDuOuuu4xx48aF9//nP/9pdOvWzZg6daqxdetWY9GiRUZ6errx+uuv2/hT+Eei79fvfvc7Y/ny5caOHTuMjz76yLjrrrsMScbSpUtt/Cn8Y//+/eF/T0ky5s6da1RUVBi7d+82DI//fhFIKTZhwgRDUtTHmjVrwvtIMp577rnw583NzcZ9991n5ObmGpmZmcaIESOMzZs32/QT+M++ffuMsWPHGt27dze6d+9ujB07NuqS45bv2cGDB43i4mLjpJNOMtLT041TTz3VmDBhglFVVWXTT+B9Tz75pFFQUGBkZGQY559/vrF27drwn02YMMEYOXJkxP5lZWXGgAEDjIyMDKN3797GU089ZcOq/SuR9+vhhx82/uVf/sXo0qWLceKJJxo//OEPjRUrVti0cv8J3WbB/DFhwgTD8PjvV8AITU8BAABA4jJ/AACAaAQSAACACYEEAABgQiABAACYEEgAAAAmBBIAAIAJgQQAAGBCIAEAAJgQSAAAACYEEgAAgAmBBAAAYPL/ARtY0M3OqNo2AAAAAElFTkSuQmCC",
|
|
"text/plain": [
|
|
"<Figure size 600x600 with 1 Axes>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"import numpy as np\n",
|
|
"import matplotlib.pyplot as plt\n",
|
|
"\n",
|
|
"# Grid\n",
|
|
"xx = np.linspace(-1.2, 1.2, 400)\n",
|
|
"yy = np.linspace(-1.2, 1.2, 400)\n",
|
|
"X, Y = np.meshgrid(xx, yy)\n",
|
|
"\n",
|
|
"# Take the $L^1$ norm\n",
|
|
"Z = np.abs(X) + np.abs(Y)\n",
|
|
"\n",
|
|
"plt.figure(figsize=(6,6))\n",
|
|
"plt.contour(X, Y, Z, levels=[1])\n",
|
|
"plt.contourf(X, Y, Z, levels=[0,1], alpha=0.3)\n",
|
|
"\n",
|
|
"plt.axhline(0)\n",
|
|
"plt.axvline(0)\n",
|
|
"plt.gca().set_aspect(\"equal\", adjustable=\"box\")\n",
|
|
"plt.title(r\"$L^1$ unit ball: $|x|+|y|\\leq 1$\")\n",
|
|
"plt.tight_layout()\n",
|
|
"plt.savefig('../images/L1_unit_ball.png')\n",
|
|
"plt.show()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ce59565a",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n",
|
|
"Consequently, optimization problems involving $L^1$ tend to produce solutions which live on the corners of this polytope.\n",
|
|
"Solutions often require linear programming or iterative reweighted least squares.\n",
|
|
"\n",
|
|
"$L^1$ based methods (such as LASSO) tend to set coefficients to be exactly zero. Unlike with $L^2$, the minimization problem for $L^1$ does not admit a closed form solution. Algorithms include:\n",
|
|
"- linear programming formulations,\n",
|
|
"- iterative reweighted least squares,\n",
|
|
"- coordinate descent methods.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "d9c50d8e",
|
|
"metadata": {},
|
|
"source": [
|
|
"## $L^{\\infty}$ norm (max/supremum norm)\n",
|
|
"The supremum norm defined as\n",
|
|
"$$ \\|x\\|_{\\infty} = \\max |x_i| $$\n",
|
|
"seeks to control the worst-case error rather than the average error. Minimizing this norm is related to Chebyshev approximation by polynomials. \n",
|
|
"\n",
|
|
"Geometrically, the unit ball of $\\mathbb{R}^2$ with respect to the $L^{\\infty}$ norm looks like a square.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"id": "2724a3bc",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAkoAAAJOCAYAAABIsiiPAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQAALdxJREFUeJzt3Xl01fWd+P9XIBCkhbggSbAIWAUZUaswCvTL0qpBXKrYUawWsS6teBwGKOLCqNBTRa1F9OdCrSiO2hmtoK1LFcaCekpcgcG6UDtFoUJkqRLcAoHP749CxjR5Q4JcouHxOCfneD95f+593Vxv8uRzP7nJy7IsCwAAamnW2AMAAHxRCSUAgAShBACQIJQAABKEEgBAglACAEgQSgAACUIJACBBKAEAJAglAIAEoQQAkCCU4DN++9vfRl5eXvzqV79Krnn//ffj+9//fpx66qlRXl4ejz32WBx99NFx3XXX7dRZP2v69OmRl5cXb7/9dkREzJs3LyZMmBAffPBBvfafMGFC5OXlxerVq3M20z9ebmquueaa2H///aOgoCBGjx5d6/OdO3eOCRMmbNd1f9H27dKlS7Rp0yZ69+4df/jDH7brurfHunXrYty4cVFaWhp777135OXlbfd9g/oSSvAZ8+fPj4iIww47LLnm+uuvj9GjR8fIkSNj6NChceONN8a9994bmzZtimeeeWYnTvt/jj/++CgrK4uSkpKIzaE0ceLEeocSn8+iRYti/PjxccABB8Rjjz0Wo0aNauyRcmrGjBkxffr0WL58eVx00UU77XbXrFkTd9xxR1RWVsbJJ5+8026XXVt+Yw8AXyTz58+Pr3zlK9GtW7fkmg8//DB69uwZsflf3Mcff3yUlJTEkCFD4ve//30MGDBgJ078d3vvvXfsvffeO/12+bvXXnstIiLGjRsX3/rWtxp7nJw7/PDD4/DDD48//OEPceutt37u69u0aVO8+OKL8bvf/S5+8IMfROfOnetc16lTp3j//ferj37eeeedn/u2YVscUYLPmD9/fhx66KHRrFn6qfHVr341Xn755Vi0aFEsX748br755li1alX85je/iYMOOmir13/22WfX+UNgy0tfdW177bXX4nvf+14UFhZGUVFRnHPOObF27doaaz/7staECRPi4osvjtj8EkleXl7k5eXF3Llzt3n/ly1bFqecckq0bds2CgsL4/vf/36sWrWqxpo///nP8YMf/CAOOOCAaN26deyzzz5x4oknxquvvrrN6/88tnw9Fi1aFKeeemoUFhbGnnvuGWPGjImqqqpYvHhxHHvssdGmTZvo3LlzXH/99Q2e+9NPP43DDjss9t9//xpf4/Ly8iguLo6BAwfGxo0ba81WWVkZERFt2rSp9/1ZsWJFfPWrX43TTz+9xvbHHnssWrRoEePHj2/Q16e+TjnllPja175Wa3tVVVV84xvfiGOOOabe19W2bdtYv379ds2xatWquO++++LMM8+M9u3bR58+feLWW2+Nqqqq5D5b/l+GnUkowWarVq2Kd999Nw4//PCtrhs3blzccsstceWVV8Ydd9wRV111VZx22mnxySef5ORo0ne/+93o2rVrzJgxIy699NL41a9+Vec5MFucd9558a//+q8RETFz5swoKyuLsrKybd6viIghQ4bE/vvvHw899FBMmDAhHnnkkRg0aFBs2LChes3y5ctjr732imuvvTaefPLJuPXWWyM/Pz+OPPLIWLx4cYPvX15eXgwcOLDe60877bQ49NBDY8aMGXH++efHjTfeGKNHj46TTz45jj/++Hj44Yfj29/+dlxyySUxc+bMBs3dqlWrePDBB2PlypVxzjnnRGw+2nHmmWdGlmXxn//5n9G8efMG38e6lJSUxLhx4+LBBx+MV155JSIi5s6dG6eeemqMGDEirr766h1yO/+of//+8e6778Y777xTY/vkyZPjzTffjNtuuy0nt7tp06Z44YUXYsKECXHEEUdEUVFRnHXWWbF48eIYMWJEzJs3L1auXBn7779/Tm4ftpeX3mCzLT+stnZ+UkTEHnvsEdOnT6++3KVLlwb9K7yhzj333OojREcffXT8+c9/jrvuuiumTZtW57+uv/a1r8W+++4bsfm+pF7GqMspp5xSfSSmtLQ0ioqK4swzz4wHH3wwzjzzzIjNP2j79+9fvc/GjRvj+OOPj4MOOih+8YtfxOTJkxt0/5o3b96g+PjhD38YY8aMidj89Zg1a1bccsstMXPmzBgyZEhERAwcODAee+yxuP/+++OUU05p0NwHHHBA3HnnnTF06NC46aab4m9/+1vMnTs3nnzyyepzwP7R3/72t4jNRxsbYuzYsfGLX/wiLrnkkpg0aVJ85zvfie9973tx0003Neh6GmLL12DevHnRqVOniIhYsmRJTJw4sfo8q/pq3bp1xOZfcNhjjz2S6xYuXBhHH310rFmzJnbfffcoLS2NCy+8MAYPHhxFRUWf+z5BLjmiBJttOZG7PkdedqbvfOc7NS4fcsgh8emnn8bKlSt3+G1tiaEtTjvttMjPz485c+ZUb6uqqoprrrkm/umf/ilatmwZ+fn50bJly3jrrbfijTfeaPBtVlVVxdNPP13v9SeccEKNy927d4+8vLwYPHhw9bb8/PzYf//9axw1acjcp512WowYMSIuvvji+OlPfxqXX355nTG8YcOGePPNN+Oee+6JoqKi2G+//Rp031u3bh0//elP4+mnn45vfetbMXjw4PjlL3+Z05eXvvGNb0Tbtm1r/LbaiBEjomPHjnHJJZc06LqOOOKIiM2/8bd69erYtGlTneuaN29eHVVVVVXx6aefVn/AF51Qgs3mz58fBQUF2zzPaGfba6+9alwuKCiIiIhPPvlkh99WcXFxjcv5+fmx1157xZo1a6q3jRkzJq644oo4+eST49FHH40XXnghXnrppTj00ENzMtM/2nPPPWtcbtmyZbRu3TpatWpVa/tnfxA3dO5zzjknNmzYEPn5+TFy5Mg6ZznggAOie/fu8d5778Wjjz4aLVu2bPD96dq1a8TmlyCnT5++w17aS2nWrFn07ds35s2bFxER999/fzz11FMxderUBs8/cODAuPbaa+PnP/957L333tVH+v7RwQcfHEuXLo1XX301rrzyyqioqIiRI0dG586d46CDDoqxY8fG008/vd3nO0EuCSXYbP78+dGjR49o0aJFzm6jVatW1Sf+ftaOfP+iz6O8vLzG5aqqqlizZk2NWLvvvvvirLPOimuuuSYGDRoURxxxRPTq1esLcx9SGjL3Rx99FMOGDYuuXbvGbrvtFuedd16d1/nII4/E/fffH82aNYsf/vCHkWVZg2ZauHBhnHDCCfHNb34zPvzww7jrrru2+/41RP/+/WPRokWxdOnSGDNmTAwfPrxB54lt8dprr8VVV10Vxx9/fMyaNSt+/OMfb3V9jx494uKLL445c+bEmjVrYubMmfHNb34zHnjggTj66KNjr732ij/96U+f457BjieUICI++OCDWLJkSc5fduvcuXOsXLky3nvvvept69evj6eeemqH3s72HnW6//77a1x+8MEHo6qqqsYP0by8vOrr3+Lxxx+Pd99993PNnGsNmfuCCy6IpUuXxsyZM2PatGnx29/+Nm688cZa677xjW/EGWecERdeeGEsXLgw/vKXv9R7nsWLF8egQYOiT58+MWfOnDjppJNiwoQJtX6jMRf69+8fGzdujBNOOCE2btwYN9xww3Zdz6xZs6KysjKmTp0axxxzTHTs2LHe+7Zp0yaGDBkSd9xxRyxbtiwWLVoUV1xxRa0jg9DYnMwNnzk/aePGjfHII4/U+ny/fv1qvQS2PYYOHRpXXnllnH766XHxxRfHp59+GjfffHOdv3L+eRx88MEREXHTTTfF8OHDo0WLFtGtW7dt/vr6zJkzIz8/P4455ph47bXX4oorrohDDz00TjvttOo1J5xwQkyfPj0OPPDAOOSQQ+KVV16Jn/3sZ3X+ynl95Ofnx4ABAxp0ntL2qO/cd955Z9x3331x9913x0EHHRQHHXRQXHTRRXHJJZfEN7/5zerzcj5rn332idgc3PXx9ttvx9FHHx3dunWLGTNmRIsWLeLaa6+NHj16xDXXXFPvd3nPy8uLAQMG1OutHz7rn//5n2O33XaLV199Ne66665o165dg/bfoqKiIiIiOnTosNV1K1asiMcff3yb17fnnntu84T43/3ud/HRRx/FunXrIiLi9ddfj4ceeigiIo477rjqc6FgRxFKEBELFiyIiIi77rqrzpc/li5dukNCqUuXLvGb3/wmLr/88viXf/mXKCkpiTFjxsSqVati4sSJn/v6txg4cGBcdtllcc8998Qvf/nL2LRpU8yZM2ebL6/MnDkzJkyYELfffnvk5eXFiSeeGFOmTKlx7spNN90ULVq0iEmTJsWHH34Yhx9+eMycOTP+/d//fbtm3bhx4w4PxbrUZ+5XX301Ro4cGcOHD4+zzz67evsNN9wQZWVlMXTo0FiwYEHsvvvuNa57y8nX9XnpbcWKFXH00UdH+/bt47HHHovddtstIiIOPPDAOOecc+Kmm26KESNGbPO3FT/88MOIzW8z0FDNmjWLPfbYI3r16lXjfjbUlvu7rZPPFy9eHOeff369rrNXr161zkP7rBEjRtQ4Sf/Xv/51/PrXv47Y/Nt7DfktT6iXDIDP5d57780iInvuueeSazp16pRdddVV23X9de37+OOPZ3l5edmiRYsavO/PfvazrGXLltnrr7/e4H0/65JLLsny8/PrdR/gy8oRJYDPactRjIceeih69OgRbdu23eq7u+8Ic+bMidNPP736ZdZt+fjjj+N//ud/4qWXXorx48fH1VdfHd27d9+u266qqorly5fHf//3f1e/FxM0VUIJ4HPq27dvDBkyJG6++ea46aab4t/+7d9iypQpOb3Nn/3sZw1aP2vWrBgyZEgUFxfH5ZdfHmPHjt3u297yHlWtW7eO22+/fbuvB74MhBLA59SsWbOYOXNm/O1vf4u//vWvW32X6sZy8sknN/jtC1Iee+yxaN68eey33361fpMQmpq8bEc9cwAAmhjvowQAkCCUAAAShBIAQEKTO5l706ZNsXz58mjTpk1O/wI3APDllWVZrFu3Ljp06LDVt/NocqG0fPnyBv29IQBg17Vs2bKt/gmmJhdKW/6W1bJly6Jt27aNPQ7QyD5eXxVHXP33vyP34vijonXLJvdtD9gOFRUV0bFjx23+Dcwm9x1jy8ttbdu2FUpA5K+vimYFf/9DqW3bthVKQA3bOk3HydwAAAlCCQAgQSgBACQIJQCABKEEAJAglAAAEoQSAECCUAIASBBKAAAJQgkAIEEoAQAkCCUAgAShBACQIJQAABKEEgBAglACAEgQSgAACUIJACBBKAEAJAglAIAEoQQAkCCUAAAShBIAQIJQAgBIEEoAAAlCCQAgQSgBACQIJQCABKEEAJAglAAAEoQSAECCUAIASBBKAAAJQgkAIEEoAQAkCCUAgISchtKzzz4bJ554YnTo0CHy8vLikUce2eY+zzzzTPTs2TNatWoV++23X0ydOjWXIwIAJOU0lD766KM49NBD45ZbbqnX+iVLlsRxxx0X/fr1iwULFsTll18eI0eOjBkzZuRyTACAOuXn8soHDx4cgwcPrvf6qVOnxr777htTpkyJiIju3bvHyy+/HDfccEN897vfzeGkAAC15TSUGqqsrCxKS0trbBs0aFBMmzYtNmzYEC1atGi02XJp0bOvx//Mea2xx4AmaX0WEdE6IiL+85qHo2VeY08ETVNR573jqO/3i+bNmzf2KDvUFyqUysvLo6ioqMa2oqKiqKqqitWrV0dJSUmtfSorK6OysrL6ckVFxU6ZdUf57/uejeuG3xKRZY09CjRJm1rkR4z+fkRE/GrSzGi2oaqxR4Ima/7Ti+KSe/418vKazr9IvlChFBG1vrjZ5oBIfdEnTZoUEydO3Cmz5cIvx90bkWWx34BuUbjPHo09DjQ5VXl58b+b//vQ0/458v2jBHa4jRuq4o8Pz4+n73su/mXMibH/N7o09kg7zBcqlIqLi6O8vLzGtpUrV0Z+fn7stddede5z2WWXxZgxY6ovV1RURMeOHXM+647yccUnERFROvGk2P1rezb2ONDkVG7YFDPv/lNERBz97ydGQQvvigK58O78pbHmf1fGR2s/buxRdqgvVCj16dMnHn300RrbZs2aFb169Uqen1RQUBAFBQU7aUIAYFeS039affjhh7Fw4cJYuHBhxOZf/1+4cGEsXbo0YvPRoLPOOqt6/QUXXBDvvPNOjBkzJt5444246667Ytq0aTF27NhcjgkAUKecHlF6+eWX41vf+lb15S0vkQ0fPjymT58eK1asqI6miIguXbrEE088EaNHj45bb701OnToEDfffLO3BgAAGkVOQ2ngwIHVJ2PXZfr06bW2DRgwIObPn5/LsQAA6sVZjQAACUIJACBBKAEAJAglAIAEoQQAkCCUAAAShBIAQIJQAgBIEEoAAAlCCQAgQSgBACQIJQCABKEEAJAglAAAEoQSAECCUAIASBBKAAAJQgkAIEEoAQAkCCUAgAShBACQIJQAABKEEgBAglACAEgQSgAACUIJACBBKAEAJAglAIAEoQQAkCCUAAAShBIAQIJQAgBIEEoAAAlCCQAgQSgBACQIJQCABKEEAJAglAAAEoQSAECCUAIASBBKAAAJQgkAIEEoAQAkCCUAgAShBACQIJQAABKEEgBAglACAEgQSgAACUIJACBBKAEAJAglAIAEoQQAkCCUAAAShBIAQIJQAgBIEEoAAAlCCQAgQSgBACQIJQCABKEEAJAglAAAEoQSAECCUAIASBBKAAAJQgkAIEEoAQAkCCUAgAShBACQIJQAABKEEgBAglACAEgQSgAACUIJACBBKAEAJAglAIAEoQQAkCCUAAAShBIAQIJQAgBIEEoAAAlCCQAgQSgBACQIJQCABKEEAJAglAAAEoQSAECCUAIASBBKAAAJQgkAIEEoAQAkCCUAgAShBACQIJQAABKEEgBAglACAEgQSgAACUIJACAh56F02223RZcuXaJVq1bRs2fPeO6555Jr586dG3l5ebU+3nzzzVyPCQBQS05D6YEHHohRo0bF+PHjY8GCBdGvX78YPHhwLF26dKv7LV68OFasWFH9ccABB+RyTACAOuU0lCZPnhznnntunHfeedG9e/eYMmVKdOzYMW6//fat7te+ffsoLi6u/mjevHkuxwQAqFPOQmn9+vXxyiuvRGlpaY3tpaWlMW/evK3ue9hhh0VJSUkcddRRMWfOnFyNCACwVfm5uuLVq1fHxo0bo6ioqMb2oqKiKC8vr3OfkpKSuOOOO6Jnz55RWVkZ9957bxx11FExd+7c6N+/f537VFZWRmVlZfXlioqKHXxPAIBdVc5CaYu8vLwal7Msq7Vti27dukW3bt2qL/fp0yeWLVsWN9xwQzKUJk2aFBMnTtzBUwMA5PClt3bt2kXz5s1rHT1auXJlraNMW9O7d+946623kp+/7LLLYu3atdUfy5Yt+1xzAwBskbNQatmyZfTs2TNmz55dY/vs2bOjb9++9b6eBQsWRElJSfLzBQUF0bZt2xofAAA7Qk5fehszZkwMGzYsevXqFX369Ik77rgjli5dGhdccEHE5qNB7777bvzHf/xHRERMmTIlOnfuHAcddFCsX78+7rvvvpgxY0bMmDEjl2MCANQpp6E0dOjQWLNmTfzkJz+JFStWRI8ePeKJJ56ITp06RUTEihUraryn0vr162Ps2LHx7rvvxm677RYHHXRQPP7443HcccflckwAgDrl/GTuCy+8MC688MI6Pzd9+vQal8eNGxfjxo3L9UgAAPXib70BACQIJQCABKEEAJAglAAAEoQSAECCUAIASBBKAAAJQgkAIEEoAQAkCCUAgAShBACQIJQAABKEEgBAglACAEgQSgAACUIJACBBKAEAJAglAIAEoQQAkCCUAAAShBIAQIJQAgBIEEoAAAlCCQAgQSgBACQIJQCABKEEAJAglAAAEoQSAECCUAIASBBKAAAJQgkAIEEoAQAkCCUAgAShBACQIJQAABKEEgBAglACAEgQSgAACUIJACBBKAEAJAglAIAEoQQAkCCUAAAShBIAQIJQAgBIEEoAAAlCCQAgQSgBACQIJQCABKEEAJAglAAAEoQSAECCUAIASBBKAAAJQgkAIEEoAQAkCCUAgAShBACQIJQAABKEEgBAglACAEgQSgAACUIJACBBKAEAJAglAIAEoQQAkCCUAAAShBIAQIJQAgBIEEoAAAlCCQAgQSgBACQIJQCABKEEAJAglAAAEoQSAECCUAIASBBKAAAJQgkAIEEoAQAkCCUAgAShBACQIJQAABKEEgBAglACAEgQSgAACUIJACBBKAEAJAglAIAEoQQAkCCUAAAShBIAQIJQAgBIEEoAAAlCCQAgQSgBACQIJQCAhJyH0m233RZdunSJVq1aRc+ePeO5557b6vpnnnkmevbsGa1atYr99tsvpk6dmusRAQDqlNNQeuCBB2LUqFExfvz4WLBgQfTr1y8GDx4cS5curXP9kiVL4rjjjot+/frFggUL4vLLL4+RI0fGjBkzcjkmAECdchpKkydPjnPPPTfOO++86N69e0yZMiU6duwYt99+e53rp06dGvvuu29MmTIlunfvHuedd16cc845ccMNN+RyTACAOuXn6orXr18fr7zySlx66aU1tpeWlsa8efPq3KesrCxKS0trbBs0aFBMmzYtNmzYEC1atKj37X+8viry11dt5/Q7z6b85rGpRX5UVmVRuWFTY48DTc5nn1eeY5A7G5v//efZp1Wb4uMvwc/f+s6Ys1BavXp1bNy4MYqKimpsLyoqivLy8jr3KS8vr3N9VVVVrF69OkpKSmrtU1lZGZWVldWXKyoqIiLiiKufjmYFrXfQvcmhC06LiIjLZ62KiFWNPQ00aaPv+3NjjwBN18mDIiJi6OylEbPrPsXmi2RT5cf1Wpfzk7nz8vJqXM6yrNa2ba2va/sWkyZNisLCwuqPjh077pC5AQBydkSpXbt20bx581pHj1auXFnrqNEWxcXFda7Pz8+Pvfbaq859LrvsshgzZkz15YqKiujYsWO8OP6oaNu27Q65L7l0avtz49NP1se5vxsdu++zR2OPA01O5YZN1UeSbvz+/lHQwruiQC7cfdL/F39bsiqueeLyOLhf98YeZ5sqKiqiZMq21+UslFq2bBk9e/aM2bNnx5AhQ6q3z549O0466aQ69+nTp088+uijNbbNmjUrevXqlTw/qaCgIAoKCmptb90yP1q3zNnd22GaVW2MZhuqoiA/zzdwyLGCFs08zyBHmm/8+8+zVvnNvhQ/f6vqOWNOv2OMGTMm7rzzzrjrrrvijTfeiNGjR8fSpUvjggsuiNh8NOiss86qXn/BBRfEO++8E2PGjIk33ngj7rrrrpg2bVqMHTs2l2MCANQpp8k3dOjQWLNmTfzkJz+JFStWRI8ePeKJJ56ITp06RUTEihUrarynUpcuXeKJJ56I0aNHx6233hodOnSIm2++Ob773e/mckwAgDrl/NjYhRdeGBdeeGGdn5s+fXqtbQMGDIj58+fneiwAgG3yYj0AQIJQAgBIEEoAAAlCCQAgQSgBACQIJQCABKEEAJAglAAAEoQSAECCUAIASBBKAAAJQgkAIEEoAQAkCCUAgAShBACQIJQAABKEEgBAglACAEgQSgAACUIJACBBKAEAJAglAIAEoQQAkCCUAAAShBIAQIJQAgBIEEoAAAlCCQAgQSgBACQIJQCABKEEAJAglAAAEoQSAECCUAIASBBKAAAJQgkAIEEoAQAkCCUAgAShBACQIJQAABKEEgBAglACAEgQSgAACUIJACBBKAEAJAglAIAEoQQAkCCUAAAShBIAQIJQAgBIEEoAAAlCCQAgQSgBACQIJQCABKEEAJAglAAAEoQSAECCUAIASBBKAAAJQgkAIEEoAQAkCCUAgAShBACQIJQAABKEEgBAglACAEgQSgAACUIJACBBKAEAJAglAIAEoQQAkCCUAAAShBIAQIJQAgBIEEoAAAlCCQAgQSgBACQIJQCABKEEAJAglAAAEoQSAECCUAIASBBKAAAJQgkAIEEoAQAkCCUAgAShBACQIJQAABKEEgBAglACAEgQSgAACUIJACBBKAEAJAglAIAEoQQAkCCUAAAShBIAQIJQAgBIyGkovf/++zFs2LAoLCyMwsLCGDZsWHzwwQdb3efss8+OvLy8Gh+9e/fO5ZgAAHXKz+WVn3HGGfHXv/41nnzyyYiI+OEPfxjDhg2LRx99dKv7HXvssXH33XdXX27ZsmUuxwQAqFPOQumNN96IJ598Mp5//vk48sgjIyLil7/8ZfTp0ycWL14c3bp1S+5bUFAQxcXFuRoNAKBecvbSW1lZWRQWFlZHUkRE7969o7CwMObNm7fVfefOnRvt27ePrl27xvnnnx8rV67M1ZgAAEk5O6JUXl4e7du3r7W9ffv2UV5entxv8ODBceqpp0anTp1iyZIlccUVV8S3v/3teOWVV6KgoKDW+srKyqisrKy+XFFRsQPvBQCwK2vwEaUJEybUOtn6Hz9efvnliIjIy8urtX+WZXVu32Lo0KFx/PHHR48ePeLEE0+M3/3ud/GnP/0pHn/88TrXT5o0qfpk8cLCwujYsWND7xIAQJ0afETpoosuitNPP32razp37hyLFi2K9957r9bnVq1aFUVFRfW+vZKSkujUqVO89dZbdX7+sssuizFjxlRfrqioEEsAwA7R4FBq165dtGvXbpvr+vTpE2vXro0XX3wxjjjiiIiIeOGFF2Lt2rXRt2/fet/emjVrYtmyZVFSUlLn5wsKCup8SQ4A4PPK2cnc3bt3j2OPPTbOP//8eP755+P555+P888/P0444YQav/F24IEHxsMPPxwRER9++GGMHTs2ysrK4u233465c+fGiSeeGO3atYshQ4bkalQAgDrl9A0n77///jj44IOjtLQ0SktL45BDDol77723xprFixfH2rVrIyKiefPm8eqrr8ZJJ50UXbt2jeHDh0fXrl2jrKws2rRpk8tRAQBqyekbTu65555x3333bXVNlmXV/73bbrvFU089lcuRAADqzd96AwBIEEoAAAlCCQAgQSgBACQIJQCABKEEAJAglAAAEoQSAECCUAIASBBKAAAJQgkAIEEoAQAkCCUAgAShBACQIJQAABKEEgBAglACAEgQSgAACUIJACBBKAEAJAglAIAEoQQAkCCUAAAShBIAQIJQAgBIEEoAAAlCCQAgQSgBACQIJQCABKEEAJAglAAAEoQSAECCUAIASBBKAAAJQgkAIEEoAQAkCCUAgAShBACQIJQAABKEEgBAglACAEgQSgAACUIJACBBKAEAJAglAIAEoQQAkCCUAAAShBIAQIJQAgBIEEoAAAlCCQAgQSgBACQIJQCABKEEAJAglAAAEoQSAECCUAIASBBKAAAJQgkAIEEoAQAkCCUAgAShBACQIJQAABKEEgBAglACAEgQSgAACUIJACBBKAEAJAglAIAEoQQAkCCUAAAShBIAQIJQAgBIEEoAAAlCCQAgQSgBACQIJQCABKEEAJAglAAAEoQSAECCUAIASBBKAAAJQgkAIEEoAQAkCCUAgAShBACQIJQAABKEEgBAglACAEgQSgAACUIJACBBKAEAJAglAIAEoQQAkCCUAAAShBIAQIJQAgBIEEoAAAk5DaWrr746+vbtG61bt47dd9+9XvtkWRYTJkyIDh06xG677RYDBw6M1157LZdjAgDUKaehtH79+jj11FNjxIgR9d7n+uuvj8mTJ8ctt9wSL730UhQXF8cxxxwT69aty+WoAAC15DSUJk6cGKNHj46DDz64XuuzLIspU6bE+PHj45RTTokePXrEPffcEx9//HH86le/yuWoAAC15Df2AJ+1ZMmSKC8vj9LS0uptBQUFMWDAgJg3b1786Ec/atT5cqF5i+YREfE///Vi7NGlXWOPA03Ohiwi4qsREfHH38yPFnmNPRE0PZs2bIx15R9ERER+iy9UWnxuX6h7U15eHhERRUVFNbYXFRXFO++8U+c+lZWVUVlZWX25oqIix1PuWCeOGBT/de3DUTZ1bmOPAk3Sphb5EaO/HxERT13xSDTbUNXYI0GT1bXX1+PAI/Zv7DF2qAaH0oQJE2LixIlbXfPSSy9Fr169tnuovLya/+TLsqzWti0mTZq0zXm+yM65+nux+95tY/7Tixp7FGiSqvKaxf9u/u9epYdGfrapkSeCpqmoU/v4wU9Pj+b5zRt7lB0qL8uyrCE7rF69OlavXr3VNZ07d45WrVpVX54+fXqMGjUqPvjgg63u95e//CW+/vWvx/z58+Owww6r3n7SSSfF7rvvHvfcc0+tfeo6otSxY8dYu3ZttG3btiF3DWiCPl5fFf905VMREfH6TwZF65ZfqAPpQCOpqKiIwsLCbfZCg79jtGvXLtq1y825NF26dIni4uKYPXt2dSitX78+nnnmmbjuuuvq3KegoCAKCgpyMg8AsGvL6W+9LV26NBYuXBhLly6NjRs3xsKFC2PhwoXx4YcfVq858MAD4+GHH47Y/JLbqFGj4pprromHH344/vjHP8bZZ58drVu3jjPOOCOXowIA1JLTY9BXXnlljZfLthwlmjNnTgwcODAiIhYvXhxr166tXjNu3Lj45JNP4sILL4z3338/jjzyyJg1a1a0adMml6MCANTS4HOUvujq+5ojsGtwjhJQl/r2gr/1BgCQIJQAABKEEgBAglACAEgQSgAACUIJACBBKAEAJAglAIAEoQQAkCCUAAAShBIAQIJQAgBIEEoAAAlCCQAgQSgBACQIJQCABKEEAJAglAAAEoQSAECCUAIASBBKAAAJQgkAIEEoAQAkCCUAgAShBACQIJQAABKEEgBAglACAEgQSgAACUIJACBBKAEAJAglAIAEoQQAkCCUAAAShBIAQIJQAgBIyG/sAXa0LMsiIqKioqKxRwG+AD5eXxWbKj+O2Px9oaplk/u2B2yHLZ2wpRtS8rJtrfiS+etf/xodO3Zs7DEAgC+BZcuWxde+9rXk55tcKG3atCmWL18ebdq0iby8vMYeZ5sqKiqiY8eOsWzZsmjbtm1jj0M9eMy+fDxmXz4esy+fL9tjlmVZrFu3Ljp06BDNmqXPRGpyx6CbNWu21TL8omrbtu2X4n8s/o/H7MvHY/bl4zH78vkyPWaFhYXbXONkbgCABKEEAJAglBpZQUFBXHXVVVFQUNDYo1BPHrMvH4/Zl4/H7MunqT5mTe5kbgCAHcURJQCABKEEAJAglAAAEoQSAECCUGoEV199dfTt2zdat24du+++e732ybIsJkyYEB06dIjddtstBg4cGK+99lrOZ+Xv3n///Rg2bFgUFhZGYWFhDBs2LD744IOt7nP22WdHXl5ejY/evXvvtJl3Nbfddlt06dIlWrVqFT179oznnntuq+ufeeaZ6NmzZ7Rq1Sr222+/mDp16k6blb9ryGM2d+7cWs+nvLy8ePPNN3fqzLuyZ599Nk488cTo0KFD5OXlxSOPPLLNfZrC80woNYL169fHqaeeGiNGjKj3Ptdff31Mnjw5brnllnjppZeiuLg4jjnmmFi3bl1OZ+XvzjjjjFi4cGE8+eST8eSTT8bChQtj2LBh29zv2GOPjRUrVlR/PPHEEztl3l3NAw88EKNGjYrx48fHggULol+/fjF48OBYunRpneuXLFkSxx13XPTr1y8WLFgQl19+eYwcOTJmzJix02ffVTX0Mdti8eLFNZ5TBxxwwE6beVf30UcfxaGHHhq33HJLvdY3medZRqO5++67s8LCwm2u27RpU1ZcXJxde+211ds+/fTTrLCwMJs6dWqOp+T111/PIiJ7/vnnq7eVlZVlEZG9+eabyf2GDx+enXTSSTtpyl3bEUcckV1wwQU1th144IHZpZdeWuf6cePGZQceeGCNbT/60Y+y3r1753RO/k9DH7M5c+ZkEZG9//77O2lCtiYisocffnira5rK88wRpS+BJUuWRHl5eZSWllZvKygoiAEDBsS8efMadbZdQVlZWRQWFsaRRx5Zva13795RWFi4za//3Llzo3379tG1a9c4//zzY+XKlTth4l3L+vXr45VXXqnx/IiIKC0tTT4+ZWVltdYPGjQoXn755diwYUNO52X7HrMtDjvssCgpKYmjjjoq5syZk+NJ+TyayvNMKH0JlJeXR0REUVFRje1FRUXVnyN3ysvLo3379rW2t2/ffqtf/8GDB8f9998fv//97+PnP/95vPTSS/Htb387KisrczzxrmX16tWxcePGBj0/ysvL61xfVVUVq1evzum8bN9jVlJSEnfccUfMmDEjZs6cGd26dYujjjoqnn322Z00NQ3VVJ5n+Y09QFMxYcKEmDhx4lbXvPTSS9GrV6/tvo28vLwal7Msq7WN+qvvYxZ1fO2jHl//oUOHVv93jx49olevXtGpU6d4/PHH45RTTvlcs1NbQ58fda2vazu505DHrFu3btGtW7fqy3369Illy5bFDTfcEP3798/5rGyfpvA8E0o7yEUXXRSnn376Vtd07tx5u667uLg4YnOdl5SUVG9fuXJlrVqn/ur7mC1atCjee++9Wp9btWpVg77+JSUl0alTp3jrrbe2a17q1q5du2jevHmtIxFbe34UFxfXuT4/Pz/22muvnM7L9j1mdendu3fcd999OZiQHaGpPM+E0g7Srl27aNeuXU6uu0uXLlFcXByzZ8+Oww47LGLza/zPPPNMXHfddTm5zV1BfR+zPn36xNq1a+PFF1+MI444IiIiXnjhhVi7dm307du33re3Zs2aWLZsWY3Y5fNr2bJl9OzZM2bPnh1Dhgyp3j579uw46aST6tynT58+8eijj9bYNmvWrOjVq1e0aNEi5zPv6rbnMavLggULPJ++wJrM86yxzybfFb3zzjvZggULsokTJ2Zf/epXswULFmQLFizI1q1bV72mW7du2cyZM6svX3vttVlhYWE2c+bM7NVXX82+973vZSUlJVlFRUUj3Ytdy7HHHpsdcsghWVlZWVZWVpYdfPDB2QknnFBjzWcfs3Xr1mU//vGPs3nz5mVLlizJ5syZk/Xp0yfbZ599PGY58F//9V9ZixYtsmnTpmWvv/56NmrUqOwrX/lK9vbbb2dZlmWXXnppNmzYsOr1f/nLX7LWrVtno0ePzl5//fVs2rRpWYsWLbKHHnqoEe/FrqWhj9mNN96YPfzww9mf/vSn7I9//GN26aWXZhGRzZgxoxHvxa5l3bp11T+vIiKbPHlytmDBguydd97Jsib8PBNKjWD48OFZRNT6mDNnTvWaiMjuvvvu6subNm3Krrrqqqy4uDgrKCjI+vfvn7366quNdA92PWvWrMnOPPPMrE2bNlmbNm2yM888s9avKX/2Mfv444+z0tLSbO+9985atGiR7bvvvtnw4cOzpUuXNtI9aPpuvfXWrFOnTlnLli2zww8/PHvmmWeqPzd8+PBswIABNdbPnTs3O+yww7KWLVtmnTt3zm6//fZGmHrX1pDH7Lrrrsu+/vWvZ61atcr22GOP7P/9v/+XPf744400+a5py1s0/OPH8OHDs6wJP8/ysi1nVgEAUIO3BwAASBBKAAAJQgkAIEEoAQAkCCUAgAShBACQIJQAABKEEgBAglACAEgQSgAACUIJACBBKAEAJPz/9l9Qm5pf7sYAAAAASUVORK5CYII=",
|
|
"text/plain": [
|
|
"<Figure size 600x600 with 1 Axes>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"import numpy as np\n",
|
|
"import matplotlib.pyplot as plt\n",
|
|
"\n",
|
|
"# Grid\n",
|
|
"xx = np.linspace(-1.2, 1.2, 400)\n",
|
|
"yy = np.linspace(-1.2, 1.2, 400)\n",
|
|
"X, Y = np.meshgrid(xx, yy)\n",
|
|
"\n",
|
|
"# Take the $L^{\\infty}$ norm\n",
|
|
"Z = np.maximum(np.abs(X), np.abs(Y))\n",
|
|
"\n",
|
|
"plt.figure(figsize=(6,6))\n",
|
|
"plt.contour(X, Y, Z, levels=[1])\n",
|
|
"plt.contourf(X, Y, Z, levels=[0,1], alpha=0.3)\n",
|
|
"\n",
|
|
"plt.axhline(0)\n",
|
|
"plt.axvline(0)\n",
|
|
"plt.gca().set_aspect(\"equal\", adjustable=\"box\")\n",
|
|
"plt.title(r\"$L^{\\infty}$ unit ball: $\\max\\{|x|,|y|\\} \\leq 1$\")\n",
|
|
"plt.tight_layout()\n",
|
|
"plt.savefig('../images/Linf_unit_ball.png')\n",
|
|
"plt.show()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "55c4ce17",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n",
|
|
"Problems involving the $L^{\\infty}$ norm are often formulated as linear programs, and are useful when worst-case guarantees are more important than optimizing average performance. "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "d393c069",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n",
|
|
"## Matrix norms\n",
|
|
"\n",
|
|
"There are also various norms on matrices, each highlighting a different aspect of the associated linear transformation.\n",
|
|
"- **Frobenius norm**. This is an important norm, essentially the analogue of the $L^2$ norm for matrices. It is the Euclidean norm if you think of your matrix as a vector, forgetting its rectangular shape. For $A = (a_{ij})$ a matrix, the Frobenius norm \n",
|
|
" $$ \\|A\\|_F = \\sqrt{\\sum a_{ij}^2} $$\n",
|
|
" is the square root of the sum of squares of all the entries. This treats a matrix as a long vector and is invariant under orthogonal transformations. As we'll see, it plays a central role in:\n",
|
|
" - least-squares problems,\n",
|
|
" - low-rank approximation,\n",
|
|
" - principal component analysis.\n",
|
|
"\n",
|
|
" In particular, the truncated SVD yields a best low-rank approximation of a matrix with respect to the Frobenius norm.\n",
|
|
"\n",
|
|
" We also that that the Frobenius norm can be written in terms of tracial data. We have that\n",
|
|
" $$ \\|A\\|_F^2 = \\text{Tr}(A^TA) = \\text{Tr}(AA^T). $$\n",
|
|
"- **Operator norm** (spectral norm). This is just the norm as an operator $A: \\mathbb{R}^p \\to \\mathbb{R}^n$, where $\\mathbb{R}^p$ and $\\mathbb{R}^n$ are thought of as Hilbert spaces:\n",
|
|
" $$ \\|A\\| = \\max_{\\|x\\|_2 = 1}\\|Ax\\|_2. $$\n",
|
|
" This norm measures how big of an amplification $A$ can apply, and is equal to the largest singular value of $A$. This norm is related to stability properties, and is the analogue of the $L^{\\infty}$ norm.\n",
|
|
"- **Nuclear norm**. The nuclear norm, defined as\n",
|
|
" $$ \\|A\\|_* = \\sum \\sigma_i, $$\n",
|
|
" is the sum of the singular values. When $A$ is square, this is precisely the trace-class norm, and is the analogue of the $L^1$ norm. This norm acts as a generalization of the concept of rank. "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "4ee62ee0",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n",
|
|
"# A note on regularization\n",
|
|
"\n",
|
|
"Regularization introduces additional constraints or penalties to stabilize ill-posed problems. From the linear algebra point of view, regularization modifies the singular value structure of a problem. \n",
|
|
"- **Ridge regression**: add a positive multiple $\\lambda\\cdot I$ of the identity to $X^TX$ which will artificially inflate small singular values.\n",
|
|
"- This dampens unstable directions while leaving well-conditioned directions largely unaffected.\n",
|
|
" \n",
|
|
"Geometrically, regularization reshapes the solution space to suppress directions that are poorly supported by the data."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "be3b8c1e",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n",
|
|
"# A note on solving multiple targets concurrently\n",
|
|
"\n",
|
|
"Suppose now that we were interested in solving several problems concurrently; that is, given some data points, we would like to make $k$ predictions. Say we have our $n \\times p$ data matrix $X$, and we want to make $k$ predictions $y_1,\\dots,y_k$. We can then set the problem up as finding a best solution to the matrix equation\n",
|
|
"$$ XB = Y $$\n",
|
|
"where $B$ will be a $p \\times k$ matrix of parameters and $Y$ will be the $p \\times k$ matrix whose columns are $y_1,\\dots,y_k$. "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "908cd528",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n",
|
|
"# What can go wrong?\n",
|
|
"\n",
|
|
"We are often dealing with imperfect data, so there is plenty that could go wrong. Here are some basic cases of where things can break down.\n",
|
|
"\n",
|
|
"- **Perfect multicolinearity**: non-invertible $\\tilde{X}^T\\tilde{X}$. This happens when one feature is a perfect combination of the others. This means that the columns of the matrix $\\tilde{X}$ are linearly dependent, and so infinitely many solutions will exist to the least-squares problem. \n",
|
|
" - For example, if you are looking at characteristics of people and you have height in both inches and centimeters.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "a1d9e0ed",
|
|
"metadata": {},
|
|
"source": [
|
|
"- **Almost multicolinearity**: this happens when one features is **almost** a perfect combination of the others. From the linear algebra perspective, the columns of $\\tilde{X}$ might not be dependent, but they will be be **almost** linearly dependent. This will cause problems in calculation, as the condition number will become large and amplify numerical errors. The inverse will blow up small spectral components. \n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "fa57c3a4",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n",
|
|
"- **More features than observations**: this means that our matrix $\\tilde{X}$ will be wider than it is high. Necessarily, this means that the columns are linearly dependent. Regularization or dimensionality reduction becomes essential.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "f5fff0b5",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n",
|
|
"- **Redundant or constant features**: this is when there is a characteristic that is satisfied by each observation. In terms of the linear algebraic data, this means that one of the columns of $X$ is constant.\n",
|
|
" - e.g., if you are looking at characteristics of penguins, and you have \"# of legs\". This will always be two, and doesn't add anything to the analysis.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ed7d745d",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n",
|
|
"- **Underfitting**: the model lacks sufficient expressivity to capture the underlying structure. For example, see the section on polynomial regression -- sometimes one might want a curve vs. a straight line."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"id": "2de2ed0c",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGdCAYAAACyzRGfAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQAALL9JREFUeJzt3X9wXOV97/HPSg6SbVYb5ES7EpaJ6ug2UYQB27Fjl8RKghQ7qQZCp0NizIQ2yYBtGFTfjrnGt5WVMlJwWtdpfFEC907iRmPgtik4GohqdWhEcm2uBUIFWTQXHIEdexdhy+wKY0lBOvcPaxetfu6uds85e877NXNmorNH2sdC2fM5z/N9nsdjGIYhAAAAk+RY3QAAAOAuhA8AAGAqwgcAADAV4QMAAJiK8AEAAExF+AAAAKYifAAAAFMRPgAAgKkWWN2AycbGxnT27Fl5vV55PB6rmwMAABJgGIYGBwdVUlKinJzZ+zZsFz7Onj2r0tJSq5sBAABScPr0aS1dunTWa2wXPrxerzTe+IKCAqubAwAAEhCJRFRaWhq7j8/GduEjOtRSUFBA+AAAIMskUjJBwSkAADAV4QMAAJiK8AEAAExF+AAAAKYifAAAAFMRPgAAgKkIHwAAwFSEDwAAYCrbLTJmhtExQ8f7BtQ/OKQib77WlBUqN4d9ZAAAMIPrwkdbT1ANrb0Khodi54p9+aqvrdDGymJL2wYAgBu4ZthldMzQ9//t/+nulq644CFJofCQtrZ0qa0naFn7AABwC1f0fLT1BLXn570KRYamfd2Q5JHU0Nqr6ooAQzAAAGSQ43s+2nqC2trSNWPwiDIkBcNDOt43YFrbAABwI0eHj9ExQw2tvTKS+J7+wdlDCgAAmB9Hh4/jfQNT6jvmUuTNz1h7AACAw2s+kunF8EgK+C5PuwUAAJnj6J6PZHsx6msrKDYFACDDHB0+1pQVqtiXr7niRKAgT81bVrLOBwAAJnB0+MjN8ai+tkIaH1aZzl/c9F/0f/7bFwkeAACYxNHhQ5I2VharectKBXzxQzDFvnz9cMtK3XdTOUMtAACYyNEFp1EbK4tVXRFgPxcAgKvZZW8zV4QPjQ/BrFu+xOpmAABgCTvtbeb4YRcAANwuutq3XfY2I3wAAOBgs632HT3X0Nqr0bFk1gOfH8IHAAAONtdq31bsbUb4AADAwRJd7dvMvc0IHwAAOFiiq32bubcZ4QMAAAeba7Vvz/isFzP3NiN8AADgYLOt9h392uy9zQgfAAA43EyrfQd8+ZbsbeaaRcYAAHAzO632nVTPx549e+TxeOKOQCAQe90wDO3Zs0clJSVauHChqqqqdOLEiUy0GwAAJCm62vfN11+tdcuXWLbNSNLDLp/61KcUDAZjxyuvvBJ7be/evdq3b58OHDigzs5OBQIBVVdXa3BwMN3tBgAAWSrp8LFgwQIFAoHY8dGPflQa7/XYv3+/du/erVtvvVWVlZU6ePCg3nvvPR06dCgTbQcAAFko6fDx2muvqaSkRGVlZfra176m3/72t5Kkvr4+hUIh1dTUxK7Ny8vThg0bdPTo0Rl/3vDwsCKRSNwBAACcK6nwsXbtWv3jP/6j/vVf/1WPPvqoQqGQ1q9fr/PnzysUCkmS/H5/3Pf4/f7Ya9NpamqSz+eLHaWlpan+WwAAQBZIKnxs2rRJf/Inf6Jrr71WN910k55++mlJ0sGDB2PXeDzxxSuGYUw5N9GuXbsUDodjx+nTp5P/VwAAgKwxr3U+Fi9erGuvvVavvfZabNbL5F6O/v7+Kb0hE+Xl5amgoCDuAAAAzjWv8DE8PKxXX31VxcXFKisrUyAQUHt7e+z1kZERdXR0aP369eloKwAAcICkFhn7y7/8S9XW1mrZsmXq7+/Xgw8+qEgkom984xvyeDyqq6tTY2OjysvLVV5ersbGRi1atEibN2/O3L8gg0bHDFssxgIAgJMkFT5+97vf6etf/7rOnTunj370o/rMZz6j559/Xtdcc40kaefOnbp06ZK2bdumCxcuaO3atTpy5Ii8Xm+m2p8xbT1BNbT2Khj+YIvhYl++6msrTF+GFgDgXk58EPYYhmFY3YiJIpGIfD6fwuGwZfUfbT1BbW3p0uRfTPQ/tRXr4AMA3CebHoSTuX+zsdwko2OGGlp7pwQPSbFzDa29Gh2zVWYDADhM9EF4YvCQpFB4SFtbutTWE7SsbfNF+JjkeN/AlP/QExmSguEhHe8bMLVdAAD3cPqDMOFjkv7BmYNHKtcBAJAspz8IEz4mKfLmp/U6AACS5fQHYcLHJGvKClXsy9dMdcSe8WKfNWWFJrcMAOAWTn8QJnxMkpvjUX1thTRhdktU9Ov62oqsn+YEALAvpz8IEz6msbGyWM1bVirgi0+UAV8+02wBABnn9Adh1vmYhRMXdgEAZA+nrvNB+AAAwMay5UE4mft3UsurAwAAc+XmeLRu+RKrm5FW1HwAAABTET4AAICpCB8AAMBUhA8AAGAqwgcAADAV4QMAAJiK8AEAAExF+AAAAKYifAAAAFMRPgAAgKkIHwAAwFSEDwAAYCrCBwAAMBXhAwAAmIrwAQAATEX4AAAApiJ8AAAAUxE+AACAqQgfAADAVIQPAABgKsIHAAAwFeEDAACYivABAABMtcDqBrjF6Jih430D6h8cUpE3X2vKCpWb47G6WQAAmI7wYYK2nqAaWnsVDA/FzhX78lVfW6GNlcWWtg0AALMx7JJhbT1BbW3pigsekhQKD2lrS5faeoKWtQ0AACsQPjJodMxQQ2uvjGlei55raO3V6Nh0VwAA4EyEjww63jcwpcdjIkNSMDyk430DprYLAAArET4yqH9w5uCRynUAADgB4SODirz5ab0OAAAnIHxk0JqyQhX78jXThFrP+KyXNWWFJrcMAADrED4yKDfHo/raCmk8aEwU/bq+toL1PgAArkL4yLCNlcVq3rJSAV/80ErAl6/mLStZ5wMA4DosMmaCjZXFqq4IsMIpAACED/Pk5ni0bvkSq5sBAIDlGHYBAACmInwAAABTET4AAICp5hU+mpqa5PF4VFdXFztnGIb27NmjkpISLVy4UFVVVTpx4kQ62goAABwg5fDR2dmpRx55RCtWrIg7v3fvXu3bt08HDhxQZ2enAoGAqqurNTg4mI72AgCALJdS+Hj33Xd1++2369FHH9VVV10VO28Yhvbv36/du3fr1ltvVWVlpQ4ePKj33ntPhw4dSme7AQBAlkopfGzfvl1f+cpXdNNNN8Wd7+vrUygUUk1NTexcXl6eNmzYoKNHj077s4aHhxWJROIOAACcYnTM0LGT53W4+4yOnTyv0THD6iZZLul1Ph5//HF1dXWps7NzymuhUEiS5Pf74877/X69+eab0/68pqYmNTQ0JNsMAABsr60nqIbWXgXDH+xeXuzLV31thatXuE6q5+P06dO677771NLSovz8mXdi9XjiV+40DGPKuahdu3YpHA7HjtOnTyfTJAAAbKmtJ6itLV1xwUOSQuEhbW3pUltP0LK2WS2p8PHiiy+qv79fq1at0oIFC7RgwQJ1dHToH/7hH7RgwYJYj0e0BySqv79/Sm9IVF5engoKCuIOAACy2eiYoYbWXk03wBI919Da69ohmKTCxxe/+EW98sor6u7ujh2rV6/W7bffru7ubv3BH/yBAoGA2tvbY98zMjKijo4OrV+/PhPtBwDAdo73DUzp8ZjIkBQMD+l434Cp7bKLpGo+vF6vKisr484tXrxYS5YsiZ2vq6tTY2OjysvLVV5ersbGRi1atEibN29Ob8sBALCp/sGZg0cq1zlN2jeW27lzpy5duqRt27bpwoULWrt2rY4cOSKv15vutwIAwJaKvDPXRaZyndN4DMOw1YBTJBKRz+dTOBym/gMAkJVGxwzd+NCzCoWHpq378EgK+PL16/u/oNyc6SdkZJtk7t/s7QIAQJrl5nhUX1shjQeNiaJf19dWOCZ4JIvwAQBABmysLFbzlpUK+OKHVgK+fDVvWenqdT7SXvMBAAAu21hZrOqKgI73Dah/cEhF3nytKSt0bY9HFOEDAIAMys3xaN3yJVY3w1YYdgEAAKYifAAAAFMRPgAAgKkIHwAAwFSEDwAAYCrCBwAAMBXhAwAAmIp1PrLE6JjBIjUAAEcgfGSBtp6gGlp7FQx/sPVysS9f9bUVrl6eFwCQnRh2sbm2nqC2tnTFBQ9JCoWHtLWlS209QcvaBgBAKggfNjY6ZqihtXfa7Zij5xpaezU6Nt0VAACrjI4ZOnbyvA53n9Gxk+f5nJ6EYRcbO943MKXHYyJDUjA8pON9A+wbAAA2wVD53Oj5sLH+wZmDRyrXAQAyi6HyxBA+bKzIm5/W6wAAmcNQeeIIHza2pqxQxb58zTSh1jPelbemrNDklgEAJktmqNztCB82lpvjUX1thTQeNCaKfl1fW8F6HwBgAwyVJ47wYXMbK4vVvGWlAr74oZWAL1/NW1ZSvAQANsFQeeKY7ZIFNlYWq7oiwAqnAGBj0aHyUHho2roPz/iDI0PlhI+skZvjYTotANhYdKh8a0uXPBOKTMVQ+RQMuwAAkCYMlSeGng8AANKIofK5ET4AAEgzhspnx7ALAAAwFeEDAACYivABAABMRfgAAACmInwAAABTET4AAICpCB8AAMBUhA8AAGAqwgcAADAV4QMAAJiK8AEAAExF+AAAAKYifAAAAFMRPgAAgKkIHwAAwFSEDwAAYCrCBwAAMBXhAwAAmIrwAQAATEX4AAAApiJ8AAAAUxE+AACAqZIKH83NzVqxYoUKCgpUUFCgdevW6Re/+EXsdcMwtGfPHpWUlGjhwoWqqqrSiRMnMtFuAACQpZIKH0uXLtV3v/tdvfDCC3rhhRf0hS98QTfffHMsYOzdu1f79u3TgQMH1NnZqUAgoOrqag0ODmaq/QAAJG10zNCxk+d1uPuMjp08r9Exw+omuYrHMIx5/cYLCwv1ve99T3/+53+ukpIS1dXV6f7775ckDQ8Py+/366GHHtJdd92V0M+LRCLy+XwKh8MqKCiYT9MAAJiirSeohtZeBcNDsXPFvnzV11ZoY2WxpW3LZsncv1Ou+RgdHdXjjz+uixcvat26derr61MoFFJNTU3smry8PG3YsEFHjx6d8ecMDw8rEonEHQAAZEJbT1BbW7rigockhcJD2trSpbaeoGVtc5Okw8crr7yiK6+8Unl5ebr77rv15JNPqqKiQqFQSJLk9/vjrvf7/bHXptPU1CSfzxc7SktLU/l3AAAwq9ExQw2tvZquuz96rqG1d95DMAzpzG1Bst/wh3/4h+ru7tY777yjn/3sZ/rGN76hjo6O2OsejyfuesMwppybaNeuXdqxY0fs60gkQgABAKTd8b6BKT0eExmSguEhHe8b0LrlS1J6D4Z0EpN0z8cVV1yhj3/841q9erWampp03XXX6fvf/74CgYAkTenl6O/vn9IbMlFeXl5s9kz0AAAg3foHZw4eqVw3GUM6iZv3Oh+GYWh4eFhlZWUKBAJqb2+PvTYyMqKOjg6tX79+vm8DAMC8FHnz03rdRGYN6ThFUsMuDzzwgDZt2qTS0lINDg7q8ccf1y9/+Uu1tbXJ4/Gorq5OjY2NKi8vV3l5uRobG7Vo0SJt3rw5c/8CAAASsKasUMW+fIXCQ9OGBI+kgC9fa8oKk/7ZZgzpOElS4eOtt97SHXfcoWAwKJ/PpxUrVqitrU3V1dWSpJ07d+rSpUvatm2bLly4oLVr1+rIkSPyer2Zaj8AAAnJzfGovrZCW1u65JnQI6Hx4CFJ9bUVys2ZuU5xJpke0nGaea/zkW6s8wEAyKS5ikJHxwwd7xtQ/+CQiryXe0LmCiTHTp7X1x99fs73fuzbn3Fsz0cy9++kZ7sAAJDNNlYWq7oiMG3ASHW2SiaHdJyIjeUAAK6Tm+PRuuVLdPP1V2vd8iWx4JHqbJXokI4mDOFEzXdIx4kIHwAA10vHbJWNlcVq3rJSAV/8bJmAL1/NW1ayzscEDLsAAFwvXbNVZhvSwQcIHwAA10vnbJXokA5mxrALAMD1MrkAGaYifAAAXC86W2WmwRHP+KwXZqukB+EDAOB6zFYxF+EDAGAJu209z2wV81BwCgAwnV23nme2ijlYXh0AYKroYl6Tbz7R2zu9DNkpmfs3wy4AANOw9TxE+AAAmCmZxbzgXIQPAIBp2HoeInwAAMzEYl4Qs13cYXTMoHIbgC2w9TxE+HA+u05nA+BO0cW8trZ0yTOhyFQs5uUqDLs4WHQ62+TirlB4SFtbutTWE7SsbQDci8W8QM+HQ801nc0zPp2tuiLAEwYA07GYl7sRPhwqmels8936mZoSAKlg63n3Inw4lFnT2agpAQAki5oPhzJjOhs1JQCAVBA+HCo6nW2mwQ/PeA9FqtPZWCIZAJAqwodDRaezacL0tah0TGdjiWQAQKoIHw6WyelsLJEMAEgVBacOl6npbCyRDABIFeHDBTIxnY0lkgEAqWLYBSnJdE0JAMC5CB9IWSI1JaNjho6dPK/D3Wd07OR5Zr8AABh2wfzMVlPCAmQAgOl4DMOw1aNoJBKRz+dTOBxWQUGB1c1BiqILkE3+44oOwrB5FAA4SzL3b4ZdkHYsQAYAmA3hA2nHAmQAgNkQPpB2LEAGAJgN4QNpxwJkAIDZED6Qdpne1A4AkN0IH0g7FiADAMyG8IGMyOSmdgCA7MYiY8iYTG1qBwDIboQPZFQmNrUDAGQ3hl0AAICpCB8AAMBUhA8AAGAqwgcAADAV4QMAAJiK8AEAAExF+AAAAKYifAAAAFMlFT6ampr06U9/Wl6vV0VFRbrlllv0m9/8Ju4awzC0Z88elZSUaOHChaqqqtKJEyfS3W4AAJClkgofHR0d2r59u55//nm1t7fr/fffV01NjS5evBi7Zu/evdq3b58OHDigzs5OBQIBVVdXa3BwMBPtBwAAWcZjGIaR6je//fbbKioqUkdHhz73uc/JMAyVlJSorq5O999/vyRpeHhYfr9fDz30kO666645f2YkEpHP51M4HFZBQUGqTQMAACZK5v49r5qPcDgsSSosLJQk9fX1KRQKqaamJnZNXl6eNmzYoKNHj077M4aHhxWJROIOAADgXCmHD8MwtGPHDt14442qrKyUJIVCIUmS3++Pu9bv98dem6ypqUk+ny92lJaWptokAACQBVIOH/fcc49efvllPfbYY1Ne83jit0w3DGPKuahdu3YpHA7HjtOnT6faJAAAkAUWpPJN9957r37+85/rueee09KlS2PnA4GANN4DUlxcHDvf398/pTckKi8vT3l5eak0AwAAZKGkej4Mw9A999yjf/mXf9Gzzz6rsrKyuNfLysoUCATU3t4eOzcyMqKOjg6tX78+fa0GAABZK6mej+3bt+vQoUM6fPiwvF5vrI7D5/Np4cKF8ng8qqurU2Njo8rLy1VeXq7GxkYtWrRImzdvztS/AQAAZJGkwkdzc7MkqaqqKu78j3/8Y915552SpJ07d+rSpUvatm2bLly4oLVr1+rIkSPyer3pbDcAAMhS81rnIxNY5wMAgOxj2jofAAAAyXJf+LBXRw8AAK6T0lTbrPXgg9Jf/ZXVrchePp/08Y9Pf/j90gxruYyOGTreN6D+wSEVefO1pqxQuTnTXwsAcD531Xy8+aZ0223S738vdXWl92cD6fLlL0u1tdKtt0pFRVa3BgASksz9213hw4nGxqSBAenUqZmPYNDqVgLWueYaqapKWrPm8nHdddKHPmR1qwDHIXzAcqNjhm586FkFw0PTvu6RFPDl69f3f8GeQzDhsPTv/y79/OdSa6t07pzVLQKsdfXV0rJll4/S0qn/e8mSGYde4Q7J3L/dVfMB0xzvG5gxeEiSISkYHtLxvgGtW77E1LYlxOeTbrnl8oH0eestqbNTOnbsg2No5r8T2MiZM5ePY8esbkl2ue66y3VxV14p/dEfSR/7mHTFFYkfublW/wsygvCBjOgfTOyGkuh1cAi/X/rjP758ID3Gxi4Prb76qtTb+8Hx6qv02NnBf/zH5UOSDh60ujUfePrpy/VlFiF8ICOKvPlpvQ7ADHJyLg+JXH21dNNNVrfGOd5/XxoclN5553JdXTLHyMgHP8frlT78Yendd6UvflH6xCcuv57KMTx8ecLE5PPvv5/8v8/iVccJH8iINWWFKvblKxQe0nRFRdGajzVlhRa0DgDmsGCBdNVVl49Jm6hi/ty3yBhMkZvjUX1thTQeNCaKfl1fW2HPYlMAQEYRPpAxGyuL1bxlpQK++KGVgC9fzVtWamNlcUI/Z3TM0LGT53W4+4yOnTyv0TFbTdACACSJYRdk1MbKYlVXBFJe4bStJ6iG1t64mTPFvnzV11YkHF4AAPbCOh+wrbaeoLa2dE2pGYnGlmR6TwAAmcU6H8h6o2OGGlp7py1WjZ574MlXdOn3YwoUsF8MAGQTwgdsaa5FyiRp4OLv9RdPdEsMxQBAVqHgFLaU7OJjofCQtrZ0qa2HfWwAwO4IH7ClZBcfiw7FNLT2MhsGAGyO8AFbii5SlkwVx8T9YgAA9kX4gC3NtkjZXNgvBgDsjfAB25ppkbK5sF8MANgbs11gaxMXKQuFL+lvnn5VFy6OsF8MAGQxwgdsLzfHo3XLl0iSFl6Rq60tXfJMKDIV+8UAQFZh2AVZJV37xQAArEPPB7LOfPeLAQBYi/CBrDRxKAYAkF0YdgEAAKYifAAAAFMRPgAAgKmo+QCALDI6ZiRUbJ3odYAVCB9wHD504VRtPUE1tPYqGP5gC4FiX77qayvippkneh1gFY9hGLbaAjQSicjn8ykcDqugoMDq5iDL8KELp2rrCWprS9eU1X2jsTq6zk2i1wHplsz9m5oPOEb0Q3di8JCkUHhIW1u61NYTtKxtwHyMjhlqaO2ddluB6LmG1l6NvD+W0HWjY7Z65oQLET7gCIl+OPOhi2x0vG9gSqieyJAUDA/pp8feSOi6430DGWopkBjCBxwh0Q9nPnSRjfoHZ/7bnujNgffS+vOATKHgFI4QCl9K6Do+dGFHcxVJF3nzZ/3+qGsKFyV03UeuzNOxk+cpyoZlCB/Iem09Qf3N068mdO1HrszLeHuswiyf7JRIkfSaskIV+/IVCg9NO7ToGd9c8Y51H9P//HXfrNd9eNGH9F//d7dCkeEZ3w/INIZdkNWiRaYDF0cS+waHlny09QR140PP6uuPPq/7Hu/W1x99Xjc+9CxFtjaXaJF0bo5H9bUV0oRZK1HRr+trK3TFgpxZrzMkXXjv93HBY7r3AzKN8IGsNVuR6UzOXRxO4Krswiyf7JRskfTGymI1b1mpgC9+CCbgy4+bPjvbdR9e9KFp20JRNszGsAuy1lxFptNJdOw8W8x1A/OM31CqKwIMwdhMMkXS0R2cN1YWq7oiMOfw2nTXjY0Zuv1//d+k3g/IFMIHslayxaNLFl+hVddclbH2WCGVGxjsIdG/38nX5eZ4EvpvOfm6w91n0touYD4YdkHWSrYX4/zFEX2m6d/0zMtnM9Yms6V6A4P1Ev37TVdvndnvB8yG8IGsFZ0BkMxgwsDF32vboZfU9ExvBltmHm4o2Wuuv1/P+CyUNWWFWfl+wGwIH8has80AmMuPnuvTMy+bV4g5Ombo2MnzOtx9RsdOnk9bUR83lOyV6AyWdNXqmP1+wGwIH8hqM1X2J+KvDveYUtmfyWmw3FCyW6IzWLL1/YCZsKstHGHiAluvvTWoA/9+MqHve+zbn8loIaZZO4yym292M3uBOBakQyYkc/9mtgscYWJl/7GT5xMOH5ksxDRzGmyiUzBhT4nOYMnW9wMmS3rY5bnnnlNtba1KSkrk8Xj01FNPxb1uGIb27NmjkpISLVy4UFVVVTpx4kQ62wzMak1ZoQoXT7+Y0mSZLMQ0e7O76A3l5uuv1rrlSwgeAGwr6fBx8eJFXXfddTpw4MC0r+/du1f79u3TgQMH1NnZqUAgoOrqag0ODqajvcCccnM8evDmyjmvy3QhJtNgrZWpIl8A85f0sMumTZu0adOmaV8zDEP79+/X7t27deutt0qSDh48KL/fr0OHDumuu+6af4uBBHx5RYnu+t07+tFzfdO+7jGhEJNpsNahBgawt7TOdunr61MoFFJNTU3sXF5enjZs2KCjR4+m862AOe36coUe3rxShYuviDtfPKGyP5NPx0yDtYZb9rqhZwfZLK0Fp6FQSJLk9/vjzvv9fr355pvTfs/w8LCGhz/Y7CsSiaSzSXCR6Sr4v7yiWF+qnL4QM9NPx9FpsFtbumI7ikYxDTYz3LLXDT07yHYZme3i8cT/n9owjCnnopqamtTQ0JCJZsBF5vownlzZP9MU2OjTcbqmwEbXVZjctoANbhROnG7phr1uzPrbBTIpreEjEAhI4z0gxcUf/PH39/dP6Q2J2rVrl3bs2BH7OhKJqLS0NJ3NgsMl+2Fs9tOxHafBOvXJ2elFvm7p2YHzpbXmo6ysTIFAQO3t7bFzIyMj6ujo0Pr166f9nry8PBUUFMQdQKLm+jDW+IfxxPFws6fAymbTYOdbE2HnWgOnF/la8bcLZELSPR/vvvuuXn/99djXfX196u7uVmFhoZYtW6a6ujo1NjaqvLxc5eXlamxs1KJFi7R58+Z0tx1IqZvd6U/Hs5nvk7Pde0yiRb6h8NC0/0bP+JBXthb5uvlvF86SdM/HCy+8oBtuuEE33HCDJGnHjh264YYb9Nd//deSpJ07d6qurk7btm3T6tWrdebMGR05ckRerzf9rYfrpfJh7PSn49nM58k5G2aROH2vGzf/7cJZkg4fVVVVMgxjyvGTn/xEGi823bNnj4LBoIaGhtTR0aHKyrkXfAJSkcqHsZunwKb65JzK8JZVnLx5mpv/duEs7O2CrJZKN7ubp8Cm+uScbbNI7Fjkmw5u/tuFs6S14BQwW6rd7E5+Op5Nqk/O2VhrYKci33Ry698unIWeD2S9VNfScOrT8WxSfXKm1sBe3Pi3C2fxGIZh/SDtBJFIRD6fT+FwmGm3SIoTF83KlGRnrYyOGbrxoWfnHN769f1f4HcOuFQy92/CB+BSyYa16GwXzdBjQpc/4G6ED8AFrOjpsfs6HwCsk8z9m5oPwIbmChZWhQBqDQCkAz0fgM3MFSxm2suG4Q8AVkrm/s1UW8BG5lpF9JmXz2bNYl9msfNeMwCmx7ALYBOJ7Lvy3w/3aODi72f8Gela7CtbZg5RgwJkJ8IHYBOJrCI6W/CYaD6LfWXLDX2m4adoL9H/2HyDrlqcZ/sABbgR4QOwiXSuDprIYl/T9W6094ZmvaHbpZ4kkb1m7nnsJU0cgbFjgALcivAB2ESiq4MWLr5CFy6OzGvL+Ol6NwIFeRp6f2zWYZ+G1l5VVwQs70GYq5dIkiaXftgtQAFuRsEpYBOJ7rvy4M2Vsa8nv64ENhabsag1Mqx33kusnsRqqfQSubUgF7AjwgdgE4lukvflFalvLDbbcEWi7LB5XKp7yNgpQAFuxrALYCOJbpKX6mJfiQxXzMUOm8dFe4lm2mtmLnYIUNPJlllGwHwRPgCbSTRYRLeMT8Z8brqJ1pOYYbbdeRNhhwA1WbbMMgLSgWEXwIaiweLm66/WuuVL0vb0m+pNN9F6klSkukhYtJdo8vDTbM2L1s3YIUBNNNficm09QcvaBmQCPR+Ai8w1XOGR5Fv0IeUvyFUoMvOwT7rM92l/ul6iCxdHtP3QzLvvZiJAzUcii8vZZZYRkC6ED8BFZhuuiN7WvnvrtaZsHjfXImGJTomdbvipOWfuuhm7SGRxuXSsWgvYCeEDcJlEi1ozeaPL9NN+qgW5VhR8JlqHY9ciWSAVhA/AhVK9OaeLGU/7yRbkWlXwmWgdjh2LZIFUET4Al0pltky62O1pP11DQKlIpA7HLrOMgHRhtgsA09npaT+RfWIyuSpqoovLUWwKJyF8ADBdokvJm/G0n8wQUKbMNG04kVVrgWzEsAtgA25b2TKRWTdmPe3bZQjI6jocwEyED8Bibl3ZMtFZN8lIJcTZaQjIyjocwEyED8BCVhY62kE6n/ZTDXEUfALmo+YDsIjVhY52kY6l5OezPDkFn4D5CB+ARexQ6OgE6QhxFHwC5mLYBbCIXQods126Fiyj4BMwD+EDsIidCh2zWTpDHAWfgDkYdgEsYqe1LrIZIQ7IPoQPwCIUOqYHIQ7IPoQPwEIUOs4fIQ7IPh7DMGw1jy8Sicjn8ykcDqugoMDq5gCmcNsKp5ng1sXaALtI5v5N+AAwrWwMRNnYZsApkrl/M9sFwBTZ2ovAbBUgO1DzASDOfFYLBYBEED4AxLDkOwAzED4AxLDkOwAzED4AxLDkOwAzUHAKIIbVQj/AzBkgcwgfAGKiq4WGwkPT1n14xhdAc/pqodk62wfIFgy7AIhhtVBm+wBmIHwAiOPmJd+Z7QOYg2EXAFNsrCxWdUXAdTUPycz2YTEzIHWEDwDTcuNqocz2AcyRsWGXhx9+WGVlZcrPz9eqVav0q1/9KlNvBQBpwWwfwBwZCR9PPPGE6urqtHv3br300kv67Gc/q02bNunUqVOZeDsASIvobJ+ZBpc847NenD7bB8i0jISPffv26Zvf/Ka+9a1v6ZOf/KT279+v0tJSNTc3Z+LtACAtmO0DmCPt4WNkZEQvvviiampq4s7X1NTo6NGj6X47AEgrN8/2AcyS9oLTc+fOaXR0VH6/P+683+9XKBSacv3w8LCGh4djX0cikXQ3CQCS4tbZPoBZMjbbxeOJ/z+pYRhTzklSU1OTGhoaMtUMAEiJG2f7AGZJ+7DLRz7yEeXm5k7p5ejv75/SGyJJu3btUjgcjh2nT59Od5MAAICNpD18XHHFFVq1apXa29vjzre3t2v9+vVTrs/Ly1NBQUHcAQAAnCsjwy47duzQHXfcodWrV2vdunV65JFHdOrUKd19992ZeDsAAJBFMhI+brvtNp0/f17f+c53FAwGVVlZqWeeeUbXXHNNJt4OAABkEY9hGLbaISkSicjn8ykcDjMEAwBAlkjm/s2utgAAwFSEDwAAYCrCBwAAMBXhAwAAmIrwAQAATEX4AAAApiJ8AAAAUxE+AACAqTK2qy2A7DY6ZrClPICMIHwAmKKtJ6iG1l4Fw0Oxc8W+fNXXVmhjZbGlbQOQ/Rh2ARCnrSeorS1dccFDkkLhIW1t6VJbT9CytgFwBsIHgJjRMUMNrb2absOn6LmG1l6NjtlqSygAWYbwASDmeN/AlB6PiQxJwfCQjvcNmNouAM5C+AAQ0z84c/BI5ToAmA7hA0BMkTc/rdcBwHQIHwBi1pQVqtiXr5km1HrGZ72sKSs0uWUAnITwASAmN8ej+toKaTxoTBT9ur62gvU+AMwL4QNAnI2VxWreslIBX/zQSsCXr+YtK1nnA8C8scgYgCk2VharuiLACqcAMoLwAWBauTkerVu+xOpmAHAghl0AAICpCB8AAMBUhA8AAGAqwgcAADAV4QMAAJiK8AEAAExF+AAAAKYifAAAAFMRPgAAgKlst8KpYRiSpEgkYnVTAABAgqL37eh9fDa2Cx+Dg4OSpNLSUqubAgAAkjQ4OCifzzfrNR4jkYhiorGxMZ09e1Zer1cej7M3sYpEIiotLdXp06dVUFBgdXNchd+9Nfi9W4ffvXXc8rs3DEODg4MqKSlRTs7sVR226/nIycnR0qVLrW6GqQoKChz9B2ln/O6twe/dOvzureOG3/1cPR5RFJwCAABTET4AAICpCB8WysvLU319vfLy8qxuiuvwu7cGv3fr8Lu3Dr/7qWxXcAoAAJyNng8AAGAqwgcAADAV4QMAAJiK8AEAAExF+LCZ4eFhXX/99fJ4POru7ra6OY73xhtv6Jvf/KbKysq0cOFCLV++XPX19RoZGbG6aY708MMPq6ysTPn5+Vq1apV+9atfWd0kx2tqatKnP/1peb1eFRUV6ZZbbtFvfvMbq5vlOk1NTfJ4PKqrq7O6KbZA+LCZnTt3qqSkxOpmuMZ//ud/amxsTD/60Y904sQJ/f3f/71++MMf6oEHHrC6aY7zxBNPqK6uTrt379ZLL72kz372s9q0aZNOnTplddMcraOjQ9u3b9fzzz+v9vZ2vf/++6qpqdHFixetbpprdHZ26pFHHtGKFSusboptMNXWRn7xi19ox44d+tnPfqZPfepTeumll3T99ddb3SzX+d73vqfm5mb99re/tbopjrJ27VqtXLlSzc3NsXOf/OQndcstt6ipqcnStrnJ22+/raKiInV0dOhzn/uc1c1xvHfffVcrV67Uww8/rAcffFDXX3+99u/fb3WzLEfPh0289dZb+va3v62f/vSnWrRokdXNcbVwOKzCwkKrm+EoIyMjevHFF1VTUxN3vqamRkePHrWsXW4UDoclib9xk2zfvl1f+cpXdNNNN1ndFFux3cZybmQYhu68807dfffdWr16td544w2rm+RaJ0+e1A9+8AP93d/9ndVNcZRz585pdHRUfr8/7rzf71coFLKsXW5jGIZ27NihG2+8UZWVlVY3x/Eef/xxdXV1qbOz0+qm2A49Hxm0Z88eeTyeWY8XXnhBP/jBDxSJRLRr1y6rm+wYif7uJzp79qw2btyoP/3TP9W3vvUty9ruZB6PJ+5rwzCmnEPm3HPPPXr55Zf12GOPWd0Uxzt9+rTuu+8+tbS0KD8/3+rm2A41Hxl07tw5nTt3btZrPvaxj+lrX/uaWltb4z6ER0dHlZubq9tvv10HDx40obXOkujvPvqhcPbsWX3+85/X2rVr9ZOf/EQ5OeTydBoZGdGiRYv0T//0T/rqV78aO3/fffepu7tbHR0dlrbPDe6991499dRTeu6551RWVmZ1cxzvqaee0le/+lXl5ubGzo2Ojsrj8SgnJ0fDw8Nxr7kN4cMGTp06pUgkEvv67Nmz+tKXvqR//ud/1tq1a7V06VJL2+d0Z86c0ec//3mtWrVKLS0trv5AyKS1a9dq1apVevjhh2PnKioqdPPNN1NwmkGGYejee+/Vk08+qV/+8pcqLy+3ukmuMDg4qDfffDPu3J/92Z/pE5/4hO6//37XD3tR82EDy5Yti/v6yiuvlCQtX76c4JFhZ8+eVVVVlZYtW6a//du/1dtvvx17LRAIWNo2p9mxY4fuuOMOrV69WuvWrdMjjzyiU6dO6e6777a6aY62fft2HTp0SIcPH5bX643V2Ph8Pi1cuNDq5jmW1+udEjAWL16sJUuWuD54iPABtzty5Ihef/11vf7661OCHp2C6XXbbbfp/Pnz+s53vqNgMKjKyko988wzuuaaa6xumqNFpzZXVVXFnf/xj3+sO++806JWwe0YdgEAAKaiqg4AAJiK8AEAAExF+AAAAKYifAAAAFMRPgAAgKkIHwAAwFSEDwAAYCrCBwAAMBXhAwAAmIrwAQAATEX4AAAApiJ8AAAAU/1/rGac3e84T+0AAAAASUVORK5CYII=",
|
|
"text/plain": [
|
|
"<Figure size 640x480 with 1 Axes>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"## Generate data\n",
|
|
"import numpy as np\n",
|
|
"import matplotlib.pyplot as plt\n",
|
|
"\n",
|
|
"# 1) Generate quadratic data\n",
|
|
"np.random.seed(3)\n",
|
|
"\n",
|
|
"n = 50\n",
|
|
"x = np.random.uniform(-5, 5, n) # symmetric, wider range\n",
|
|
"\n",
|
|
"# True relationship: y = ax^2 + c + noise\n",
|
|
"a_true = 2.0\n",
|
|
"c_true = 5.0\n",
|
|
"noise = np.random.normal(0, 3, n)\n",
|
|
"\n",
|
|
"y = a_true * x**2 + c_true + noise\n",
|
|
"\n",
|
|
"# find a line of best fit\n",
|
|
"a,b = np.polyfit(x, y, 1)\n",
|
|
"\n",
|
|
"# add scatter points to plot\n",
|
|
"plt.scatter(x,y)\n",
|
|
"\n",
|
|
"# add line of best fit to plot\n",
|
|
"plt.plot(x, a*x + b, 'r', linewidth=1)\n",
|
|
"\n",
|
|
"# plot it\n",
|
|
"plt.show()\n",
|
|
"\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "dbb01960",
|
|
"metadata": {},
|
|
"source": [
|
|
"- **Overfitting**: the model captures noise rather than structure. Often due to model complexity relative to data size. Polynomial regression can give a nice visualization of overfitting. For example, if we worked with the same generated quadratic data from the polynomial regression section, and we tried to approximation it by a degree 11 polynomial, we get the following.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"id": "43ab6a3f",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGxCAYAAADCo9TSAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAaDxJREFUeJzt3Xd4U2X/x/F3WmhLoRTLaMuuKEItG0EQBUEQFERxb5wIOBBRRB4FHCwVERFUVOCR4WSIIj9RoC4QEKosfQSLrJYy2zLaQnt+f5wmaWgLnTkZn9d15eI+J3dOvok1+eaeNsMwDERERETcJMDqAERERMS/KPkQERERt1LyISIiIm6l5ENERETcSsmHiIiIuJWSDxEREXErJR8iIiLiVko+RERExK2UfIiIiIhbKfkQnzRr1ixsNpvjFhISQlRUFFdeeSXjxo0jJSXF6hDdYs+ePQwZMoTOnTtTrVo1bDYbs2bNKrDuV199xT333EOzZs2oWLEiNputWM+V9/222WyEh4fTpUsXvv7662LHbf/vt3PnzmI/1hM1bNiQ/v37l+tjGzZsmO+/gf127Ngx+vfvT8OGDV0eM3bsWBYtWlSiuERKQ8mH+LSZM2eyevVqli9fzttvv03Lli2ZMGECTZs25bvvvrM6vHK3fft25s6dS1BQENdcc81Z6y5cuJA1a9YQGxtLixYtSvR8N910E6tXr+bnn3/m7bffJjk5mT59+pQoAfElCxcu5Pnnny/357nssstYvXp1vltoaCjPP/88CxcudKmv5EOsUsHqAETKU1xcHG3btnUc33jjjTz55JN06tSJfv368ffffxMZGenWmE6ePEmlSpXc8lxXXHEFBw4cAGD9+vXMnz+/0LozZswgIMD8PfLoo4/y22+/Ffv5IiMjufTSSwHo2LEjHTp04IILLmDy5Mlce+21JX4d3q5Vq1ZueZ5q1ao53v8zNWrUyC0xiBSFWj7E79SvX5/XX3+d9PR03n33XZf71q9fz3XXXUdERAQhISG0atWKTz/9NN81fvrpJzp06EBISAh16tTh+eef5/3338/XVdCwYUN69+7NggULaNWqFSEhIYwZMwaA5ORkBgwYQN26dQkKCiImJoYxY8Zw+vRpl+fKysri5ZdfpkmTJgQHB1OzZk3uu+8+R1JxNvZkoiiKU7eoGjVqRM2aNfn3338d57788ks6dOhAaGgoYWFhdO/endWrV5/1Oi+99BIVKlRg9+7d+e67//77qV69OhkZGZDnPV+2bBmtW7emUqVKNGnShA8//DDfYzdv3kzfvn0577zzCAkJoWXLlsyePdulzqpVq7DZbMybN4/hw4cTHR1NlSpV6NOnD/v37yc9PZ2HH36YGjVqUKNGDe677z6OHTvmco0zu04yMjJ46qmnaNmyJeHh4URERNChQwcWL15cjHe3eM7sdrHZbBw/fpzZs2c7ume6dOlSbs8vkpdaPsQvXXPNNQQGBvLDDz84zq1cuZKePXvSvn173nnnHcLDw/n444+59dZbOXHihOPL448//qB79+40btyY2bNnExoayjvvvMOcOXMKfK4NGzawbds2/vOf/xATE0PlypVJTk6mXbt2BAQE8MILL9CoUSNWr17Nyy+/zM6dO5k5cyYAOTk59O3blx9//JFnnnmGjh078u+//zJq1Ci6dOnC+vXr3daKUhJHjhzh0KFDXHjhhQDMmzePO++8kx49ejB//nwyMzOZOHEiXbp04fvvv6dTp04FXmfAgAG88sorvPvuu7z88suO84cPH+bjjz/m0UcfJSQkxHH+999/56mnnuLZZ58lMjKS999/nwceeIALLriAK664AoC//vqLjh07UqtWLaZMmUL16tWZM2cO/fv3Z//+/TzzzDMuMTz33HNceeWVzJo1i507dzJs2DBuv/12KlSoQIsWLZg/fz4bN27kueeeIywsjClTphT6vmRmZnL48GGGDRtGnTp1yMrK4rvvvqNfv37MnDmTe+65p0Tvt2EY+ZLXgICAAhPL1atX07VrV6688kpHl1DVqlVL9LwixWaI+KCZM2cagLFu3bpC60RGRhpNmzZ1HDdp0sRo1aqVcerUKZd6vXv3NqKjo43s7GzDMAzj5ptvNipXrmwcOHDAUSc7O9uIjY01ACMxMdFxvkGDBkZgYKDx119/uVxzwIABRpUqVYx///3X5fxrr71mAMaWLVsMwzCM+fPnG4DxxRdfuNRbt26dARjTpk0r8ntif8zMmTPPWXfw4MFGcT8eAGPQoEHGqVOnjKysLGPbtm1Gr169DMB4++23jezsbKN27dpGs2bNHO+lYRhGenq6UatWLaNjx46Oc/b/fnnfy3vvvdeoVauWkZmZ6Tg3YcIEIyAgIN97HhIS4vLenjx50oiIiDAGDBjgOHfbbbcZwcHBxq5du1xeR69evYzQ0FDj6NGjhmEYxsqVKw3A6NOnj0u9IUOGGIDx+OOPu5y//vrrjYiICJdzDRo0MO69995C37vTp08bp06dMh544AGjVatWxXps3npAvtvIkSMNI/f9a9CggctjKleuXKRri5Q1dbuI3zK/L03bt2/nzz//5M477wTg9OnTjts111xDUlISf/31FwDx8fF07dqVGjVqOB4fEBDALbfcUuDzNG/enMaNG7uc++qrr7jyyiupXbu2y3P16tXL8Rz2etWqVaNPnz4u9Vq2bElUVBSrVq0qh3em5KZNm0bFihUJCgqiadOm/PLLL7z44osMGjSIv/76i3379nH33Xe7/BKvUqUKN954I2vWrOHEiROFXvuJJ54gJSWFzz77DHJbhaZPn861116bbxZHy5YtqV+/vuM4JCSExo0bu3T/rFixgm7dulGvXj2Xx/bv358TJ07k6wrq3bu3y3HTpk0B8o1ladq0KYcPH87X9XKmzz77jMsuu4wqVapQoUIFKlasyAcffMC2bdvO+riz6dSpE+vWrXO5DRo0qMTXEykv6nYRv3T8+HEOHTpEs2bNANi/fz8Aw4YNY9iwYQU+5uDBgwAcOnSowEGqhQ1cjY6Ozndu//79LFmyhIoVK571ufbv38/Ro0cJCgo6az1Pccstt/D0009js9kICwujUaNGBAYGQu77RiHvR+3atcnJyeHIkSOEhoYWeO1WrVpx+eWX8/bbb3PnnXfy1VdfsXPnznzjdgCqV6+e71xwcDAnT550HB86dKjQWPLGaxcREeFybP9vUtj5jIwMqlSpUuBrWbBgAbfccgs333wzTz/9NFFRUVSoUIHp06cXODalqMLDw10GWIt4KiUf4pe+/vprsrOzHQPs7K0YI0aMoF+/fgU+5qKLLoLcLzZ7spJXcnJygY8raL2MGjVq0Lx5c1555ZUCH2P/AqxRowbVq1dn2bJlBdYLCwsr5BVao2bNmoV++dkTgqSkpHz37du3j4CAAM4777yzXv/xxx/n5ptvZsOGDUydOpXGjRvTvXv3EsVavXr1QmMhz99EeZgzZw4xMTF88sknLn8fmZmZ5facIp5EyYf4nV27djFs2DDCw8MZMGAA5CYWF154Ib///jtjx4496+M7d+7M0qVLOXjwoOMLKicnx9EdUBS9e/dm6dKlNGrU6KxfuL179+bjjz8mOzub9u3bF/n6nuiiiy6iTp06zJs3j2HDhjm+dI8fP84XX3zhmAFzNjfccAP169fnqaeeIj4+njfeeKPYi6HZdevWjYULF7Jv3z5Hsgfw3//+l9DQ0EKnrJYFm81GUFCQS+zJycnlOtulIGe2Bom4i5IP8WmbN292jJNISUnhxx9/ZObMmQQGBrJw4UJq1qzpqPvuu+/Sq1cvrr76avr370+dOnU4fPgw27ZtY8OGDY7kYuTIkSxZsoRu3boxcuRIKlWqxDvvvMPx48ehiFNWX3zxRZYvX07Hjh15/PHHueiii8jIyGDnzp0sXbqUd955h7p163Lbbbcxd+5crrnmGp544gnatWtHxYoV2bNnDytXrqRv377ccMMNZ32uzz//HIB//vkHcqcT27sDbrrpJke9f//9l3Xr1gGwY8cOl8c2bNiw1M35AQEBTJw4kTvvvJPevXszYMAAMjMzefXVVzl69Cjjx48/5zUCAwMZPHgww4cPp3LlyiVeNRRg1KhRjrE3L7zwAhEREcydO5evv/6aiRMnEh4eXuJrn4t9+vWgQYO46aab2L17Ny+99BLR0dH8/fff5fa8Z2rWrBmrVq1iyZIlREdHExYW5mjhEylXVo94FSkP9tkS9ltQUJBRq1Yto3PnzsbYsWONlJSUAh/3+++/G7fccotRq1Yto2LFikZUVJTRtWtX45133nGp9+OPPxrt27c3goODjaioKOPpp582JkyYYACOWRJG7gyEa6+9tsDnOnDggPH4448bMTExRsWKFY2IiAijTZs2xsiRI41jx4456p06dcp47bXXjBYtWhghISFGlSpVjCZNmhgDBgww/v7773O+FwXNgLDfzvae5b0VZUYEYAwePPic9RYtWmS0b9/eCAkJMSpXrmx069bN+PnnnwuMJe8sFrudO3cagPHII48UeP3C3vPOnTsbnTt3djm3adMmo0+fPkZ4eLgRFBRktGjRIt9sIPtsl88++6zAGM+cUTVq1CgDcJkNVdCMlfHjxxsNGzY0goODjaZNmxozZsxwPPbM11PU2S6F/a0Zhcx2SUhIMC677DIjNDTUAPK9PyLlxWbkHfIvIiXWo0cPdu7cyf/+9z+rQ/Fpb731Fo8//jibN2/m4osvtjocESkBdbuIlMDQoUNp1aoV9erV4/Dhw8ydO5fly5fzwQcfWB2az9q4cSOJiYm8+OKL9O3bV4mHiBdT8iFSAtnZ2bzwwgskJydjs9mIjY3lo48+4q677rI6NJ91ww03kJyczOWXX84777xjdTgiUgrqdhERERG30gqnIiIi4lZKPkRERMStlHyIiIiIW3ncgNOcnBz27dtHWFhYiVcuFBEREfcyDIP09HRq1659zsUWPS752LdvX75dJkVERMQ77N69m7p16561jsclH/aNsnbv3k3VqlWtDkdERESKIC0tjXr16hVpw0uPSz7sXS1Vq1ZV8iEiIuJlijJkQgNORURExK2UfIiIiIhbKfkQERERt1LyISIiIm6l5ENERETcSsmHiIiIuJWSDxEREXErJR8iIiLiVko+RERExK2UfIiIiIhbKfkQERERt1LyISIiIm6l5ENERMSXJCfDgAEQFwf9+sGmTVZHlI/H7WorIiIiJbRvH3TuDNu3m8dbtsDKlfDzzxAba3V0Dmr5EBER8QWGAfff70w87I4ehXvvNe/3EEo+REREfMEXX8D//Z9ZrlPH7G6xt3asXw9ff21peHkp+RAREfF2hgEvvug8njrVHPMxdqzz3OTJloRWECUfIiIi3m7ZMufA0vbtoW9fs3zddXD++WZ55UpISbEuxjz8MvnIzjFYveMQixP2snrHIbJzPKcfTEREpNjee89ZHj4cbDazbLPBLbeY5Zwcs2vGA9gMw4NGoABpaWmEh4eTmppK1apVy+y62TkGm7+OJ/2/c3m82U0cPnHacV90eAij+sTSMy66zJ5PRETELZKToW5dyM6G6GjYtQsq5JnMunEjtG5tlnv3hiVLyiWM4nx/+0XLx7LNSXx6xc20uO5KOn3+Pk23rne5Pzk1g4FzNrBsc5JlMYqIiJTI55+biQdA//6uiQdAy5ZQo4ZZ/uEHZ10L+XzysWxzEgPnbODn6o0c5wat+dSljr3pZ8ySreqCERER77JwobN8223577fZoEsXs5yWBgkJ7outED6dfGTnGIxZshUDWHrRZSSeZ3arXPbvH7TY95dLXQNISs1gbeJhi6IVEREppkOHID7eLJ9/PjRrVnA9e/IBsGqVe2I7C59OPtYmHiYpNQOAnIBA3m13o+O+QWs+K/AxKekZbotPRESkVL76ytmN0q+fc6DpmTp1cpbXry+4jhv5dPJxZiKxIK4byVUiALj67zVccHBXvsfUCgtxW3wiIiKlkrfL5YYbCq8XGwvBwWZ5w4byj+scfDr5ODORyKpQkfcvud5xPPDXzx1lW+6sl3YxEW6NUUREpEROnHCuaBoVBZdeWnjdihWheXOz/L//mWM/LOTTyUe7mAiiw0PI2wg1v0VPjoZUAeC6rfHUSXUuuDKqTyyBAYU0WYmIiHiSH3+EjNwW/j59IOAcX+n26bYAv/9evrGdg08nH4EBNkb1Mde1t6cUx4NDmd26DwAVc7J5aO0CosNDmH5Xa63zISIi3uO775zl7t3PXT9v8mHxjBefTj4AesZFM/2u1kSFO7tgZrbtw8mKZt/XPVu+46f+FyvxEBER72JPPmw2uPLKc9e/+GJHMfGn3yxd4btCEep4vZ5x0XSPjWJt4mFS0jOoFRZCcMBAmDyZgMwMeHMyjB9vdZgiIiJFc/Cgs/WiVSvnImJn8b1RjW655aQ1G7ljxhrLVvj2+ZYPu8AAGx0aVadvyzp0aFSdgKefhqAg886334bDWt9DRES8xIoVznK3bmerCbkLbj741U4OhoYD0OjwHrBwhW+/ST7yqV0bHnjALB87BlOmWB2RiIhI0eQd73HVVWetmnfBzR3V6wEQeewwYZnHLVvh23+TD4BnnnGugf/mm5ZPPRIRESmS7783/w0Kcl1ArAB5F9zcEVHXcb7RIbP1w4oVvv07+WjYEO6+2ywfPWp2v4iIiHiyvXvhn3/M8qWXQmjoWavnXXBze27LB8AFh3YXWq+8+XfyAfDss8650ZMmwfHjVkckIiJSuJ9+cpYvv/yc1fMuuLkpqhHfN7qE9y65gb/zJCJn1itvfjHb5awaN4Zbb4X5883Rw++9B08+aXVUIiIiBcubfJyjy4U8C24mp2awrl4c6+rFudxvA6LcvMK3Wj4AnnvOWX71VeeKcSIiIp7mxx/NfwMCoGPHc1YvaMFNO/uxu1f4VvIBEBfn3JAnKQlmzrQ6IhERkfxSU+GPP8xy8+ZQtWqRHlbQgpvktnhYscK3ul3sRo507g44fjw8+KC5EY+IiIinWL0ajNwpsUXocsmroAU328VEWLKnmVo+7Nq0gV69zPKuXTBnjtURiYiIuLJ3uVC0waZnOnPBTas2Uy1W8jF69GhsNpvLLSoqynG/YRiMHj2a2rVrU6lSJbp06cKWLVvKI+7y8fzzzvLYsZCdbWU0IiIirn75xVm+7DIrIymVYrd8XHzxxSQlJTlumzZtctw3ceJEJk2axNSpU1m3bh1RUVF0796d9PT0so67fHToAF27muXt2+HTT62OSERExJSdDevXm+X69aFOHasjKrFiJx8VKlQgKirKcatZsybktnpMnjyZkSNH0q9fP+Li4pg9ezYnTpxg3rx55RF7+fjPf5zlV16BnBwroxERETH9+ae5HQhAu3ZWR1MqxU4+/v77b2rXrk1MTAy33XYb/+SuspaYmEhycjI9evRw1A0ODqZz5878kreZ6AyZmZmkpaW53CzVpYtz6tKWLbBokbXxiIiIAKxd6yxfcomVkZRasZKP9u3b89///pf/+7//Y8aMGSQnJ9OxY0cOHTpEcnIyAJGRkS6PiYyMdNxXkHHjxhEeHu641atXr9C6bmGzubZ+vPyyc2SxiIiIVfImH/7U8tGrVy9uvPFGmjVrxlVXXcXXX38NwOzZsx11bDbXkbOGYeQ7l9eIESNITU113Hbv3l1oXbfp2RNatzbLGzfCN99YHZGIiPg7e/Jhs5kzNL1YqabaVq5cmWbNmvH33387Zr2c2cqRkpKSrzUkr+DgYKpWrepys9yZrR8vvaTWDxERsc7Jk87FxWJjISzM6ohKpVTJR2ZmJtu2bSM6OpqYmBiioqJYvny54/6srCzi4+PpWITlXz1O374YF19sltesYcvcRWTnKAERERELJCTA6dNm2cu7XChu8jFs2DDi4+NJTEzk119/5aabbiItLY17770Xm83GkCFDGDt2LAsXLmTz5s3079+f0NBQ7rjjjvJ7BeVk2db9jIrr6zhOHTmaThNWsGxzkqVxiYiIf8nOMUhcutJxnNPWuwebUtzkY8+ePdx+++1cdNFF9OvXj6CgINasWUODBg0AeOaZZxgyZAiDBg2ibdu27N27l2+//ZYwL2seWrY5iYFzNjCnfnv+Oa82AB13/UGdLRsYOGeDEhAREXGLZZuT6DRhBb8v+NZx7r6tNq//HrIZhmcNZkhLSyM8PJzU1FRLxn9k5xh0mrCCpFRzZ9ubNn3Ha0snA7Dy/Dbcf/MYosJD+Gl4V8uWpRUREd9n/yFsACvfe4iYI0lkBlYk7snPOB1YwZIN4c6mON/f2tvlDGsTDzsSD4BFsV3YU7UWAFf+8xsXJ28nKTWDtYmHLYxSRER8WXaOwZglWzGA8JPpxBwxWzo2RzbiVKC5J+yYJVu9diyiko8zpKRnuByfDqzA9Etvchw/uvqTAuuJiIiUlbw/hFsk/c9x/vfoxgAY4NU/hJV8nKFWWEi+c583u4rkKhEA9Pzfahof2FlgPRERkbKQ9wdu3uQjoXbjQut5EyUfZ2gXE0F0eAh5R3NkVgjivXY3Oo6Hrf+CdjERlsQnIiK+L+8P3ObJfzvK9paPgup5EyUfZwgMsDGqTyyASwIyv8XVHAwNB6D75ngCt/9dyBVERERKx/FD2DBomdvycTSkCv9WMweY2oDo8BCv/SGs5KMAPeOimX5Xa6LCnRnlyaAQPr3cHPthy8mBiRMtjFBERHyZ/Ydw7fQD1Dx+FOytHjab44fxqD6xXjvrsoLVAXiqnnHRdI+NYm3iYVLSM6gVFkK76p2g4eeQmgoffWRuOpe7rLyIiEhZ6hkXTeQFpx3HCbldLlHhIYzqE+tR02yLS8nHWQQG2OjQqLrryYcfhldfhawsePttc98XERGRctAqz3iP9rf2ZP61l9IuJsJrWzzs1O1SXI8/DhVyc7Zp0+D4casjEhERX2XfyRa49NaedGhU3esTD5R8lEDdunD77Wb58GGYPdvqiERExBdlZ8P69Wa5QQM4yw7x3kbJR0k89ZSzPGmS+QciIiJSlrZtc7auX+L9m8nlpeSjJFq0gKuuMss7dsDixVZHJCIiviZPlwvt2lkZSZlT8lFSw4Y5y6+9ZmUkIiLii5R8SD49ekBcnFlevRp++cXqiERExJfYk4+AAGjTxupoypSSj5Ky2VzHfrz+upXRiIiILzl5Ev74wyzHxkKVKlZHVKaUfJTG7bdDdO4iLwsXwvbtVkckIiK+YONG52QGH+tyQclHKQUHw2OPmWXDgMmTrY5IRER8gQ+P90DJRxkYMAAqVzbLH35orv0hIiJSGko+5KwiIuC++8zyyZPwwQdWRyQiIt7OnnyEhDgnN/gQJR9lwd71AjB1Kpw+fbbaIiIihTt82FxDCqB1a6hY0eqIypySj7LQuDFcc41Z3rULvvzS6ohERMRbrVvnLPtglwtKPsrQ4487y1OmWBmJiIh4Mx8f74GSjzLUowc0aWKW4+MhIcHqiERExBsp+ZAis9lcx3689ZaV0YiIiDcyDGfyEREB559vdUTlQslHWbrnHggPN8tz58KBA1ZHJCIi3mTXLkhJMcvt2pk/bH2Qko+yVKUKPPCAWc7MhBkzrI5IRES8iR90uaDkoxw8+qi5CRDAtGlw6pTVEYmIiLdQ8iElEhMD111nlvfuhQULrI5IRES8Rd7k45JLrIykXCn5KA95p92++aaVkYiIiLc4fRrWrzfLDRpArVpWR1RulHyUhy5doFkzs7x6teuCMSIiIgXZvBlOnDDLl15qdTTlSslHebDZtOiYiIgUz6+/Osvt21sZSblT8lFe7rzTnKMN8MknZO/dx+odh1icsJfVOw6RnWNYHaGIiHiSNWucZR9v+ahgdQA+q1IlePhhGD8eTp3iw3uf45W2Nzvujg4PYVSfWHrGRVsapoiIeAh7y0fFitCqldXRlCu1fJSngQMxcqfd9vl1CRWynbvdJqdmMHDOBpZtTrIwQBER8QhHj8K2bWa5ZUsICbE6onKl5KMcZdetxw9NOgAQdeww3f92NqnZO13GLNmqLhgREX+Xd4qtj3e5oOSjfK1NPMx7zXo5ju/Z+LXL/QaQlJrB2sTDFkQnIiIew48Gm6Lko3ylpGfwc4MW7IioC0CHXZu48MC/BdYTERE/5keDTVHyUb5qhYWAzcZHra5xnLt749KC64mIiH8yDGfLR40aPruTbV5KPspRu5gIosNDWNCsG8crmglGvy0rqJJpLiJjy5310i4mwuJIRUTEMjt2wKFDZrl9e5/dyTYvJR/lKDDAxqg+saQHV2bRxV0AqJJ1khu2rMD+pzWqTyyBAb7/hyYiIoXI2+XiB+M9UPJR/nrGRTP9rtZ8c0U/x7l7NnxNVNVgpt/VWut8iIj4u7yDTf1gvAdKPtyjZ1w0s994kLRLzD+qCw/t5qf2AUo8RETE2fJhs0G7dlZH4xZKPtwkMMBG1aeGOI+nT7M0HhER8QAnT0JCgllu0gTCw62OyC2UfLjTDTdAVJRZXrQI9uyxOiIREbHSxo1wOnf1az/pckHJh5sFBZn7vQBkZ8O771odkYiIWMkPB5ui5MMCDz8MgYFmecYMyMqyOiIREbHKzz87yx06WBmJW5Uq+Rg3bhw2m40hQ5xjGQzDYPTo0dSuXZtKlSrRpUsXtmzZUhax+oY6dczuF4D9++GLL6yOSERErGAY8NNPZjk8HOLirI7IbUqcfKxbt4733nuP5s2bu5yfOHEikyZNYurUqaxbt46oqCi6d+9Oenp6WcTrGwYPdpbfftvKSERExCrbt0NKilm+7DII8J/OiBK90mPHjnHnnXcyY8YMzjvvPMd5wzCYPHkyI0eOpF+/fsTFxTF79mxOnDjBvHnzyjJu79a5M1x8sVn++Wf4/XerIxIRkXKSnWOweschFifsZfWOQ86dzO2tHgCdOlkWnxVKlHwMHjyYa6+9lquuusrlfGJiIsnJyfTo0cNxLjg4mM6dO/PLL78UeK3MzEzS0tJcbj7PZoNBg5zHav0QEfFJyzYn0WnCCm6fsYYnPk7g9hlr6DRhBcs2Jyn5KI6PP/6YDRs2MG7cuHz3JScnAxAZGelyPjIy0nHfmcaNG0d4eLjjVq9eveKG5J3uvhvCwszy3Llw9KjVEYmISBlatjmJgXM2kJTqunN5cmoGA+ds4PiKVeaJoCC45BJrgrRIsZKP3bt388QTTzBnzhxCQgrfidV2xqY4hmHkO2c3YsQIUlNTHbfdu3cXJyTvFRYG995rlk+cgFmzrI5IRETKSHaOwZglWzEKuM8Aqh8/SuWd/5gn2raFs3yn+qJiJR+//fYbKSkptGnThgoVKlChQgXi4+OZMmUKFSpUcLR4nNnKkZKSkq81xC44OJiqVau63PxG3q6XadMgJ8fKaEREpIysTTycr8UjrzZ7tzoP/KzLheImH926dWPTpk0kJCQ4bm3btuXOO+8kISGB888/n6ioKJYvX+54TFZWFvHx8XTs2LE84vduTZtC165m+e+/4bvvrI5IRETKQEp64YkHQNs9/p18VChO5bCwMOLOmIdcuXJlqlev7jg/ZMgQxo4dy4UXXsiFF17I2LFjCQ0N5Y477ijbyH3F4MGwYoVZfvttyDNYV0REvFOtsLN3o1ySN/nwwx/nxUo+iuKZZ57h5MmTDBo0iCNHjtC+fXu+/fZbwuyDK8XVdddB3brmPi9ffQU7d0LDhlZHJSIipdAuJoLo8BCSUzPyjfuolJXBxft3AGDExmKrXt2SGK1U6hVNVq1axeTJkx3HNpuN0aNHk5SUREZGBvHx8flaSySPChVgwACznJMD77xjdUQiIlJKgQE2RvWJBeDM6RZt9v1JxZxs8z4/7HJBe7t4iIcegooVzfL775tbLIuIiFfrGRfN9LtaExXu2gXTPTnPliNXXun+wDxAmXe7SAlERsItt5jrfRw6BJ98Av37Wx2ViIiUUs+4aLrHRrE28TAp6RnUCgvh0lUvOSv4afKhlg9P8eijzvJbb5kbDomIiNcLDLDRoVF1+rasQ4eaFbGtX2/ecfHF5o9PP6Tkw1O0bw9t2pjlDRvg11+tjkhERMrajz9Ctjnew19bPVDy4UFsNtfWj6lTrYxGRETKg31pBXCu8+SHlHx4kltvBfuUq08/hf37rY5IRETKkj35sNnMHc79lJIPT1KpEjz4oFk+dQpmzLA6IhERKSuHDkFCgllu1QoiIqyOyDJKPjzNI49AQO5/lnfeMZMQERHxfqtWOct+PN4DJR8eqGFD6NPHLO/dC4sXWx2RiIiUhZUrnWU/Hu+Bkg8PpYGnIiK+5/vvzX8DA+Hyy62OxlJKPjxRt25w0UVmOT4eNm0iO8dg9Y5DLE7Yy+odh8jO0TogIiJeY+dO+PNPs9y+Pfj5fmda4dQT2afdPvYYALteepVbW91LUqpzi+bo8BBG9YmlZ1y0hYGKiEiRfPONs3zNNVZG4hHU8uGp7rkHqlQBoMaiTzm+/6DL3cmpGQycs4Flm5MsClBERIps6VJnWcmHkg+PVbUqOffcA0DoqUxu2vS9y932TpcxS7aqC0ZExMPk7Spfs3Uvhn19j6goaNnS6vAsp+TDg/1x3V2O8t0bv8Jm5LjcbwBJqRmsTTxsQXQiIlKQZZuT6DRhBbfPWMMTHyfw9pgPsZ04Yd7Zq5fZte7nlHx4sH8j6/Nzg+YAxBxJ4orEjQXWS0nPKPC8iIi417LNSQycs8FljF6Xf35zlDdefKlFkXkWJR8erFZYCP9t3dtxfM+GrwqtJyIi1srOMRizZCtndoR3+cfcxfa0LYBnUiPVVa7kw7O1i4lgS+vO7A2rCcCVO9ZT/4hzgKktd9ZLuxj/XaJXRMRTrE087NLiAVD/SBKNDu8F4Lc6Tfk7q4K6ypV8eLbAABv/ub4Zc1v1AiAAg7s2miOm7T2Go/rEEhig/kMREasV1AVub/UAWNWobaH1/I2SDw/XMy6aNi89TVZgRQBu/eNbQk5lEBUewvS7WmudDxERD1FQF/hV29c6yqvOb1NoPX+j5MMLdLsijgp33A5AeOZxvj5vJz8N76rEQ0TEg7SLiSA6PMTRMh1+Mp0Ou/4AYHd4JH/WjFFXeS4lH14i4InHHeVG8z4gUD0tIiIeJTDAxqg+sZDbNd5tx1oq5mQD8H+NO4DNpq7yXEo+vEWbNtCpk1netg2WL7c6IhEROUPPuGim39WaqPAQev5vteP8utZd1FWeh80wDI+a85OWlkZ4eDipqalUrVrV6nA8yxdfwE03meVevVyX6xUREY+RnZaOrVYtAjIzyKpZi8B9+wisEGh1WOWqON/favnwJn37QoMGZvmbb8wWEBER8TiB/7eMgExzVktQvxt8PvEoLiUf3qRCBXjcOfaDKVOsjEZERArz8cfO8o03WhmJR1K3i7dJTYW6deHYMahUCfbsgQiNnBYR8RhHj0JkJGRlmf/u2WP+ePRx6nbxZeHhcN99ZvnkSZgxw+qIREQkrwULzMQD4NZb/SLxKC4lH97oscecuyJOnQqnTlkdkYiI2M2b5yzfcYeVkXgsJR/e6MILoXfuhnN79phZtoiIWG/fPlixwiw3agTt2lkdkUdS8uGtnnzSWZ482cpIRETEbvZssA+lvOMOZyu1uFDy4a26dIHmzc3ymjXmTURErGMY8MEHzmP7+DzJR8mHt7LZYMgQ57FaP0RErBUfDzt2mOVu3SAmxuqIPJaSD292++1Qs6ZZ/vxz2L3b6ohERPzX++87yw8+aGUkHk/JhzcLCYGBA81ydja89ZbVEYmI+KeDB80fgWCuvXT99VZH5NGUfHi7gQMhKMgsv/supKVZHZGIiP95913IzDTL/fubPw6lUEo+vF1UFNx9t1lOS9OiYyIi7paVBW+/bZYDAuDRR62OyOMp+fAFTz3lLE+erEXHRETc6bPPICnJLPftq4GmRaDkwxc0bQp9+pjlPXtcNzQSEZHyYxjw+uvO47yzEKVQSj58xdNPO8uvvupc5EZERMrP0qWwcaNZbt0aLr/c6oi8gpIPX9GpE7Rvb5Y3bYJvv7U6IhER32YY8OKLzuPnn9eKpkWk5MNX2Gz5Wz9ERKT8fPstrF1rlps3h+uuszoir6Hkw5dcfz1ccIFZ/v572LDB6ohERHxTTg6MGOE8fv55c6aLFIneKV8SGAhDhzqPX3vNymhERHzXRx85x3q0bAn9+lkdkVdR8uFr+veHGjXM8qefwr//Wh2RiIhvOX4cRo50Hr/+ulo9iknvlq+pVMm5wE12NrzxhtURiYj4lldegb17zXKfPtC1q9UReZ1iJR/Tp0+nefPmVK1alapVq9KhQwe++eYbx/2GYTB69Ghq165NpUqV6NKlC1u2bCmPuOVsBg82kxByNzo6dMjqiEREfENCAkycaJaDgjS4v4SKlXzUrVuX8ePHs379etavX0/Xrl3p27evI8GYOHEikyZNYurUqaxbt46oqCi6d+9Oenp6ecUvBalRA+6/3ywfPw5TplgdkYiIR8nOMVi94xCLE/ayeschsnOKsDbS6dPmbrXZ2ebxyJFw0UXlHqsvshlG6VajioiI4NVXX+X++++ndu3aDBkyhOHDhwOQmZlJZGQkEyZMYMCAAUW6XlpaGuHh4aSmplK1atXShObf/v3XnPly+jRUqwa7dkFYmNVRiYhYbtnmJMYs2UpSaobjXHR4CKP6xNIzLrrwB44cCWPHmuWLLzZnFNo39pRifX+XeMxHdnY2H3/8McePH6dDhw4kJiaSnJxMjx49HHWCg4Pp3Lkzv/zyS6HXyczMJC0tzeUmZaBBA7jrLrN89ChMn251RCIillu2OYmBcza4JB4AyakZDJyzgWWbkwp+4PLlMG6cWa5QAT74QIlHKRQ7+di0aRNVqlQhODiYRx55hIULFxIbG0tycjIAkZGRLvUjIyMd9xVk3LhxhIeHO2716tUryeuQgjz7rHO1vUmT4ORJqyMSEbFMdo7BmCVbKai5335uzJKt+btg/vkH7rzTuW3FK684V5Qu5HmK3aXjZyoU9wEXXXQRCQkJHD16lC+++IJ7772X+Ph4x/22M5aWNQwj37m8RowYwdA8a1OkpaUpASkrF10EN91k7ri4fz98+KE5GFVExA+tTTycr8UjLwNISs1gbeJhOjSqbp48cgSuvRYOHDCPe/aEYcMKvUaJu3T8TLFbPoKCgrjgggto27Yt48aNo0WLFrz55ptERUUB5GvlSElJydcakldwcLBj9oz9JmXoueec5YkT4dQpK6MREbFMSnrhiUeB9dLTzam0f/5pHjdpAvPmFbqmR4m7dPxQqdf5MAyDzMxMYmJiiIqKYvny5Y77srKyiI+Pp2PHjqV9Gimpli3hmmvM8q5dMHeu1RGJiFiiVlhI0eulpsLVV8PPP+eerGXuYHveeQU+psRdOn6qWMnHc889x48//sjOnTvZtGkTI0eOZNWqVdx5553YbDaGDBnC2LFjWbhwIZs3b6Z///6EhoZyxx13lN8rkHPLuxLf+PHOaWIiIn6kXUwE0eEhFDYQwJbbRdKOVLjsMli92rwjIgKWLYOYmEKvXZwuHSnmmI/9+/dz9913k5SURHh4OM2bN2fZsmV0794dgGeeeYaTJ08yaNAgjhw5Qvv27fn2228J0xRPa3XsCJ07Q3w8/PUXLFgAN99sdVQiIm4VGGBjVJ9YBs7ZgC1PiwS5iQfAWxEpBHa43TnGo3p1c6POFi3Oeu1id+n4uVKv81HWtM5HOVm+HOzToFu0MDdEOstAYBERX1XQoNBGwdl8sONLGs79wHHuWIPz+fuD+TS/8hICA87+ebl6xyFun7HmnM89/6FLnYNZfUxxvr+VfPgLwzCnhq1bZx5/9ZU5gltExA9l5xisTTxMSuoJLv52IY3eeAVbSorj/lUxbXj8uqdJC6lSpNkq2TkGnSasIDk1o8BxHzYgKjyEn4Z3PWci463cssiYeBmbzXXmy+jRzjnrIiJ+JvBUFh1++JK+d/fkghFPOBKPjApBjLpqAP1vHk1aSBUo4mwVe5cOebpw7OzHo/rE+mziUVxKPvzJddc5+y3Xr4evv7Y6IhER9/rrL3MQfkyMuQfW1q2Ou1bEXcFVD0xjdps+Lt3SRZ2t0jMumul3tSYq3HVWTVR4CNPvaq11PvJQt4u/WbQIbrjBLLdpY3bDaOyHiPiyQ4fgk09g9mxYuzb//Z06seWRYVy76dxzMIoyZsPRpZOeQa2wENrFRPhFi0dxvr+LvcKpeLm+faFVK3PA6W+/mWM/+vSxOioRkbKVng6LF8P8+fDtt+Ymm3kFBpqtwU8/DR06sD1hL2xKOOdlizJbJTDA5rODSsuKkg9/Y7OZ4z369jWPR4+G3r3V+iEi3u/UKbM7ed4884dVQftZtWwJ99wDt98OuStzU9wFyKTUlHz4oz59oHVrczvoDRvgyy+dyYiIiLfZswdmzID33oOCNjKtWxduuw3uvhuaNy/wEvYFyM41W6VdTETZx++HNODUH9lbP+w080VEvNGmTXDrrdCwIbz4omviUaMGDBwIP/wA//4Lr75aaOKBZqu4nZIPf9W7N7Rta5YTEsyBqCIiblTiree3bTNXaW7eHD791LllRGAg9Otn7sGybx9MmwaXX17oRnBn0mwV99FsF3/29ddmEgLm/8QbNxb5f1IRkdIo0dbz6ekwZgy8+abrANLISBgwAB56yOxiKSV/na1SWlrhVIrmzFVPP/8cbrzR6qhExMfZt54/88vH/vVeYCvDl1/CI49AUp6FviIjYfhw83ylSuUet5ydVjiVoilo7Id2vBWRclTsreePHzdbNfr2dSYewcEwahT88w88+aQSDy+k5MPf9epltn4AbN5szokXESknxdp6futWc2bee+85K1xzjXl+9GgIDXVP0FLmlHz4O5sNxo51Hr/wAmRlWRmRiPiwIm8pv/RruPRS+N//zOPQUDMJ+eorOP/8co1Ryp+SD4GuXaF7d7OcmGjOlxcRKQdFWaTrgbULufSJ/uYAUzD3pNq40RxQqgURfYKSDz9QpOlseVs/XnoJjh1za4wi4h/si3kVmEIYBk/Hz+b5lR9gs8+FuPFG+PlnaNzYzZFKeVLy4eOWbU6i04QV3D5jDU98nMDtM9bQacKK/FtDt20LN91klvfvN6eyiYiUscIW87IZOYz+/j0Gr/nMefKFF8x1PCpXdn+gUq401daHFXs6219/wcUXmzNeqlY1R5JX1+ZIIlL2XNb5MAxe+fZt7kxY5qzw9tswaJCVIUoxaaqtFH86G8BFF0H//mY5LQ0mTHBLrCLif3rGRfPT8K7Mf+hSlqd840w8AgJg9mwlHj5OyYePKtZ0trxGjTLn0AO89Rbs3XvO5yrxEski4tcCA2x0+Px9Lpw1zTxhs8Hcueaus+LTtKutjyrqdLZ89erVg8GDYdIkyMgwN2t6991CH1+iJZJFRMBs4Xj2Wefx9Onm7rPi89Ty4aOKMp2t0HojRkBYmFn+4APnPPsz2MeUnNnCkpyawcA5G/IPahURsfv5Z3PqrN3YseZKpuIXlHz4qLNOZ8sddBodbm6YlE+NGvD002Y5O9vcO+EMJRpTIiICsHMn3HADnDplHg8c6NoCIj5PyYePKmw6W97jUX1iC9+pcehQiM7tNlm0CH74weXuEo8pERH/dvw4XHcdHDhgHnfrZk7t1+JhfkXJhw/rGRfN9LtaExXu2rUSFR5S8K6ReVWubC42ZvfUU5CT4zgs8ZgSEfFfhmG2cmzaZB5feCF89hlUrGh1ZOJmGnDq43rGRdM9Noq1iYdJSc+gVpjZ1VJoi0de/fubv0g2bYL16+Hjj+GOO6C0Y0pExD99+CF89JFZrlIFvvwSzjvP6qjEAmr58AOBATY6NKpO35Z16NCoetESD4DAQHjtNefxiBFw8iSUdkyJiPif33+HRx91Hr//PjRpYmVEYiElH3J2PXpAz55medcumDIFijGmBNAaICL+7sQJuPVWc/o+mAuI3Xqr1VGJhbS8upzb5s3mrpI5Oeay69u3Q82acI51Psid8aI1QET83OOPm4sWArRubU6zDVGXrK8pzve3kg8pmocfhhkzzPLgwTB1quOu7Bwj35iS5VuTi7evjIj4pu++g+7dzXJICCQkmFs5iM9R8iFlLzkZLrjAnCYXGAhbthT6AZKdY9BpwopCp+Lacmfc/DS8a9HHn4iI9zl6FJo1gz17zOMpU+Cxx6yOSsqJNpaTshcV5VxsLDsbhg0rtKrWABERyO1usSce3bqZraYiSj6kWIYOhTp1zPJXX8E33xRYTWuAiAgLFzqn1YaHw8yZ5o61Iko+pFgqV4aJE53HQ4ZAVla+aloDRMTPpaa6tnK89Za5aaVILiUfUjy33w6XXWaW//c/5wj2PLQGiIife+45SMrdWPLaa+Guu6yOSDyMkg8pHpvNHDRm34dhzBhzMGoepd5XRkS81+rVMH26Wa5cGaZN074tko+SDym+1q2dW2Gnp5u/cs5Qqn1lRMQ7nTplTsu3T6J86SWoX9/qqMQDaaqtlMyBA+amUKmp5vGvv0K7dvmqFbQGiFo8RHzUuHHOHyOtW5ufCxW0hZi/0Dof4h5TpsATT5jldu3M5laNZhfxT4mJEBtrLqEeEADr1pkJiPgNrfMh7jFwoPlhA7B2rXNanYj4n2HDnHu3PP64Eg85KyUfUnIVK8KbbzqPhw93dsOIiP/4/ntYsMAsR0aaA9FFzkLJh5TOVVdBv35mef9+eP55qyMSEXc6fdrZ/Qowfry5AaXIWSj5kNKbNAlCQ83y22/Dhg1WRyQi7jJ9urnXE7ljv+65x+qIxAso+ZDSa9AAXnjBLOfkmGNBsrOtjkpEytvBg87/98kdhK5B51IE+iuRsvHkk66DT2fMsDoiESlv//mPuXMtwL33Qvv2VkckXkLJh5SNoCBzJUO7ESPMMSAi4psSEuC998xylSrmGh8iRaTkQ8pO587O/t6jR+Hpp62OSETKg2HAU085VzJ9/nmI1qrFUnTFSj7GjRvHJZdcQlhYGLVq1eL666/nr7/+cqljGAajR4+mdu3aVKpUiS5durDFPhhJfN+rr8J555nljz6C+HirIxKRsrZ8OaxYYZYbNXKd7SJSBMVKPuLj4xk8eDBr1qxh+fLlnD59mh49enD8+HFHnYkTJzJp0iSmTp3KunXriIqKonv37qSnp5dH/OJpatVybX4dOBCysqyMSETKUk4OPPus8/jllyE42MqIxAuVann1AwcOUKtWLeLj47niiiswDIPatWszZMgQhg8fDkBmZiaRkZFMmDCBAQMG5LtGZmYmmZmZjuO0tDTq1aun5dW9WU4OdOhgDjwFeOWVAjefExEv9PHHcPvtZrlVK1i/XjNcBNy5vHpq7mqWERERACQmJpKcnEyPHj0cdYKDg+ncuTO//PJLgdcYN24c4eHhjlu9evVKE5J4goAAeOcd5wfSiy/C//5ndVQiUlpZWeYMF7tx45R4SImU+K/GMAyGDh1Kp06diIuLAyA5ORmAyMhIl7qRkZGO+840YsQIUlNTHbfdu3eXNCTxJK1awdChZjkz09xmOyfH6qhEpDQ++AB27DDLV14JeX5oihRHiZOPRx99lD/++IP58+fnu89mc90y3TCMfOfsgoODqVq1qstNfMSYMXD++WY5Pt784BIR73T8uNmKaTd+PBTyuS5yLiVKPh577DG+/PJLVq5cSd26dR3no6KiIE8LiF1KSkq+1hDxA6Gh8O67zuOnn4Z9+6yMSERKavJksH+233ijuZS6SAkVK/kwDINHH32UBQsWsGLFCmJiYlzuj4mJISoqiuXLlzvOZWVlER8fT8eOHcsuavEeV10F/fub5dRUeOwxqyMSkeI6dAgmTjTLgYHmIHKRUihW8jF48GDmzJnDvHnzCAsLIzk5meTkZE6ePAm53S1Dhgxh7NixLFy4kM2bN9O/f39CQ0O54447yus1iKd77TVzCi6Y227bt94WEe8wdiykpZnl++6Diy6yOiLxcsWaalvYuI2ZM2fSP/fXrWEYjBkzhnfffZcjR47Qvn173n77bceg1HMpzlQd8SKffAK33WaWo6Nh61aoVs3qqETkXHbtgsaNzYHjISGwfTvUqWN1VOKBivP9Xap1PsqDkg8fZRhw3XXw1Vfm8UMPOfeFEBHPdf/9MHOmWX7mGZgwweqIxEMp+RDPtHu3ufPtsWPm8fLl5pgQEfFMW7dCs2bmNPlq1eCff5zbJ4icwW2LjIkURXaOweodh1h8KIB/nn7BeccDDzj7kUXE8zz3nHN9nmefVeIhZaaC1QGIb1u2OYkxS7aSlJoBgM1owmfnt6LtPxvNvuRhw9T9IuKJfvkFFi82y7Vra6aalCm1fEi5WbY5iYFzNjgSDwDDFsCQHo9yLKiSeWLGDPi//7MuSBHJzzBcN48bNcpct0ekjCj5kHKRnWMwZslWChpQtCc8krFX3u888eCD5hogIuIZvvkGfvzRLDdubA46FSlDSj6kXKxNPOzS4nGmeS168mODlubBnj3OfWBExFo5OTBihPP4lVeggnropWwp+ZBykZJeeOIBgM3G8Gse51TlKubxhx/C0qVuiU1EzmLePPjjD7Pctq25lLpIGVPyIeWiVljIOevsq1qLXc+95Dzx0ENw5Ej5BiYihcvMhOefdx5r8zgpJ0o+pFy0i4kgOjyEwj62bEB0eAgNhz8OV19tnty3Dx591J1hikhe770HO3ea5e7doVs3qyMSH6XkQ8pFYICNUX1iITfRyMt+PKpPLIGBAeaMF/tS6/PmmTcRca/0dHgpT0vk+PFWRiM+TsmHlJuecdFMv6s1UeGuXTBR4SFMv6s1PeOizRP16sH06c4KgwaZa4DkcixSlrCX1TsOkZ3jUYvyiviGSZPgwAGzfOut0Lq11RGJD9Py6lLusnMM1iYeJiU9g1phIbSLiSAwoIAOmbvugrlzzXLnzvD99yzbluKySBm53TWj+sQ6kxcRKZ2UFGjUyNz6oEIF2LYNLrjA6qjEy2hvF/FOR49C8+bmHjDAn0P/Q6+Kl+ZbK8Setri0nohIyQ0ZAm++aZYHDoRp06yOSLyQ9nYR71StGvz3v47R9Y3eHE+TlH/yVTNyb88t3MTCjeqKESmVnTud3Z6hoa6zXUTKiZIP8SxdusDTTwNQMfs0k5e8RvCpzAKrHj5+iic/SeD2GWvoNGEFyzYnuTlYER/wwguQlWWWhwyBaLUmSvlT8iGe58UXOXrRxQBcdHAXz8bPOudDklMzGDhngxIQkeL44w+YM8csR0TAM89YHZH4CSUf4nmCg9k5+T0yKgQBcN9vS7jq71/P+hB7p8uYJVvVBSNSVM89Z24iZy+Hh1sdkfgJJR/ikZr16MDUXgMcx68tfYPaaSlnfYwBJKVmsDbxsBsiFPFyP/4IX39tluvWhcGDrY5I/IiSD/FIgQE24l55lm8adwSgWsYx3vzyNQJzss/52HPuKyPi7wwDnn3WeTxmDISce0sEkbKi5EM8Vs9mtak48wOSqkUCcMnerTz509xzPq4o+8qI+LXFi+GXX8xy06Zwzz1WRyR+RsmHeLSrOjah1tcLycnd0nvQms+4PHFjgXXt+8W0i4lwc5QiXuT0aRgxwnk8bpy5sJiIGyn5EI8X2LEDAWPHAhBgGLzx1evUPOa6+63LfjEFrZ4qIqaZM+HPP83yZZfBdddZHZH4ISUf4h2eegp69QKgxomjTFs2iYA84z/y7RcjIvkdPw6jRjmPJ050LOon4k5qaxPvEBAAs2dDixaQlMQlOzbyw8kf+O2BJ8++X4yIOE2eDEm5a+HccAN07Gh1ROKntLeLeJdVq6BbN8jJMY+XLIHeva2OSsTzHThgbh6Xng6BgbB5MzRpYnVU4kO0t4v4ri5dYPx45/Fdd8H27VZGJOIdXn7ZTDwAHnhAiYdYSsmHeJ9hw+DGG81yair06wcnTlgdlYjn+ucf183jRo+2OiLxc0o+xPvYbOaIffsvt02bYMAA5zLRIuLqP/+BU6fM8tCh2jxOLKfkQ7xTWBgsWABVqpjHc+bA229bHZVIucvOMVi94xCLE/ayesehQvcystdbNe8bmD/fPFmjhmPXaBErabaLeK+mTc0WkJtvNo+ffBJatSK7Q0fWJh4mJT1DM2HEpyzbnMSYJVtJSnVuIRAdHsKoPrEu08wd9Y6e5JN5zmXUtz34OE01kF88gJIP8W433WSOAXntNTh9mszrrufWB98kwebcnbOgD2cRb7NscxID52zgzHaO5NQMBs7Z4FjnJm+9a/76mfZ7tgDwT0Qd+p5uxpTNSfp/QSynbhfxfuPGQdeuAAQfPsjYWf8hNOuk4277h/OyzUkWBilSctk5BmOWbM2XeJC7mzPAmCVbyTqd46gXfDqL51Z+6Kj38pUPcCqwImOWbC20q0bEXZR8iPerUIHsjz9hT0RtAGJTEpn09SRshrkWiJF704eueKu1iYddulrOZABJqRl8tHqno94D6xZRNy0FgB8atmJFo0sc9dYmHnZb7CIFUfIhPmHNUbj3hudJCwoFoOf/VjP0R9cdcPWhK57qXINIU9ILTzzy+vewOeW85rHDDF79qXltWwAvd33AZRn1ol5PpLxozId4vWWbkxj+xSZSa9Tjsb7D+fDzMQQaOTy2+hP+rlGPL2O7OOomp+lDVzxLUQaR1goLKdK1GkSYyffTP/yXyqfM681t2Yv/1WzoUq9GlWBW7zikQdliGSUf4tXOHIQXf34bXrnyAV5YMQOAV5e+yd6qtfitbiwAh49lWhht+crOMTTLx8sUdRBpu5gIosNDSE7NKHDchy13c8W7OzTkx0/+j5s2fQ9AWnBl3uh0h0u9aqEVeerTBJLTnP8vaFC2uJu6XcRrFTYI78O21/Fx8x4ABGef4v0vXqLRod0ARFQOsiDS8rdscxKdJqzg9hlreOLjBG6fsYZOE1ZokK0HK+og0uwcg8AAG6P6mAn0memk/XhUn1iCAm1MWj2LgNwrvHnZ7RwJDXfUM4AjJ065JB5oULZYQMmHeK1CB+HZbDzfYyA/NGwFwHkZ6cz6bDQ1jx0hKryS+wMtZ/Zfz2e+F/pC8WxFHURqH6fUMy6a6Xe1JirctQsmKjzE0ULCRx8RsWEtALuq1+G/ra91qVcttGKhz4UGZYsbqdtFvNbZBs2dCqzIoOtH8Mm8Z7k45R/qpe7nvwvG0OilPm6Nsbyd69ezLfcLpXtslLpgPExRB33mrdczLprusVEFd68dOWKueZOrzkfv89/Glzjq5eQY3PnBr4U+T95kp0Oj6qV8dSJnp5YP8VrnGoR3LDiU+24axZ6qNQFomrSd9e268c1v/7opwvJX3F/P4jmKOoj0zHqBATY6NKpO35Z16NCoujOp/M9/4MABs3zTTQT26ulS7+Dxoo130kwYcQclH+K17IPwzvZ7PiWsOv1vHkNqcGUAOm7/jRP33se4rza7Lc7yVJJfz+IZzvX3a8sdCNouJuLcF1u/3rlrbeXK8MYb+aqUNNkRKQ9KPsRrnW0QXl7ba9TnoRufJzPQ7GW8cctK6r4wnKW/73NTpEXfDKy49IXivYo6iPSc3WXZ2TBokHNX51GjoG7dfNXKNNkRKSUlH+LVChuEd6a19eJ47LrhnLaZf/J3b1xKyqNPkp2dU+4xludMFH2heLciDSI9l3ffhXXrzHJsLAwZUmC1Mkt2RMqAzTAMjxranJaWRnh4OKmpqVTV7otSRHnXuPh7fzpTV+4osF7fLSt546tJjqmIu54cQf1JY8strsLWcbB/vBf5C6YIz0GeWQtl/RxSvkq8RsuuXXDxxXDsmHm8ahV07nzWhxR1Z1yR4irO97eSD/E5q3cc4vYZawq9//aEZYz7v6nOE2+8UeivxdLIzjHoNGFFoQNC7QtD/TS8a6l/beoLxQ8ZBlx7LXzzjXn84IMwY0aRHqoF6aQ8FOf7W1Ntxee0i4kgonJFDh8/VeD981v2JDTrJM+v/MA88eST5r9lnIAUZyZKaac2nnUKpvimuXOdiUft2vDqq0V+qH3GjIhVij3m44cffqBPnz7Url0bm83GokWLXO43DIPRo0dTu3ZtKlWqRJcuXdiyZUtZxixyVoEBNl7uG3fWOh+0u4EZ3e5xnnjySZgwoUzjcPdMlEKnYIrvSUmBJ55wHk+fDtWqWRmRSLEUO/k4fvw4LVq0YOrUqQXeP3HiRCZNmsTUqVNZt24dUVFRdO/enfT09LKIV6RIrmlemwFXxBR6vw2o98Y4GD3aefLZZ2HMGOesgVLSTBRrldcMI4/w+ONwOHftlltvheuuszoikWIpdrdLr1696NWrV4H3GYbB5MmTGTlyJP369QNg9uzZREZGMm/ePAYMGFD6iEWKaMQ1sbSoex7/WbyZw8ezHOfzjoXIvvgF9hzLpsFrL5l3jh4NmZnwyisuW5CXRFE3A9NMlLLn02NgPv4YPvkEgFPnRfDdwyOotuOQutnEq5TpmI/ExESSk5Pp0aOH41xwcDCdO3fml19+KTD5yMzMJDPTufJeWlpaWYYkfqSgQXTXNI/m6riCx0I4vqAC23N/14ccO+EybhykpcGbb0JgYInjsU9tHDhng2NTLztNbSw/Rd0p1ivt2gWPPOI4fKrTA3z57R5gj+8kV+IXyjT5SE5OBiAyMtLlfGRkJP/+W/CS1uPGjWPMmDFlGYb4oXP90j1zcN2ZX1AfXtKXrAoVefnbaeaJt9+GpCSYMwcqlXwzOvs6DmfGFuUBXxS+OOPBp/e6yc6Ge+6B1FQAFsV25stY57Ran0iuxG+Uy2wX2xnN1YZh5DtnN2LECIYOHeo4TktLo169euURlvio4v7SLewLak6razhRMZgJ30yhYk42LFgAPXrA4sUQUfKuEU+cieKr3RLunGHkdq+/DvHxAOypWpMXug90udvrkyvxK2W6wmlUVBTkaQGxS0lJydcaYhccHEzVqlVdbiJFda5fuhSwTfjZvqAWxHXjgRtf4HjF3EGgP/0El10GOwpetKyoPGkmij1ZO/M9sCdr51p51ZMHcvrsXjcbN5obxwE52Hjq2qGkhVTJV00bCYq3KNPkIyYmhqioKJYvX+44l5WVRXx8PB07dizLpxKBEu7qeq4vnh/Ob8Otd4wno7q5Gy5//gmXXALff192gVukJMlaXuW5VHxZ8MkZRqmpcMstcMpct+adS2/k1/rNzvoQr0uuxO8UO/k4duwYCQkJJCQkQO4g04SEBHbt2oXNZmPIkCGMHTuWhQsXsnnzZvr3709oaCh33HFHecQvfq4kv3SL8sWzOeoCtn6+DJo0MU8cOQJXXw1vvVVmU3GtUJJkza60LSbu4HN73RgG3H8/bN8OwLG4FrzR6c5zPsyrkivxS8VOPtavX0+rVq1o1aoVAEOHDqVVq1a88MILADzzzDMMGTKEQYMG0bZtW/bu3cu3335LWFhY2Ucvfq8kv3SL+gXV4opWsGaNuYQ1uQP+Hn8cHngATpwoi/DdrqTdEqVtMXEXn9s8bdIkc+wRwHnnUWnRAmpEhPlOciV+q9jJR5cuXTAMI99t1qxZkDvYdPTo0SQlJZGRkUF8fDxxcWdfbVKkpEryS7dYX1Dh4eaA0+HDnZVmzjS7Ybxw5d6SdkuUpsXE3cpkp1hPEB/v+nf30UcENjrft5Ir8VtlOuZDxN1K+ku3WF9QgYEwfry5l0ZoqHlu61YzAfngA6/qhilpt4S3DeTsGRfNT8O7Mv+hS3nztpbMf+hSfhre1XsSjx074MYbzdY2gOeec7TA+UxyJX5Nu9qKTyjp1NFir3Xx55/m4L9Nm5znbrjB3FujkBldnsY+doNCFj4r6AvsXDsF281/6FLvm8LqaVJToUMH2LbNPL76avjqK6jgujKCL67TIt6tON/fSj7EZ7jtw/jkSXMjunffdZ6LiICpU+G220q9LLs7FDdZy84x6DRhxTmXiv9peFd9AZbGqVPmPi3LlpnHTZvC6tVm95+Ih1PyIeIOX3xhLnV98KDzXN++5rLsDRpYGVmRFDdZK0mLiRRDTg7cdx/897/mcfXq8Ouv0KiR1ZGJFImSDxF3OXAABg+Gzz5znqtUydwh9+mnS7U0+7lY0ezuqyujWs4wYNgwc3YLQHAwfPstXHGF1ZGJFJmSDxF3++wzePRRSElxnouJMTepu/lmCCje2O5zJRZWJgEaa1AOxo0zB5WC+bfy+efmWCIRL6LkQ8QKqakwerS5EJl9lgJAixbw0kvQu3eRxoOcK7EobC8bdX94qbFjYeRI5/GMGfDgg1ZGJFIixfn+1lRbkbISHg5vvAG//w5duzrP//67OYiwfXuzheT06UIvca5VRJf+sc8rFvtyJ0/ea+acXnzRNfGYMEGJh/gFtXyIlAfDgOXLzc3A1q1zva9BA+dKqXlmMdhnlBS2mJcNOK9yRQ4fP3XOpy/tlFdv6Vo5WyuRp+0k7CInx1xA7LXXnOcmTjTHCYl4KXW7iHgKw4Avv4RRo8wWkLxCQsx+/f79oVs3Vu88WqS1NIrizdta0rdlnRI91lsGlZ6t+8kAqoVW5OgJZ6LmMa8hI8P8b/7JJ85zkyaZ07dFvJi6XUQ8hc1mTr/duNFsCbnmGud9GRkwf765iFT9+kSOfIrLEzdQMfvcLRvnUpRl1AvqrvCGzeMo4l4zeRMPPOU1pKRAjx7OxCMgwFwvRomH+Bm1fIi427Zt5oqo8+bBoUP57k4LCuXnhi35tV4cv9aP48+aDTFszt8JEZWDOHI8q1SLfRXUuhFVNZiM0zn5vrSLe213KOqKq2ey9DX8/LO5Ou6+feZxaCh8+qlz40IRL1ec7+8KZ71XRMpe06YwZYrZ3//11zBrlrmiZVYWAFWzTtDrf7/Q63+/AHA0pAqbIi9gW+T57G14EVfdchUP/5JBRsWQAhf7OtfGYoV1VySnZZ417Lybx1m9hHpJ95Cx5DVkZ8PkyebaL/bBxtHRZndc27buiUHEwyj5ELFKUJA55uOGGyA9Hb79lr2zPyZ0+f9xXka6o1q1jGNc/m8Cl/+bAGuBT8exDTgYFkFi1Sh2VYvkQJUIsiJq0KVTLM13B8DxXVC5svMWGgqhoWRjK7S7oqg8YfO4ou7OWxi3vYa//zZXLf35Z+e5Ll3M7raoqHzVvWWgr0hpqdtFxMMs+30P897/mkbbfqP97s202bONmieOlsm1DZuN07YAcmw2jNx/c2wBuTeb41/DZsPAdsY5899a4ZWoFFzR3O23UiXXBMdePu88qFEDatZ0/TcyskxWfT3XXjPnUu4b4GVkmNOuX3rJ3AvIbvhwePnlfJvE4UUDfUUKo9kuIl4u3y/gkEwCf08wB67+9Ze55fo//0BystWhFl/NmlC/vjnl2H5r2BAaNzb3MQkKKtJlCttr5mzKfcxHTo65Oukzz8C//zrPn38+zJxZ6HLpWjhOfIGSDxF/cfw47NxpzqJISTH3mjlwAA4fhhMnzNvx4+btxAmOncjkn5R0AgyDACOHgJwcZ9nIwYaRe2xgM3Jy/zUIwKxTNTiQYFvul+zp0+b1867mWkpGYCA0aoStSRPIe2vaFKpVy1e/oNYC+xRbmzs3wDt1Cj7+GMaPh61bnecDAsxl98eONVuEClCU9V08ZaCvyNko+RCRAp2ru8IGhIdWJKRCIMlpRWj+Nwzzi9ee4Nhvhw+bu/0eOOD898ABSEoyWwT27TMTmOKIjoaLL4bYWPOWW86udl6+cRLLtya7pwtj+3b48EOYPds5i8Wue3dz/Y64uLNeoqgzd8q9q0iklDTbRUQKFBhgY1SfWAbO2VBoy8D4fs2KvjqozWZ2kwQFmeM8iurUKeJXbGT67BXUTkuh3tH9nH94L40O76HRoT1UOl3AzJukJPP23Xeurykykg55khFiY+kZG0v34V2LPXjznAM+T50yu76+/hq++go2bMh/kU6dzCXTr766SHv5FHXwqycM9BUpK0o+RPxMz7hopt/VOv86H2e0DJTnr+zswAo8uyGdpPrN8t1nM3KonX6QthkpvNE8mIA//zTXRtmypcB1Udi/37ytXOlyOrBmTTMpadwY6taFOnWc/553nrm0feXKjgTB3oVz4PAxqmWkc96JNJpwnEdqZ3NxerKZaKxf7zqA1PFkgebGgU89BZdfXqz3oqgzd0o7w0fEk6jbRcRPWTmts8RdDSkp5piKvLctW8zzJZE7Y+d0dg6Zp80xL6Gnzr7eiYtWreD22+HuuwucOlsURekK05gP8QbqdhGRcwoMsFk2hqDEXQ21apm3Ll1czx88aLaO5E1Itm41u2nOJjsbjh2jQlE/DBs2hA4doHNns6WjTsn2z8mrKF1h51o4TsTbKPkQEbcr866GGjXM7o4zuzyOHDFnA+3ZY9727jVvR49CaiqkpnIi9Rh7jmZg2MDAxvGgShypFMaRSuEcCq3Kv+fVJjGiNsOeuJ627ZuW4NWeW1G7wkR8hZIPEQ/gbytbtouJIDo85JxdDe1iIkr3ROedZ95atSq0yvKEvTzxccI5L7U3uCrluRh6z7joog/0FfFySj5ELOaPK1uWV1dDSZI4TxrwaWVXmIg7KfkQsVChm7zlbv/uyytblnVXQ0mTOLe1woiIg2a7iFhEK1uayqLLqbTLkxe2VLuWNxcpuuJ8fwe4LSoRcbE28XChiQdnbP/uy+xdDX1b1qFDo+ol6mopbKde+7kxS7aSnVP47yx7K0xUuGvXSlR4iBIPkXKgbhcRi2hly7JRnCTubOMpNOBTxH2UfIhYxJMGOnqzskziNOBTxD3U7SJiEftAx8J+V9tyB0xqoOPZKYkT8T5KPkQsYp9uSp6BjXZa2bLolMSJeB+v7XbJzs7m1KlTVochHq5ixYoEBgZaHUahtLJl6Wl5chHv43VTbQ3DIDk5maNHj1oSn3ifatWqERUVha0I25tbxRNXOPXEmM7GHxdrE/EkPr2xnD3xqFWrFqGhoR79hSLWMgyDEydOkJK742l0tOd+AXnaQEdv/CLXbBUR7+FVyUd2drYj8ahe3XM+qMVzVapUCYCUlBRq1arl0V0wnsKbV131tCRORArmVQNO7WM8QkNDrQ5FvIj970VjhM6tLBbsEhE5F69KPuzU1SLFob+XotOqqyLiDl6ZfIhI+dCqqyLiDko+RMRBC3aJiDso+XCT/v37Y7PZsNlsVKxYkcjISLp3786HH35ITk5Oka8za9YsqlWrVq6xiv/Sgl1O2TkGq3ccYnHCXlbvOKRxLiJlyKtmu5QlK9Yw6NmzJzNnziQ7O5v9+/ezbNkynnjiCT7//HO+/PJLKlTw2/8c4iG0YJfJG6cai3gTv2z5WLY5iU4TVnD7jDU88XECt89YQ6cJK1i2Oalcnzc4OJioqCjq1KlD69atee6551i8eDHffPMNs2bNAmDSpEk0a9aMypUrU69ePQYNGsSxY8cAWLVqFffddx+pqamOVpTRo0cDMGfOHNq2bUtYWBhRUVHccccdjvUtRIrD37eXt081PnPgrX2qcXl/Toj4A79LPjztg6Vr1660aNGCBQsWABAQEMCUKVPYvHkzs2fPZsWKFTzzzDMAdOzYkcmTJ1O1alWSkpJISkpi2LBhAGRlZfHSSy/x+++/s2jRIhITE+nfv79bX4v4jp5x0fw0vCvzH7qUN29ryfyHLuWn4V19PvHQVGMR9/Crdv5zfbDYcj9YusdGubVZuUmTJvzxxx8ADBkyxHE+JiaGl156iYEDBzJt2jSCgoIIDw/HZrMRFRXlco3777/fUT7//POZMmUK7dq149ixY1SpUsVtr0V8hz8u2FWcqcb+9t6IlCW/avnw1DUMDMNwrEWxcuVKunfvTp06dQgLC+Oee+7h0KFDHD9+/KzX2LhxI3379qVBgwaEhYXRpUsXAHbt2uWW1yDiCzTVWMQ9yi35mDZtGjExMYSEhNCmTRt+/PHH8nqqIvPUD5Zt27YRExPDv//+yzXXXENcXBxffPEFv/32G2+//TacY3XO48eP06NHD6pUqcKcOXNYt24dCxcuhNzuGBEpGk01FnGPckk+PvnkE4YMGcLIkSPZuHEjl19+Ob169bL8V7gnfrCsWLGCTZs2ceONN7J+/XpOnz7N66+/zqWXXkrjxo3Zt2+fS/2goCCys7Ndzv35558cPHiQ8ePHc/nll9OkSRMNNhUpAU01FnGPckk+Jk2axAMPPMCDDz5I06ZNmTx5MvXq1WP69Onl8XRFZvUHS2ZmJsnJyezdu5cNGzYwduxY+vbtS+/evbnnnnto1KgRp0+f5q233uKff/7ho48+4p133nG5RsOGDTl27Bjff/89Bw8e5MSJE9SvX5+goCDH47788kteeumlcnkNIr7MPtWYPFOL7fxpqrFIeSvz5CMrK4vffvuNHj16uJzv0aMHv/zyS776mZmZpKWludzKi9UfLMuWLSM6OpqGDRvSs2dPVq5cyZQpU1i8eDGBgYG0bNmSSZMmMWHCBOLi4pg7dy7jxo1zuUbHjh155JFHuPXWW6lZsyYTJ06kZs2azJo1i88++4zY2FjGjx/Pa6+9Vi6vQcTX+ftUYxF3sBmGUaZzxvbt20edOnX4+eef6dixo+P82LFjmT17Nn/99ZdL/dGjRzNmzJh810lNTaVq1aou5zIyMkhMTHSMJSkpLSDkX8rq70b8ixULEYp4s7S0NMLDwwv8/j5TuU21PXMn0bwzOvIaMWIEQ4cOdRynpaVRr1698goLcn/ZdI+N0geLiBTKH6cai7hLmScfNWrUIDAwkOTkZJfzKSkpREZG5qsfHBxMcHBwWYdxTvpgERERsUaZj/kICgqiTZs2LF++3OX88uXLXbphRERExD+VS7fL0KFDufvuu2nbti0dOnTgvffeY9euXTzyyCPl8XQiIiLiRcol+bj11ls5dOgQL774IklJScTFxbF06VIaNGhQHk8nIiIiXqTcBpwOGjSIQYMGldflRURExEv51d4uIiIiYj0lHyIiIuJWSj5ERETErZR8iEcbPXo0kZGR2Gw2Fi1aRP/+/bn++uutDktEREpByYeb9O/fH5vNhs1mo2LFikRGRtK9e3c+/PBDcnJyrA6vVH744Qf69OlD7dq1HUnCmRYsWMDVV19NjRo1sNlsJCQknPO627ZtY8yYMbz77rskJSXRq1cv3nzzTWbNmuWo06VLF4YMGVLmr0lERMqPkg836tmzJ0lJSezcuZNvvvmGK6+8kieeeILevXtz+vTpcn3urKyscrv28ePHadGiBVOnTj1rncsuu4zx48cX+bo7duwAoG/fvkRFRREcHEx4eDjVqlUrk7hFRMQaSj7cKDg4mKioKOrUqUPr1q157rnnWLx4Md98843Lr/nU1FQefvhhatWqRdWqVenatSu///67y7VefvllatWqRVhYGA8++CDPPvssLVu2dNxv754YN24ctWvXpnHjxgDs3buXW2+9lfPOO4/q1avTt29fdu7c6XLtmTNn0rRpU0JCQmjSpAnTpk076+vq1asXL7/8Mv369Su0zt13380LL7zAVVddVaT3avTo0fTp0weAgIAAx75Aebtd+vfvT3x8PG+++aajVenM1yIiIp5HyYfFunbtSosWLViwYAHkbsB37bXXkpyczNKlS/ntt99o3bo13bp14/DhwwDMnTuXV155hQkTJvDbb79Rv359pk+fnu/a33//Pdu2bWP58uV89dVXnDhxgiuvvJIqVarwww8/8NNPP1GlShV69uzpaBmZMWMGI0eO5JVXXmHbtm2MHTuW559/ntmzZ7v1fRk2bBgzZ84EICkpiaSkpHx13nzzTTp06MBDDz3kqFPemxL6k+wcg9U7DrE4YS+rdxwiO6dMN8AWET9WbouMuVXbtnDGRnZuERUF69eX+jJNmjThjz/+AGDlypVs2rSJlJQUx4Z7r732GosWLeLzzz/n4Ycf5q233uKBBx7gvvvuA+CFF17g22+/5dixYy7XrVy5Mu+//z5BQUEAfPjhhwQEBPD+++87WhJmzpxJtWrVWLVqFT169OCll17i9ddfd7RixMTEsHXrVt59913uvffeUr/WoqpSpYqjeyUqKqrAOuHh4QQFBREaGlpoHSmZZZuTGLNkK0mpGY5z0eEhjOoTS8+4aEtjExHv5xvJR3Iy7N1rdRQlZhiGIxn47bffOHbsGNWru+64e/LkSccYiL/++ivf6rHt2rVjxYoVLueaNWvmSDzs196+fTthYWEu9TIyMtixYwcHDhxg9+7dPPDAAzz00EOO+0+fPk14eHgZvmLxZMs2JzFwzgbObOdITs1g4JwNTL+rtRIQESkV30g+rPrVW0bPu23bNmJiYgDIyckhOjqaVatW5auXd6ClPVmxM4z8TeKVK1d2Oc7JyaFNmzbMnTs3X92aNWuSkWH+yp0xYwbt27d3uT8wMLDYr0u8T3aOwZglW/MlHgAGYAPGLNlK99goAgNsBdQSETk330g+yqDrwyorVqxg06ZNPPnkkwC0bt2a5ORkKlSoQMOGDQt8zEUXXcTatWu5++67HefWF+E9aN26NZ988oljIOuZwsPDqVOnDv/88w933nlnqV6XuwQFBZGdnW11GD5jbeJhl66WMxlAUmoGaxMP06FR9ULriYicjW8kH14iMzOT5ORksrOz2b9/P8uWLWPcuHH07t2be+65B4CrrrqKDh06cP311zNhwgQuuugi9u3bx9KlS7n++utp27Ytjz32GA899BBt27alY8eOfPLJJ/zxxx+cf/75Z33+O++8k1dffZW+ffvy4osvUrduXXbt2sWCBQt4+umnqVu3LqNHj+bxxx+natWq9OrVi8zMTNavX8+RI0cYOnRogdc9duwY27dvdxwnJiaSkJBAREQE9evXB+Dw4cPs2rWLffv2QW7XEbnjOUozXqNhw4b8+uuv7Ny5kypVqhAREUFAgMZRl1RKeuGJR0nqiYgURJ/SbrRs2TKio6Np2LAhPXv2ZOXKlUyZMoXFixc7ujVsNhtLly7liiuu4P7776dx48bcdttt7Ny5k8jISMhNIkaMGMGwYcNo3bo1iYmJ9O/fn5CQkLM+f2hoKD/88AP169enX79+NG3alPvvv5+TJ086WkIefPBB3n//fWbNmkWzZs3o3Lkzs2bNcnQLFWT9+vW0atWKVq1aATB06FBatWrFCy+84Kjz5Zdf0qpVK6699loAbrvtNlq1asU777xTqvd02LBhBAYGEhsbS82aNdm1a1eprufvaoWd/W+ouPVERApiMwoaLGChtLQ0wsPDSU1Nzdc1kJGRQWJiIjExMef8ovU33bt3Jyoqio8++sjqUDyO/m6KLjvHoNOEFSSnZhQ47sMGRIWH8NPwrhrzISIuzvb9fSZ1u3ihEydO8M4773D11VcTGBjI/Pnz+e6771i+fLnVoYmXCwywMapPLAPnbMCWO8bDzp5qjOoTq8RDREpF3S5eyN41c/nll9OmTRuWLFnCF198UeTVQ0XOpmdcNNPvak1UuGsrUVR4iKbZikiZUMuHF6pUqRLfffed1WGID+sZF0332CjWJh4mJT2DWmEhtIuJUIuHiJQJJR8iUqDAAJum04pIuVC3i4iIiLiVVyYfOTk5VocgXkR/LyIinsWrul2CgoIICAhg37591KxZk6CgoHzLjIvYGYZBVlYWBw4cICAgwGWfGxERsY5XJR8BAQHExMSQlJTkWClT5FxCQ0OpX7++Vj4VEfEQXpV8kNv6Ub9+fU6fPq09PeScAgMDqVChglrIREQ8iNclH+Suc1GxYkUqVqxodSgiIiJSTGqHFhEREbdS8iEiIiJupeRDRERE3MrjxnzYN9lNS0uzOhQREREpIvv3tv17/Gw8LvlIT08HoF69elaHIiIiIsWUnp5OeHj4WevYjKKkKG6Uk5PDvn37CAsL8/npkWlpadSrV4/du3dTtWpVq8PxK3rvraH33Tp6763jL++9YRikp6dTu3btc66r5HEtHwEBAdStW9fqMNyqatWqPv0H6cn03ltD77t19N5bxx/e+3O1eNhpwKmIiIi4lZIPERERcSslHxYKDg5m1KhRBAcHWx2K39F7bw2979bRe28dvff5edyAUxEREfFtavkQERERt1LyISIiIm6l5ENERETcSsmHiIiIuJWSDxEREXErJR8eJjMzk5YtW2Kz2UhISLA6HJ+3c+dOHnjgAWJiYqhUqRKNGjVi1KhRZGVlWR2aT5o2bRoxMTGEhITQpk0bfvzxR6tD8nnjxo3jkksuISwsjFq1anH99dfz119/WR2W3xk3bhw2m40hQ4ZYHYpHUPLhYZ555hlq165tdRh+488//yQnJ4d3332XLVu28MYbb/DOO+/w3HPPWR2az/nkk08YMmQII0eOZOPGjVx++eX06tWLXbt2WR2aT4uPj2fw4MGsWbOG5cuXc/r0aXr06MHx48etDs1vrFu3jvfee4/mzZtbHYrH0DofHuSbb75h6NChfPHFF1x88cVs3LiRli1bWh2W33n11VeZPn06//zzj9Wh+JT27dvTunVrpk+f7jjXtGlTrr/+esaNG2dpbP7kwIED1KpVi/j4eK644gqrw/F5x44do3Xr1kybNo2XX36Zli1bMnnyZKvDspxaPjzE/v37eeihh/joo48IDQ21Ohy/lpqaSkREhNVh+JSsrCx+++03evTo4XK+R48e/PLLL5bF5Y9SU1MB9DfuJoMHD+baa6/lqquusjoUj+Jxu9r6I8Mw6N+/P4888ght27Zl586dVofkt3bs2MFbb73F66+/bnUoPuXgwYNkZ2cTGRnpcj4yMpLk5GTL4vI3hmEwdOhQOnXqRFxcnNXh+LyPP/6YDRs2sG7dOqtD8Thq+ShHo0ePxmaznfW2fv163nrrLdLS0hgxYoTVIfuMor73ee3bt4+ePXty88038+CDD1oWuy+z2Wwux4Zh5Dsn5efRRx/ljz/+YP78+VaH4vN2797NE088wZw5cwgJCbE6HI+jMR/l6ODBgxw8ePCsdRo2bMhtt93GkiVLXD6Es7OzCQwM5M4772T27NluiNa3FPW9t38o7Nu3jyuvvJL27dsza9YsAgKUl5elrKwsQkND+eyzz7jhhhsc55944gkSEhKIj4+3ND5/8Nhjj7Fo0SJ++OEHYmJirA7H5y1atIgbbriBwMBAx7ns7GxsNhsBAQFkZma63OdvlHx4gF27dpGWluY43rdvH1dffTWff/457du3p27dupbG5+v27t3LlVdeSZs2bZgzZ45ffyCUp/bt29OmTRumTZvmOBcbG0vfvn014LQcGYbBY489xsKFC1m1ahUXXnih1SH5hfT0dP7991+Xc/fddx9NmjRh+PDhft/tpTEfHqB+/foux1WqVAGgUaNGSjzK2b59++jSpQv169fntdde48CBA477oqKiLI3N1wwdOpS7776btm3b0qFDB9577z127drFI488YnVoPm3w4MHMmzePxYsXExYW5hhjEx4eTqVKlawOz2eFhYXlSzAqV65M9erV/T7xQMmH+Ltvv/2W7du3s3379nyJnhoFy9att97KoUOHePHFF0lKSiIuLo6lS5fSoEEDq0PzafapzV26dHE5P3PmTPr3729RVOLv1O0iIiIibqVRdSIiIuJWSj5ERETErZR8iIiIiFsp+RARERG3UvIhIiIibqXkQ0RERNxKyYeIiIi4lZIPERERcSslHyIiIuJWSj5ERETErZR8iIiIiFv9PyazBuvdi4E5AAAAAElFTkSuQmCC",
|
|
"text/plain": [
|
|
"<Figure size 640x480 with 1 Axes>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"import numpy as np\n",
|
|
"import matplotlib.pyplot as plt\n",
|
|
"\n",
|
|
"# 1) Generate quadratic data\n",
|
|
"np.random.seed(3)\n",
|
|
"\n",
|
|
"n = 50\n",
|
|
"x = np.random.uniform(-5, 5, n)\n",
|
|
"\n",
|
|
"a_true = 2.0\n",
|
|
"c_true = 5.0\n",
|
|
"noise = np.random.normal(0, 3, n)\n",
|
|
"\n",
|
|
"y = a_true * x**2 + c_true + noise\n",
|
|
"\n",
|
|
"# 2) Fit degree 11 polynomial\n",
|
|
"coeffs = np.polyfit(x, y, 11)\n",
|
|
"\n",
|
|
"# Create polynomial function\n",
|
|
"p = np.poly1d(coeffs)\n",
|
|
"\n",
|
|
"# 3) Sort x for smooth plotting\n",
|
|
"x_sorted = np.linspace(min(x), max(x), 500)\n",
|
|
"\n",
|
|
"# 4) Plot\n",
|
|
"plt.scatter(x, y, label=\"Data\")\n",
|
|
"plt.plot(x_sorted, p(x_sorted), 'r', linewidth=2, label=\"Degree 11 fit\")\n",
|
|
"\n",
|
|
"plt.legend()\n",
|
|
"plt.title(\"Degree 11 Polynomial Fit\")\n",
|
|
"plt.show()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "3aa62d78",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n",
|
|
"- **Outliers**: large deviations can dominate the $L^2$ norm. This is where normalization might be key.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "86606942",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n",
|
|
"- **Heteroscedasticity**: this is when the variance of noise changes across observations. Certain least-squares assumptions will break down."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "04e122fb",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n",
|
|
"- **Condition number**: a large condition number indicates numerical instability and sensitivity to perturbation, even when formal solutions exist."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "b5e424fd",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n",
|
|
"- **Insufficient tolerance**: in numerical algorithms, thresholds used to determine rank or invertibility must be chosen carefully. Poor choices can lead to misleading results."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "8d1dd798",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n",
|
|
"\n",
|
|
"The point is that many failures in data science are not conceptual, but they happen geometrically and numerically. Poor choices lead to poor results. \n",
|
|
"\n"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.14.3"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|