DownloadsDownload削減不能な誤差の推定mitamaバイアス‐バリアンス分解¶証明Err(x0)=E[(Y−f^(x0))2∣X=x0]=σε2+[Ef^(x0)−f(x0)]2+E[f^(x0)−Ef^(x0)]2=σε2+Bias2(f^(x0))+Var(f^(x0))= Irreducible Error +Bias2+ Variance. \begin{aligned} \operatorname{Err}\left(x_0\right) &= \mathrm{E} \left[\left(Y-\hat{f}\left(x_0\right)\right)^2 \mid X=x_0\right] \\ & =\sigma_{\varepsilon}^2+\left[\mathrm{E} \hat{f}\left(x_0\right)-f\left(x_0\right)\right]^2+E\left[\hat{f}\left(x_0\right)-\mathrm{E} \hat{f}\left(x_0\right)\right]^2 \\ & =\sigma_{\varepsilon}^2+\operatorname{Bias}^2\left(\hat{f}\left(x_0\right)\right)+\operatorname{Var}\left(\hat{f}\left(x_0\right)\right) \\ & =\text { Irreducible Error }+\operatorname{Bias}^2+\text { Variance. } \end{aligned}Err(x0)=E[(Y−f^(x0))2∣X=x0]=σε2+[Ef^(x0)−f(x0)]2+E[f^(x0)−Ef^(x0)]2=σε2+Bias2(f^(x0))+Var(f^(x0))= Irreducible Error +Bias2+ Variance. (1)もし、現状のデータとモデルではそれ以上誤差を削減させられないなら、モデルのフィッティングを頑張るよりも特徴量を増やす努力をしたほうがよいかもしれない。削減不能な誤差の推定¶削減不能な誤差をどう推定するか？Liitiäinen, E., Corona, F., & Lendasse, A. (2008). On nonparametric residual variance estimation. Neural Processing Letters, 28, 155-167.Devroye, L., Györfi, L., Lugosi, G., & Walk, H. (2018). A nearest neighbor estimate of the residual variance. Electronic Journal of Statistics, 12, 1752-1778.