Rearuh

In Anscombe's quartet the two datasets on the bottom both contain influential points. All four sets are identical when examined using simple summary statistics, but vary considerably when graphed. If one point were removed, the line would look very different.

In statistics, an influential observation is an observation for a statistical calculation whose deletion from the dataset would noticeably change the result of the calculation.^[1] In particular, in regression analysis an influential point is one whose deletion has a large effect on the parameter estimates.^[2]

Assessment

Various methods have been proposed for measuring influence.^[3][4] Assume an estimated regression y=Xb+e{displaystyle mathbf {y} =mathbf {X} mathbf {b} +mathbf {e} }, where y{displaystyle mathbf {y} } is an n×1 column vector for the response variable, X{displaystyle mathbf {X} } is the n×k design matrix of explanatory variables (including a constant), e{displaystyle mathbf {e} } is the n×1 residual vector, and b{displaystyle mathbf {b} } is a k×1 vector of estimates of some population parameter β∈Rk{displaystyle mathbf {beta } in mathbb {R} ^{k}}. Also define H≡X(XTX)−1XT{displaystyle mathbf {H} equiv mathbf {X} left(mathbf {X} ^{mathsf {T}}mathbf {X} right)^{-1}mathbf {X} ^{mathsf {T}}}, the projection matrix of X{displaystyle mathbf {X} }. Then we have the following measures of influence:

1. 
   DFBETAi≡b−b(−i)=(XTX)−1xiTei1−hi{displaystyle {text{DFBETA}}_{i}equiv mathbf {b} -mathbf {b} _{(-i)}={frac {left(mathbf {X} ^{mathsf {T}}mathbf {X} right)^{-1}mathbf {x} _{i}^{mathsf {T}}e_{i}}{1-h_{i}}}}, where b(−i){displaystyle mathbf {b} _{(-i)}} denotes the coefficients estimated with the i-th row xi{displaystyle mathbf {x} _{i}} of X{displaystyle mathbf {X} } deleted, hi=xi(XTX)−1xiT{displaystyle h_{i}=mathbf {x} _{i}left(mathbf {X} ^{mathsf {T}}mathbf {X} right)^{-1}mathbf {x} _{i}^{mathsf {T}}} denotes the i-th row of H{displaystyle mathbf {H} }. Thus DFBETA measures the difference in each parameter estimate with and without the influential point. There is a DFBETA for each point and each observation (if there are N points and k variables there are N·k DFBETAs).^[5]
   


2. DFFITS


3. 
   Cook's D measures the effect of removing a data point on all the parameters combined.^[2]
   

Outliers, leverage and influence

An outlier may be defined as a surprising data point. Leverage is a measure of how much the estimated value of the dependent variable changes when the point is removed. There is one value of leverage for each data point.^[6] Data points with high leverage force the regression line to be close to the point.^[2] In Anscombe's quartet, only the bottom right image has a point with high leverage.

See also

• Outlier


• 
  Leverage
  
  • Partial leverage
  
  


• Regression analysis


• Cook's distance#Detecting highly influential observations


• Anomaly detection

References

Further reading

• 
  Dehon, Catherine; Gassner, Marjorie; Verardi, Vincenzo (2009). "Beware of 'Good' Outliers and Overoptimistic Conclusions". Oxford Bulletin of Economics and Statistics. 71 (3): 437–452. doi:10.1111/j.1468-0084.2009.00543.x.
  


• 
  Kennedy, Peter (2003). "Robust Estimation". A Guide to Econometrics (Fifth ed.). Cambridge: The MIT Press. pp. 372–388. ISBN 0-262-61183-X.