Uniform post selection inference for LAD regression and other Z-estimation problems

We develop uniformly valid confidence regions for regression coefficients in a high-dimensional sparse median regression model with homoscedastic errors. Our methods are based on a moment equation that is immunized against non-regular estimation of the nuisance part of the median regression function by using Neyman’s orthogonalization. We establish that the resulting instrumental median regression estimator of a target regression coefficient is asymptotically normally distributed uniformly with respect to the underlying sparse model and is semiparametrically efficient. We also generalize our method to a general non-smooth Z-estimation framework with the number of target parameters p₁ being possibly much larger than the sample size n. We extend Huber’s results on asymptotic normality to this setting, demonstrating uniform asymptotic normality of the proposed estimators over p₁-dimensional rectangles, constructing simultaneous confidence bands on all of the p₁ target parameters, and establishing asymptotic validity of the bands uniformly over underlying approximately sparse models.