Many modern estimation methods in econometrics approximate an objective function, for instance, through simulation or discretisation. These approximations typically affect both bias and variance of the resulting estimator. We provide a higher-order expansion of such 'approximate' estimators that takes into account the errors due to the use of approximations. This expansion allows us to establish general conditions under which the approximate estimator is first-order equivalent to the exact estimator. Moreover, we use the expansion to propose adjustments of the approximate estimator that remove its first-order bias and adjust its standard errors. These adjustments apply to a broad class of approximate estimators that includes all known simulation-based procedures. We also propose another approach to reduce the impact of approximations, based on a Newton-Raphson adjustment. A Monte Carlo simulation on the mixed logit model shows that our proposed adjustments can yield spectacular improvements at a low computational cost.