Local projections (LPs) are a popular tool in applied macroeconomic research. We survey the related literature and find that LPs are often used with very small samples in the time dimension. With small sample sizes, given the high degree of persistence in most macroeconomic data, impulse responses estimated by LPs can be severely biased. This is true even if the right-hand-side variable in the LP is iid, or if the data set includes a large cross-section (i.e., panel data). We derive a simple expression to elucidate the source of the bias. Our expression highlights the interdependence between coefficients of LPs at different horizons. As a byproduct, we propose a way to bias-correct LPs. Using U.S. macroeconomic data and identified monetary policy shocks, we demonstrate that the bias correction can be large.