In order to capture the environmental externalities of urban agglomeration, we use an agglomeration indicator referred to as economic density, which can be decomposed into the average population density and the concentration of population distribution. Our benchmark regressions use the ordinary least squares method and grid-level panel data for China and for the period 2000–2016. We find that economic density is positively correlated with PM2.5 concentration, and the effects of the two components of economic density are opposite. To address the endogeneity issues, the causal effect of economic density on PM2.5 concentration is estimated with an instrumental variable method. Empirical results show that the PM2.5 concentration increases with economic density, and the associated elasticity is between 0.045 and 0.079. The findings of our benchmark regressions are also supported by a variety of robustness checks. Moreover, while economic growth, the development of secondary industry, and the presence of coal-driven power plants explain why cities with more dense population are more polluted, residential energy use is an opposite channel through which cities with higher economic density can reduce air pollution. Overall, the total effect is a trade-off, and the negative environmental externalities of agglomeration are larger than positive environmental externalities.