In recent years, a vast array of literature has shown that more and more city centres are affected by retail decline, be it in Japan, Belgium, Britain, the United States, or in France, while urban peripheries benefit from expanding retail activities. In this paper, we first rely on an international state of the art review, which allows us to identify recurring main factors that explain retail decline (such as sectoral concentration and competition from e-commerce), as well as more contextual factors bringing changes in urban settings and consumption practices (like demographic and labour market dynamics, the size of cities, or the location of municipalities within cities). Secondly, we examine how these factors, when combined, contribute to explain the decline in a superior diversity mix of retail activities in small and medium-sized French towns and cities, for four periods of about ten years each, from 1975 to 2014. The decline in shop diversity affects about a third of the municipalities observed in each period. Our hypotheses regarding the role of demographic size, employment mobility and location are then discussed and compared with the information given by the literature review.