Across the world, the SARS-CoV-2 (COVID-19) pandemic has disproportionately affected economically disadvantaged groups. This differential impact has numerous possible explanations, each with significantly different policy implications. We examine, for the first time in a low- or middle-income country, which mechanisms best explain the disproportionate impact of the virus on the poor. Combining an epidemiological model with rich data from Bogotá, Colombia, we show that total infections and inequalities in infections are largely driven by inequalities in the inability to work remotely and in within-home secondary attack rates. Inequalities in isolation behavior are less important but non-negligible, while access to testing and contract-tracing plays practically no role. Interventions that mitigate transmission are often more effective when targeted on socioeconomically disadvantaged groups.