This paper pursues three goals: (1) determining the relative importance of built environment barriers limiting walkability, (2) analyzing the existence of an asymmetry in the way people evaluate positive and negative built environment characteristics, and (3) identifying solutions to tackle the main barriers and quantify their impact in walkability. A best–worst scaling survey was developed to compare the importance of eight different attributes of the built environment regarding walkability. Model results show an asymmetry negative–positive in the judgment and choice of built environment characteristics that promote and impede walkability. The most important barriers, obtained from worst responses, are connectivity, topography, sidewalk surface and absence of policemen. Walkability scores were computed for different neighborhoods and different policy scenarios were forecasted. Simulation results from the worst responses indicate that improvements in sidewalk quality, along with an increase in the number of police officers, lead to an 85% increase in the walkability score for the lower income neighborhoods.