In this paper, we study the determinants of rental values in urban housing markets in Kigali, Rwanda. In particular, we study the value of access to piped water; due to the high costs associated with installing new piped connections, renting a property with an existing connection is often the only way for low income households to access piped water. Our results indicate that extending the piped network to a new house will in many cases raise the rental value of the house enough to pay for the cost of installing the new connection in less than two years.