New data and technologies, in particular machine learning, may make it possible to forecast neighbourhood change. Doing so may help, for example, to prevent the negative impacts of gentrification on marginalised communities. However, predictive models of neighbourhood change face four challenges: accuracy (are they right?), granularity (are they right at spatial or temporal scales that actually matter for a policy response?), bias (are they equitable?) and expert validity (do models and their predictions make sense to domain experts?). The present work provides a framework to evaluate the performance of predictive models of neighbourhood change along these four dimensions. We illustrate the application of our evaluation framework via a case study of Buffalo, NY, where we consider the following prediction task: given historical data, can we predict the percentage of residential buildings that will be sold or foreclosed on in a given area over a fixed amount of time into the future?