From Neurons to Neighborhoods