Is it uptown or downtown? Who decides? Does it really matter?
There isn't a rule on this one. It is a perspective thing. Does the speaker think of the location as "up" from here or "down" from here? Up and down can be seen from lots of different ways.
And why not over to London, say? It all depends on how you choose to think of the movement. Down on a map, down in topography, down a river, up a river, up a map, up in social status, up in importance? which and what and why?