Disable Shifting Viewport Based on Selection

I noticed the behaviour in the relationship between the selection and the viewport position changed recently.

When I have something selected in one master, and then change to another master, the position often jumps around to a weird place that’s not really useful for example:

Link to video

It’s really jarring and means I have to deselect points before changing masters or suddenly my glyph is out of view. Can this be turned off or fixed? I’m using the CMD+Num shortcut to change masters.