You might switch to using kerning groups.
I doubt that left side bearing of B will be different on its alternative. So name left kerning group that consists of B, B.ss01 or B.alt as Bleft (or to whatever makes sense to you) and the kerning should work the same way as it does for A and B.
for example, in most of my designs, Cyrillic A is exactly the same as latin A, so they have common left and right kerning group values. I certainly doubt that one will ever use cyrillic letter and latin letter together, but if they do, they’ll have that choice of having kerned pair. It’s better maybe to kern every script separately, but I don’t see a problem when they’re kerned together instead (to save time).
But your situation is different and I believe that regular-alternative kerning is just as important as regular-regular.