Currently in Color layer one can not use colors from color palette and must always set the actual color, but I want to define several color palettes in the font and be able to switch between them.
SVG table presumably can use CPAL table, in addition to using glyphsLib and ufo2ft to general COLRv1 table from Color layers, so being able to use colors (fill color, stroke color, gradient stops and so on) should be generally useful.
FWIW, as Khaled and I were discussing here, this is also something I would be very interested in using in Glyphs.
I recently produced COLRv0, COLRv1, and SVG-in-OT fonts for my Bradley Initials revival, and it would be nice to handle these all from a single source with a unified set of color palettes (especially now that we are starting to see limited support for multiple palettes.)