The main problem is that there are several formats of color fonts and most browsers do support only some of them. Firefox does support Colr/Cpal and SVG fonts, Internet explorer (win10) supplest those and Apples sbix and Safari only sbix. Chrome in win10 supports Colr/Cpal but not its own color format.
So you need sbix and Colr (or SVG) table to support most users.