At the moment if a font includes at least one glyph from one of the matching language groups, then that group will always be visible even if turned off in the active language setting.
In some cases it would be useful to turn off categories such as music, math or emoji if only a handful of glyphs from those categories are present in the font. At the moment the only way to do this would be to modify the category a glyph belongs to in the glyphdata xml file.
This would also be useful for those that add their own custom language categories based on the standard one. At the moment even if a “Latin - Custom” was available in the options menu and activated, the normal “Latin” category could not be turned off.
At presents it seems that you can only add to a language category and not out right replace it. As such, duplicate categories for scripts such as latin or greek are sometimes necessary for larger modifications.