I’m playing around with this a bit more. And now it works in one line but not the other. 
It seems that I set my Preferences > Type to show the Arabic and Indic composer and that one is used for a new document/text box. And with that composer, rlig works fine in Illustrator 2023 and 2024.
Not sure why I get this:
(Please give feedback.)
And when I click it, I get to this page:
And if I define a paragraph style from that line of text, I get this:
“Use defined” instead of one of the available composers.