After playing around a bit more, I found out that the second step above is not necessary to cause the problem. Just deleting a bunch of exports (at the end of the list) fails.
One thing I noticed: If I select the exports immediately after opening the Font Info and then wait for a few seconds then they get deselected. After this, I can delete them without problems.
So, maybe the conclusion is that something is running in the background that shouldn’t (i.e. something that should block the UI until finished)?