Python’s default encoding for reading a file becomes ASCII in macOS Sonoma

Scripting
1

Just upgraded to macOS Sonoma, and my Python scripting started to fail because I haven’t specifying encoding="UTF-8" explicitly for functions like file.read() (we all know it’s supposed to default to “UTF-8” on macOS, right?):

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe3 in position 0: ordinal not in range(128)

Can reproduce with a simple line in the Macro panel:

open("path to non-ASCII file").read()

Platform:

  • both the latest stable and cutting edge versions of Glyphs: 3.1.2 (3151), 3.2 (3226)
  • Python version: “3.9.1 (Glyphs)” for both Glyphs installations
  • macOS Sonoma 14.1.1 (23B81) with system language and region: English (US), Netherlands
2

Which Python version are you using?

3

Oh yeah, forgot to mention. It’s the built-in one for both installations. Added to the post.

4

Interesting, the preferred encoding is different when printed in Glyphs versus command line Python.

import locale
print(locale.getpreferredencoding())

prints UTF-8 when run in the command line, but US-ASCII in the Glyphs Macro Panel.

So I guess it is safer to add encoding="utf-8" to the open call:

open("path to non-ASCII file", encoding="utf-8").read()
5

Of course that’s the workaround. This bug report is about the unexpected change of the default encoding.

6

Alright. It seems not only the Glyphs-supplied Python is affected, but also the versions from the python.org installers.

1 Like