It may have to do with an outdated ufoLib and its installation. Why do you import/need that one? Not sure I understand exactly what the DigestPen does, but I believe the function of the script can be coded simpler without need for loading extra libraries.
DigestPen just returns a glyph in a simplified list of values so it’s easy to compare. You’re totally right I should be make this script with Glyphs App native API.
There is a fast convenience function that checks for compatibility: GSLayer.compareString(). You could use that one in a first loop, and if it is True continue with more complex (and slower) checks.