A lookup can only contain substitutions of one kind, so I guess that
sub a by X;
sub a b by Y;
sub a b c by Z;
is internally split into 2 lookups, one with a single substitution type:
sub a by X;
and one with a ligature substitution type:
sub a b by Y;
sub a b c by Z;
The rule in the first lookup is evaluated before the second lookup is being processed, so the a
never appears in the input glyph sequence for the second lookup.
This may give the impression that the rules are not reordered when in fact they are, per lookup.