I ran into this problem where custom marks (in my case, Armenian punctuation with non-unicode .comb marks) were not combining with their base glyphs. After quadruple checking that .comb glyphs had the correct anchor names and were set as non-spacing marks, I found that having autohint set explicitly to off, either as a custom parameter in the instance or from the export window, would cause the .comb glyph to not attach correctly.
I should also mention that I am noticing this issue when the .comb glyphs have a set width (which automatically reduce to zero on export). However, if I explicitly set the width to zero (which makes it very difficult to select) then the marks are in the correct places.
The x values of the anchors are too far to the left. It appears that the full width of the combining mark plus the distance from the RSB is subtracted, instead of just the distance from the RSB.