If you use automatic feature code, then R_y.swsh_ will be placed in the dlig feature. By default, this feature is ordered before (over) the swsh feature, so rearrange the features such the swsh is before dlig.
For manual feature code, you could consider adding the ligature rule directly into the swsh feature so that the user does not need to activate dlig separately from swsh:
sub R y by R_y.swsh_;
sub R by R.swsh;
The glyph name R_y.swsh_ is following Glyphs’s naming scheme where first the ligature parts are named (R_y) and then any modifiers separated by a dot. The modifier parts (just swsh_ in this case) also use an underscore to describe separate modifiers for the ligature parts (swsh for R and nothing for y, in this case).