I've been looking at the nitrate calculator. Does the 3x5 subsampling option have lots of moire and aliasing?
You mean
bitrate calculator? Which one? There are many online.
Please explain what you mean by 3x5 subsampling option. Is this a menu option?
Perhaps you're talking about mv720 mode in raw video or the 1280x720 H.264 setting which skips every 3 pixels horizontally and 5 pixels vertically. Yes, this does have lots of moiré and aliasing on the EOSM as it does on other cameras except for the 5D3 which uses pixel binning instead of skipping.
The EOSM doesn't have mv1080 mode raw video, 3x3 sampling, which is far superior. However it does have crop mode which shows virtually no moiré and aliasing on most shots. If your subject has lots of fine patterns and details you may still have some problems but that is an issue with pretty much all digital cameras.