I just recently installed Myst III (DVD version that comes packaged with games 1-5) successfully with ResidualVM. It boots up no problem in both full screen and non-full screen modes. However, when in full screen my mouse doesn't work properly. That is, it moves around so fast to an uncontrollable degree making it impossible for me to even select any options on the main menu screen of the game when it boots.
I have tried lowering the DPI and windows mouse pointer speed to basically "zero", however it doesn't fix the problem (my mouse is a Roccat Nyth). So, I can only assume this is a resolution bug. It's a shame because the game seems to be running absolutely fine, and like I said, in the non-full screen mode the mouse pointer and the game runs absolutely fine, it's just too small to be playable.
The only other info I can give is my monitor is 2560 x 1440 native res. I don't think this issue is impacted by any other hardware, however if you want more info on that, I can provide it. I was surprised it installed and booted so well with ResidualVM for such an old game - the devs from ResidualVM did an amazing job!
Appreciate anyone who can help!