Did you set the executable themselves to compatibility mode, or just KS? You have to find where SM is installed and change the setting directly on the EXE or it'll fail- that's what fixed it for me on XP, at any rate...
Windows XP is NOT Windows Vista! No Wing Commander games will work on Windows Vista X64! Microsoft website will confirm! And regards to Windows Vista X86 nothing is working!
I can definitely refute that with Vista on an x86, having played through it during the community replay a short while back.
Make sure you're using the Kilrathi Saga version of SM1 (Available from the CIC) and set the SM1.exe file to have the exact same compatability settings that worked for Kilrathi Saga. For me, it was Windows 98/Me, Disable Visual Themes and run as Administrator.
The two unregistered posters seem like different people. I don't think the original poster has the 64 bit edition. But like Max said, it'd help to know what exactly happens when the games are executed.