How are some apps able to simply unzip and run on Windows PCs without SxS problems?

It's a common complaint from certain groups that installing on Windows spreads stuff all over the filesystem and registry, and some apps provide the option to simply unzip the files so you can 'uninstall' just be deleting. But how do they avoid all the ugliness of Visual C++ Redist, Side-by-Side (SxS), etc?

I have a little utility and really I'd prefer to distribute just a single .EXE which is run - no installer at all or even unzipping since it is small. Is this feasible?

edit: As suggested, I tried building with static dependency on run-times (/MT instead of /MD). I got the following (middle part truncated for clarity):

1>L开发者_运维问答inking...
1>MSVCRT.lib(MSVCR90.dll) : error LNK2005: _free already defined in LIBCMT.lib(free.obj)
1>MSVCRT.lib(MSVCR90.dll) : error LNK2005: _malloc already defined in LIBCMT.lib(malloc.obj)
1>MSVCRT.lib(MSVCR90.dll) : error LNK2005: __mbschr already defined in LIBCMT.lib(mbschr.obj)
1>MSVCRT.lib(MSVCR90.dll) : error LNK2005: _realloc already defined in LIBCMT.lib(realloc.obj)
1>MSVCRT.lib(MSVCR90.dll) : error LNK2005: __strdup already defined in LIBCMT.lib(strdup.obj)
1>MSVCRT.lib(MSVCR90.dll) : error LNK2005: _abort already defined in LIBCMT.lib(abort.obj)
1>MSVCRT.lib(MSVCR90.dll) : error LNK2005: __stricmp already defined in LIBCMT.lib(stricmp.obj)
1>MSVCRT.lib(MSVCR90.dll) : error LNK2005: _setlocale already defined in LIBCMT.lib(setlocal.obj)
1>MSVCRT.lib(ti_inst.obj) : error LNK2005: "private: __thiscall type_info::type_info(class type_info const &)" (??0type_info@@AAE@ABV0@@Z) already defined in LIBCMT.lib(typinfo.obj)
1>MSVCRT.lib(ti_inst.obj) : error LNK2005: "private: class type_info & __thiscall type_info::operator=(class type_info const &)" (??4type_info@@AAEAAV0@ABV0@@Z) already defined in LIBCMT.lib(typinfo.obj)
1>LINK : warning LNK4098: defaultlib 'MSVCRT' conflicts with use of other libs; use /NODEFAULTLIB:library

You can distribute a single EXE if you statically link to the VC runtime - on the project's Property Page -> Configuration Properties -> C/C++ -> Code Generation and set "Runtime Library" to a "non-dll" option.

Depending on your target audience, distributing as a .exe may be problematic as .exe files are sometimes blocked from downloading in corporate environments, so providing a .zip file as well is advisable.