How do I use RestartManager to restart explorer.exe with Windows Installer custom action?

I have an installer that prompts users to restart their computer after an install. I would rather not have the user restart their computer in this case, and have explorer.exe just restart itself using the RestartManager API provided with Windows Vista and up.

I've created a separate executable that gets copied to the local computer during install and runs after that. The separate executable registers explorer.exe, shuts it down, and restarts it based on this code: http://msdn.microsoft.com/en-us/library/aa373681%28v=VS.85%29.aspx. When the executable is run separately from the installer, it works as designed. But when it runs as a custom action as part of an MSI package created with InstalShield, it shuts down explorer.exe but does not restart it.

I always get a 160 error code for RmRestart when it runs with the installer. The docs say it's an error co开发者_如何学Cde meaning there were invalid arguments provided. (http://msdn.microsoft.com/en-us/library/aa373665%28v=vs.85%29.aspx). I'm fairly positive that my arguments are not invalid as they work when the executable runs separately from Windows Installer.

I'm stuck at this point and not sure what else to do to get this working. The only thing I'm uncertain of is if "0" can be a proper session handle returned from RmStartSession() with error code of 0 (Success). Assuming this was wrong, I set up my executable to also take in the RmSessionKey that's created by Windows Installer before InstallValidate. And I use that to call my executable as a deferred action. I get an error of 4c3 for RmShutdown in this case, which seems to be an invalid error code.

Cliffs: Have separate .exe that uses RestartManager API to shutdown, restart explorer.exe that works when not run with Windows Installer, but when combined, it breaks. Seeing error code of 160 for RmRestart(). Ran out of ideas to try to get this working. I can provide code snippets if people want...

Thanks for any suggestions/comments.

I ended up reaching a solution to this...

Rather than creating a separate executable that registers explorer.exe and shuts it down, create a MSI DLL Custom Action. All this DLL has to have is a single function that registers explorer.exe to be restarted and use the existing restart manager session provided by Windows Installer (by default). Then in your installer, add the MsiFilesInUse dialog and you'll be good to go.

Now when the installer runs, it starts the restart manager session, and calls your MSI DLL CA, and adds explorer.exe to the list. The list gets displayed and the user is given options to close or defer closing of the applications.

Using this method allows you to avoid having to distribute a pointless executable to the user, as well as simplifies the amount of code written greatly.