What's a good way to do testing a plug-in on multiple Windows and Outlook versions?

We're building a plug-in for Outlook that should work on multiple Windows versions (XP, Vista, 7) and also with different Outlook versions (2003, 2007, 2010).

The testing problem I am facing right now, is that I can't figure out a good/convenient/thorough way to test the application on multiple Windows and Outlook versions.

At the moment, I have a VirtualBox which runs many virtual machines, with different Windows versions and Outlook versions. So I would have a virtual machine with Windows 7 testing Outlook 2010, and another one with Windows 7 testing Outlook 2007, Windows Vista with Outlook 2010 and so on, going through some of the possible combinations. It kind of gets the job done, although it is cumbersome and takes a long time to test.

Some of the testing included in the application is unit testing, but this is also rather tied in with the machine I test it on (windows 7 with outlook 2010). For example, I was using ManagementObject recently, which worked fine on my system (and thus passed the unit test for that method), however, usin开发者_如何学Cg that object threw an exception in another person's system, which crashed the application.

I work on Visual Studio 2010 Ultimate.

The questions: Is there a more elegant way to make the testing process more streamline and more efficient? Any other testing methods you recommend? How would you deal with this problem?

Thanks! Looking forward to your replies.

I've worked on similar situations (in my case is to test 20 different languages of Windows). The basic idea is the same: each virtual machine maps to a specific condition that you need to test.

The cumbersome part comes from inappropriate tool usage. AFAIK, there are several ways to achieve automation (to a certain degree):

• Use VMWare ESX which supports nice scripting. It costs a little fortune unfortunately.
• Use some screen record/replay tools to automate what you are doing by hand right now. There are quite a few on the market and the price difference is huge. Only you can tell what fits you best.
• VirtualBox can be reverted via command-line. Therefore, if you can setup a bunch of machines hosting VBox, each configured to revert VBox during reboot, and each guest image is scripted to load your test, then the problem is to reboot these machines as you wish. This is an easier problem since there are quite a few ways to reboot machine remotely.