Best practices for unit testing integration with an unpredictable third party resource that you don't have control over

I've written an application that interacts with at least one third party resource (in my case it's a website and there are many websites that need to be tested against) that I do not have control over. Part of the interaction requires logging in with real user credentials and interacting with data that is transient.

As such, I have the following problems:

• My test(s) must include private data (username and password) in order to log in
• The data I'm looking for will no longer be valid anywhere from 24 to 48 hours after the identifier pointing at that data has been coded into the test(s) and new data matching the test scenario will need to be chosen

Which gives rise to the following questions:

• How and where should I reference this data to prevent it from accidentally ending up in source control?
• Is there a way to request this input as a precondition of running the relevant tests?
• How should I deal with the problem that automated build scenarios wil开发者_JAVA技巧l fail when the test data becomes outdated every couple of days?
• What are the best practices for writing tests that deal with this sort of scenario?

I'm using Microsoft's unit testing framework with .Net 4.

I've run into this general class of problems before when writing automated tests. General they have to do with depending on an unmanaged resource. This resource can be a SOAP service, network or in your case a website.

My general approach is to have tests that can run in 2 modes.

Unmanaged Mode

I want to leverage the real resource during testing, this ensures that the code actually works with the actual resource. This is also useful when you need to extend the code for a new resource, or a change in the structure of the resource.

Managed Mode

I want to capture the transient data and use that as a fixture in the mock of the unmanaged resource. This ensures that the code still works against a particular real world example (although static), and it gives me the fine grained control that I get from using a managed (mock) resource. I can also run this test if the resource in question becomes unavailable for some time, or is simply unavailable in general (ie. being run from the build server).

It's not an exact solution for your problem, but have you considered testing against a mock website? i.e. write your own "website" that exists as part of the test framework, and behaves in a predictable and consistent manner. All it needs to do is respond to a minimal set of requests.

With this approach, you still demonstrate that your code works as expected (and doesn't regress, etc.), you can clearly identify the source of any regression as being from your code vs. the unpredictable 3rd-party stuff, and you eliminate the privacy concerns.