When doing Acceptance Testing and following BDD, do you assert database changes or just what the user sees?

I am doing acceptance testing with Steak on a Ruby on Rails application. Imagine I w开发者_JS百科ant to test the functionality of a form.

• It should create an user if all the fields are correct.
• It should not create a user if any of the fields are incorrect.

In the first case, a message will inform that the user has been created: 'The user has been created'

In the second case error messages will indicate the errors.

I can base my tests on the information displayed. That is, the first test pass if the correct message is displayed.

Or I can base my tests on the changes on the database. That is, the first test pass if the database has a new user which contains the data entered.

Or I can assert for both conditions, database should change and the appropriate message should be displayed.

What is the conceptually adequate way of testing this kind of behaviour, according to Behaviour Driven Development?

What is the practical way of testing this kind of behaviour, according to being a Pragmatic Programmer?

Generally, keep to what the user sees.

However, sometimes there's more than one user of the information. Perhaps, once the data has reached the database, another application then uses that data (fairly typical for internal projects!)

If you can't automate with both applications together, then asserting the contents of the database, or perhaps verifying resources on a RESTful URL, can be a good way to go.

If you do this, it will probably change the nature of your scenarios. Rather than saying, Then the database should contain XYZ, try something like:

Given that Fred Brown lives at 25 Warrington Grove, Springfield
When Fred orders a deckchair
Then the warehouse should receive an order:
    Deckchair
    Quantity: 1
    To: Fred Brown
    25 Warrington Grove
    Springfield

So you're always looking at the capabilities of the system, rather than just the data - even if the warehouse doesn't really receive it in your scenario, you're showing why the data has to be in that particular place and form.

This way, you also leave yourself the option of using a real UI, or a RESTful URL, etc. later - the step is independent of the implementation you choose.

My BDD practices involve using Cucumber. I try my hardest to keep it to what the user sees. That is, any When and Then statements should reflect user input or user visuals.

In your case I would make sure that a success message appears afterwards, and then I would (if possible) look for data on the screen indicating the application is aware of the object. For instance, if you added a comment to a blog post you should fill in the post, hit the 'accept' button, see a 'Message Posted' success message, and then also see your content on the page as a new post.

You should back up your user-driven BDD tests with unit and sometimes controller tests to make sure everything behind-the-scenes is functioning correctly as well.

I tend to use RSpec and verify that things get added to the database, methods are returning expected results etc. But I also use Cucumber to make sure the user sees what the user is expecting to see (But I don't really let cucumber care about what's in the DB).

I hope this helps. If you aren't using cucumber yet I highly recommend it for BDD testing in conjunction with RSpec. You should check out the book (comes in electronic PDF form) - I started with that and it helped a ton with my testing practices:

http://www.pragprog.com/titles/achbd/the-rspec-book

I would agree with keeping with what the user see's, up to a point.

If there is some kind of layer that could potentially persist a value in memory, such that it might show up on the site as if it was in the db, when it's not actually been written out TO the db.. then it might be a good idea to back up your 'what the user's see's' with some checks at the db level.