Should BDD scenarios include actual test data, or just describe it?

We've come to a point where we've realised that there are two options for specifying test data when defining a typical CRUD scenario:

Option 1: Describe the data to use, and let the implementation define the data

Scenario: Create a region
    Given I have navigated to the "Create Region" page
      And I have typed in a valid name
      And I have typed in a valid code
    When I click the "Save" button
    Then I should be on the "Regions" page
     And the page should show the created region details

Option 2: Explicitly state the test data to use

Scenario: Create a region
    Given I have navigated to the "Create Region" page
      And I have filled out the form as follows
        | Label | Value  |
        | Name  | Europe |
        | Code  | EUR    |
    When I click the "Save" button
    Then I should be on the "Regions" page
     And the page should show the following fields
        | Name   | Code开发者_运维知识库 |
        | Europe | EUR  |

In terms of benefits and drawbacks, what we've established is that:

Option 1 nicely covers the case when the definition of say a "valid name" changes. This could be more difficult to deal with if we went with Option 2 where the test data is in several places. Option 1 explicitly describes what's important about the data for this test, especially if it were a scenario where we were saying something like "has typed in an invalid credit card number". It also "feels" more abstract and BDD somehow, being more concerned with description than implementation.

However, Option 1 uses very specific steps which would be hard to re-use. For example "the page should show the created region details" will probably only ever be used by this scenario. Conversely we could implement Option 2's "the page should show the following fields" in a way that it could be re-used many times by other scenarios.

I also think Option 2 seems more client-friendly, as they can see by example what's happening rather than having to interpret more abstract terms such as "valid". Would Option 2 be more brittle though? Refactoring the model might mean breaking these tests, whereas if the test data is defined in code the compiler will help us with model changes.

I appreciate that there won't be a right or wrong answer here, but would like to hear people's opinions on how they would decide which to use.

Thanks!

I would say it depends. There are times when a Scenario might require a large amount of data to complete a successful run. Often the majority of that data is not important to the thing we are actually testing and therefore becomes noise distracting from the understanding we are trying to achieve with the Scenario. I started using something I call a Default Data pattern to provide default data that can be merged with data specific to the Scenario. I have written about it here:

http://www.cheezyworld.com/2010/11/21/ui-tests-default-dat/

I hope this helps.

I prefer option 2.

To the business user it is immediately clear what the inputs are and the outputs. With option 1 we don't know what valid data is, so your implementation may be wrong.

You can be even more expressive by adding invalid data too, when appropriate

Scenario: Filter for Awesome
    Given I have navigated to the "Show People" page
    And I have the following data
    | Name  | Value  |
    |  John | Awesome|
    |  Bob  | OK     |
    |  Jane | Fail   |
When I click the "Filter" button
Then the list should display    
    | Name   | Value   |
    |  John  | Awesome |

You should however keep the data so its described in terms of the domain, rather that the specific implementation. This will allow you to test at different layers in your application. e.g. UI Service etc..

Every time I think about this I change my mind. But if you think about it - the test is to prove that you can create a region. A Criteria met by both options. But I agree that the visual cues with option 2 and developer friendliness are probably too good to turn down. In examples like this, at least.

I would suggest you take a step back and ask what stories and rules you are trying to illustrate with these scenarios. If there are rules about what makes a valid or invalid region code, and your stakeholders want to describe those using BDD, then you can use specific examples of valid and invalid region codes. If you want to describe what can happen after a region is created, then the exact data is not so interesting.

Your "Create a region" is not actually typical of the scenarios that we use in BDD. It can be characterised as "when I create a thing, then I can see the thing". It's not a useful scenario in that it doesn't by itself deliver anything valuable to the user. We look for scenarios in which something interesting or valuable is delivered to the end-user. Why is the user creating a region? What is the end goal? So that another user can assign other objects to that region, perhaps?

Example mapping, where stories are linked with rules and examples (where the examples become scenarios), is described in https://cucumber.io/blog/bdd/example-mapping-introduction/