[tlohmar]

What type of PR is this?

• documentation

What this PR does / why we need it:

The PR suggests a clarification of the Test Definitions, matching the verification checks. A statement is added, that the Scenario name must be unique.
Tooling is throwing a warning, when more than 50 Scenarios are within one feature file.

Which issue(s) this PR fixes:

Fixes #709

Does this PR introduce a breaking change?

• Yes
• No

Special notes for reviewers:

Changelog input

Clarifying that the Scenario names must be unique. Clarifying the upper limit of scenarios per feature file

Additional documentation

This section can be blank.

docs

[tanjadegroot]

Suggested change

	Each defined error response need its own scenario with a unique scenario name per operation. Alternatively, multiple, operation specific feature files can be created as single feature files can get very large, making them difficult to maintain.

[tlohmar]

Isnt that clear from the structure of steps, i.e. error response is checked with a Then statement, meaning, each error code require an own Scenario?

Or, is that prohibiting the definition of a single Scenario Outline, checking for different error codes?

[tanjadegroot]

It was just a reminder to make it explicit, but feel free to adapt or not use my proposed text.

On the Scenario Outline approach, Eric had responded as follows:

"A third approach was used by IoTDeviceManagement, which was to define the operations as different examples for a common scenario. This works well when the behaviour is largely independent of the operation (e.g. 401 UNAUTHENTICATED - see example below), but would quickly become cumbersome for operation specific behaviour.

My own preference is to have separate feature files for each operation, as single feature files can get very large, making them difficult to maintain and review.

@esim_profile_errors_401_01_no_valid_token
Scenario Outline: is rejected without a valid access token
Given the resource "/esim-profile-management/vwip/"
And the header "Authorization" is set to an invalid access token
When the request "" is sent
Then the response status code is 401
And the response property "$.status" is 401
And the response property "$.code" is "UNAUTHENTICATED"
And the response property "$.message" contains a user friendly text

Examples:
  | endpoint       | operationId            |
  | download       | downloadEsimProfile    |
  | enable         | enableEsimProfile      |
  | disable        | disableEsimProfile     |
  | delete         | deleteEsimProfile      |
  | set-fallback   | setFallbackEsimProfile |
  | retrieve-status| getEsimProfileStatus   |

So I would not put the Scenario Outline in the recommendations.

[tlohmar]

Thanks for the clarification. Could make sense to also describe the "templating across operation's".

I am realizing, that this templating approach goes a bit against the statement "Operations that execute an action or retrieve information from the operator without modifying a resource will usually be tested in its own dedicated feature file." (which could be read as recommendation for operation-specific feature files).

Is there a preference for operations and endpoints into own feature files?

[tlohmar]

@tanjadegroot @eric-murray Please check the updated wording,

[tanjadegroot]

I had a preference for NOT mentioning the outline approach, as it contradicts a bit (as you also say) other statements. This does not mean people cannot use it.

For me the preference is to focus on files (as decided by the team, and not too big per line 96)

but if you want to add it as a possibility I can live with that :-)

[tlohmar]

We can simplify and remove the mentioning of "operations".

ie. remove "or operations (e.g. 401 UNAUTHENTICATED for all operations)".

[tanjadegroot]

works for me !

[tlohmar]

Note, there are multiple occurrences of "must" (all small letters) in the document and this PR introduces now inconsistency with the capitalization.