Skip to content

'allure-pytest-bdd' report marks feature as 'Fail' even when all steps pass when test is using 'Scenario Outline' #455

@ricwal-richa

Description

@ricwal-richa

I'm submitting a ...

  • bug report
  • feature request
  • support request => Please do not submit support request here, see note at the top of this template.

What is the current behavior?

When generating an allure report for pytest-bdd test; the report shows test feature marked as Fail although all steps in the test are marked as Pass.
The test is using Scenario Outline and running for 12 data sets.

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

  1. Create a test in pytest-bdd feature file. sample from my test -
@regression
Feature: Some description
  As blah

  Scenario Outline: value comparison
    Given the <feed_file> exists
    When the <attribute_name> with the <attribute_value> for the <customer_id> is provided in the <feed_file>
    Then verify that CDP mapping for <attribute_name> with <attribute_value> matches for the <customer_id>

    Examples:
      | feed_file | attribute_name  | attribute_value | customer_id |
      | Customer.csv  | FirstName  | Taylor  | 8470939 |
      | Customer.csv  | LastName  | Perry | 8470936 |
      | CustomerEmail.csv  | Email  | justsomeemail@gmail.com | 8470939 |
  1. From terminal, run pytest --activedir=<absolute path to report folder>
  2. Verify that multiple JSON files are generated under report folder when execution completes.
  3. Open any one JSON file.
  4. The report JSON shows feature status as 'Fail' even when all test step status shown as 'Pass'

What is the expected behavior?

Feature status should be marked as 'Pass' when all steps inside the test are 'Pass'.

What is the motivation / use case for changing the behavior?

Bug Fix

Please tell us about your environment:

  • allure-pytest-bdd version: 2.8.6b0
  • pytest version: 5.3.2
  • pytest-bdd version: 3.2.0

Other information

. When same test is executed and report is generated using pytest-html, all tests show as Pass.

. Allure report screenshot

Allure_report_feature_fail

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions