rspec-search-and-destroy

KO | EN

gitlite — search

#hacktoberfest #ai #typescript #llm #android #machine-learning #rust #macos #nodejs #cli #awesome-list #pytorch #ios #open-source

rspec-search-and-destroy

(★ 18)

Automatically finds test-ordering bugs in RSpec test suites

rspec-search-and-destroy Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

RSpec Search-and-Destroy (SAD) runs your RSpec suites to automatically
find the root causes of test ordering bugs.

# Background

Ideally, tests are independent of each other, but sometimes global
state can leak out of one test and cause another test to fail. When
you have many tests, these types of failures can be insidious to
debug.

# Usage

First, add the appropriate hooks in your `spec_helper.rb`:

```ruby
require 'rspec-search-and-destroy'

RSpec.configure do |config|
  RSpecSearchAndDestroy.configure(config)
end
```

Then run the driver program:

```
# Search and destroy mode
rspec-sad

# If you have a particular ordering that creates issues
SPEC_OPTIONS="--seed 12345" rspec-sad

# If you have a complicated script
rspec-sad --rspec-command "/path/to/script"
```

# Search and destroy mode

In the search phase, SAD will run your test suite until a test failure
occurs, then switch to the destroy phase. In the destroy phase, the
contributing tests are narrowed down until a single test is found that
causes the failure.

# Caveats

RSpec SAD requires that your test suite not have any flaky tests. Any
intermittently failing tests will cause false positives or false
negatives. In this case, the results will not provide any useful
information.

# How it works

During the search phase, your test suite will be run once to get the
set of tests that could contribute to the test failure. The order of
tests will be saved and passed off to the destroy phase.

During each step of the destroy phase, half of the remaining tests
will be disabled. If the test continues to fail, these disabled tests
can be ignored. If the test stops failing, then the currently enabled
tests can be enabled.

## Implementation notes

The driver process runs RSpec as a subprocess. The two processes
communicate using environment variables and files. The driver program
specifies a set of tests to run, and RSpec reports the results back to
the driver.

The test ordering and filtering is done via a custom
`config.order_examples` block, and the results are saved using a
custom formatter.

rspec-search-and-destroy

rspec-search-and-destroy Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

rspec-search-and-destroy

rspec-search-and-destroy Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README