Seed parameter to reproduce a failed test

[lake-effect]

Say you have a lot of data as your test input. When you get a failure message, rantly/rantly-rspec ends up telling you so much that you can't use it for reproducing regressions:

3 successful tests, failed on:
       [#<ActiveRecord::Relation [#<SomeRecord id: nil, label: "some_label", value: 0.1241e4>, #<SomeRecord id: nil, label: "some_other_label", value: 0.2577e4>, #<SomeRecord id: nil, label: "some_label", value: 0.1233e4>, #<SomeRecord id: nil, label: "some_other_label", value: 0.1143e4>, #<SomeRecord id: nil, label: "some_label", value: 0.2001e4>, #<SomeRecord id: nil, label: "some_other_label", value: 0.596e3>, #<SomeRecord id: nil, label: "some_label", value: 0.1926e4>, #<SomeRecord id: nil, label: "some_other_label", value: 0.1309e4>, #<SomeRecord id: nil, label: "some_label", value: 0.2292e4>, #<SomeRecord id: nil, label: "some_other_label", value: 0.1136e4>, ...]>, "day"]

...

Randomized with seed 61833

It turns out that running rspec --seed 61833 doesn't reproduce this input set either.

Is there a shorthand way to tell rspec + rantly to run this exact test again?

[abargnesi]

Hi @lake-effect. Good question.

Rantly uses Kernel#rand to generate random data so you should be able to use Kernel#srand to seed random number generation from a numeric seed value.

This example will generate the same string b|1sYv for each run because the random seed value is always 123:

require 'rantly'

Kernel.srand(123)
puts Rantly { string }
# => b|1sYv

Looking at rspec docs (Randomization can be reproduced across test runs) it seems you could map the rspec seed value to srand.

Can you try the following in your rspec setup before test execution?

srand RSpec.configuration.seed

That should make your rspec and rand seed match which should give you reproducible randomization in rantly.

[lake-effect]

I made the following change to spec/spec_helper.rb:

RSpec.configure do |config|
+  srand RSpec.configuration.seed

Ran rspec --seed 61833 spec/classes/library_spec.rb and still got a successful test, where I didn't expect one.

Assuming that's correct, maybe the seed Rantly was using at the time was something else and I can't reproduce it yet?

[lake-effect]

I noticed that the seed gets you different results if you're running on a specific test spec/classes/some_spec.rb:34 versus the whole test file without the line number. This can be punishing if you have a massive test file.