How much is too much time for a rake test to be ran? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.

---

Want to improve this question? Upd开发者_开发技巧ate the question so it can be answered with facts and citations by editing this post.

Closed 9 years ago.

Improve this question

I have a project that the tests are far from ideal, It takes around 15 min to run all tests, and for me it is a problem. What is the maximum that a rake test should take?

I understand that this a personal take, please write a little bit of why as well, or paste a link to justify your answer, please.

---

Just as a baseline, my current application has 2 sets of unit tests (for non-rails-dependent code and for rails-dependent code). 2 of the most time-consuming things in running rake in a rails environment is loading up rails and bundle exec. This is why we are moving as much of our testing into a non-rails-dependent, non-bundle-exec environment. So, as we touch parts of our existing system, we are moving the specs into the non-rails-dependent section.

Here is the breakdown of timing our current test suite:

time rake spec:no_rails Finished in 0.14229 seconds 141 examples, 0 failures

real 0m0.981s user 0m0.783s sys 0m0.168s

time rake spec Finished in 11.86 seconds 677 examples, 0 failures, 2 pending

real 0m28.613s user 0m23.435s sys 0m3.180s

I'm sure you can see why we are moving non-rails-dependent code examples away from the rails-dependent ones. There are a few other benefits to keeping your code from being dependent on rails, but the speed of tests is one of the most important. Minimizing the feedback cycle is imperative to get the most benefit out of your tests as regression.

UPDATE Just as an update, as we move things over, here are the current stats for our specs. We definitely are finding both the speed to be an advantage, as well as the design pressure to isolate ourselves from rails. When you have your specs running this quickly, you can run them more frequently, which minimizes the feedback loop between doing something and seeing if the system still works. time rake spec:no_rails Finished in 0.36083 seconds 316 examples, 0 failures, 1 pending real 0m1.019s user 0m0.858s sys 0m0.147s

time rake spec:with_rails Finished in 12.86 seconds 706 examples, 0 failures, 2 pending real 0m21.316s user 0m18.610s sys 0m1.954s

---

There's no way to answer without knowing the particulars. Do you have many test cases? I've heard of large systems with hundreds or even thousands of tests that need hours to complete a full suite.

OTOH you might have only a few tests that are slow. Have you tried segmenting the test cases to see if any take a particularly long time? Do you have tests that depend on potentially slow resources, like network or file system operations?