Release build vs nightly build [closed]

Closed. This question is off-topic. It is not currently accepting answers.

---

Want to improve this question? Update the question so it's on-topic for Stack Overflow.

Closed 11 years ago.

Improve this question

A typical solution is to have a CI (Continuous Integration) build running on a build server: It will analyze the source c开发者_开发问答ode, make build (in debug) and run tests, measure test coverage, etc.

Now, another build type usually known is "Nightly build": do slow stuff like create code documents, make a setup package, deploy to test environment, and run automatic (smoke or acceptance) tests against the test environment, etc.

Now, the question:

• Is it better to have a third separate "Release build" as release build?
• Or do "Nightly build" in release mode and use it as a release?

What are you using in your company?

(The release build should also add some kind of tag to source control of potential product version.)

---

Usually the way I do it is my nightly builds get promoted to Release builds. You usually don't want to create a separate release build. Most QA teams should test nightly builds (hopefully they aren't testing from your CI builds). Once they have deemed it good enough for release, you promote it to release status. You determine what that means. Move it to another location, rename it, tag it, label it, burn it, etc.

You don't want to have QA testing a nightly, then when they deem it good, build another one, that you say is the same. You never know, they can be different. An OS patch may have been applied to your build machine, a third party tool may have been updated, etc. You don't want to make your QA teams work twice to test the "same exact build". It might be from the same source, but there is no guarantee it is the same exact build.

---

The answer to your question depends highly on the project you're working on and the goals you want to set.

In general, (trivially true for small projects) a build should be very fast and it should include everything needed for deployment. This is for me always the goal even if I don't reach it - at least not at once. It just keeps me looking at what can be improved all the time.

I know from working on big legacy projects that there are so many accumulated problems slowing things down that it might not be feasible. At least not as an immediate target at least. In large legacy projects compiling and linking usually takes too long, tests (if existing) might also run too long and generating all the required information for deployment might also be slow and even manual. Also build hardware might be insufficient. There are many other things to add to this incomplete list.

When working on a project like this, I try to have separate cycles doing things.

First cycle, a solid CI server which builds, runs automated unit tests, packages and archives builds. This must be fast to give fast feedback to development on changes made. If this is slow, get better hardware for building, sort out dependencies and fix slow unit tests etc. You want this to be as fast as possible. The builds are all deployable builds.

The second cycle would be a slower cycle only picking up builds made by the CI system. It does not work with source code as input, but rather release builds. These are picked up as you want (every build produced) or latest available when ready to do another cycle. This longer cycle would consist of deploying the build onto a test server, run automated functional tests and do other things which are "too slow", "not yet fast" or something else you want which takes a long time. Depending on your organization, you can now add to the deployable package (docs etc.), rename the release according to something visible to clients or things like that. Builds passing here could be good-to-go-live.

If you also have performance tests to run, you might want a third cycle which works with the second cycle's builds as input.

This is briefly described, but the main point here is to separate things, so you can have everything in the chain while getting feedback quicker than having one cycle. I find this a good approach as it's possible to get the benefits of speed (feedback) as well as a natural place to do things.

Finally, I want to mention that the way to go about this would vary from project to project as well, especially if you retrofit CI. You may even want to have a separate continuous build with only build and unit tests, and have a one a day (or something) build which feeds the releases and testing. This would of course mean that only development uses the fast CI builds, because they're incomplete and not suitable for deployment. Still, long-term this is not where you want to be. You want to have the whole chain automated.

---

Over the years I've done this a number of ways. The first was that the release build would happen IF and ONLY IF the 'sanity test' passed for the debug build. It would also auto-deploy to our pre-production environment for user-driven validation.

I've also seen this done where release builds are treated almost as sacred, and that they are only made when it is deemed 'time to get ready to really deploy'. Along with that comes some 'paperwork' and approvals, and then the release build is made (manually) and then it is sanity checked and then deployed.

From my experience it doesn't really matter as long as you are consistent and that it works with the way the company/team understands it should work. Going against the grain is easy at first but then results in what happened at one client which is they actually abandoned a structured build/deployment approach all together (a $100M company did that, imagine, but they did).