Insights

News and information about the Hard Tech world

Common Errors in Continuous Integration

by: Inmetrics in

| September 24, 2018 at 11:37 am

Some examples of errors we should avoid if we seek an efficient Continuous Integration.

Have you ever wondered why Continuous Integration, despite having all the necessary tools and infrastructure, is not adding the expected value to your project? Perhaps the problem is in other stages of the software life cycle, or even in the team’s processes and culture in the way they work.

To begin with, we need to understand the main purpose of this software development practice so we know what we can expect from it.

The goal of Continuous Integration is to shorten the time between developing and releasing a software update running on production, providing quick feedback on if the package can cause any type of problem for the application.

In this post, I will explain some techniques, processes and also some common mistakes that should be avoided in order to achieve better results with Continuous Integration.

Untrusted Feedback

One of the advantages of Continuous Integration is in having multiple pipelines that provide quick feedback to save time and enable the person responsible to take action as soon as possible.

There is no benefit in having a complete Continuous Integration framework if the feedback is not reliable.

For example, the automated test pipeline breaks, but is ignored because it is not reliable enough to identify the reason for the fault from feedback alone. The reason may be only the instability of the environment, the mass used, or the application code itself that has an error.

Another example would be at building time, when we have feedback informing where the error is, but the responsible team does not have the culture to fix it immediately, leaving the fix for a later time, along with many other errors.

There is no sense in having quick feedback if it is not reliable or if no action is taken immediately.

Slow Feedback

Only using a good Continuous Integration tool does not guarantee that there will be fast and reliable feedback. It is also necessary to apply good practices of its concept.

Imagine working without an organized structure, without pipelines to perform automated tests, without parallelism, and receiving error feedback only after finishing running the entire set of tests. Was it really necessary to await all this process to receive feedback? Of course not!

To make matters worse, what if this set of tests had more UI tests than unit tests? Besides not being recommended, it could drastically increase execution time and waiting for feedback.

Techniques such as the Test Automation Pyramid (with more Unit tests than Services tests, and these more than UI), Smoke Test (run key performance functionalities prior to the set of regression tests) and utilize resources like parallelism in running tests are examples that help to receive quick feedback. Because if we find a problem early on, we can stop the testing and focus on the fix as soon as possible, and in this manner save something very valuable in any project: time.

Local code accumulation pending integration

A culture that does not go well with Continuous Integration is that of the team that does not integrate its code with the repository at least once a day. Integrating large code or old branches increases the chances of generating conflict with the main code, increasing the time, complexity and cost to correct it.

One of the advantages of Continuous Integration is to enable developers to quickly refactor and integrate their code, since each integration is analyzed by an automated build that checks if the new package will not generate any conflict or code break, which can be done through static code analysis or tests (from units to UI). And a small and recently updated package makes it easier to search for the source of a code integration failure, should it occur.

With this, the culture of the entire team always performing code integration at least once a day is highly encouraged, avoiding working with long-lived branches.

Specific Environments

Running automated tests in an environment that is shared with other teams such as in manual or development testing is very risky, as changes in the environment for other purposes may lead to false negative test results (e.g., the test fails because of the automation script, and not because of the tested application), making the team lose confidence in feedbacks, or even in false positives (e.g., it passes the test, however there is a bug that was not found because it was camouflaged by a change in the environment), which means that the tests are not being efficient in finding real bugs.

Is Continuous Integration ideal for every situation?

Despite the numerous benefits offered by this development practice, based on what has been written so far, some minimum requirements are recommended for its implementation:

  • Running automated tests in all of the application’s layers
    * Without automated testing, Continuous Integration will not be able to verify if the new package could degrade the application when deployed.
  • Using Code Versioning
    * A versioner is necessary for integrating the code, which can also be the trigger for the test pipelines
  • Strive to discover problems (errors, bugs, faults…) as soon as possible, so they may be dealt with immediately
    * There is no point in deploying Continuous Integration if the team does not have the agile mentality to solve the problems and analyze the feedback
  • The purpose of performing continuous delivery and/or deployment
    * Continuous delivery or deployment turns out to be a consequence of an efficient Continuous Integration, in this manner making almost the entire process automatic
  • The goal of obtaining infrastructure for creating environments automatically
    * Creating environments must also be automatic for ensuring that the tests will run in a unique environment and there will be no unknown factors that could influence the results, making it more reliable.

… and thinking beyond Continuous Integration

After Continuous Integration, as mentioned earlier, it is also a good idea to think about deploying Continuous Delivery (a manually-initiated automated procedure for implementing a new version in a given environment) or even Continuous Deployment (similar to Delivery, but the process is initiated automatically after Continuous Integration).
All these tips and mistakes that must be avoided are for improving the efficiency of Continuous Integration because, in addition to the tool, the team must know its practice and work under its concept.

Thanks to Walmyr Filho.