Quality Testing

Code is Guilty Until Proven Innocent

The importance of defensive programming

Defensive programming is a commonly used term. Wikipedia defines this as

A form of defensive design intended to ensure the continuing function of a piece of software under unforeseen circumstances. Defensive programming practices are often used where high availability, safety or security is needed.

This is one of the most important principles to keep in mind when building something that will be released into the wild.

The first example that immediately pops into my mind is Jenkins. When Jenkins executes a build process it defaults to passing. Unless the process encounters an error, or failing tests appear in the JUnit XML, the build was always be marked as passed. This is not defensive programming.

This has previously been the cause of a number of problems for me. Builds that should have been marked as failed were actually marked as passed. Builds that did nothing were marked as passed, and so I was able to continue thinking that everything was OK. Everything was not OK.

The way this currently works can look something like this:

Start job
Jenkins build result = passed
Pull down some code
Unsuccessfully execute some script

The correct way to handle this is to have the system fail everything by default, and don’t let anything pass until a condition is met. Jenkins should go through a code flow, and the flow should explicitly set the build to passed upon successful completion.

Start job
Pull down code
Execute some script
If a specific condition == true
    Jenkins build result = passed

If the boolean condition doesn’t match what is expected, then the build won’t be set to passed. This would indicate that something went wrong during the process. Again, this would be the ideal way to do it, not what actually happens.

The way that Jenkins currently does this can be problematic for anyone, but especially problematic for anything mission critical.

Example: A healthcare company performs batch processing for blood results every 15 minutes. The processing job will open a file containing information about blood-work. In the file is:

  • the reason for the blood draw
  • the expected range of values of the measured information is
  • the actual value was

Each person’s test result will be Positive, Negative, or Other (indicating a problem with the procedure). As pat of the example, imagine that someone performs a bad update on this script and now it no longer writes out each individuals’ test result.

When Jenkins runs this batch processing job, it will pass. The job will only fail if the files to be written out are being used later on and are missing (causing a FileNotFoundException that goes uncaught).

However, if that’s not the case, everything will continue on and (hopefully) this failure will be caught somewhere down the line.

In Jenkinsfile (a DSL on top of Groovy that codifies a Jenkins), the parameter is currentBuild.result and it defaults to passed. If the code is ever set to anything other than passed, it cannot be set to passed again during the entire duration of that build.

Cloudbees has made it more difficult to do defensive programming. The workaround for this system is to create a new variable, actualResult and default it to false. Code the job in Jenkinsfile as it normally would happen. At the condition where it is known if the build should be passed or failed, actualResult should be true or false. At the end, set currentBuild.result = actualResult.

Jenkins is not the only case. There are many other examples that can be used. Another one is a simple phone number input field. As with any input field, it is best practice to have validation around it to make sure that it's actually a valid phone number By default, the field should say that the input (or lack thereof) is invalid. It should go through validation to prove that it is good.

Without any sort of validation on the front-end, “abcd” would be a valid phone number. That’s not what the database field will want to accept, and it should error out. Instead, the field should determine the following to make a judgement:

  • Correct amount of numbers?
  • Extension?
  • Country code included?
  • Any non-digits?

By assuming that code is guilty and forcing it has to prove its innocence and checking expected conditions before blindly stamping a seal of approval (a passing build), an increased amount of safety and security will be baked into applications. This is especially important for mission critical systems, and it’s not any more difficult to put into smaller applications. Follow these practices and save yourself the headache later on. Always prove innocence in code.