Principles are those kinda things that you don’t have to believe in for them to exist.  For that matter they will still push you around without your consent.  I have never seen a set of Continuous Integration (CI) principles so I thought that I could share the ones that I have found through out my career.  The first one I elect to share is on task size…

Below is a loop diagram that shows some of the forces and effects task size can exert on CI.  If you have never seen a loop diagram before you can read this diagram by ideas relating to each other either in the same way or in the oppisite way.

As task size increases changeset size will increase as well.  People tend to commit changes when they have completed a task.

As changeset size increases so does the average time to fix a build.  When a build breaks and the changeset size was small it is generally easy to know what went wrong and fix it.  If the changeset size was large there are more things to suspect, investigate, and debug.  This takes longer.

As the average time to fix a broken build increases the build availability decreases.  This is assuming that you don’t allow committing changes to a broken build…the more often you have hard to fix builds the less available the build will be.

As the build is less available the changeset size will increase.  This is because developers will continue working while the build is not available, thus increasing the size of their changesets.

As changeset size increases the build stability will decrease. This is because the build breaks more often with large changesets than small changesets.

As build stability decreases the development environment tractability will decrease.  It is more difficult to work on a codeline that is unstable than a stable codeline.

As the environment tractability decreases the rate of change to the codeline will decrease.  Again it is more difficult to work on an unstable codebase.

As the rate of change decreases there will be less broken builds.  If you have a high rate of change you will have more broken builds.

As the number of broken builds increases the build availability will decrease.

I don’t mean to show this loop diagram as a standalone or complete system.  There are other forces that can come into play here to address the number of build breaks, how long it takes to fix a broken build, or even isolating build breaks.  These are important.  But they are not the core system.  This is the core system and the root input affecting the system’s output is the size of tasks.  If people are working with small tasks they will break the build far less often and when they do it will be easy to fix.  If people are working with large tasks they will break the build often and when they do it will be difficult to fix.  From what I have seen it works best when one or more small tasks can be completed in a day.  You will need to figure out what works on your project. 

You can also add to this diagram mitigating forces such as a precommit developer build to help control the number of broken builds.

P.S. If you are having a hard time getting task sizes down maybe you should draw a loop diagram of the forces at play keeping task sizes large on your project.

I will be speaking at AgileDC this week on agile testing on government contracts. I have not been to this even before but it look like it will be fun with all the great speaks showing up.  I did not see where my slide have ben posted on the agiledc.org site so I will link them in here.  Hope to see you there.

The use of the depends attribute on targets is prevalent in many project Ant scripts.  I believe that depends is the bane of reuse.  Lets take the name of a simple target like deploy.  In the context of a developers environment deploy would likely depend on package, package on unittest, unittest on compile…  This is fine until you want to reuse deploy in a different context, say to a different environment like production.  In that context deploy has no business with such a dependency chain.

That is not a clear enough picture though.  Depends is not just for defining a dependency chain, it also includes the much desired feature of run once.  That is to say when I call the target deploy, compile will only been run once even though both recompile and unittest depend on it.

The output from calling the target deploy would be:

c:\>ant -f scratch.build.xml
Buildfile: scratch.build.xml

clean:
[echo] clean missile

compile:
[echo] compile the missile

recompile:

unittest:
[echo] test the missile

package:
[echo] package the missile

deploy:
[echo] deploy the missile

BUILD SUCCESSFUL
Total time: 0 seconds

Sidenote: you should almost never use antcall.  It violates the principle of least astonishment.  You should think of it like you are ask for Ant to execute the target in isolation.  You should use runtarget from Ant contrib.

I have found the use of orchestration targets to be far more powerful when I can ask for a target to be executed and optionally specify that it only be run once.  The depends run once is not as robust as you might think.  If you call multiple targets from the command line run once does not always work:

c:\>ant -f scratch.build.xml recompile unittest
Buildfile: scratch.build.xml

clean:
[echo] clean missile

compile:
[echo] compile the missile

recompile:

compile:
[echo] compile the missile

unittest:
[echo] test the missile

BUILD SUCCESSFUL
Total time: 0 seconds

What would be ideal would be the following.

With the output:

c:\>ant -f scratch.build.xml recompile unittest
Buildfile: scratch.build.xml

recompile:

clean:
[echo] clean missile

compile:
[echo] compile the missile

unittest:
[echo] test the missile

BUILD SUCCESSFUL
Total time: 0 seconds

I implemented the call task with a macrodef that checks if the target has been run; this is tracked by checking the existence of a property.

This macrodef depends on a property being set after the successful execution of every target.  To accomplish this I created a simple logger to set a property for each target called; source is at the end of the post.  I also made sure that the logger was loaded with the script task at the beginning of the Ant project file.

Once you have this functionality you can begin to orchestrate in more robust ways.  With depends you can only string targets together, one after the other with no customization between the execution of each target.  Now you have the opportunity to wrap each and every target call with any customization you can write.  You can reuse in any way you want.  For this example we only want to compile and package on a developers machine (e.g. env.dev==true).

c:\>ant -f scratch.build.xml -Denv.dev=false
Buildfile: scratch.build.xml

deploy:
[echo] deploy the missile

BUILD SUCCESSFUL
Total time: 0 seconds

package com.agilex.ant;

import java.lang.reflect.Field;
import java.util.Enumeration;
import java.util.Hashtable;
import java.util.Vector;

import org.apache.tools.ant.BuildEvent;
import org.apache.tools.ant.BuildListener;
import org.apache.tools.ant.Project;
import org.apache.tools.ant.ProjectHelper;
import org.apache.tools.ant.Target;

public class TargetListener implements BuildListener {

@Override
public void buildFinished(BuildEvent arg0) {
// TODO Auto-generated method stub
}

@Override
public void buildStarted(BuildEvent arg0) {
// TODO Auto-generated method stub
}

@Override
public void messageLogged(BuildEvent arg0) {
// TODO Auto-generated method stub
}

@Override
public void targetFinished(BuildEvent event) {
Target target = event.getTarget();
if (event.getException() != null)
this.forceProperty(event.getProject(), event.getProject().getName() + “.Target.” + target.getName() + “.State”, “Failed”);
else
this.forceProperty(event.getProject(), event.getProject().getName() + “.Target.” + target.getName() + “.State”, “Success”);
this.forceProperty(event.getProject(), event.getProject().getName() + “.Target.” + target.getName() + “.Executed”, Boolean.toString(true));
}

@Override
public void targetStarted(BuildEvent event) {
Target target = event.getTarget();
this.forceProperty(event.getProject(), “Target.” + target.getName() + “.State”, “Running”);
}

@Override
public void taskFinished(BuildEvent arg0) {
// TODO Auto-generated method stub
}

@Override
public void taskStarted(BuildEvent arg0) {
// TODO Auto-generated method stub
}
private Object getValue( Object instance, String fieldName ) throws IllegalAccessException, NoSuchFieldException {
Field field = getField( instance.getClass(), fieldName );
field.setAccessible( true );
return field.get( instance );
}
private Field getField( Class thisClass, String fieldName ) throws NoSuchFieldException {
if ( thisClass == null ) {
throw new NoSuchFieldException( “Invalid field : ” + fieldName );
}
try {
return thisClass.getDeclaredField( fieldName );
}
catch ( NoSuchFieldException e ) {
return getField( thisClass.getSuperclass(), fieldName );
}
}

private void forceProperty(Project project, String name, String value) {
try {
Hashtable properties = (Hashtable) getValue(project, “properties”);
if ( properties == null ) {
project.setUserProperty(name, this.parseProperty(project, value));
}
else {
project.setProperty(name, this.parseProperty(project, value));
}
}
catch ( Exception e ) {
project.setUserProperty(name, this.parseProperty(project, value));
}
}

@SuppressWarnings(“deprecation”)
private String parseProperty(Project project, String value){
Vector fragments = new Vector();
Vector propertyRefs = new Vector();
ProjectHelper.parsePropertyString(value, fragments, propertyRefs);

if (propertyRefs.size() != 0) {
StringBuffer sb = new StringBuffer();
Enumeration i = fragments.elements();
Enumeration j = propertyRefs.elements();
while (i.hasMoreElements()) {
String fragment = (String)i.nextElement();
if (fragment == null) {
String propertyName = (String)j.nextElement();
fragment = project.getProperty(propertyName);
}
sb.append( fragment );
}
return sb.toString();
}
return value;
}
}