Demanding of Ant: 2 Run Once

The use of the depends attribute on targets is prevalent in many project Ant scripts.  I believe that depends is the bane of reuse.  Lets take the name of a simple target like deploy.  In the context of a developers environment deploy would likely depend on package, package on unittest, unittest on compile…  This is fine until you want to reuse deploy in a different context, say to a different environment like production.  In that context deploy has no business with such a dependency chain.

That is not a clear enough picture though.  Depends is not just for defining a dependency chain, it also includes the much desired feature of run once.  That is to say when I call the target deploy, compile will only been run once even though both recompile and unittest depend on it.

<?xml version=1.0 encoding=UTF-8?>

<project name=scratch default=deploy basedir=. >

  <target name=deploy depends=recompile,package>

    <echo>deploy the missile</echo>

  </target>

  <target name=package depends=unittest>

    <echo>package the missile</echo>

  </target>

  <target name=unittest depends=compile>

    <echo>test the missile</echo>

  </target>

  <target name=compile >

    <echo>compile the missile</echo>

  </target>

  <target name=clean>

    <echo>clean missile</echo>

  </target>

  <target name=recompile depends=clean,compile/>

</project>

The output from calling the target deploy would be:

c:\>ant -f scratch.build.xml
Buildfile: scratch.build.xml

clean:
[echo] clean missile

compile:
[echo] compile the missile

recompile:

unittest:
[echo] test the missile

package:
[echo] package the missile

deploy:
[echo] deploy the missile

BUILD SUCCESSFUL
Total time: 0 seconds

Sidenote: you should almost never use antcall.  It violates the principle of least astonishment.  You should think of it like you are ask for Ant to execute the target in isolation.  You should use runtarget from Ant contrib.

I have found the use of orchestration targets to be far more powerful when I can ask for a target to be executed and optionally specify that it only be run once.  The depends run once is not as robust as you might think.  If you call multiple targets from the command line run once does not always work:

c:\>ant -f scratch.build.xml recompile unittest
Buildfile: scratch.build.xml

clean:
[echo] clean missile

compile:
[echo] compile the missile

recompile:

compile:
[echo] compile the missile

unittest:
[echo] test the missile

BUILD SUCCESSFUL
Total time: 0 seconds

What would be ideal would be the following.

<target name=deploy>

  <call target=recompile once=true/>

  <call target=package once=true/>

  <echo>deploy the missile</echo>

</target>

<target name=package>

  <call target=unittest once=true/>

  <echo>package the missile</echo>

</target>

<target name=unittest>

  <call target=compile once=true/>

  <echo>test the missile</echo>

</target>

<target name=compile >

  <echo>compile the missile</echo>

</target>

<target name=clean>

  <echo>clean missile</echo>

</target>

<target name=recompile>

  <call target=clean once=true/>

  <call target=compile once=true/>

</target>

With the output:

c:\>ant -f scratch.build.xml recompile unittest
Buildfile: scratch.build.xml

recompile:

clean:
[echo] clean missile

compile:
[echo] compile the missile

unittest:
[echo] test the missile

BUILD SUCCESSFUL
Total time: 0 seconds

I implemented the call task with a macrodef that checks if the target has been run; this is tracked by checking the existence of a property.

<macrodef name=call>

  <attribute name=target/>

  <attribute name=if default=true/>

  <attribute name=unless default=false/>

  <attribute name=once default=false/>

  <sequential>

    <if>

      <and>

        <istrue value=@{if} />

        <isfalse value=@{unless} />

        <or>

          <and>

            <istrue value=@{once}/>

            <not>

              <isset property=${ant.project.name}.Target.@{target}.Executed/>

            </not>

          </and>

          <isfalse value=@{once}/>

        </or>

      </and>

      <then>

        <runtarget target=@{target}/>

      </then>

    </if>

  </sequential>

</macrodef>

This macrodef depends on a property being set after the successful execution of every target.  To accomplish this I created a simple logger to set a property for each target called; source is at the end of the post.  I also made sure that the logger was loaded with the script task at the beginning of the Ant project file.

<?xml version=1.0 encoding=UTF-8?>

<project name=scratch default=deploy basedir=. >

  <taskdef resource=net/sf/antcontrib/antcontrib.properties />

  <typedef resource=AgilexAnt.properties />

  <script language=javascript>

    <![CDATA[

      importClass(Packages.com.agilex.ant.TargetListener);

      var targetListener = new TargetListener();

      project.setProjectReference(targetListener);

      project.addBuildListener(targetListener);

    ]]>

  </script>

  <target name=deploy>

    <call target=recompile once=true/>

    <call target=package once=true/>

    <echo>deploy the missile</echo>

  </target>

  …

Once you have this functionality you can begin to orchestrate in more robust ways.  With depends you can only string targets together, one after the other with no customization between the execution of each target.  Now you have the opportunity to wrap each and every target call with any customization you can write.  You can reuse in any way you want.  For this example we only want to compile and package on a developers machine (e.g. env.dev==true).

<property name=env.dev value=true/>

<target name=deploy>

  <call target=recompile once=true if=${env.dev}/>

  <call target=package once=true if=${env.dev}/>

  <echo>deploy the missile</echo>

</target>

c:\>ant -f scratch.build.xml -Denv.dev=false
Buildfile: scratch.build.xml

deploy:
[echo] deploy the missile

BUILD SUCCESSFUL
Total time: 0 seconds

package com.agilex.ant;

import java.lang.reflect.Field;
import java.util.Enumeration;
import java.util.Hashtable;
import java.util.Vector;

import org.apache.tools.ant.BuildEvent;
import org.apache.tools.ant.BuildListener;
import org.apache.tools.ant.Project;
import org.apache.tools.ant.ProjectHelper;
import org.apache.tools.ant.Target;

public class TargetListener implements BuildListener {

@Override
public void buildFinished(BuildEvent arg0) {
// TODO Auto-generated method stub
}

@Override
public void buildStarted(BuildEvent arg0) {
// TODO Auto-generated method stub
}

@Override
public void messageLogged(BuildEvent arg0) {
// TODO Auto-generated method stub
}

@Override
public void targetFinished(BuildEvent event) {
Target target = event.getTarget();
if (event.getException() != null)
this.forceProperty(event.getProject(), event.getProject().getName() + “.Target.” + target.getName() + “.State”, “Failed”);
else
this.forceProperty(event.getProject(), event.getProject().getName() + “.Target.” + target.getName() + “.State”, “Success”);
this.forceProperty(event.getProject(), event.getProject().getName() + “.Target.” + target.getName() + “.Executed”, Boolean.toString(true));
}

@Override
public void targetStarted(BuildEvent event) {
Target target = event.getTarget();
this.forceProperty(event.getProject(), “Target.” + target.getName() + “.State”, “Running”);
}

@Override
public void taskFinished(BuildEvent arg0) {
// TODO Auto-generated method stub
}

@Override
public void taskStarted(BuildEvent arg0) {
// TODO Auto-generated method stub
}
private Object getValue( Object instance, String fieldName ) throws IllegalAccessException, NoSuchFieldException {
Field field = getField( instance.getClass(), fieldName );
field.setAccessible( true );
return field.get( instance );
}
private Field getField( Class thisClass, String fieldName ) throws NoSuchFieldException {
if ( thisClass == null ) {
throw new NoSuchFieldException( “Invalid field : ” + fieldName );
}
try {
return thisClass.getDeclaredField( fieldName );
}
catch ( NoSuchFieldException e ) {
return getField( thisClass.getSuperclass(), fieldName );
}
}

private void forceProperty(Project project, String name, String value) {
try {
Hashtable properties = (Hashtable) getValue(project, “properties”);
if ( properties == null ) {
project.setUserProperty(name, this.parseProperty(project, value));
}
else {
project.setProperty(name, this.parseProperty(project, value));
}
}
catch ( Exception e ) {
project.setUserProperty(name, this.parseProperty(project, value));
}
}

@SuppressWarnings(”deprecation”)
private String parseProperty(Project project, String value){
Vector fragments = new Vector();
Vector propertyRefs = new Vector();
ProjectHelper.parsePropertyString(value, fragments, propertyRefs);

if (propertyRefs.size() != 0) {
StringBuffer sb = new StringBuffer();
Enumeration i = fragments.elements();
Enumeration j = propertyRefs.elements();
while (i.hasMoreElements()) {
String fragment = (String)i.nextElement();
if (fragment == null) {
String propertyName = (String)j.nextElement();
fragment = project.getProperty(propertyName);
}
sb.append( fragment );
}
return sb.toString();
}
return value;
}
}

Demanding of Ant: 1 scriptdef

I hate Ant. I like NAnt better, and I hate NAnt too.  The intent of this series is to share how I have demanded more out of Ant.

Did you know that you can mess with the Java objects of Ant in Ant script?  You can create tasks on the fly. You can even create a logger on the fly.  I have used the following example to insure that a log file is written to without having to count on a user supplying the needed command line arguments.

If you are not familiar with the scriptdef task in Ant you should read the help for the task here.  It is part of the optional Ant tasks, so you will need to have gotten your Ant install setup correctly for the optional tasks.  The scriptdef task supports several languages, here I will be using JavaScript.

<?xml version=1.0 encoding=UTF-8?>

<project name=scratch default=play basedir=. >

  <target name=play>

    <say.hello/>

  </target>

  <scriptdef name=say.hello language=javascript>

    <![CDATA[

      self.log(”hello”);

    ]]>

  </scriptdef>

</project>

Executing this script will output:

Buildfile: scratch.build.xml

play:
[say.hello] hello

BUILD SUCCESSFUL
Total time: 0 seconds

You can import Java classes into the JavaScript with the importClass method: importClass(Package.[Java Full Class Name]);.  When you combined this with your access back into the Ant object graph through self you can do some interesting things.  To explore what is available to me I used this trick to iterate over what self provides:

<scriptdef name=explore.self language=javascript>

  <![CDATA[

  for (member in self)

  {

    self.log(member);

  }

  ]]>

</scriptdef>

[explore.self] project
[explore.self] getText
[explore.self] createDynamicElement
[explore.self] toString
[explore.self] getTaskType
[explore.self] wait
[explore.self] bindToOwner
[explore.self] log
[explore.self] getClass
[explore.self] getLocation
[explore.self] taskType
[explore.self] hashCode
[explore.self] fail
[explore.self] reconfigure
[explore.self] class
[explore.self] notify
[explore.self] description
[explore.self] maybeConfigure
[explore.self] location
[explore.self] setTaskType
[explore.self] dynamicAttribute
[explore.self] getTaskName
[explore.self] clone
[explore.self] getOwningTarget
[explore.self] getProject
[explore.self] setLocation
[explore.self] setDescription
[explore.self] getRuntimeConfigurableWrapper
[explore.self] taskName
[explore.self] text
[explore.self] equals
[explore.self] setRuntimeConfigurableWrapper
[explore.self] setTaskName
[explore.self] setDynamicAttribute
[explore.self] owningTarget
[explore.self] getDescription
[explore.self] execute
[explore.self] setOwningTarget
[explore.self] perform
[explore.self] notifyAll
[explore.self] init
[explore.self] addText
[explore.self] setProject
[explore.self] runtimeConfigurableWrapper

Self provides a reference to the Ant object project; pretty much the root of the object graph.  You can use this same trick on other objects you need to get more information on.  Sometimes I need to understand the inner workings so I use a decompiler or read the Ant source.  The last critical piece of information you need to unlock the door to productivity is the method project.createTask(String taskName).  The argument taskName is the friendly name not the class name of the task to be created. For example project.createTask(“mkdir”) will create an instance of the mkdir task.  Armed with this knowledge you can experiment and learn to apply it in many helpful and time saving ways.  Here is an example that will use the default logger to output to the console and to a dynamically named log file.

<?xml version=1.0 encoding=UTF-8?>

<project name=scratch default=play basedir=. >

  <taskdef resource=net/sf/antcontrib/antcontrib.properties />

  <target name=play>

    <var name=Log.Directory.Path value=C:\Temp\Play/>

    <start.logger/>

    <echo>T minus</echo>

    <echo>3</echo>

    <echo>2</echo>

    <echo>1</echo>

    <echo>blast off…</echo>

  </target>

  <scriptdef name=start.logger language=javascript>

    <![CDATA[

        importClass(Packages.org.apache.tools.ant.DefaultLogger);

        importClass(Packages.java.io.File);

        importClass(Packages.java.io.FileOutputStream);

        importClass(Packages.java.io.PrintStream);

        importClass(Packages.org.apache.tools.ant.BuildEvent);

        importClass(Packages.org.apache.tools.ant.Target);

        mkdir = project.createTask(”mkdir”);

        mkdir.setDir(new File(project.getProperty(”Log.Directory.Path”)));

        mkdir.execute();

        tstamp = project.createTask(”tstamp”);

        format = tstamp.createFormat();

        format.setProperty(”now”);

        format.setPattern(”MM.dd.yyyy-HH.mm.ss-zz”);

        tstamp.execute();

        var logger = new DefaultLogger();

        var out = new PrintStream(new FileOutputStream(project.getProperty(”Log.Directory.Path”) + “/” + project.getProperty(”ant.project.name”) + “-” + project.getProperty(”now”) + “.log”));

        logger.setOutputPrintStream(out);

        logger.setMessageOutputLevel(2);

        project.setProjectReference(logger);

        project.addBuildListener(logger);

        logger.buildStarted(new BuildEvent(project));

        logger.targetStarted(new BuildEvent(self.getOwningTarget()));

        logger.taskStarted(new BuildEvent(self))

    ]]>

  </scriptdef>

</project>

This is the console output:

c:\>ant -f scratch.build.xml play
Buildfile: scratch.build.xml

play:
[echo] T minus
[echo] 3
[echo] 2
[echo] 1
[echo] blast off…

BUILD SUCCESSFUL
Total time: 0 seconds

And here is the output to the logfile:

C:\Temp\play\scratch05.01.2010-10.00.33-EDT.log

play:
[echo] T minus
[echo] 3
[echo] 2
[echo] 1
[echo] blast off…

BUILD SUCCESSFUL
Total time: 0 seconds

Automated Workspace Management

Recently I was at CITCON 2010 (Continuous Integration and Testing Conference) in Raleigh-Durham. One of the sessions I participated in was on automated workspace management.  I thought I would share more on my related experiences here.workbench

First what is automated workspace management?  Think of a new person on your project.  How do they get a workspace created?  Most projects have a document or documents, maybe on the wiki, that is supposed to give instructions on how to get setup.  We all know that these are never up to date.  Often they lie and confuse.  In the end it always requires someone on the team to help them figure out how to get them up and running.   And I would put money on that most times, in the end, when it starts working no one is really sure why it finally starts working.  In essence private workspaces are works of art.  We are not in the business of making works of art.  Our workspaces should be easy to create, destroy, and recreate: they should be disposable.  A new person should be able to click a button to setup their workspace.  All team members should be able to click a button to update their workspaces’ to be in sync with the latest from source control.

In CI Factory there is a Package dedicated to this; it is named…you guessed it Workspace.  It is organized into to basic types of scripts software and configuration.  Here are a couple of simple examples of each:

Software – The Subversion script will check that you have at least a minimum version of TortoiseSVN installed and prompt you for permission to download and install it if you do not have it installed.

Configuration – The TimeSync script will check that your w32time service is configured to sync with the Navy time server Tock and if not prompt you for permission to adjust the configuration.

In the session at CITCON I discovered that in the Linux OPs world there are tools written around doing this very thing.  People have begun using them to manage not only operations centers but personal workspaces for developers and testers.  Some of the tools mentioned in the session were:

I have not had a chance to look deeply into these tools yet…especially to see if they can better satisfy my need on a Windows platform.  I am most keen to see how they address the myriad of ways that software for the Windows platform is offered for download and the technologies used to install.  There is no RPM or YUM for Windows…

In CI Factory we have employed several techniques to over come the variations.

First you may not be able to reliably, or legally, script the download of an installer (take the JDK for example).  In this case we downloaded the JDK to a server(http or ftp) that everyone on the team has access to and script the download from there.

<property name=Java.Installer.File.Name value=jdk-6u13-windows-i586-p.exe/>

<property name=Java.Installer.File.Path value=C:\Temp\${Java.Installer.File.Name}/>

<property name=Java.Installer.Download.URL value=${Workspace.Ftp.Url}${Java.Installer.File.Name}/>

<get

  src=${Java.Installer.Download.URL}

  dest=${Java.Installer.File.Path}

  unless=${file::exists(Java.Installer.File.Path)}

/>

Second you need to silently execute the installer.  Most installer technologies offer some form of silent install.  You just need to read up on how they work.

<property name=Java.Install.Path value=C:\Java\jdk1.6.0_13/>

<exec

  workingdir=${path::get-directory-name(Java.Installer.File.Path)}

  program=${Java.Installer.File.Path}

  commandline=/s /v “/qn INSTALLDIR=${Java.Install.Path}”

  verbose=true

/>

You will eventually run into a situation where you need to reverse engineer an installer.  I recommend that you keep it simple, create a zip.  I doubt that you would ever need anything fancier than a SFX (self-extracting zip); both 7Zip and WinRar are good tools for creating SFX installers.  Here is a simple NAnt script I have used to create SFX files with WinRar:

<largeproperty name=SfxConfig.Content>

  <value expand=true xml=false>

    <![CDATA[;The comment below contains SFX script commands

Path=C:\FuzzyBunnies

Setup=echo Hippied Hop

Overwrite=1

Title=My Example Installer]]>

  </value>

</largeproperty>

<property name=SfxConfig.FilePath value=C:\Temp\SfxConfig.txt/>

<echo message=${SfxConfig.Content} file=${SfxConfig.FilePath}/>

<property name=InstallerZip.FileName value=Example-Installer.zip/>

<property name=InstallerZip.FilePath value=C:\Temp\${InstallerZip.FileName}/>

<zip zipfile=${InstallerZip.FilePath} verbose=True >

  <fileset>

    <include name=some stuff/>

  </fileset>

</zip>

<exec

  program=${WinRarProgramPath}

  commandline=s -ibck “${InstallerZip.FilePath}” c -z”${SfxConfig.FilePath}”

  workingdir=C:\Temp

  verbose=true

/>

The end result would be C:\Temp\Example-Installer.exe.  I would unzip to the directory C:\FuzzyBunnies and echo Hippied Hop in a command window.  The title of the install window would be My Example Installer.  You can read the help for WinRar or play with the GUI to see all the options.

WinRar SFX

The Workspace Package in CI Factory creates a SFX that bundles up NAnt and all the scripts needed to bootstrap the process of creating a new workspace.

Workspace Setup