Recently I was at CITCON 2010 (Continuous Integration and Testing Conference) in Raleigh-Durham. One of the sessions I participated in was on automated workspace management.  I thought I would share more on my related experiences here.

First what is automated workspace management?  Think of a new person on your project.  How do they get a workspace created?  Most projects have a document or documents, maybe on the wiki, that is supposed to give instructions on how to get setup.  We all know that these are never up to date.  Often they lie and confuse.  In the end it always requires someone on the team to help them figure out how to get them up and running.   And I would put money on that most times, in the end, when it starts working no one is really sure why it finally starts working.  In essence private workspaces are works of art.  We are not in the business of making works of art.  Our workspaces should be easy to create, destroy, and recreate: they should be disposable.  A new person should be able to click a button to setup their workspace.  All team members should be able to click a button to update their workspaces’ to be in sync with the latest from source control.

In CI Factory there is a Package dedicated to this; it is named…you guessed it Workspace.  It is organized into to basic types of scripts software and configuration.  Here are a couple of simple examples of each:

Software – The Subversion script will check that you have at least a minimum version of TortoiseSVN installed and prompt you for permission to download and install it if you do not have it installed.

Configuration – The TimeSync script will check that your w32time service is configured to sync with the Navy time server Tock and if not prompt you for permission to adjust the configuration.

In the session at CITCON I discovered that in the Linux OPs world there are tools written around doing this very thing.  People have begun using them to manage not only operations centers but personal workspaces for developers and testers.  Some of the tools mentioned in the session were:

• cfengine
• puppet
• chef

I have not had a chance to look deeply into these tools yet…especially to see if they can better satisfy my need on a Windows platform.  I am most keen to see how they address the myriad of ways that software for the Windows platform is offered for download and the technologies used to install.  There is no RPM or YUM for Windows…

In CI Factory we have employed several techniques to over come the variations.

First you may not be able to reliably, or legally, script the download of an installer (take the JDK for example).  In this case we downloaded the JDK to a server(http or ftp) that everyone on the team has access to and script the download from there.

Second you need to silently execute the installer.  Most installer technologies offer some form of silent install.  You just need to read up on how they work.

You will eventually run into a situation where you need to reverse engineer an installer.  I recommend that you keep it simple, create a zip.  I doubt that you would ever need anything fancier than a SFX (self-extracting zip); both 7Zip and WinRar are good tools for creating SFX installers.  Here is a simple NAnt script I have used to create SFX files with WinRar:

The end result would be C:\Temp\Example-Installer.exe.  I would unzip to the directory C:\FuzzyBunnies and echo Hippied Hop in a command window.  The title of the install window would be My Example Installer.  You can read the help for WinRar or play with the GUI to see all the options.

The Workspace Package in CI Factory creates a SFX that bundles up NAnt and all the scripts needed to bootstrap the process of creating a new workspace.

For a long time now CI Factory has had the concept of a Package: a set of NAnt targets, properties, tasks, and functions that can be loaded by the Main.Build.xml or the Post.Build.xml and later call targets on them.

<include buildfile="${PackagesDirectory}MSBuildCompile.Target.xml" />

……

<include buildfile="${PackagesDirectory}MSBuildCompile.Target.xml" />

Most packages include:

Properties file to tweak the package to your specific needs.

Targets file provides.. well targets that you can call from the main build file or from custom build files.

Xsl files to provide web page reports by transforming xml output from the build process.

And sometimes a bin folder containing NAnt task dlls and or console applications.

By convention the target file would include the properties file.  There was no thought to how one could safely customize a Package beyond changing the values of properties in the properties file.  This has cause some issues when trying to upgrade to a new version of CI Factory.  I think the new version of CI Factory has over come this issue and generally created a more powerful concept of Package.

To load packages there is a new task:

<loadpackages>

  <package name="Publish" />

  <package name="Subversion" type="SourceControl" />

  <package name="SourceModificationReport" />

  <package name="TargetProcess" />

  <package name="Simian" />

  <package name="Ant" />

  <package name="Selenium" />

  <package name="FitNesse" />

  <package name="JUnit" type="UnitTest" />

  <package name="IntegrationTest" />

  <package name="Corbertura" type="Coverage" />

  <package name="Workspace" />

  <package name="GlassFish" />

  <package name="SoapUI" />

</loadpackages>

When a Package is loaded these types of property are set, for example the Ant Package:

Package.Ant.Custom.File.Loaded=False

Package.Ant.Custom.File.Path=c:\Projects\Chapter33\Trunk\Build\Packages\Ant\Ant.Custom.xml

Package.Ant.Directory.Path=c:\Projects\Chapter33\Trunk\Build\Packages\Ant

Package.Ant.MacroDefs.File.Loaded=True

Package.Ant.MacroDefs.File.Path=c:\Projects\Chapter33\Trunk\Build\Packages\Ant\Ant.MacroDefs.xml

Package.Ant.Name=Ant

Package.Ant.Properties.File.Loaded=True

Package.Ant.Properties.File.Path=c:\Projects\Chapter33\Trunk\Build\Packages\Ant\Ant.Properties.xml

Package.Ant.Targets.File.Loaded=True

Package.Ant.Targets.File.Path=c:\Projects\Chapter33\Trunk\Build\Packages\Ant\Ant.Targets.xml

Notice there is the optional attribute type.  Type is something similar to an interface, it says that I offer these common pieces of functionality for you to make use of. This will likely become something that is enforced in a future version.  At the moment how much of that interface is implemented is up to the creator/maintainer.  The concept of a type of Package allows us to do things like:

<loadpackages>

  <package name="${package::find-name-by-type(’SourceControl’)}" type="SourceControl" />

</loadpackages>

The loadpackages task knows of a Packages directory from the property Common.Directory.Packages.Path; this property is set by a core CI Factory script.  It will try and load these files in this order:

1. [PackageName/PackageType].Properties.xml
2. [PackageName/PackageType].MacroDefs.xml
3. [PackageName/PackageType].Targets.xml
4. [PackageName/PackageType].Custom.xml

The old convention was to name the file containing the targets [PackageName].Target.xml, singular; this has change to plural [ProjectName].Targets.xml.

There are two new files that can optional be included in a Package: [PackageName].MacroDefs.xml and [PackageName].Custom.xml.  The fork of NAnt included with CI Factory now includes the NAnt task macrodef; used for creating new tasks from nant script.  The [PackageName].MacroDefs.xml is where you can create new tasks.  The [PackageName].Custom.xml is where you can customize an existing Package.  Customization is really made possible with the new NAnt features that allow you to override an existing target and call a target by full name.

Here is an example to illustrate the override and call by full name.  Let say the following target was defined in the package Example targets file Example.Targets.xml:

<?xml version="1.0" encoding="utf-8"?>

<project name="Example" xmlns="http://nant.sf.net/schemas/nant.xsd">

  <target name="Hello">

    <echo message="Hello World"/>

  </target>

</project>

We could then override that target in the Example.Custom.xml file of the package like so:

<?xml version="1.0" encoding="utf-8"?>

<project name="Example.Custom" xmlns="http://nant.sf.net/schemas/nant.xsd">

  <target name="Hello" override="true">

    <call target="Example::Hello"/>

    <echo message="Bend to my will"/>

  </target>

</project>

If you were to call the target Hello the output would be:

[echo] Hello World

[echo] Bend to my will

Notice in the example of the original target I included the entire file.  The project tag requires a name, in this case Example.  So when calling a target you now have two options for the name of the target to call:  the short name (e.g. Hello) and the full name (e.g. Example::Hello and Example.Custom::Hello).

The MacroDefs file opens the door for sharing not just targets between Packages, now you can share tasks.  The Ant package defines the following macro:

<?xml version="1.0" encoding="utf-8"?>

<project xmlns="http://nant.sf.net/schemas/nant.xsd" name="Ant.MacroDefs">

  <macrodef name="ant">

    <attributes>

      <attribute name="antbat" default="${Ant.Bat}" type="string"/>

      <attribute name="logfile" type="string"/>

      <attribute name="target" type="string" require="true"/>

      <attribute name="buildfile" type="string" require="true"/>

    </attributes>

    <elementgroups>

      <elementgroup name="args" type="NAnt.Core.Types.Argument" elementname="arg"/>

    </elementgroups>

    <elements>

      <element name="environment" type="NAnt.Core.Types.EnvironmentSet"/>

    </elements>

    <sequential>

      <ifthenelse test="${property::exists(’logfile’)}">

        <then>

          <property name="Ant.LogParams" value=‘-logger com.agilex.ant.GoodXmlLogger -logfile "${logfile}"‘ overwrite="true"/>

          <ifnot test="${directory::exists(path::get-directory-name(logfile))}">

            <mkdir dir="${path::get-directory-name(logfile)}"/>

          </ifnot>

        </then>

        <else>

          <property name="Ant.LogParams" value=‘’ overwrite="true"/>

        </else>

      </ifthenelse>

      <exec program="${antbat}" failonerror="true" verbose="true">

        <element name="environment"/>

        <arg line=‘${target} -buildfile "${buildfile}"‘ />

        <arg line=‘${Ant.LogParams}‘ />

        <arg line=‘-Dprogress-filepath="${CCNetListenerFile}"‘ />

        <arg line=‘-DProductVersion="${CCNetLabel}"‘ />

        <arg line=‘-Ddebug="${Ant.Debug}"‘ />

        <elementgroup name="args"/>

      </exec>

    </sequential>

  </macrodef>

</project>

The ant macro is used by the JUnit Package as well as several others.  See how easy it is to call ant now:

<ant

  target="unittest.run"

  buildfile="${Ant.Build.File.Path}"

  logfile="${Ant.Log.Directory.Path}unittest_log.xml"

>

  <environment refid="${Common.EnvironmentVariables.RefId}"/>

  <args>

    <arg line="-Dcompile.debug=${Compile.Debug}"/>

  </args>

</ant>

The macrodef task in CI Factory is improved over the original created by Eoin Curran.  It will allow you to set an existing property in the macro and have the new value persist after the macro has completed.  Normally all property values are returned to the same value before the macro was executed.  You can create an element group as well as just one element.  Lastly elements are referenced in a way that does not prevent the use of the same task/type name in the macro itself.

All these improvements to the concept of a Package in CI Factory should make it easier to do great things.  If you see something else that can be done to make it easier I am all ears…

CI Factory and I have been stuck in the Java world for over a year now.  I have not had much reason to keep up with the Dot Net packages.  I know there are new versions of most of the Dot Net tools CI Factory offers packages for.  I have let this keep me from releasing a new version of CI Factory.  There have been many new features in the core of CI Factory and several new packages around Java tools.  I will not sacrifice my personal life to maintain all of CI Factory.  As well I see no good reason for delaying a release of a new version of CI Factory for out-of-date Dot Net packages.  I welcome anyone’s help in updating the Dot Net packages.

That said I hope to finish a release version of 1.2 in August.

Here are some of the features in the new version:

Core

• Default CCNet projects are now Build Scripts, Dev, Heavy, Release, Deploy, and Test.
• Improved directory structure, cleaner, more intuitive.
• Ground work for 64 bit support.
• Improved property names.
• Better support for environment variables.

CCNet

• Real time log messages on the dashboard.
• Improved how CCNet kills a process tree when a timeout occurs.
• Improved dashboard layout.
• Add WCF REST interface to CCNet server.
• Added sounds to CCTray installer.

NAnt

• Added task deleteregistry.
• Added task macrodef (credit to Eoin Curran).
• Improved xsd intellisense to include properties, target names, functions, and more.
• Added task propertystructure and property structure iterator for loopthrough task.
• Added task stringadd to add values to a string list.
• Added task stringsplit to convert a delaminated string into a string list.
• Added process functions: get-current-pid, get-parent-pid, and get-command-line.
• Improved code to allow overriding a target and calling a target by full name: full name format = [project name]::[target name].
• Added task loadpackages to load CI Factory packages, packages are now a baked in part of NAnt.
• Improved saveproperties task to accept property structure iterators.
• Added package functions: find-name-by-type.
• Added TargetProcess tasks.
• Added property functions: destroy and value.
• Added scriptfile functions: exists, loaded, get-file-Path, get-directory-Path, get-name, get-current-name, get-current-file-path, and get-current-directory.

Packages

• Added TargetProcess Package, mine commit/checkin log message/comment for TargetProcess Story/Task/Bug id to display title and description on summary build report.
• Upgrade to new version of Subversion 1.6.
• Improved Ant package, includes Ant scripts for Compile, Coverage Instrumentation with Corbertura, JUnit, and Packaging.
• Added FitNesse Package.
• Added Selenium Package.
• Added JUnit Package.
• Added Corbertura Package.
• Added JUnit Integration Test Package.
• Added Personal Tracking Package.
• Added Eclipse Package.
• Added support for VS 2008.
• Analytics: general improvements plus new graphs and measures for FitNesse, individual developers, Code Coverage.