September 3, 2006 Uncategorized

Custom CCNET Branch – New Features That Could Not Wait!

So I have written in previous posts about a project that I am helping automate the build.  It is a rather large project, in terms of code, people, and process.  We are trying to become more agile but there are some forces that we can not change (eg medical software).  CCNET is close to handling all our build needs.  It has been lacking some crusal functionality.  The ability to create relationships between builds; this is partially implemented.  One build can trigger another or watch another.  This is good for no traffic regulation situations, small teams.  We needed the ability to control when builds could not happen.  For example project experimental1 should not be forcible when experimental2 is building or checking modifications:

   <projectForceFilter>

     <projectFilters>

       <projectFilter

            serverUri=tcp://localhost:21247/CruiseManager.rem

            project=experimental2>

         <exclusionFilters>

           <activities>

             <activity>Building</activity>

             <activity>CheckingModifications</activity>

           </activities>

         </exclusionFilters>

       </projectFilter>

     </projectFilters>

   </projectForceFilter>

Nor should 1 be triggerable when 2 is building, checking modifications, or failed.

   <projectTriggerFilter>

     <trigger type=intervalTrigger seconds=60 />

     <projectFilters>

       <projectFilter

            serverUri=tcp://localhost:21247/CruiseManager.rem

            project=experimental2>

         <exclusionFilters>

           <conditions>

             <condition>Failure</condition>

           </conditions>

           <activities>

             <activity>Building</activity>

             <activity>CheckingModifications</activity>

           </activities>

         </exclusionFilters>

       </projectFilter>

     </projectFilters>

   </projectTriggerFilter>

We had need for this type of relationship because we have multiple build servers building the same codeline concurrently.  Each build server its own list of users that can trigger a build by committing changes to source control.  Only one user can be building at a time on a build box.  There is a long discussion about this on the yahoo XP group entitled “Build Contention Equation” if you want more details.  The point I am trying to show is that if one build is broken they will all be broken so there is no point in building.  In fact it would be more informative to the team if all the build servers broke when one broke but I have not taken it that far yet.  At any rate you can now create blocking relationships between builds.  I bet a lot of you just cringed when you read “blocking”.  If that is what the process is then the tool needs to support it.  Tools should not drive/dictate process.  As I mentioned earlier they type of project this is for demands this type of process.  We don’t all get to work on small teams.

But wait there’s more!

The build servers that I mentions above are working in an incremental fashion, not something that you would wont to release but it was needed to bring the build times lower.  This means we now have a release build server.  It is not trigger based: it is force only.  This build is controlled by the build team.  It’s automation includes things that only a build team member has the authority to do.  So we needed a way to restrict who could force the build.  I have seen this feature asked for many times of that few years, I was one of those people.  Well here it is:

   <userForceFilter

            domainName=chcsii

            userName=build

            password=password>

     <includedUsers>

       <includedUser>jflowers</includedUser>

     </includedUsers>

     <includedGroups>

       <includedGroup>Build-Masters</includedGroup>

     </includedGroups>

   </userForceFilter>

This force filter uses a Windows domain for managing who can and can not force a build.  There are included and excluded listings avalible (the example just uses the included).  To get the group info an account in needed to login to the AD.  This filter works for both cctray and the web dashboard.  For the web dashboard you will need to change the authentication of the virtual directory from anonymous to basic  You will also need to add <identity impersonate=true/> to the web.config file.

Beware!

Currently there is no negative feedback on cctray that a force did not occur.  This will be rectified in the near future.  The web dashboard will let you know that a force did not happen but it will not tell you why.  Again this will be fixed in soon.  Force filters must interact will all remoting clients, some might require the client to provide info (eg userForceFilter).  This means that both cctray and the web dashboard now need all the assemblies that contain the implementations of all the types serialised in the project config.  The entire project is serialised, sent to the remoting client, and deserialized.  It is the deserialization that causes the dependency on all implementing assemblies.  This is a first cut and I am sure that better designs will present them selves (fire away).

Eric Hexter is helping me with this.  We are trying to get these changes in to the trunk of the CCNET project.  I think this may take sometime and the project I am working on can’t wait that long.  Untill then I will maintain a branch of the 1.0 release of CCNET.  The source is avalible from http://svn2.cvsdude.com/jflowers/CCNET.

Download

Here is a complete example:

<?xml version=1.0 encoding=utf-8?>

<!DOCTYPE cruisecontrol [

  <!ENTITY labeller '<labeller type="defaultlabeller">

      <prefix>1.0.0.</prefix>

    </labeller>'>

]>

<cruisecontrol>

  <project name=experimental1>

    <state type=state />

    <triggers>

      <projectTriggerFilter>

        <trigger type=intervalTrigger seconds=60 />

        <projectFilters>

          <projectFilter

            serverUri=tcp://localhost:21247/CruiseManager.rem

            project=experimental2>

            <exclusionFilters>

              <conditions>

                <condition>Failure</condition>

              </conditions>

              <activities>

                <activity>Building</activity>

                <activity>CheckingModifications</activity>

              </activities>

            </exclusionFilters>

          </projectFilter>

        </projectFilters>

      </projectTriggerFilter>

    </triggers>

    <forceFilters>

      <projectForceFilter>

        <projectFilters>

          <projectFilter

            serverUri=tcp://localhost:21247/CruiseManager.rem

            project=experimental2>

            <exclusionFilters>

              <activities>

                <activity>Building</activity>

                <activity>CheckingModifications</activity>

              </activities>

            </exclusionFilters>

           </projectFilter>

         </projectFilters>

       </projectForceFilter>

       <userForceFilter

            domainName=chcsii

            userName=build

            password=password>

         <includedUsers>

           <includedUser>jflowers</includedUser>

         </includedUsers>

         <includedGroups>

           <includedGroup>Build-Masters</includedGroup>

         </includedGroups>

       </userForceFilter>

     </forceFilters>

     <integrationFilter>

       <triggeredIntegrationFilter>

         <allowed>

           <modificationsRequired condition=true />

         </allowed>

       </triggeredIntegrationFilter>

       <forcedIntegrationFilter>

         <blocked>

           <trackerRequired condition=false>

             <connectionInformation

               username=build

               password=xxx

               dbmsserver=jupiter

               dbmstype=Tracker SQL Server Sys

               dbmsloginmode=2

               dbmsusername=tracker

               dbmspassword=password

               projectname=CHCSII

             />

             <query name=For Me/>

           </trackerRequired>

         </blocked>

       </forcedIntegrationFilter>

     </integrationFilter>

     &labeller;

     <tasks>

       <nant>

         <executable>C:\Temp\TestProjects\nAnt\bin\nant.exe</executable>

         <baseDirectory>C:\Temp\TestProjects</baseDirectory>

         <logger>NAnt.Core.XmlLogger </logger>

         <buildFile>scratch.build.xml</buildFile>

 

         <buildTimeoutSeconds>5400</buildTimeoutSeconds>

       </nant>

     </tasks>

     <publishers>

       <xmllogger />

     </publishers>

   </project>

   <project name=experimental2>

     <state type=state />

     <triggers>

       <projectTriggerFilter>

         <trigger type=intervalTrigger seconds=60 />

         <projectFilters>

           <projectFilter

            serverUri=tcp://localhost:21247/CruiseManager.rem

            project=experimental1>

             <exclusionFilters>

               <conditions>

                 <condition>Failure</condition>

               </conditions>

               <activities>

                 <activity>Building</activity>

                 <activity>CheckingModifications</activity>

               </activities>

             </exclusionFilters>

           </projectFilter>

         </projectFilters>

       </projectTriggerFilter>

     </triggers>

     <forceFilters>

       <projectForceFilter>

         <projectFilters>

           <projectFilter

            serverUri=tcp://localhost:21247/CruiseManager.rem

            project=experimental1>

             <exclusionFilters>

               <activities>

                 <activity>Building</activity>

                 <activity>CheckingModifications</activity>

               </activities>

             </exclusionFilters>

           </projectFilter>

         </projectFilters>

       </projectForceFilter>

       <userForceFilter

            domainName=chcsii

            userName=build

            password=password>

         <includedUsers>

           <includedUser>joflowers</includedUser>

         </includedUsers>

       </userForceFilter>

     </forceFilters>

 

     <integrationFilter>

       <triggeredIntegrationFilter>

         <allowed>

           <modificationsRequired condition=true />

         </allowed>

       </triggeredIntegrationFilter>

       <forcedIntegrationFilter>

         <allowed>

           <modificationsRequired condition=false />

         </allowed>

       </forcedIntegrationFilter>

     </integrationFilter>

     &labeller;

     <tasks>

       <nant>

         <executable>C:\Temp\TestProjects\nAnt\bin\nant.exe</executable>

         <baseDirectory>C:\Temp\TestProjects</baseDirectory>

         <logger>NAnt.Core.XmlLogger </logger>

         <buildFile>scratch.build.xml</buildFile>

 

         <buildTimeoutSeconds>5400</buildTimeoutSeconds>

       </nant>

     </tasks>

     <publishers>

       <xmllogger />

     </publishers>

   </project>

</cruisecontrol>

4,125 Total Views

10 to “Custom CCNET Branch – New Features That Could Not Wait!”

Trackbacks/Pingbacks

  1. Custom CCNET Branch – New Features That Could Not Wait!…

    You’ve been kicked (a good thing) – Trackback from DotNetKicks.com…

  2. CCNetConfig says:

    CCNet Build Restrictions and Project Relationships…

  3. CCNetConfig says:

    Support for extended CCNet Configuration…

  4. CCNetConfig says:

    0.2.1.3 Released…

  5. [...] Another solution is to head over to Jay Flowers. He has branched off the official 1.0 version of ccnet and created a ccnet that allows for a lot more constraints than just serializing builds. It happens at the expense of running the non-official, not-recent version of ccnet. [...]

  6. [...] There are other filters in CI Factory like the most popular user force filter that I blogged about here. [...]

  1. DotNetKicks.com says...

    Custom CCNET Branch – New Features That Could Not Wait!…

    You’ve been kicked (a good thing) – Trackback from DotNetKicks.com…

  2. CCNetConfig says...

    CCNet Build Restrictions and Project Relationships…

  3. CCNetConfig says...

    Support for extended CCNet Configuration…

  4. Moshe Hajaj says...

    How will it work for more than two projects? How is the setup for experimental1 to be dependant on experimental2 & experimental3 and vice versa?

  5. jflowers says...

    Just add another projectFilter element. I assume you mean how would the config look.

  6. CCNetConfig says...

    0.2.1.3 Released…

  7. Jon says...

    Is this functionality going to make ccnet 1.1+? I noticed that some of this functionality is in the plugins at: http://ccnetplugins.sourceforge.net/

    However, your functionality seems more capable at first glance, and it would be nice to have it in the official ccnet install.

    Thanks,
    Jon

  8. jflowers says...

    I would like it to get into the CCNET trunk and continue to be maintained by the CCNET dev team.
    Grant Drake is trying to add some complementary behavior to the trunk right now. The discussion thread can be found on google groups ccnet-devel titled “Implementing build queues – is this wanted?”.
    Thanks for prodding me I was about to forget that I needed to do something about this. I will try to get on the ball. :-)

  9. Nerdyville » Blog Archive » CruiseControl.net and build queues says...

    [...] Another solution is to head over to Jay Flowers. He has branched off the official 1.0 version of ccnet and created a ccnet that allows for a lot more constraints than just serializing builds. It happens at the expense of running the non-official, not-recent version of ccnet. [...]

  10. JayFlowers > Force Filters says...

    [...] There are other filters in CI Factory like the most popular user force filter that I blogged about here. [...]

Leave a comment

*

here