DevOps – The Last Mile – Workspace Management

Here is my presentation from today’s AgileDC session.

Devops – The Last Mile

Here are the links to the setup.exe and the GitHub project:

https://github.com/jflowers/virgo.sample-greenpages
http://jayflowers.com/Misc%20Downloads/Workspace-Setup.exe

138,863 Total Views
October 23, 2012 Uncategorized

Principles, Practices, and Processes

I love it when I am reading a book and it brings something I have only had a tacit understanding of to a cognitive understanding.  Apparently I have been running around without truly understanding the difference and relation between principles, practices, and processes.  I have understood the definition of principle.  I gained a clear understanding of it from Steven Covey:

Principles are like lighthouses. They are natural laws that cannot be broken. As Cecil B. deMille observed of the principles contained in his monumental movie, The Ten Commandments, “It is impossible for us to break the law. We can only break ourselves against the law.” — Stephen R. Covey 

Principles don’t require our belief for them to work.  I have also understood that practices and processes flow from principles.  What I just recently learned from the book Lean IT is about practices and processes.

…we use the term process to describe a series of actions or operations supported by structured information. Processes are activities that are generally repetitive, well-defined, routine, controllable, and standardized. In contrast, practices are nonroutine, highly variable, loosely defined, and require a degree of judgment and experience to carry out.

…information used in practice work is typically unstructured, difficult to define, and often experiential. When improving practices, the focus is on supporting a learning organization through an accessible collection of knowledge of past experiences to enable situation-specific decision making. To support practice work, knowledge management systems should be designed to manage both unstructured data (searchable content, documents, and images) and structured data (information in transactional databases, drilldown reports, trend analyses, etc.). They may also provide access to collaborative social networks, forums, blogs, wikis, and other sources of free-form knowledge sharing. Over time, some of the elements of practice work may be simplified and standardized into processes, effectively creating more efficiency without sacrificing agility.

CropperCapture[1]

Why would this be important?  Well, many reasons… For me, most recently, it has been important in trying to help lead a change in our business sector.  I have found that change is most likely to adopted when the parties involved understand the WHY first.  Simon Sinek holds this same view as you can see from the picture of his golden circle and/or from his book: Start with Why: How Great Leaders Inspire Everyone to Take Action.  Why maps back to principles and value systems, where as how maps back to practices and processes.

CropperCapture[2]

I have also found these concepts useful in quantifying skill level of personnel.  Those with Wisdom and Understanding are capable of performing practices.  Those with Wisdom are able to define new processes.  Those with Knowledge and Information are able to perform processes. 

107,659 Total Views
August 16, 2012 Uncategorized

Switching to Jenkins–Download and Install Artifact Script for Tester

One of the things that I really liked about the version of CCNet that I was using was the ability to publish artifacts for download on the build report web page.  Here is a snapshot of the featured artifacts section of the summary report:

image

Beyond the nice GUI you could count on the url for the artifacts to conform to a pattern.  For example the weblogic release installer exe can be found at this address:

http://cir51-build03/Chapter33-5.2/Artifacts/20120314114423/Chapter33-Weblogic-Release-5.2.1.40468.exe

This enabled me to write a script, Chapter33.Deploy.Release.For.Tester.bat, that would download the http content, database installer, and weblogic installer and install the Chapter33 application in a tester’s private(local) workspace.  Well I am getting a little ahead of myself, the script also read from a CCNet REST interface.  It would gather up the build numbers for the last 6 successful Release Builds and present a GUI to the tester asking them which version of the application they would like to install.

Jenkins offers the analogous features, both artifacts for download and a REST interface, that will enable an analogous script to be written.  First lets look at the REST interface.  You can quickly see the xml from a REST request in you browser by adding /api/xml to any Jenkins page you are on.  So if you were at the following URL(a project dashboard):

image

 

Just adding /api/xml to the URL will call the REST interface:

 

image

If you want some more info on options for calling the REST API just /api to the URL…  Now the Jenkins xml REST interface is pretty cool in that you can specify the depth of information as well as tune it with an xpath query.  We will need a deeper depth of information as the current depth does not show the result, success or failure, for each build, nor does it give us the build display name.  By adding the query param depth=1 we get:

image

 

Finally we just need the build number for the last 6 successful build.  Note on my projects where we set the build display name to the SVN revision number we select the fullDisplayName as opposed to the build number here.  By adding the this xpath query xpath=(/*/build[result='SUCCESS'])[position()<=6]/number we get the following results: 

 

image

 

Only 5 results are returned here because there are only 5 successful builds retained on this job.  With this REST query we have enough to get started on a Groovy script to download the artifacts.

def protocol = ‘http://’
def serverName = ‘ci.jruby.org’
def port = ”
def jobName = ‘jruby-dist-release’
def resultSetSize = 6

def restEndPointBuildList = "${protocol}${serverName}${port}/job/${jobName}/api/xml?depth=1&xpath=(/*/build[result='SUCCESS'])[position()<=${resultSetSize}]/number&wrapper=builds"
def slurper = new XmlSlurper()
def buildList = slurper.parse(restEndPointBuildList)

def goodBuilds = new ArrayList<String>()

buildList.number.each {
    goodBuilds.add(it.text())
}

This code will create an array of the build numbers returned from the REST query.  It uses the XmlSlurper to accomplish this, both downloading the REST result and parsing it so that we can loop over it creating the array of good build numbers.  Next we need to present the user with a UI so that they may choose a build that they wish to download and install.  I did not care about making the UI pretty so I opted to use a basic antforms UI.

image

def goodBuildsFlat = ”

goodBuilds.sort().each {
    if (goodBuildsFlat == ”){
        goodBuildsFlat = it
    }else{
        goodBuildsFlat = it + "," + goodBuildsFlat
    }
}

def ant = new AntBuilder()
ant.taskdef(
    name:"antform",
    classname: "com.sardak.antform.AntForm",
    classpath: System.getenv(‘ANT_HOME’) + "/lib/antform.jar"
);

ant.antform(title: "Choose Which Build to Deploy"){
   ant.label("Choose a build")
    ant.radioSelectionProperty(label: "Builds: ", property: "buildChosen", values: "${goodBuildsFlat}")
}
def buildNumber = ant.project.properties.buildChosen;

The antforms task will accept a comma separated value string of options to make radio buttons out of.  I wanted them to be sorted, that is first bit of code here.  Next we need to load the antforms task.  You can see here that the jar is loaded from the ant lib dir.  You could load it from anywhere…  After the ant task has been loaded we call to display the form passing in several options like a title and labels.  When the user clicks the OK button the ant property buildChosen will contain the value they chose.  This is placed in the local variable buildNumber.

def restEndPointBuild = "${protocol}${serverName}${port}/job/${jobName}/api/xml?depth=1&xpath=/*/build[number='${buildNumber}'][1]"

def build = slurper.parse(restEndPointBuild)

build.artifact.each {
    def artifactFileName = it.fileName.text()
    def artifactRelativePath = it.relativePath.text()
    def artifactUrl = "${protocol}${serverName}${port}/job/${jobName}/${buildNumber}/artifact/${artifactRelativePath}"
    download(artifactUrl, "C:\\Temp\\${artifactFileName}")
}

Next the script makes a REST query getting the build details for the build chosen.  It then loops over the artifacts published in that build downloading each.  This is the part of the script that you will need to start tailoring to your needs.  Where do you want to download to?  Do you want to download all the artifacts or just a specified few?  What are you going to do after downloading the artifacts?  On the project that I am working on at the moment we publish full on headless installers.  After downloading these installers we execute them.  This provides a push button script for the testers or anyone on the project to deploy a personal instance of the application any time they wish.  They even get to choose the version they want to install.

 

Ooh before I forget here is the last bit of the script, the download method.  I used Curl so that if the download fails it will pick back up where it left off we you retry.  This assumes that Curl is in your PATH.

def download(url, destination){
    String command = "-0 -m 86400 -S -C – -o \"${destination}\" -k -L –retry 5 ${url}"

    def ant = new AntBuilder()
    ant.exec(executable: "curl"){
        arg(line: "${command}")
    }
}

101,803 Total Views
March 16, 2012 Continuous Integration

here