As part of a recent agile transformation project, I was tasked with improving the deployment cycle times and asked to institute a release process suitable for fast paced delivery schedule. The client, a large blue-chip organisation, already had in place an established manual release control process. This was a well thought through process it was fully audited and included review and quality gates. However as with all manual processes it was resource intensive and unsuitable for supporting an agile process.To ensure that any process undertaken provided value we measured the existing process over a number of deployments.
The application being deployed needed to be transitioned through 4 environments:
An uncontrolled single server environment where only the developers and testers had access. This environment was there so that things could be deployed reactively to feedback.
Test / Quality Assurance
A controlled single server environment, the test manager made the call on when drops were made to this environment.
Staging / Model Office
This replicated the distributed redundant hardware used in live. This provided a location to stress test the application and ensure that if could be split of a load balanced set of servers.
UAT was also performed in this environment.
The end goal environment. Entry to this environment was based on successful deployments to all the previous environments. Finally any deployment documentation had to be reviewed by a panel to ensure any plans met with the corporate SLA’s and security requirements.
When measuring the deployment we found a number of factors firstly no one person knew how to deploy the entire application. The instructions though verbose were incomplete and each deployment seemed to include an extended dialogue between the development team and operations while problems were addressed.
As for resource utilised, here is the breakdown:
As you can see a month of developer and over half a month from operations time was burned in simply deploying the application. This assumed nothing went wrong with the deployment at any stage.Given our planed deployment velocity this current approach was untenable. We looked to automate and started to examine our options. Whilst there are a number of potential tools out there the clients existing eco-system and experience meant a Microsoft based solution was preferable. Release Management for Team Foundation Server had just been released and this seemed to fulfil all of our immediate problems, whilst maintaining the audit and control the business so desired.
The proposed plan was to automate from the ground up, i.e. start and Development then automated the deployment into each environment, thus reducing the overhead to the project.
The development environment was a relatively quick win with a near fully automated approach complete within a single 2 week sprint. This was then proved whilst work began to integrate operations into the approval process.
In all the process took around 3 months, although the results were astounding:
As you can see a deployment which manually took days and hours could be completed within minutes. The direct output of this meant that the application was delivered to test and to UAT earlier and more consistently allowing for early feedback and reduced cycle times. On top of the increased velocity the direct savings in resource accounted for 1.5 people.
Given that the timings and numbers here are based on one project imagine the impact that this approach could have across an entire portfolio?
If you'd like to know more about this or our other life-cycle management offerings please don't hesitate to contact us.
N.B.to get to the figure of £90,000 we took an conservative annual contract developer rate. Current figures from IT jobs watch put the average developer rate in the uk at £350 per day.
About the author
Experienced Developer / Architect with a background in consultancy working with public and private sector client including Local Government Ombudsman, Legal Ombudsman, Virgin Atlantic and Willis Group