Fork me on GitHub

Drew's World

Rants, News, Etc on my Life and Projects

Git Based Workflow Revised

by Andrew De Ponte (@cyphactor)

If you happened to read my previous post An Ideal Git Based Team Workflow this posting is a result of taking that concept and discussing it heavily with my team as well as further contemplating the issues and solutions.

The Real Requirements

After reviewing my thoughts and discussing things further with my team I determined that a number of aspects of the previously defined workflow are not needed. In fact they were overcomplicating the workflow due to a lack of understanding of the actual requirements. One such requirement was the concept that branches should parallel deployment environments. This is not actually a requirement and I was sadly imposing it as one in my initial workflow. In hindsight this was primarily due to the fact that it was the focal point of a number of discussions with one of my developers, Michael Genereux. This singular mistaken requirement imposes a huge amount of complexity and constraint into the workflow for no significant value in my opinion. The set of requirements that I now believe are true requirements of our workflow are presented in the list below.

  • Overhead on the developers should be as minimal as possible
  • Overhead on the integrators should be as minimal as possible
  • Hotfix deployments should be able to trivially be made
  • Content deployments should be able to trivially be made
  • Iterative development cycles can happen parallel to Hotfix and Content development & deployment
  • Code review and feedback is needed for our inexperienced developers so that they can learn

The Two Roles

After, isolating the correct requirements I determined that in order to satisfy the minimalistic overhead requirements as well as the code review/feedback requirements we were going to need two different roles within our workflow. The first role is what I will from this point on refer to as an “Integrator”. In reality an “Integrator” in my mind is simply a developer that is skilled enough to follow carefully designed policy, doesn’t need much guidance, and fully understands the core concepts of Git and why the workflow policies exist. The second role is what I will refer to as a “Newb”. A “Newb” in this case is simply a less experienced developer that may need more guidance, heavier code review, and more direction in general.

The Integrator

The group of developers which fall within the “Integrator” title are responsible for not only doing their normal iteration development but also for performing code review and providing feedback to the “Newbs”. Beyond that they are responsible for following the provided policies to manage Hotfixes, Content changes, and the normal iterative development cycle. The policies for each of these areas with respect to “The Integrator” role are provided below.

Hotfixes

In this workflow I am defining hotfixes simply as changes that have to be made and deployed abruptly out side of a normal iteration deployment. These are normally severe bugs that drastically hinder the user experience and must be fixed asap.

Developers of the “Integrator” role are required when creating a hotfix or series of hotfixes to create a remote branch for the hotfixes within the current weeks iteration. The branch should be based on the release that the hotfixes are going to apply to. Secondarily the name of the branch should be of the following format, "week_<week number>_hotfixes". Once the remote branch exists development of the hotfixes should occur on the newly created hotfixes branch. Once the branch reaches a state in which it is ready to be released into the accelerated pipeline it should be tagged using the release naming scheme and deployed.

Content Changes

In this workflow I am defining content changes simply as changes which need to be deployed on a more frequent basis than the normal iteration deployments. Beyond that they are generally changes to assets or textual changes needed by marketing, etc. which have less of a functional involvement.

Developers of the “Integrator” role are not usually the developers making these changes directly. However, they are responsible for reviewing, integrating, and providing feedback to the “Newbs” that are making these changes through the “Newb” workflow as defined below. Hence, the “Integrators” should pull in the “Newbs” content changes after reviewing them and merge them into the current weeks remote branch for content changes. This branch should have a name of the following format, "week_<week number>_content". If this branch doesn’t exist at the time in which an “Integrator” needs it they are responsible for creating it. Once this branch reaches a state in which it is ready to be deployed, it should be tagged using the release naming scheme and deployed into the pipeline.

Normal Iteration

In this workflow a normal iteration is simple a release cycle of two weeks. Within a normal iteration “Integrators” are responsible for not only managing content and hotfixes from “Newbs” but also managing normal iteration development from “Newbs”. Beyond that “Integrators” are responsible for performing their normal development work for the current iteration. All development changes should be performed inside specific topic branches and never directly on the master branch. If features/bugs are large enough that they may consume multiple days of development they should have their own remote topic branch. This allows other developers to participate or at a minimum obtain the partially completed changes which you have made. Beyond that never developing directly on master makes abrupt context switching trivial. Once a topic branch has reached a point in which it is ready to be merged into the current weeks iteration and ready for the pipeline the “Integrator” should merge it into the master branch and push the updated master branch to the central repository.

The Newb

The developers which fall within the “Newb” title are responsible for performing their development tasks and learning the process and value of the workflow policies as well as the technologies and languages which they are using. Ideally the “Newbs” progress to “Integrators” eventually. Their tasks similar to the “Integrators” may consist of development work for Hotfixes, Content Changes, or Normal Iteration development. However, one way they differ is in the process and policies they have to follow when performing these tasks. The workflow for each of these types of changes with respect to “Newbs” is provided below.

Hotfixes

A “Newb” similar to an “Integrator” is required to do all development within topic branches. However, when a “Newb” believes they are ready to have their changes included in the current weeks hotfixes branch they must make a Pull Request via GitHub to their upstream repository. GitHub will then notify all of the “Integrators” of the new pull request and allow them to review the changes, provide any feedback and merge the changes in. GitHub will also allow the “Integrators” to throw the changes back to the “Newb” with feedback rather than merging the changes in.

Content Changes

With respect to a “Newb” content changes work exactly the same way as Hotfixes. They simply develop their content changes and submit an appropriate Pull Request via GitHub to their upstream repository so that an “Integrator” can appropriately deal with their changes and provide any feedback.

Normal Iteration

“Newbs” follow the same process for Normal Iteration development as they did Hotfixes and Content Changes. Develop the changes in a topic branch and submit a Pull Request via GitHub.

Requirements Review

I believe that the above workflows and policies provide a solid basis for a well rounded team with very little overhead and a decent amount of flexibility with respect to parallel releases being made at different rates. Please note additional parallel releases can be added simply by adding additional conceptual types of changes to my current list of Hotfixes, Content Changes, and Normal Iteration Development.

Release Naming Scheme

Given that we are developing by either week or two week iterations I have decided that we should use the following naming scheme for tagging releases:

release-YY.<week_num>-rc<release candidate counter>

The YY represents the year for 2010 it should be 10. The <week_num> is the week number and the <release candidate counter> is a counter for this iteration that reset at the beginning of each iteration. Once a final tagged release for production is ready to be made it should use the following naming scheme.

release-YY.<week_num>-rf<release candidate counter>

All the variables in the naming scheme for final tags are the same as those in the latest rc for a weekly iteration. For example if the latest rc is release-10.34-rc6 and no changes have been made since that tag and the code is ready to be tagged for a production release it should be tagged as release-10.34-rf6.

Topic Branch Naming Scheme

When naming topic branches it is required that you use the following naming scheme.

<type char><ticket id>_<name>

The <type char> is a character that defines if it is a task (t), bug (b), or feature (f). The ticket id is the unique identifier of the ticket in the ticketing system that corresponds to this topic branch. The <name> is a short but meaningful name that describes the topic branch.

Don’t Tie Deployment Environments to Branches

So, as I stated in the opening I was assuming in the previous article that each deployment environment should have a branch that parallels it. This was an idea that was presented to me by one of my developers and I mistakenly ran with it. The following is an explanation of why I believe doing this is not valuable and potentially costly. Also, I am not pointing this out to pick on my developer. In fact it is quite the opposite. I am glad that he forced me to think about alternatives in great depth because the process has only more solidified my belief in the workflow and policies presented above. Beyond that, the explanation of my opinion may help him or others come to the same or even a better conclusion/workflow.

For some reason I couldn’t initially place my finger on what was wrong with using branches that paralleled the deployment environments of development, qa, staging, and production. I knew that I really hated the overhead of having to deal with cherry picking changes from one branch to the next to simulate the pipeline but I still simply saw that as its own issue which inspired me to come up with the previous and very flawed workflow, An Ideal Git Based Team Workflow.

Today after an insane amount of contemplation, discussion, and fiddling with the workflow the answer finally came to me. The problem with the Deployment Environments mapping to branches is that from a workflow perspective we don’t really care about the deployment environments other than the tagged release that is currently deployed in each environment. This information can easily be obtained without branches that map to the environments by correlating the deployed tag in a given environment to the matching tag in Git.

What we do care about with respect to a workflow however is the ability to be able to cleanly share code changes as well as be able to handle a number of parallel releases. I have chosen to represent these parellel releases via different classifications of changes in the above workflow. Specifically, Hotfixes, Content Changes, and Normal Iteration Development.

When you test these two models it is very interesting because in the model where you have branches for each deployment environment you end up spending a huge amount of overhead simulating something for no reason. The primary cost in overhead is the amount of merging and cherry-picking that is necessary to keep the states of the branches in line. Secondly, it does NOT allow you to make parallel deployments of various types unless those changes happen to fall within the various stages of your deployment pipeline.

The second model in which you have classifications of changes that map to parallel releases, the overhead of maintaining the deployment environment branches is completely eliminated. It also models the actual situation with respect to code and the natural requirements surrounding what needs to be done with/to that code. Therefore, it requires no additional overhead or maintenance.

Conclusion

The above is why I believe the model in which branches parallel deployment environments is costly and has no perceived value in comparison to its costs. As always I am interested to hear peoples thoughts and opinions with respect to my postings, so don’t hesitate to share.

A Quick Multi-Source Ticket Dashboard

by Andrew De Ponte (@cyphactor)

Today was one of those days where I woke up and something was just off with respect to my workflow for some reason. I was frustrated with the fact that I had to load a browser and go to a website to see what my tickets were for the day. Instinctively I just wanted my tickets to magically be in my head so I didn’t have to go through all those steps. So, I started thinking about this problem more and more as the day went by and I came to the conclusion that I really have two problems.

  1. I have tickets dispersed through multiple ticketing system.
  2. I hate having to go through so many steps to see my tickets.

When I started thinking about my two problems I realized that I have my tickets in roughly four different applications/services. These applications/services are as follows:

  • Trac - Web based project management/task management tool
  • Lighthouse - Web based project management/task management tool
  • GitHub - Web based Git repository management system and issue tracking tool
  • Things - Desktop task management system

Beyond having multiple ticket sources, most of them have multiple accounts for various reasons (contract jobs I have done, etc.). Being in a specific environment while working within that scope/environment is fine. For example if I am working on Project X it makes sense that I be in project X’s environment. However, the situation that I am frustrated with is the initialization and transition periods which exist before you start working on a project and when you are switching from one project to the next project.

Based on my issues and the situation I am frustrated with I have come up with a small set of requirements for a solution. These requirements are listed below.

  1. Tickets from all ticket sources must exist in a dashboard
  2. This dashboard must require zero effort/steps to view it
  3. This dashboard must automatically update with zero effort/steps

As many of you know despite the fact that I write software for various platforms and in various languages my platform for development is generally Mac OS X. Given that Mac OS X was my environment, I started thinking about this requirement set and thinking about how I could make something that required zero effort/steps to view it. The closest thing I could come up with was having my tickets displayed on my desktop. After a few minutes of googling based on that idea I found an awesome tool for Mac OS X called GeekTool. Among other things, GeekTool will run shell scripts/console applications for you on a set frequency and display their output to a defined section of your desktop. This was perfect in my opinion so I played with it for a few minutes and decide that I was going to use GeekTool.

Given the functionality of GeekTool all that was really left was to write shell scripts/console applications that output all of my tickets from the various ticket sources and I would have a very quick and functional solution to my issue. Hence, I got to writing and pumped out the following short little Ruby scripts, and one AppleScript script to provide output of all my tickets from the various sources.

The following is a ruby script that pulls all my tickets in from all of my various Lighthouse accounts and projects.

#!/usr/bin/env ruby
require 'rubygems'
require 'lighthouse-api'

include Lighthouse

accounts = ['domain one', 'domain two', 'domain three']
Lighthouse.token = 'your lighthouse All Accounts token'

accounts.each { |a|
  Lighthouse.account = a
  Project.find(:all).each { |p|
    if (!p.archived)
      ticks = p.tickets({:q => "responsible:me sort:priority state:open"})
      if (ticks.length > 0)
        puts p.name
        puts "-------------------------"
        puts "    id | pri | title"
        puts "-------------------------"
        ticks.each { |t|
          puts t.id.to_s.rjust(6) + " | " + t.priority.to_s.center(3) + " | " + t.title
        }
        puts "\n"
      end
    end
  }
}

The following is a ruby script that pulls all my tickets in from all of my various GitHub accounts and projects.

#!/usr/bin/env ruby

require 'rubygems'
require 'octopi'

include Octopi

creds = [
  { :login => "login one", :token => "API token tied to login one" },
  { :login => "login two", :token => "API token tied to login two" },
  { :login => "login three", :token => "API token tied to login three"}
]

creds.each { |cred|
  authenticated_with :login => cred[:login], :token => cred[:token] do
    user = User.find(cred[:login])
    repos = user.repositories
    repos.each { |r|
      ticks = Issue.find_all(:repository => r, :state => "open")
      if (ticks.length > 0)
        puts r.name
        puts "-------------------------"
        puts "    id | pri | title"
        puts "-------------------------"
        ticks.each { |t|
          puts "n/a".rjust(6) + " | " + t.votes.to_s.center(3) + " | " + t.title
        }
        puts "\n"
      end
    }
  end
}

The following is a AppleScript script that pulls all my tickets in from Things.

tell application "Things"
  set nl to "\n"
  set output to "Things Tasks in Today\n--------------------------\n"
  repeat with todayToDo in to dos of list "Today"
    set curTaskName to name of todayToDo
    set output to output & " - " & curTaskName & nl
  end repeat
end tell
output

The following is a Python script that pulls all my tickets in from Trac when your Trac environment has the Trac XML RPC plugin installed.

#!/usr/bin/env python

import xmlrpclib

username = "loginname"
password = "password"
port = None # if need to specify a port set this to the port number
host = "domain-name-where-trac-hosted" # trac.rippletv.com at my work
path = "/path-to-the-environment"

url = "https://%s:%s@%s" % (username, password, host) 
if (port):
    url += ":%s" % (str(port))
url += path + "/login/xmlrpc"

svr = xmlrpclib.ServerProxy(url)

print "-------------------------"
print "    id | pri | title"
print "-------------------------"

ticket_ids = svr.ticket.query("owner=" + username + "&status!=closed&order=priority")
for ti in ticket_ids:
  cur_tick = svr.ticket.get(ti)
  cur_tick_hash = cur_tick[3]
  cur_id = ti
  cur_summary = cur_tick_hash['summary']
  cur_priority = cur_tick_hash['priority']
  pri_num = 0
  if (cur_priority == "trivial"):
    pri_num = 0
  elif (cur_priority == "minor"):
    pri_num = 1
  elif (cur_priority == "major"):
    pri_num = 2
  elif (cur_priority == "critical"):
    pri_num = 3
  elif (cur_priority == "blocker"):
    pri_num = 4
  print str(cur_id).rjust(6) +  " | " + str(pri_num).center(3) + " | " + str(cur_summary)

After coding the above scripts I simply set them up through the GeekTool UI and like magic within 15 mins or so I had a working solution to my issue, thanks to the power of REST based Web Services, Ruby, the Ruby Community, and Things AppleScript integration. Checkout a screenshot here.