Tuesday, August 08, 2006

Patterns for issue tracking

I recently spent some time on devising methodologies for software development lifecycle in my company, dealing with SCM (Version Control) and Issue Tracking.

I'm a big fan of patterns. my first encounter with them was with the POSA series (Pattern-Oriented Software Architecture, Volume 1: A System of Patterns
/ http://www.cs.wustl.edu/~schmidt/patterns-ace.html) when working on distributed systems.

As a fan of reuse, this was quite an important finding.

Later I encountered the SCM patterns. I read http://www.cmcrossroads.com/bradapp/acme/branching/ by Brad Appleton and understood, yet again, that much of what we were doing good was a pattern, and what we were doing wrong and were looking to improve was an anti-pattern. I also read his book Software Configuration Management Patterns: Effective Teamwork, Practical Integration

Software Reconstruction Patterns (http://www.cmcrossroads.com/bradapp/acme/repro/SoftwareReconstruction.html) are a related useful family of patterns.

I also encountered organizational/process patterns, but I admit to not grepping the concept fully so far (in the todo list...). See http://www.ambysoft.com/processPatternsPage.html#FAQ%20What%20are%20Process%20Patterns.

Now while trying to devise the Issue Tracking methodology, starting with a baseline documentation of how each group (recall we are one R&D group acquired by another) does its work, I felt the need for patterns in this domain, and wasn't able to find any so far.
So, I decided that while keeping on the lookout for a pattern repository for this domain, I will start documenting patterns on my own, and try to come up with a draft of the issue tracking pattern family. I'm sure it will be useful to myself in the future. Hopefully via discussion in the right community, it can evolve into a public body of knowledge.

Anyhow - the patterns I've thought of so far are below. I now see that one of the greatest challenges are naming them right - so they can be generic enough and still specific to what the context you are talking about. I'm trying to take some guidelines from the Gang of Four definition (see http://en.wikipedia.org/wiki/Design_pattern_%28computer_science%29)

Taxonomoy - Categories

v Deliverables Generation

Ø AutoInternalReleaseNotes

Ø AutoApplicableWorkaroundsList

v Process Flow

Ø Heirarchical Triage for Incoming issues

aim to make a distributed decision on 80% of the issues according to pre-discussed policies, but have a streamlined process for tracking and reaching a wise decision on the rest 20%.

Ø Resolve->Integrate->Release completed issues

Ø Understand scope and impact before committing to Schedule

Be able to track issues which need work in order to schedule, but are NOT to be solved unless are really trivial, instead are to be raised for a schedule decision/discussion

Ø Close everything

Having a closure phase for completed (fixed) issues as well as duplicates, invalids, wontfixes, etc.

Ø Match and document the actual workflow between people

§ Give leads/managers the ability to review work by their people and sign off on it (or reject it)

§ QA Lead confirms new bugs from QA

§ DEV Lead integrates fixes resolved by his people

Ø commit approval / CCB activity / code review process should be enabled by the issue tracker workflow

Ø Ownership is NOT a state. Current action phase IS.

Waiting for QA Reproduction - State or Ownership?

v Relationships between issues

Ø Track symptoms separately from change tasks ?

Ø How/When to divide issues

Ø Issue equivalent of "Release branch"

How to deal with issues that are relevant on multiple version, where their state might be different for each version, but most of the data is shared?

v Issue Meta-Data

Ø Track "resolved in" version automatically

Ø Establishing priority based (among other things) on Severity

Ø Track stage at which the bug was opened

Allows understanding of QA/DEV effectiveness at developing/releasing quality software.

Ø Track reproducability and reproduces of the issue

reproduce cases - via link to the test management ? Via sub-issues linked to the parent?

Ø Keywords might be better than Custom Fields

Ø Discern "introduced in" from "detected in"

v Interface to other processes

Ø Interface to SCM

§ Integration with Task-Level Commit

Ø Interface to Test case management

§ Track test case for each issue

· if test case opened the issue - to know what to run to test/verify/close

· if from field or exploratory - to track process of adding this to regression suite

Ø Interface to Project Management / Use as project management

Ø Interface to CRM

v Usability

v Useful Metrics

Ø AutoCustomerReleaseNotes

Ø FaultFeedbackRatio (Regression rate)

Ø Rate of bugs fixed in version they were opened for (>70%)

Ø Rate of bugs detected in the field (<5%?)

v Anti-patterns

Ø Metering people via bug counts

Ø Overloading states/fields for multiple purposes

Ø Over centralization of decision making

Don't let a workflow with many steps fool you into thinking that it requires many people. Use steps to track where you are. Use assignment to track who holds the issue. Don't assign upwards unless necessary.

Ø Tracking and metering by Components?

see http://www.anyware.co.uk/2005/2006/07/27/jira-issue-tracking-meets-tagging/

This is still very much a work in progress, but any comment or help is very much welcome.

1 comment:

Yuval said...

http://c2.com/cgi/wiki?DefectTrackingPatterns seems to be some prior art relating to this, done by Steven Newton from io.com

I might decide to document the patterns there as well.