Building a Test Case Management solution

I've recently been looking at how to build a reasonable test case management solution (good!=word documents) for our company. I quickly learned this is not a very developed field. Mercury TestDirector seems to dominate the commercial field, with the other QA product companies (CompuWare, IBM-Rational) following suite, but not there yet.

Test Case Management as I understand it deals with the following objects:
* Test Plans
* Test Cases
* Test Labs
* Test Schedule

The main deliverables expected from a Test Case Management solution are:
* It should provide visibility into the testing process - both the plan and the actual execution.
* Is risk-oriented, focusing on the riskier aspects of the system first.
* Facilitates day to day management of the testing team.
* It should present a coherent picture connected to the Issue Tracking tool (what bugs are blocking tests, what tests need to be ready for a certain release, etc.)
* It should present a coherent picture connected to the Automated testing frameworks (If the smoke test / sanity failed during the night, the Test Case Managment should reflect that without requiring the QA engineer to manually copy+paste the information).
* Tracking pass/fail status for each test
* Tracking pass percentage per module per version per milestone requirements
* Dynamic priority management which affects to do list for testers
* Coverage of requirements by test cases (each change request should be linked to at least one test case ? Closing a change request requires successful test run ? )
* Management of test beds relevant for each test case
* Manage test cases that are blocked by other change requests (bugs/enhancements)
* Accessibility to test case information from each bug / change request.
* Accessibility to test logs from the relevant test case instance
* Manage the relationship between different test cases - its quite useful to create dependencies - e.g. run the login test case then run the change password test case.
* MANY test case instances for ONE test case in ONE version
* ONE requirement can be tested by MANY test cases
* ONE test case may test MANY requirements ?
* ONE test script may be used in MANY test cases
* ONE test case may run in MANY configurations
* Time estimate for coverage of a version
* Last time a specific test was run, by who, what results
* specific test case results across builds
* Ability to share the test cases with an OEM or remote team
* Version control for test scripts (link to the SCM)
* What version of the test script was used for each test case instance ?
* What version of the test case was used for each test case instance ? (if we added a sequence, and suddenly tests started to fail, it doesn't mean regression in software)

My understanding of this is based on some resources I've been monitoring (see the full list on ).
Some of the noteworthy ones are:
* StickyMinds.com : Article info : Reengineering Test Management
* StickyMinds.com: Bringing Your Test Data to Life
* Rhonabwy writes about his own experience with the open source test case management tools from time to time
* OpenSourceTesting - Test Management Tools is the list of test case management tools everyone refers to.

Based on the information I found I've been looking at TestMaster, TestLink and QATraq, but didn't install any of them yet. The other ones really seem either dead or not ready yet.

I'm still trying to understand whether the correct approach is to get a test management tool, and try to connect it to your issue tracker, or to get a really customizable issue tracker (e.g. JIRA) and build what you need of a test management tool there. I'm still contemplating the pros and cons, and trying to understand how much of test case management is actually an issue tracking type activity, and what are the parts that are not. This is quite uncharted ground from what I've found so far, and I understand that part of having a "reasonable" solution is to skip some of the requirements and vote for simplicity.

A good friend which knows what he's doing when it comes to managing QA efforts repeatedly tells me to aviod the bells and whistles and the complex reports metrics and processes, and go for simple worthwhile metrics, the reports and flows that are necessary to support them, and to focus on the substance. Thats a big part of what I consider to be a "reasonable" solution.

Customer Relationship Management for a small growing startup

Every startup which reaches a stage where you have customers, realizes at some point that managing the customer relationship throughout the sales life cycle (not just pre-sale but also post-sale) is a process which requires attention.
I've personally seen a few cases where this realization comes as a reflection on some dropped balls and hurt feelings on all sides.

Anyhow, we are now looking for a solution that will allow tracking customer issues, known product issues, and will also interface with the internal issue tracker (we use bugzilla but are considering JIRA for that). We are looking at SupportForce since we already use SalesForce for the sales tracking aspects.
While reading I found SugarCRM which seems to be an Open-Source alternative. No idea how it compares to the big-league players yet, and no idea what we really need.
I admit this CRM area is kind of new to me, and I think I need to learn some more about it in order to make sure we build the right foundation here so everyone is satisfied.

In another company I worked for the financial/IT guys chose PriorityCRM which all of us engineering guys thought was quite pathetic and hopeless. Sort of like Magic on bad drugs. I guess the choices in the CRM space is affected by the other adjacent modules, and all too often the choice is made according to the convinience of the financials/operations people which need to track bills, inventory, etc., and not sufficiently according to the requirements of the Professional Services and Engineering departments. I honestly cannot tell what is more important based on my current perspective. Need to take a more complete look at the picture to really say.

I decided I will try to learn a bit about SalesForce from one of our sales guys, and try to see what he gains from using such a product, and what are his expectations. That will give me some perspective.

I'll probably continue this thread as we progress.

JIRA - Bug Tracking, issue tracking and project management software

JIRA - Bug Tracking, issue tracking and project management software
Recently we've been looking at overhauling the issue tracker we use for our software development life cycle (SDLC) where I work.
We currently use Bugzilla but we think it has serious shortcomings when starting to talk about multiple versions being maintained concurrently, custom fields necessary to track stuff we care about (such as metrics, release version for a bug, etc.).
I guess bugzilla is better suited to projects which care less about old versions, and more about the tip.

My research took me thru several tools (Mantis, Eventum, TestTrackPro, Trac) and currently JIRA, although commercial, seems like the best fit.

They also provide Confluence which is a very nice enterprise wiki, which has us quite drooling here.

I'll go into details about what we care about and what we think about the other tools sometime else when I have more time for this...