The challenge is to come up with a way of marking out the importance of a bug, without having to spend time thinking about important levels. A simple one to five or one to ten system is quite vague, and one person's 4 is another person's 2. So numbers aren't ideal.
I've trialled a simpler system: Critical, Need and Want. That works pretty well, although the Need category is quite overpopulated. It's clear three levels isn't really enough once you get past 30 or so active bugs - of my current list, around 25 are "Need", with 2 "Critical" and 3 "Want".
The second issue is that, while a client might think a particular bug is critical because it occurs in their favourite part of a site, I might think of it as relatively minor as it's not affecting sales.
Other bug systems have different levels of success with this, and among them Drupal has probably the best system: Critical, Major, Normal and Minor. That's close. But it's not quite there.
I want a system where each level ties to a concept of importance. With moderately clear delineation between levels. Where I could let a client could see the prioritisation of a bug without worrying it would upset them.
Here are the contenders I'm looking at:
- Critical
- Important
- Need
- Want
Not bad, but people disagree about whether "Need" or "Important" (and even "Critical") are highest importance. Not quite intuitive enough.
- Completely Broken
- Partially Broken
- Not Ideal
- Request
Clear delineation, but only works for bugs, not other issues, feature requests and additions.
- Urgent
- Must Fix
- Should Fix
- New Item
I dislike "urgent" as it conveys an importance of time, not priority. And a client will always think everyhing is "must fix", but on the plus side the delieation is clear.
Please feel free to add your thoughts or your own system in the comments.
25 Comments
While thinking about how we do things at my very large corporate company in regards to this, I realized that we do things very similarly to this - with one very important distinction, I think. There is a bug list and a request list. If our clients find a bug, a real one, something that doesn't work the way we intended it to... It goes in that stack. If it's something that they would like to see, would make their lives easier, etc., it goes into the request list, along with all the things that we want to do for the product.
We attempt to squash bugs rapidly - we want to get the corrective code into the code tree as fast as we can to make sure that it doesn't break things, and we generally backport things as far as possible as long as it doesn't take more than an additional couple of hours of dev time.
Requests go into the current development tree. One thing that we've implemented is an anonymous voting system - well, anonymous to users, anyhow. Your "support ID" gets one vote on each feature request, and it's a 1-10 thing. About 25% of our population use this functionality of our support system (usually the 25% that are most vocal). It helps us prioritize, and, shows our customers that we are listening to them, and giving them input on the next version.
And now that I've written a book... Have a great day.
#1, Andy, US, 6 October 2010. Reply to this.
We're going through this at MODx too, and it's really challenging to get right. Since we'll be rolling out commercial support with a real honest to goodness SLA, this is even more important for us to get right out of the gate. Here's a high-level overview of where we are today:
Emergency (Critical Impact)
+ Impacts a production environment.
+ Disabled critical business processes for end customers or for multiple accumulated customers of service providers.
+ Continuous or near continuous interruption of service.
+ No workaround available.
High (Major Issue)
+ In development environment jeopardizes production rollout.
+ In production environment causes a serious, but non-critical impact, for end-customers or for a service provider due to multiple customers experiencing the same issue.
+ Intermittent service disruption.
+ No stable workaround.
Medium (Problem)
+ In development environment creates a minor impact on activity, but does not jeopardize production rollout.
+ In production environment creates a minor impact on business operations of end customer or for a service provider due to multiple customers experiencing the same issue.
Low (Costmetic or Inquiry)
+ In development, creates little or no impact on activity.
+ In production environment, creates little or no impact on business operations of end users or of service provider.
+ Documentation errors or clarity that impacts efficiency.
+ New or enhanced feature requests that extend or improve functionality of the core framework product.
#2, Ryan Thrash, USA, 6 October 2010. Reply to this.
Here's my contribution :)
http://sigstp.blogspot.com/
Cheers
#3, Jerome Eteve, UK, 6 October 2010. Reply to this.
I don't know if your ever heard of ITIL. That is a concept to handle all incidents and service requests that ever reach the inbox of an IT department.
When I was trained for that kind of concept we were thaught that you have to seperate in these two categories:
Urgency: Is the boss of some department calling or a trainee?
1. very "urgent"
2. normal
3. kinda ... well not important, maybe only "planning"
and now: IMPACT
1. impacts EVERYONE in the company
2. impacts a WHOLE department in the company
3. impacts only a few people
4. impacts only one single person
So now you define your priority due to this two. URGENCY: 1 AND IMPACT: 1 is clearly very very high-priority.
But which one is more important Urg: 1 and Impact:4 or Urg: 4 and Impact 1.
You could discuss hours about it.
Best practice in a company in my humble opinion is, to set up Service Level Agreements that deal with that. I have no idea how to fix that in an other enviroment for example the open source community.
The reason is simple: you can't decide which impact there really is. There could only be 3 similiar bug reports and its really urgent because it is a main feature and everbody uses it once in a while but there could be like 42 feature requests that request all the same thing.
I think this is a very difficult issue.
#4, oktarinen, Germany, 6 October 2010. Reply to this.
We run into this all this time at my work. What we have worked out seems to be straight forward, though definitely NOT perfect. It's a sort of grid.
The bottom line is it's all about the bottom line. We work off of three priorities across the top ranked below high to less high (nothing's ever really low ;-) )
1. Customer Experience/Impact
2. Revenue / Profitability
3. Productivity Impact
Then down the left:
1. Major Impact (many)
2. Substantial Impact (some)
3. Minor Impact (few)
Anything that affects many or some customers is "Severe". Anything that affects Many Revenue / Profitability is "Severe". "High" is a status that affect Many Productivity, some revenue / profitability or some customers experience. Some productivity or a few revenue impact and it's medium, the other being low for a few productivity.
Overall it makes sense, and gives a nice guideline for prioritizing.
#5, Thomas, USA, 6 October 2010. Reply to this.
I don't remember which repository I originally saw it in, but I saw a project that used ? for enhancements and would tag it with a ?/??/??? depending upon the size of the enhancement. Then for bugs they would mark it with ?/??/??? depending upon the severity of the bug.
From there its it just up to deciding what the priority of the bugs are and acting accordingly when planning.
I liked it because it gave a quick summary of the type of issue and the size of the issue without having a complex system to do so.
#6, Randy Merrill, 6 October 2010. Reply to this.
Good ideas, all, and thanks :)
Andy: I like the idea of voting on priority for new work.
Ryan: Thanks, I like that categorisation. Something like "high, medium, low" or "major, normal, minor" is pretty clear. My only objection is that those words aren't intrinsically linked to priority, unlike words like "urgent".
Jerome: Thanks. I would worry there would be quite a lot of overlap between the top two levels, and between the bottom two levels though.
oktarinen: Not heard of ITIL, no. I do agree that both urgency and impact go into deciding which issue is the next (of a collection) to handle, but want to have a single metric to convey that. It's my gut feeling that every field you add to something like a bug tracker will increase the likelihood you'll get bugs not filed, or filed badly.
Thomas: I see your point, but I dislike the idea of adding a second metric to something that should translate into a linear scale. Even in your system, you should be able to see which bugs are next to process, and I want to combine the factors that go into that metric into a singe number.
Randy: I like the idea of a system that makes it quick to see the severity of things.
#7, DaveChild, United Kingdom, 6 October 2010. Reply to this.
Everyone in the English-speaking world knows how to grade: A, B, C, D, F. So that suggests five grades.
My personal priorities are: 1. THE one and only thing I'm working on at the moment. 2. Candidates for 1. 3. Deferred.
Severity: A. Keeps other people from working. B. Prevents shipping my product. C. Major functions don't work. D. Minor functions don't work. F. Appearance and convenience.
Replies: #12.
#8, Bill Drissel, USA, 6 October 2010. Reply to this.
At a previous company where I was the IT Manager and R&D Manager simutaneously, one of my subordinates suggested a system with 10 priority levels which he built for us and it worked fairly well, though a lot of good ideas didn't make it to the top of the list. We kept only a handful of tasks at each of the top five levels. He also rigged it so that when something broke on our web site, it automatically logged it in the database at priority level zero (more important than anything).
I'm now working as a member of a larger IT Department in another company. Trying to establish some semblance of order has been difficult, but I've made some headway. The company-wide workflow system only has five priority levels. I was able to implement <a href="http://30secondblogs.blogspot.com/2006/10/t-shirt-estimates.html">t-shirt size estimates</a>, which have helped get the short tasks done more quickly. Now I'm trying to implement something along the lines of <a href="http://30secondblogs.blogspot.com/2010/03/two-vs-three-column-planning.html">three-column planning</a>, using the top four priority levels for in-progress and the lowest level (priority 5) for incoming requests. The fact that we don't automatically do everything that everyone asks of us is taking a while to sink in with some people, but it's really helped my stress level.
We don't have an established method yet for people to indicate to us how important they think their requests are, but we've decided that nobody outside our department is truly qualified to judge how important their requests are in relation to other people's requests. The IT Manager determines priority and assigns tasks to each member of the department. We're slowing forcing the other departments to send us requests on the workflow system instead of e-mails, phone calls, post-it notes, or wandering in to our area.
Replies: #12.
#9, Scott, Canada, 8 October 2010. Reply to this.
Great ideas.
I agree that urgent becomes a problem when the customer always think something is difficult.
That is just not fair to anyone. Really important issues not reported by the customer and fairly treated dont stand a chance since they are correctly evaluated.
Thanks for sharing.
#10, centrex, Sweden, 9 October 2010. Reply to this.
The difficulty you/I/everyone has to cope with is that there are at least two dimensions to the problem of prioritization: importance and urgency.
Items with high importance and urgency are easy to deal with (web site down, roof leaking, customer cannot run software, credit card cannot be processed). They are always at the top of the list.
Items with low importance and low urgency are also easy to deal with - they only get done if there isn't anything else to do. Possibly they will never get done.
The problem is with stuff that is important (but not urgent) or urgent (but not very important). I wish there was an easy solution to this, but I don't think there is. The precedence should probably be given to the stuff that is important (but not urgent - yet). But the problem is that we aren't comparing like with like so it's difficult to keep a good balance.
Replies: #12.
#11, MZB, Canada, 20 October 2010. Reply to this.
#8 Bill: The problem with simple letters and numbers is that not everyone will assign the same importance level to each. Your B might be my D. I think each level of a successful system needs to have an intrinsic quality that means that everyone views it the same way.
#9 Scott: I love the T-shirt size estimates system. Estimates are a pain, and I've been trying to come up with a way to make them more usable, and hence actually used. That might just do the trick! I also agree that importance has to be judged internally for there to be any value in the attribute.
#11 MZB: Quite right, and that's what makes this tricky.
Perhaps instead of assigning a level, I should look a assigning it to a conceptual space in the work queue ... so, for example, use something like:
- Drop Everything!
- Jump the Queue
- Normal Flow
- No Hurry
#12, DaveChild, United Kingdom, 21 October 2010. Reply to this.
Hi Dave,
for new features, I've come across a MuSCW (pronounced Moscow):
Mu - Must
S - Should
C - Could
W - Would be nice.
This is very effective at sorting out features for clients when the budget doesn't cover everything. It also helps prioritise feature development, making the product viable even before work finishes.
As for bugs, I fix them as they come in - no lists. But then it's just me here, and I don't have mountains of bugs to deal with.
To me it's clear that the larger the volume of tasks (development or bug fixing or anything else) the more levels of ordering/prioritisation are required.
Replies: #17.
#13, Luis, UK, 30 October 2010. Reply to this.
As for me, I mostly weigh my decision through 2 options, the long term and the short term. However, I always make sure that most of my short term actions will help in getting me closer to my long term goals/objectives. Well, it actually comes out to these:
1. Need = crucial (to prolong and sustain the flow of business)
2. Important (related to future expansions)
3. Want (incentives but must be relevant)
Replies: #17.
#14, Seth, Philippines, 15 November 2010. Reply to this.
I would argue that anything that effects your day to day business operations should fall under critical.
Replies: #17.
#15, Thomas Craig Consulting, Canada, 16 November 2010. Reply to this.
Sometimes my team and I make our biggest gains when we sit down and really focus on whats important, this is such a great post to drive home this fact. Get organized and get your work done. Get the stuff that needs to be done first, and also construct a long term strategy, shareing this on twitter as well, thanks!
Replies: #17.
#16, commenter, canada, 21 November 2010. Reply to this.
#13 Luis: Do you not end up with most things under "Should"?
#14 Seth: A lot of people have been rating "Need" above "Important" - it seems they aren't distinct and obvious enough for prioritisation where lots of people can see the levels.
#15 Thomas: But if everything is critical, how do you decide what to fix first?
#16 commenter: All makes sense :)
#17, DaveChild, United Kingdom, 24 November 2010. Reply to this.
I've had a couple more suggestions from people:
- OMGWTFBBQ!
- WTF!
- OMG!
- BBQ!
And something very bug-specific:
- System Unresponsive
- System Error
- Unexpected Behaviour
- Edge Case
Finally, someone suggested using Dante's circles of hell - I fear for their company if their bugs fit into this kind of system:
- Limbo
- Lust
- Gluttony
- Greed
- Anger
- Heresy
- Violence
- Fraud
- Treachery
#18, DaveChild, United Kingdom, 24 November 2010. Reply to this.
Or how about a more time-focused list?
- Urgent
- Quick
- Normal
- Eventual
#19, DaveChild, United Kingdom, 24 November 2010. Reply to this.
The difficulty you/I/everyone has to cope with is that there are at least two dimensions to the problem of prioritization: importance and urgency.
#20, aptiv, 26 January 2011. Reply to this.
It seems to me that everyone agrees, important and urgent first. The debate is which is a higher priority those that are urgent but not important, or those that are important but not urgent.
Although I see some good suggestions, I am with you on the time scale. This forces you to choose the level of importance in terms of urgency.
#21, Ann Michelle Marks, USA, 31 May 2011. Reply to this.
I stumbled upon ur site here and noticed this page, im no IT guy or anything, im just building my own website, and i thought i might comment on this.
what about this:
1. today (something that affects us now)
2. tomorrow (something that might affect us tomorrow)
3. next week (you get the point...)
4. next month
I dunno its just a thought.
Replies: #23.
#22, Paul, Canada, 21 July 2011. Reply to this.
Like #22 said, this is also the system used by lazymeter and works quite wel.
#23, gnur, Netherlands, 12 August 2011. Reply to this.
@gnur but lazymeter do have a lot of marketing power behind them. i have seen a lot of PPC ads that they are running so they must have a good marketing budget. I suspect they have refined and streamlined their process as time has gone on.
#24, eric, United Kingdom, 2 November 2011. Reply to this.
I agree that urgent becomes a problem when the customer always think something is difficult.
That is just not fair to anyone. Really important issues not reported by the customer and fairly treated dont stand a chance since they are correctly evaluated.
Thanks for sharing.
#25, jhoncena, pakistan, 30 March 2012. Reply to this.