Tuesday, September 23, 2008

What's a defect? What's a missing feature?

I was reading this about a nice idea some bloke had about developers fixing bugs in their spare time. Most of the serious developers I know are already working in their spare time- let's not make all defects their problem too.

Many software testers cause great frustration among software developers.  One of the biggest issues that arises on agile projects is that testers have a hard time distinguishing between defects and features that have not yet been implemented.

It's always a bit of a challenge to deal with bugs- even outside of agile.  What to some people is an obvious bug is a feature that was never requested to someone else.

For example, I was called into a meeting today where a bug that had been discovered during a user demo. The search was not "not finding telephone numbers in documents". The search term entered was 5551212. Some documents contained 555 1212. Some contained 555-1212. Some contained 1(703)555-1212.

They tried 555*1212, still not working. Search is broken. Developer suggests- try "555 1212", magic happens. * matches any character, not word boundaries...

It was an obvious problem...to the developers who understood that searching for "breakup" was not going to match documents containing "break up".  With text, it's obvious, but I can certainly see where people might not see the issue with phone numbers.

We'll add normalized versions of phone numbers into the search index and we'll normalize search terms that look like phone numbers or something like that, but...this is not an insignificant effort. (Even though there is some decent code out there to handle it.) There are trade-offs in performance that have to be considered.  Ask a tester though- they'll say found a bug; it's their job, and they want to be able to show how good they are at finding them.  If it's a bug, we can't even mark our existing search work as complete.  If it's a missing feature, we have to allocate it to the next release.  Fortunately, a tester didn't find this one, so we don't quite have that problem, but the users do want the feature of being able to match many different formats of phone numbers.

It seems like a simple semantic difference, and it seems like developers are being too sensitive, but it's actually a big deal.  Some "bugs" might actually end up costing a huge amount of money and not be worth fixing- particularly if they aren't really bugs.  I have seen teams show excessive deference to testers and spend a time equal to the time spent working on the basic features to handle some edge cases that would never really occur and were better handled by error messages than by trying to do something useful. Meanwhile, the project sponsor is seldom asked to decide whether a bug should be fixed- few are even estimated as to the cost.

I say- if you want to fix a bug, you have to pay up. If you are really smart, you do do the five whys and find the root cause, but if it happens to be something that it never occured to anyone to ask for, try not to ask the developers to take care of it on their own time.

blog comments powered by Disqus