A trail of ants came marching though my kitchen last week. I found the hole they came in and plugged it up. There, that’s sorted them. Two days later they were back, this time through a different hole. Bugs – I hate them. Just like software.
There are probably as many different approaches to software bugs as there are software applications. And none of them are perfect – I can’t think of a single application or OS that hasn’t crashed on me at least once over the years. So given that bugs are a reality in most (all?) software products, what is the best approach to reducing their impact once the product has been released? As a Product Manager over the past 10+ years, I’ve been involved in 20 to 30 major releases and probably twice as many minor ones across a variety of products and platforms. They all had one thing in common – within 2 weeks of release, at least one customer reported a bug that, had we known about it before release, we would have fixed before it went out.
At one company I was at, the policy was to “test, test, test” with the QA department somewhat removed from the R&D department. A huge battery of mostly manual tests was run on every release. If a single line of code was changed, all had to be run again, on every combination of OS and platform supported. Each round of testing took a minimum of a week with every developer pulled off coding to help. Finally, the product was released and, inevitably, a bug report would come in almost straight away.. But, because of the extended release cycle, fixing that bug would take all of the developers and testers time for two weeks, during which time other bug reports would come in. In this way, four months pass before a patch was released, during which time no progress has been made on the next major release. This starts a vicious cycle where at the release post mortem, the QA department insists on more testing for the next version, so that release gets delayed even longer and when the inevitable bug reports come in, an even longer delay before that patch is released.
At MacVector, Inc, we’ve taken a more streamlined approach. We certainly do a lot of testing, but we rely on a much tighter integration between the QA and R+D departments to only retest areas that the developer believes would have been affected by the bug fix. That reduces our bug fix development cycle time dramatically, so we can have patch releases posted on the web site within a week of the initial report. The risks, of course, are those “Whack-a-Mole” bug fixes where the code to fix one bug reveals another bug elsewhere in the application. But if that happens, we know we can turn around a second bug fix in a week or so, not the six months that my old company took.
So the take home lessons are (a) check back to the MacVector web site often to pick up any new patches and (b) I really need to find out where those ants are coming from…