0.9.2 Branch and Beyond
Sunday June 24th, 2001
Chris Blizzard has posted an update on current tree management plans for the 0.9.2 branch and the 0.9.3 trunk. The plan calls for mozilla.org to drop the requirement of firstname.lastname@example.org approval for check ins to the trunk, while continuing it till 0.9.2 is finished on the branch. Chris felt that using 0.9.2 as a stability milestone was a success, partly thanks to the drivers requirement, but mostly due to better self policing by those who were checking in.
#18 Misunderstanding of software development process
Monday June 25th, 2001 1:12 PM
You are replying to this message
The problem with the kind of "analysis" used by MozillaQuest is that it misunderstands the software development process. Many of these bugs that were targetted for the 0.9.2 milestone have in fact been in Mozilla for a long time, as can be easily verified with bugzilla. So the previous milestones had them as well, but just left them unfixed. This hardly means that the new milestone is "more buggy" than the previous one, simply that the team had hoped to fix more of the outstanding bugs for the milestone than they may actually accomplish.
This is quite typical in software development - you target the most critical and/or most visible things first. It's not at all unusual to ship something with "known problems" - it's typical. At some point you have to decide that the existing bugs are of a nature that they do not seriously impair "typical" use of the product and that holding the product back in order to fix them will either be likely to introduce more (and possibly worse) bugs into the product, or seriously delay the product introduction. Obviously you need to get more bugs out with a "final" release than you do with a "beta" but the point is the same. If you think there is a single software development house on the face of this earth that operates any other way then you are living in a dream world.
Unfortunately this kind of statistic has the (false) ring of truth, like Mark Twain's dictum of there being "lies, damn lies, and statistics." If you don't understand how software teams have to triage bugs then you are likely to totally miss the point, as the MozillaQuest author does and as he invites his readers to do.
And this doesn't even address the point that in Bugzilla (as in many -- possibly even most -- bug tracking systems), "enhancements" are lumped in there with "bugs," as are misspelled words in the dialogs and all sorts of other things. In this kind of system, you even have to be careful about making incautious conclusions based on the number of unresolved bugs (not just the number targetted for a particular milestone) -- over time, you can always think of enhancements faster than you can add them, and often you get bugs added into the system that are later found to be problems restricted to earlier releases or to misconfigured software or malfunctioning hardware. This is especially true if your number of users is going up. One example of a more reliable statistic that Asa mentioned is Mean Time Between Failure (MTBF).
Unfortunately most of this kind of distinction will sail over the heads of the general public, who are not familiar with the workings of software development efforts.