Mozilla 0.9.6 Released
Tuesday November 20th, 2001
mozilla.org today made available for download binaries of the Mozilla 0.9.6 Milestone. The builds are available on the releases page, or you can get them directly from the ftp site.
#85 Re: Re: Re: Re: Re: Re: Re: Re: What happened to m
Wednesday November 21st, 2001 11:09 PM
You are replying to this message
> I don't think I've seen a single post that claims that his bug counts are wrong.
I have. I've responded to one this month. When I pointed out the numbers were correct and gave the search links to demonstrate it, the claimant simply fell silent.
> The arguments against Mangelo's methods that have been posted here can be summed up in two words: Sampling error.
That applies somewhat to the overall counts, but it applies considerably less to the bugs that are targeted for specific releases, which have presumably been reviewed. In addition, it is not much of an excuse to say that there are a lot of unclassified, junk, and duplicate bugs in the database, because that means (as you note above) that it is hard for anyone to know what the actual situation is. Actually you said it was hard for outsiders to know, but I doubt there is anyone, inside or out, who has much of an initimate familiarity with some fifteen thousand outstanding bugs. To me this seems to indicate that the bug database is not a very useful tool for understanding progress toward the goal, which is a very dangerous situation -- it's like flying with no instruments. Late in a project's life cycle, the defect lines are the main tools for guiding the project.
Nonetheless, there are some metrics which seem to bear on reviewed bugs, and so are somewhat more reliable, although the actual situation might be worse than these measures indicate because of unreviewed bugs. One is bugs targeted at a specific milestone. One expects some of these to slip, but the slip numbers are large, in the three figures per build. Another is the bounceback rate, that is, the reopened bug line, a standard indicator of overall system stability. That just continues to climb at about a forty-five degree angle. (Actually there is a decline in the last month, but it's within the line's normal wiggle -- still, it may mean that the feature freeze is having some effect.) So the indicators that are more reliable seem to be saying pretty much the same thing as the overall bug numbers, which is that the stability situation is not under control. If I were in a position of QA authority on a project with these metrics, my first priorities would be to get all the bugs reviewed and to do a root-cause analysis to understand the reopened rate. When I suggest those things here, well, you see the kinds of responses I get....