Mozilla 0.9.3 Released!
Thursday August 2nd, 2001
mozilla.org today released Mozilla milestone 0.9.3, continuing to work towards a 1.0 release. Preliminary data is showing that 0.9.3 is up to 35% more stable than 0.9.2, thanks to increased focus on stability bugs this milestone. Along with that, the newest incarnation of the Modern has had some additional work since 0.9.2.
#103 Re: Re: Re: Re: Re: Re: bugs? what bugs? i see no
Sunday August 5th, 2001 3:21 PM
You are replying to this message
So you are suggesting that if a product has no critical, highly visible, broken functionality issues but does have 4 trivial, edge case, polish-needed issues that it is worse than that same product a month ago when it had 3 critical, highly visible, broken primary functionality bugs?! You're over simplifying this to the extreme (and so am I to make the point).
You also assume that because the total bug count is growing that the total number of bugs in the product is growing. This is a broken assumption especially in large and complex pieces of software where all bugs are not known on the first day. Feel free to scream 'feature creep' but you're also leaving out that there are scores of major features which did not exist at the beginning of the project and do now. With new code you're likely to have new bugs. There is nothing wrong with that.
In a hypothetical simple product it we knew that a list of 3 critical defects was the complete and definitive list of all existing problems in the product and a month later we had 4 critical defects in that product and we knew that list was the definitive list of all existing problems then I would agree that a change from 3 to 4 would be a bad sign. If the 4 defects were the same severity as the previos 3 then this would suggest that the product was regressing rather than progressing.
But this isn't the case. We do not and have never had a complete and definitive list of all of the problems in the product. No project of this size ever does. We are, however, getting closer to knowing most of the important problems.
To go back to my earlier example, if our simple project was a week old and we knew of 3 bugs which had existed since the beginning of the product and a day later we discovered a 4th bug which had also existed since the beginning of the product it cannot be said that our quality is worse because of the discovery. In this case our quality would not have changed at all, we would just have a better understanding of it.
In the same example, if we had one feature with 3 bugs and we added another feature of equal complexity and value and this new feature had 2 bugs bringing our total to 5 bugs it isn't fair to say that the product has regressed.
In a project this large it is probably impossible to know all defects in the product. I believe with the 15,000 active bugzilla accounts and the hundreds of thousands of Milestone and nightly downloads which result in 100 to 300 new bug reports a day that we will get as close to knowing all bugs as is possible in a project this size.
One year ago I was seeing about 35% of incoming bugs resolved as Duplicates. Six months ago about 40% of the bugs reported on Mozilla were Duplicates. Today it looks like over 50%. I expect to see this trend continue until the vast majority of reported bugs are Duplicates. When we aren't finding any new bugs except regressions then you can say that an increase in the slope is a bad sign. Until then we're actually improving our Quality by identifying more of the problems rather than less. Identification of problems in existing code helps to prevent the addition of problems in new code and helps identify areas of code which are better served with a total rewrite rather than lots of smaller patching. The closer we approach all known issues the better off we are. Mozilla has been wildly successful compared to all other major open source projects (and probably most closed projects) when it comes to broad testing and issue reporting. We could have just as well not opened Bugzilla to the world and would probably have about half of the open bug reports we currently have but we'd still be just as buggy, probably considerably more so.
We have a higher quality product than we had 2 years ago, 6 months ago, 5 weeks ago.
For example, as a percentage of the not duplicate, invalid and worksforme bugs filed in the first half of 2000 compared to the not duplicate, invalid and worksforme bugs filed in the first half of 2001 the number of blocker, critical and major bugs has dropped and the number of minor, trivial and enhancement bugs has grown. There are more known issues in Mozilla but the severity of those issues is definitely lower than it was. Ad-hoc testing and anectdotal evidence (read slashdot's story on 0.9.3 and compare that to 0.9.2 or 0.8) suggest that product is considerably better than it was a year ago.
Our MTBF has doubled in the last 3 months, you can read about it in the talkback reports posted to the newsgroups. The number of users being affected by our topcrash bugs is plummetting. We have important new features like LDAP addressing, offline mail support, bi-di hebrew and arabic, automatic proxy configuration, and much more that all happened this year. We not only have a much more complete feature set but fewer important bugs in those features and any suggestions that an auto-generated graph of bug counts in our database proves that our product is getting worse is absurd.
If you have specific and concrete suggestions about how we can improve the process of making this product better please send email to <email@example.com> (bugzilla and QA/Testing issues,) <firstname.lastname@example.org> (larger project process issues) or <email@example.com> (code issues) and I'll do my best to see that those suggestions are discussed. Posts to mozillaZine.org discussions full of vague and misleading interpretations of bug data doesn't help make anything better.