Friday December 14th, 2001
As you may have noticed, we've taken the forums down, as we are hitting our bandwidth limit with them. We're working on getting them up and running again as soon as possible.
#22 Re: Re: Re: Rating
Sunday December 16th, 2001 11:59 AM
You are replying to this message
> Keep in mind that not every program is available on the daily basis as Mozilla is. > I'm sure that in the development process many programs have prominent features > broken by new changes -- we just don't get to SEE them because most companies > don't release their programs in such a way.
I'm going on the basis of my own professional experience over the last twenty years, which has included many years of experience in alpha and beta cycles of software with nightly builds. If major features are dropping in and out on a nightly basis, you're nowhere near the stability that will be required to ship the software. It's characteristic of pre-alpha or early-to-mid-alpha software. Beta software needs to be highly solid -- very few changes should cause regressions. Instead, Mozilla experiences changes that cause major regressions every two or three days at the most. That means current schedule estimates which predict only about another four months in beta are probably optimistic, since alpha is stretching over more than a year, and that some process changes will be required in order to get this very complex system to market at all.
> I do agree, though, that modificiations need to be evaluated THOROUGHLY before being checked in.
Which, of course, is very hard to do for software with such a huge feature set. Developers don't have time to check each of a hundred web pages and a hundred user-accessible features for every change. This is where automatic testing could become critical. Changes could be made on a submission basis, and then run through automatic testing before being approved for checkin. Of course, this itself will take time and money -- there would need to be a fairly large server farm, or else the automatic testing would become a process bottleneck, and creation of the automated tests would be labor-intensive. But without it, the software may never attain stability.
> Another thing -- you? Too positive? ;-)
Sometimes, yes. I was recently quite conciliatory on the issue of the software having made a lot of progress, based on my use of the 0.9.6 milestone build, and I was willing to go along with Asa on the issue of forward progress being constant. When people in the forums started observing, almost unanimously, that the builds had been crap for weeks, then I realized I was being overly positive. Mozilla seems to take one step back for every two steps forward, and the ratio may be worse than that. Destabilizing changes that set back the program for weeks still seem to happen once or twice a month, and there's no apparent end in sight to these "crash landings." That's not really a recipe for forward progress.
Again, please bear in mind that I am noting these risks and raising these issues because I think mitigating the risks and addressing the issues is critical to the success of this program. It is unfortunate that some people seem to be able to see expressed concerns and potential solutions only as nay-saying. This sort of feedback is critical to the success of any large software program. Cheerleading approaches that casually dismiss concerns and risks lead to massive crash-and-burns like Apple's Copland -- which during its late builds, was in a similar state of stability as Mozilla is today.