We're not at Beta yet, but with any luck, we'll be far enough along at the M12 release, or soon after, that we're all happy calling it "Mozilla Alpha". How will we know when we're there? Of course, there's the general "I'll know it when I see it" test. But we'd also like to have some more objective Alpha criteria to work with.

So here's our proposal for Mozilla Alpha release criteria. Let us know what you think. We're especially interested in whether you believe these are appropriate and adequate criteria, or whether we should use other and better metrics. And please tell us anything else important that we haven't thought of so far.

We think there are three Mozilla Alpha criteria: Quality, Architecture, and User Acceptance.

1. Quality: Mean Time Between Failures should be at least 1 hour. The MTBF for the M11 release was 1.09 hours according to talkback reports (the "fullcircle" in (LINK) means that build reports crash events via a "talkback" component).

2. Architecture: Alpha should be "almost architecturally complete". "Almost" is a recognition that there will be some exceptions; it's not an excuse for large design holes. "Architecturally complete" means that all public XPIDL interfaces have been reviewed for correctness, completeness and aesthetics; and have then been blessed by mozilla.org.

3. Acceptance: There's a consensus in the Mozilla community that M12 is usable. Ideally, a majority of M12 users will try to live in Mozilla as their primary browser and mail program, ane restart it when it crashes. We don't want to set an unrealistic goal just yet, but we will measure and poll after M12 is out, in order to decide whether it was in fact "usable".

QUALITY

We hope to talkback-enable nightly M12 candidate builds, and we can also get MTBF numbers by running browser buster (see LINK). More on this next week.

ARCHITECTURE

To track architectural completeness (2), we've set up a bugzilla product called Architecture, and filed bugs on all the public XPIDL files we could find. We excluded the DOM IDL files because they're public already, thanks to the W3C. We also excluded private XPIDL files whose names begin "nsPI". Here's a bugzilla query for all open Architecture bugs:

(http://bugzilla.mozilla.org/ buglist.cgi? bug_status=NEW& bug_status=ASSIGNED& bug_status=REOPENED& product=Architecture)

If you click on a bug in the query response, say bug 19769 (LINK), you'll see that its URL field is an lxr link, e.g.:

(http://lxr.mozilla.org/ seamonkey/ source/ netwerk/ base/ public/ nsIChannel.idl)

Follow that link, and you'll see a pretty picture showing the inheritance hierarchy of the interface. At the bottom of the picture, just above the usual lxr-generated source listing, you'll see a link to generated interface documentation, in the example:

(http://unstable.elemental.com/ mozilla/ build/ latest/ mozilla/ netwerk/ dox/ class_nsIChannel.html)

The pretty picture and generated documentation come from Brian W. Bramlett's (bramlett@elemental.com) site, which uses Doxygen (LINK) to cobble up documentation from source code syntax and Java-style "doc comments" such as those used by convention in Mozilla XPIDL.

We've assigned all Architecture bugs to nobody@mozilla.org to avoid spamming module owners. We'd prefer it if owners looked over the architecture bugs for their interfaces and reassigned the bugs to themselves or peers, with Status Whiteboard keywords such as PUBLIC or PRIVATE.

In due course, after everyone has had a chance to review Architecture bugs, we'll rename all PRIVATE interfaces to begin with "nsPI" rather than "nsI", and we'll freeze PUBLIC interfaces, including their IIDs. What does "freeze" mean? We aren't sure, but it could mean holding CVS locks on the .idl files, to prevent accidental or malicious changes to the interface's IID or method names and signatures.

(It's vital to binary compatibility of components, and to component based software in general, to avoid changing IIDs and interfaces once binary components have been distributed far and wide.)

We encourage debate in the mozilla.* newsgroups about Architecture bugs, but we want to capture the essence of newsgroup discussions in the bug system, for posterity, easy querying, status tracking, and owner accountability.

ACCEPTANCE

Finally, before we go measure Mozilla user acceptance (3), we want your votes for bugs to fix by M12. For example, bug 17325 (LINK) has 51 votes, and because it impairs usability for so many members of the community, we intend to see that it's fixed by M12.

On the other hand, bug 3013 (LINK) is very old, yet a little too forward-looking (it's about transparency) to be required for a "usable alpha". We want this bug to be fixed, but we wouldn't hold up a Mozilla Alpha over it.

Again, let us know what you think of all this. Thanks,