Now that WebKit has its own web site on OpenDarwin, it's time for this blog to change. For starters, Surfin' Safari has now moved to here. Another big change is that I will no longer be the only person talking to you about Safari and WebKit changes. Some of the other members of the team will be posting about what they're working on as well.

I will leave this blog up here so that all the links remain valid, but all subsequent posts will be to the new blog. Update your bookmarks. :)

Nokia uses WebCore in a new mobile browser for the Series 60 platform.

We've already received and committed several patches from external contributors and the repository has only been live for a few hours!

As some of you have already noticed (those of you that built), the new Web Kit not only passes Acid2, but it's also substantially faster at loading Web pages and at handling JavaScript. It contains a number of additional performance improvements that went in post-Tiger.

One question people have asked is "Does this have to replace my system frameworks?" The answer is "No." You can run this custom version of Web Kit with a particular instance of Safari without replacing your system frameworks. The run-safari script we provided does this for you. If you study what it does, you'll see that you can easily try out your own WebKit apps with the new frameworks as well. We in fact encourage you to do this so that you can make sure your apps are functioning properly with the latest WebKit.

As some of you may have heard at WWDC Monday, the Safari team is proud to announce that we are making significant changes in the way we operate, and these changes start today.

Here is what we are launching:

1. webkit.opendarwin.org, the new web site for WebKit, WebCore and JavaScriptCore.
2. Full CVS access to WebCore and JavaScriptCore, our frameworks based on khtml and kjs. This repository includes the complete history of the project, so all patches past and present can be viewed.
3. WebKit, the Objective-C API that wraps WebCore, is also being open sourced. It is in the same CVS repository.
4. This repository is live. You can pull and build it today. As we improve the frameworks you can pull and run the latest and greatest. If you want to run a version of Safari that passes the Acid2 test, now you can.
5. This repository is open. We welcome contributions.
6. From now on bugs in these frameworks will be tracked in public at bugzilla.opendarwin.org. You can submit bugs in the open, view the status of our work, attach patches to bugs, and test code fixes to those bugs.
7. A new public mailing list, webkit-dev@opendarwin.org, for development discussion of WebKit, WebCore, and JavaScriptCore.
8. A new public IRC channel - #webkit on irc.freenode.net.

And finally, going forward we will be engaging actively with the community. Find us on IRC and on the mailing list, jump in, and get involved!

Build. Run. Test. It's live!

One of the most interesting problems (to me at least) in browser layout engines is how to implement a style system that can determine the style information for elements on a page efficiently. I worked on this extensively in the Gecko layout engine during my time at AOL and I've also done a lot of work on it for WebCore at Apple. My ideal implementation would actually be a hybrid of the two systems, since some of the optimizations I've done exist only in one engine or the other.

When dealing with style information like font size or text color, you have both the concept of back end information, what was specified in the style rule, and the concept of front end information, the computed result that you'll actually use when rendering. The interesting problem is how to compute this front end information for a given element efficiently.

Back end information can be specified in two different ways. It can either be specified using CSS syntax, whether in a stylesheet or in an inline style attribute on the element itself, or it is implicitly present because another attribute on the element specified presentational information. An example of such an attribute would be the color attribute on the font tag. Both WebCore and Gecko use the term mapped attribute to describe an attribute whose value (or even mere presence) maps to some implicit style declaration.

A rule in CSS consists of two pieces. There is the selector, that bit of information that says under what conditions the rule should match a given element, and there is the declaration, a list of property/value pairs that should be applied to the element should the selector be matched.

All back end information can ultimately be thought of as supplying a declaration. A normal rule in a stylesheet that is matched has the declaration specified as part of the rule. An inline style attribute on an element has no selector and is simply a declaration that always applies to that element. Similarly each individual mapped attribute (like the color and face attributes on the font tag) can be thought of as supplying a declaration as well.

Therefore the process of computing the style information for an element can be broken down into two phases. The first phase is to determine what set of declarations apply to an element. Once that back end information has been determined, the second phase is to take that back end information and quickly determine the information that should be used when rendering.

WebCore (in upcoming Safari releases) has a really cool optimization that I came up with to avoid even having to compute the set of declarations that apply to an element. This optimization in practice results in not even having to match style for about 60% of the elements on your page.

The idea behind the optimization is to recognize when two elements in a page are going to have the same style through DOM (and other state) inspection and to simply share the front end style information between those two elements whenever possible.

There are a number of conditions that must be met in order for this sharing to be possible:
(1) The elements must be in the same mouse state (e.g., one can't be in :hover while the other isn't)
(2) Neither element should have an id
(3) The tag names should match
(4) The class attributes should match
(5) The set of mapped attributes must be identical
(6) The link states must match
(7) The focus states must match
(8) Neither element should be affected by attribute selectors, where affected is defined as having any selector match that uses an attribute selector in any position within the selector at all
(9) There must be no inline style attribute on the elements
(10) There must be no sibling selectors in use at all. WebCore simply throws a global switch when any sibling selector is encountered and disables style sharing for the entire document when they are present. This includes the + selector and selectors like :first-child and :last-child.

The algorithm to locate a shared style then goes something like this. You walk through your previous siblings and for each one see if the above 10 conditions are met. If you find a match, then simply share your style information with the other element. Such a system obviously assumes a reference counting model for your front end style information.

Where this optimization kicks into high gear, however, is that it doesn't have to give up if no siblings can be located. Because the detection of identical style contexts is essentially O(1), nothing more than a straight pointer comparison, you can easily look for cousins of your element and still share style with those elements.

The way this works is that if you can't locate a sibling, you can go up to a parent element and attempt to find a sibling or cousin of the parent element that has the same style pointer. If you find such an element, you can then drill back down into its children and attempt to find a match.

This means that for HTML like the following:

<table>
<tr class='row'>
<td class='cell' width=300 nowrap>Cell One</td>
</tr>
<tr class='row'>
<td class='cell' width=300 nowrap>Cell Two</td>
</tr>

In the above example, not only do the two rows share the same style information, but the two cells do as well. This optimization works extremely well for both old-school HTML (in which many deprecated presentational tags are used) and newer HTML (in which class attributes might figure more prominently).

Once the engine determines that a style can't be shared, i.e., that no pre-existing front end style pointer is available, then it's time to figure out the set of declarations that match a given element. It is obvious that for inline style attributes and mapped attributes that you can find the corresponding declaration quickly. The inline style declaration can be owned by the element, and the mapped attributes can be kept in a document-level hash. WebCore has a bit of an edge over Gecko here in that it treats each individual mapped attribute on an element as a separate declaration, whereas Gecko hashes all of the mapped attributes on an element as a single "rule." This means that Gecko will not be able to share the mapped attribute declaration for the following two elements:

<img width=300 border=0>
<img width=500 border=0>

WebCore creates three unique declarations and hashes them, one for a width of 300, one for a width of 500, and one for a border of 0. Gecko creates two different "rules," one for (width=300,border=0) and another for (width=500,border=0). As you can see in such a system, you will frequently not be able to treat the identical border attributes as the same.

Aside from this difference in mapped attribute handling, the two engines employ a similar optimization for quickly determining matching stylesheet rules called rule filtering. All rules that are potentially matchable by any element (i.e., that have the correct media type) are hashed based on the contents of the rightmost simple selector in the rule.

A selector in CSS can be either simple (meaning that all of the contents of that selector apply only to a single element) or compound (meaning that you may examine multiple elements like parents or siblings of that element). A compound selector is essentially a chain of simple selectors, so the following rule:

tr > td { color: blue }

has two simple selectors, tr and td. The rightmost simple selector in the rule is the one that we will use for the rule filtering optimization.

The rightmost simple selector falls into four categories.

(1) The selector uses an ID. (Example: #foo)
(2) The selector doesn't have an ID but uses a class. (Example: .foo)
(3) The selector has no class or ID but specifies a tag name. (Example: div)
(4) The selector specifies none of these things. (Example: *[disabled])

The rule is placed into one of four hashtables depending on which category it falls into. The idea behind these categorizations is to always filter out more specific information first. For example, if an element has a specific ID, then obviously any rules whose rightmost selector uses a different ID cannot match. Technically the last category can just be a list and not a hashtable, since those rules must always be examined by all elements.

Each hashtable, therefore, consists of a mapping from a given atomic string to a set of rules that match. The class attribute is exceptional in that you must put the rule into the hashtable multiple times if multiple class attributes are used.

When determining the set of rules that match a given element, you only examine rules that correspond to the correct hash entry based off your ID, classes and tag name. This optimization basically eliminates 95+% of the rules up front so that they need not even be considered during the matching process.

Each rule is then examined in detail, with all selectors being checked, to determine if it is a match, and the set of matches is collected. The set of matches can then be sorted by priority and specificity such that all the declarations are in the proper application order.

This brings us to the final phase of the style computation, which is taking the set of matches and quickly computing the appropriate front end style information. It is here that Gecko really shines. What I implemented in Gecko was a data structure called the rule tree for efficient storing of cached style information that can be shared *even when* two elements are not necessarily the same.

The idea behind the rule tree is as follows. You can think of the universe of possible rules in your document as an alphabet and the set of rules that are matched by an element as a given input word. For example, imagine that you had 26 rules in a stylesheet and you labeled them A-Z. One element might match three rules in the sheet, thus forming the input word "C-A-T" or another might form the input word "D-O-G."

There are several important observations one can make once you formulate the problem this way. The first is that words that are prefixes of a larger word will end up applying the same set of rules. All additional letters in the word do is result in the application of more declarations. Thus the rule tree is effectively a lexicographic tree of nodes, with each node in a tree being created lazily as you walk the tree spelling out a given word.

This system allows you to cache style information at each node in the tree. This means that once you've looked up the word "C-A-T-E-R-W-A-U-L", and cached information at all of the nodes, then looking up the word "C-A-T" becomes more efficient.

In order to make the caching efficient, properties can be grouped into categories, with the primary criterion for categorization being whether the property inherits by default. It's also important to group properties together that would logically be specified together, so that when a fault occurs and you have to make a copy of a given struct, you do so knowing that the other values in the struct were probably going to be different anyway.

Once you have the properties grouped into categories like the border struct or the background struct, then you can either store these structs in the rule tree or as part of a style tree that more or less matches the structure of the document. Inheritance has to apply down the style tree and tends to force a fault, whereas non-inherited properties can usually be cached in the rule tree for easy access.

WebCore doesn't contain a rule tree, but it is smart enough to refcount the structs and share them as long as no properties have been set in the struct. In practice this works pretty well but is not as ideal as the rule tree solution.

KHTML developers respond to my posting of the WebCore Acid2 patches here and here.

For what it's worth, the patches I posted are to WebCore, which consists of both KHTML and KWQ (our port of Qt). They are posted to illustrate all the WebCore bugs that had to be fixed in Safari to pass the Acid2 test. They are not solely KHTML patches. The antialiasing bug was in KWQ, and so doesn't even apply to KHTML. The better object element support necessarily involves KWQ as well, since the plugin code is (obviously) platform-specific.

What do you think Apple could be doing better here? Comment or trackback. I'll read it all.

Safari now passes the Acid2 test. There were two issues left that needed to be resolved.

The first issue involved implementing a few enhancements to the object element. I needed to support fallback content when invalid MIME types were specified or when bad status codes were returned for HTTP requests (like 404). After fixing these bugs and a couple of other problems with intrinsic sizing of plugins, the eyes of the face showed up.

The second issue involved improper antialiasing of the border corners. Antialiasing was enabled for the drawing of the corner polygons, and this resulted in a bleed-through of the background. Because the two corners were drawn separately, the antialiasing was actually incorrect, since it was disrupting the join of the corners by letting the background show.

Here are the patches for all of the problems fixed in Safari to make the test pass.

Fix parsing of the REL attribute on links.

Disallow TABLE inside P in strict mode.

Add support for min/max-width/height for positioned elements.

Fix the rendering glitch that causes the reference page to paint garbage.

Make sure that percentages that go to auto don't mess up the self-collapsing block check.

Implement SGML-style comment parsing for HTML in strict mode.

Make sure empty tables honor CSS-specified height in strict mode.

Fix baseline alignment within table cells to use the bottom of empty blocks. Fix floats to not grow if child floats overhang but the height of the outer float is auto.

Make sure percentage min-height goes to 0 and not auto when the percentage does not apply.

Implement fallback content for the object element and fix intrinsic sizing to work properly when images are specified in the object element.

Disable antialiasing for the drawing of polygons.

Victory!

The Acid2 test has been updated to version 1.1 in order to fix the bug I outlined in my previous blog entry. Here is how the Safari build with all of my fixes renders version 1.1. As you can see now we're just down to better object element handling.

Astute viewers pointed out that there was still a rendering glitch in row ten. It turns out this was actually a problem with rows six to nine. The block above ended up being too tall because min-height specified using a percentage was resolving to auto instead of to 0 when the containing block had no specified height. I fixed this problem, but check out the rendering now.

Observe how the smile is now positioned too high relative to the reference rendering. I spent a very long time investigating this problem and determined that it is in fact a bug in the test. At this point I am halting work on Acid2 until a revised test has been posted.

The purpose of this blog entry is for you to track back to it with regressions you have discovered going from 1.2 to 1.3. It would be especially helpful if you can test up front for whether this is a user agent bug (by spoofing as another browser), since changes in browser version numbers often cause regressions even when nothing is wrong with the browser itself.

The more clear and concise the reduction, the better the chances are that we can address the issue quickly (thus increasing the odds it can make it into a software update sooner rather than later).

Please include in these trackbacks only regressions from 1.2. If you included something in the earlier blog entry's comments section, please post it again as a trackback. Thanks!

(1) The feed URL dialog that tells you 10.4 must be installed to view RSS feeds is simply a bug and not part of a master plan for global domination.
(2) The View Source shortcut was changed to match Mail.app.
(3) The default bookmarks reappearing after being removed won't happen going forward now that the way this is handled has been changed. See (1) above re: global domination.
(4) The selection extends to the edges of lines in the new Safari just as it does in other Mac apps like TextEdit. This change had to be made so that editing selection would behave like NSTextView. It was a challenge translating this to the Web space, but I will blog more about this in a future entry.
(5) When saving links to the desktop from the context menu, you can hold down Option to change the menu item so that you can pick a location.

Even though I consider row 14 to be ambiguous, I went ahead and modified the Safari code to yield the correct expected behavior. It isn't so much that the test is wrong as that it is testing unspecified behavior.

I also noticed thanks to Ian Hickson that the smile was not in fact rendering correctly. The reason is that Safari will expand floats to encompass overhanging child floats even when the outer float has specified a height explicitly in CSS. I changed the code so that this is now only done when height is auto, and the smile now renders as it should.

Updated screenshot below. All that I have left is fixing the object element's behavior in order to completely pass the test.

Those of you running Panther can now update to 10.3.9. This update includes Safari 1.3 and new versions of WebKit, WebCore, and JavaScriptCore that contain thousands of improvements we've made to the engine since Safari 1.2.

What you are getting is all of the new standards support, new WebKit capabilites, site compatibility fixes and performance optimizations that are also present in Safari 2.0 for Tiger. The layout engines for the two are virtually identical.

Here are some of the highlights:

Page Load Performance
Safari 1.3 loads pages overall 35% faster than 1.2 as measured by IBench. In addition to improving the overall page load, Safari 1.3 will display content sooner than 1.2 did, so that subresources don't hold up the initial display of the page.

JavaScript Performance
We have substantially improved the performance of the JavaScript engine in Safari. I encourage you to check out Safari 1.3 on this benchmark for example to see the improvement relative to 1.2.

HTML Editing
Safari 1.3 supports HTML editing, both at the Objective-C WebKit API level and using contenteditable and designMode in a Web page. The new Mail app in Tiger uses WebKit for message composition. You can write apps that make use of WebKit's editing technology and deploy them on Panther and Tiger.

Compatibility and Security
Compatibility and security are our number one priority in WebCore, and Safari 1.3 has many important compatibility fixes. For example, percentage heights on blocks, tables and cells now work much better in Safari 1.3. min/max-width/height support has been added. More of the table-related CSS properties are now supported. DOM methods like getComputedStyle are now supported.

The DOM Exposed
The entire level 2 DOM has been exposed a public API in Objective-C. This means various holes have been filled in Safari's DOM level 2 support. In addition to exposing the DOM to Objective-C, the JS objects that wrap DOM objects can also be accessed from Objective-C, allowing you to examine and edit the JS objects themselves to inject properties onto them that can then be accessed from your Web page.

XSLT
Safari 1.3 on Panther now supports XSLT. 10.3.9 includes libxslt, and Safari uses this excellent library to handle XSLT processing instructions it encounters in Web pages.

Plugin Extensions
For those of you writing WebKit apps, a new Objective-C WebKit plugin API is supported that lets you put Cocoa widgetry into the Web page more easily. In addition enhancements to the Netscape Plugin API (made in conjunction with Mozilla Foundation) have been implemented for plugins that require cross-browser compatibility.

Did I mention it's really really fast? :)

In case you're curious about differences between the Tiger and Panther versions of the engine, they mostly have to deal with frameworks that changed underneath WebKit. For example we have new faster image decoders on Tiger (that also handle PNGs correctly), so you'll find that Tiger fixes some of the PNG gamma issues that will still exist on Panther. In addition the new decoders are incredibly fast and are now run on a separate thread on multi-processor machines on Tiger.

The network layer has also been improved on Tiger, so this may be another source of differences in behavior between the two operating systems. Overall, however, it's likely that content and applications you develop with WebKit will behave identically on the two operating systems.

Let us know what you think.

I believe row 14 is ambiguous and needs to be amended. Safari makes this row too tall for the same reasons Firefox does.

See https://bugzilla.mozilla.org/show_bug.cgi?id=289480#c14 for more details.

Earlier I asserted that Safari passed rows 6-9. Now I'm not so sure. As someone in the comments pointed out, Safari has a 1px golden ring around the black nose that is not there in the reference rendering. I will have to figure out what causes this to see if it's a bug in Safari.

Ok, I now pass row thirteen. In strict mode I just made comment parsing flip a canClose bit as it encountered pairs of --. For an even number of --, you can close when you encounter a >. That fixes the rendering. I'm not sure this is correct, but hopefully it is. :)

I must admit that I am completely baffled by row thirteen. The row includes a comment that Safari appears to be handling incorrectly, but I have no idea why. There is apparently something I don't know about comment handling. Can someone from WaSP clarify and explain what the heck I'm supposed to do with this line?

<!-- -- --->ERROR<!- ------ ></div></div> <!-- two dashes is what delimits a comment, so the text "->ERROR<!-" earlier on this line is actually part of a comment -->

This particular comment test seems particularly esoteric and pointless to me. It's clear that I'll have to limit this parsing rule to strict mode anyway, since I doubt other browsers make you count hyphens when constructing comments.

I'm a bit perplexed that a test that is supposedly testing features that WaSP considers "essential" for the future Web contains data URLs and esoteric SGML comment rule tests. What about inline-block? I consider that far more essential than whether or not a browser can handle some obscure comment parsing rule.

I spent some time studying rows four and five of the test and realized it would take a while to fix. Those rows are testing using the object element without specifying a MIME type and making sure that if the types are unsupported that you fallback to nested objects. I have ideas for how to fix this, but I decided to save this part of the test for last.

Instead I focused next on why there was an odd six pixel gap between rows nine and ten in Safari. It turns out that my check for whether or not a block could collapse its own margins together was missing a case, and the Acid2 test had a nice comment explaining what I was missing.

.empty { margin: 6.25em; height: 10%; /* computes to auto which makes it empty per 8.3.1:7 (own margins) */ }

Basically I needed to improve my self-collapsing block code to be smarter about noticing when percentage heights end up turning into auto. Once I fixed that, the gap went away. Below is the current rendering after the bug fix.

I also noticed that the test and the reference rendering apply overflow:hidden to a root element and expect that to be applied to the viewport. Safari 2.0 does this for HTML documents with the body element but did not ever do this with the root element. I fixed Safari to actually behave according to the latest internal draft of CSS2.1 and now the documents don't have scrollbars.

I have finally completed the implementation of the CSS properties min-width, max-width, min-height and max-height in WebCore. Safari 2.0 (and Safari 1.3) both support these properties on everything but positioned elements. It figures then that the use of these properties on the Acid2 test would occur on positioned elements (thanks Hixie!).

Included below is a screenshot of the rendering in Safari with this bug fixed.

As you can see, rows 1-3 and rows 6-9 all render correctly. I'm going to work on rows 4 and 5 next and see if I can get the eyes of the face to show up.

The Web standards project has released the Acid2 test for Web browsers. It is a pretty crafty HTML+CSS test designed to ensure that browsers are properly implementing support for those standards.

Every browser fails it spectacularly. :)

I started work today on making Safari pass the test, and I thought I'd blog my progress as I fix bugs in the test. This will be a fairly slow process as whole features may have to be added simply to make one row of the test render correctly.

For those using Safari, I have fixed two bugs so far. The ugly red background that covers the test is caused by a bug in Safari's parsing of the rel attribute on links. Safari had some hard-coded checks for keywords rather than doing proper space-based tokenization of the attribute. I fixed this and now the red background is gone.

Row one in Safari mis-renders for two reasons. The first is that the test wants to make sure table elements are not allowed inside paragraph elements in strict mode. Safari did not have a tight check here, and allowed tables inside paragraphs in all modes. You have to allow this in quirks mode because other browsers do, but I went ahead and tightened the check for strict mode. With this bug fixed, the first row's maroon line became black as it should.

The second problem is that the first row's black line is the wrong height and the wrong width. This is because Safari 2.0 supports min/max-width/height on everything but positioned elements. I will have to add this support to make row one finally render correctly, so that's what I'll be working on next.

Sometimes trying to support the standards can be a real pain.

While trying to improve our CSS2 compliance, I recently did a big cleanup of our block layout code, including the code for handling floats. I made what I believed to be a fairly innocuous correction to follow the CSS2 specification. Here's the scenario.

Lets say you have a div that is set to 300 pixels in CSS. You then put a 250 pixel wide float inside that div. Immediately after that you have a 100 pixel wide overflow:hidden div. All sizes have been specified in CSS.

Now here's the pop quiz. What do you think the layout should be? Should the overflow div:
(a) Be on the same line with the float and spill out of the enclosing 300 pixel div
(b) Be placed underneath the float, automatically clearing it because there is insufficient space for
the overflow div next to the float

Before I give an answer, lets see what the CSS specification has to say on this issue. Section 9.5 on floats, fifth paragraph.

"The margin box of a table or an element in the normal flow that establishes a new block formatting context (such as an element with 'overflow' other than 'visible') must not overlap any floats in the same block formatting context as the element itself. If necessary, implementations should clear the said element by placing it below any preceding floats, but may place it adjacent to such floats if there is sufficient space."

My interpretation of this language is that there must be sufficient space for the table or overflow:hidden element to fit within the containing block. If not, you should clear. That's what I implemented. So in my opinion the correct answer to the question above is (b).

I decided to see what other browsers did. I started with Gecko. Gecko chose (a). Gecko always does (a). It is at least consistent if - in my humble opinion - incorrect. Gecko chooses (a) regardless of whether you pick strict, almost strict or quirks mode.

Next I tried WinIE, and this is the part that blew my mind. Depending on whether the float was an image or a table, the float was left or right aligned, the table specified that it floated via the align attribute or the float CSS property, and on whether or not the normal flow element was declared as a sibling or not of the float, I could get completely different results! The level of inconsistency was astonishing.

I was able to watch WinIE do clipping in one case, to wrap in a second case, to not wrap in a third case, to overwrite content in a fourth case, all by just tweaking the parameters outlined above. It's no wonder Web designers have no idea how to code a page to standards when they have to deal with a layout engine that is so horribly inconsistent and buggy.

Naively I opted to implement (b) and to hope for the best. Unfortunately the bugs immediately started pouring in. finance.yahoo.com was broken for example because it used an old-style align table and relied on it not wrapping underneath the float. Undaunted, I simply added a strict mode/quirks mode check and opted to do (a) in quirks mode and (b) in strict mode.

The bugs kept coming in though. Next was versiontracker.com, a page that is actually in strict mode and relies on an overflow:hidden div to spill out of a containing block rather than wrapping.

So now I really have no choice. This is an example of where the CSS2 standard simply can't be followed because buggy layout engines have set a bad precedent that the rest of us have no choice but to follow.

It's a shame that Gecko does not do the right thing in strict mode at least, but I suppose they had no choice in the matter either.

Over on webstandards.org I found a link to a guide to z-index according to the CSS2.1 specification. Click here to see this guide.

I mention it because it accuses both the Gecko and KHTML engines of violating the CSS spec, but in reality the author simply does not understand how auto z-index works in CSS2.1.

Stacking contexts are established in three ways in modern browsers:

(1) The root element gets a context.
(2) Elements with opacity < 1.0 establish a stacking context (and a z-index of auto is changed to 0).
(3) A positioned element with a z-index other than auto establishes a stacking context.

In other words, being positioned (absolute/fixed/relative) does not mean you establish a stacking context! You must also have a non-auto z-index.

The normal flow contents of positioned elements with auto z-index are sorted in an enclosing stacking context as though they had z-index 0. Document order breaks ties. This is specified in section 9.9.1 of the CSS2.1 specification.

... a stacking level for positioned descendants with 'z-index: auto', and any descendant stacking contexts with 'z-index: 0'...

So in this example you have 4 blocks, two of them relatively positioned and two absolutely positioned.

The two relative positioned blocks each contain absolute positioned descendants. Because the two RP blocks have the same z-index (auto), they are sorted in document order. RP1 is therefore below RP2. AP1, the absolute positioned block inside RP1, has a z-index of 1. This guarantees it will be above both RP1 and RP2, since it is sorted in the root element's stacking context at a z-index of 1. RP1 and RP2 render just above the normal flow (z-index 0) level of the root stacking context, and therefore will always be below AP1 if it has a positive z-index defined.

Therefore the assertion that AP1 should render below RP2 is false, and Mozilla and Safari have the correct rendering.

In this example, Safari 1.2 on Panther does violate the spec regarding the positioning of AP2. AP2 should render above RP1 and RP2 in the example but below AP1. Internal Safaris actually render this correctly (we fixed this bug a long time ago actually), so future releases of Safari will be correct.

In the next example, both Mozilla and Safari have the correct rendering. The claim that only IE gets it right is false. IE gets it wrong. The RP blocks have a z-index of auto, and so they are below all of the absolute positioned blocks with positive z-indices in the root's stacking context.

The following quote should additionally make this clear. It's also from the CSS2.1 spec, section 9.9.1.

The root element forms the root stacking context. Other stacking contexts are generated by any positioned element (including relatively positioned elements) having a computed value of 'z-index' other than 'auto'.

In other words, when a block has auto z-index, it does not act as a stacking context for other positioned descendants.

So in the next example, Mozilla and Safari are once more correct. Because RP1 now establishes a stacking context, AP1 is sorted only within RP1's stacking context. RP1 is then sorted with AP2 in the root's stacking context. RP1 has a z-index of 11 and AP2 has a z-index of 20, so RP1 is below AP2. RP1 and AP2 are both above RP2 of course because RP2 has an auto z-index, and therefore is just above normal flow content in the root stacking context (at the 0 z-index level).

I hope this helps clarify how z-index works in modern browsers. As usual, track back if you have questions.

Some time ago we switched over to libxml in Safari for the processing of XML (and XHTML) files. I'm happy to report that we now have basic XSLT support working in Safari using libxslt. You can style your XML using xml-stylesheet processing instructions. I don't yet have a programmatic JS API working for transforming documents, but that shouldn't be too difficult to add. What I really need are XSLT test cases that use xml-stylesheet. Track back or comment if you know of some good test cases online that I can use, or just generally have suggestions to make regarding XSLT support.

Ian Hickson comments on Apple's HTML extensions and points out that all of the solutions suggested so far are inappropriate. I'll be eagerly awaiting the responses of both Tim and Eric to this entry.

Ian also takes Apple to task for inventing new tags without discussing them in a standards forum first. I think Ian has gotten his wires crossed a bit. For whatever reason, he (and others) are equating a beta release of Tiger with a shipping release of Tiger. Until Apple actually ships Tiger, then we have not yet extended HTML. What you see in the Tiger beta is a proposal. The syntax of these tags is not frozen, as is evidenced by the fact that I'm willing to move them into a completely different namespace! The documentation delivered to Dashboard authors during WWDC even warns that the syntax of these tags has not yet been finalized.

To criticize Apple for not discussing the syntax of tags prior to releasing a piece of beta software is unfair. We are willing to discuss these elements in an open forum like the WHAT-WG. You can see our proposal for canvas and for the new form controls, since we released the complete source of these extensions in WebCore-146, and anyone can download it to view the current APIs. I believe we are being quite open in our process regarding these extensions.

Several people questioned the presentational aspect of the composite attribute. In an earlier blog entry I had mentioned we would like to ultimately standardize our extensions. This attribute is an exception. We are aware that this functionality should be in CSS rather than in the HTML itself.

We didn't want to introduce a new CSS property, however, because how to composite should be specifiable anywhere you use an image in CSS properties. Examples of such properties are content, background-image, and list-style-image. However, normal specification of the foreground URL of an img tag is done in the HTML itself... so there was no consistent solution that could be employed.

Ideally you'd like the composite operation to be part of the image value, just as you might like cropping (indicating a sub-region of the image to use) to be part of the image value, but you are prevented from doing this with the img tag by the fact that a portion of the image value, the URL, is specified in the HTML itself.

It made little sense to invent a new CSS property that would only apply to this tag just to specify the compositing operation when the better way to do this from a purist CSS perspective would be more fundamental, affecting any place where a URL value could be used to represent an image.

I invented -moz-image-region in CSS to solve a similar problem in Mozilla (the sub-region problem), and this property was largely derided by the CSS WG, and so I knew better than to repeat the same mistake with a -khtml-composite property.

Ok, here is what I have implemented to handle the four HTML extensions that Apple has made so far. I implemented Eric Meyer's default namespace proposal.

In HTML, in order for Safari or other Web Kit applications to recognize the new extensions, you must declare a default namespace on your root html element. This namespace's value at the moment is:

http://www.apple.com/2004/xhtml-extended/

Feedback on the above namespace identifier is welcome, if you think there's a better choice. Note that in HTML this namespace support isn't real. The default namespace declaration is all that will be checked, and it will act as a global switch that will just turn on all the HTML extensions.

However, the benefit comes when you switch to real XML. In the XML implementation, the namespace is completely real and effectively maps to a new language, Apple Extended XHTML, that has all the functionality of XHTML plus the extensions. In other words, in XML you can declare the namespace for real, use it with prefixes, use it only on canvas tags to keep the rest of the doc pure XHTML, etc.

I have not yet committed this solution, because I want to see if this meets with public approval first. Please trackback with suggestions and/or amendments.

I received more feedback on our HTML extensions, and some people made some good suggestions for how Safari should handle extensions to HTML. There were essentially three good ideas that were pointed out to me (along with a host of really bad ones).

Tim Bray suggested namespacing the extensions we've made to HTML. Eric Meyer suggested this as well. The idea would be that you could feed your HTML with the namespace declaration to an HTML parser and it would essentially have namespace support and understand how to handle the namespaced content. This is my favorite of the suggestions, since the namespace could effectively be hacked and only allowed on the root element. This seems like a minor cut-and-paste requirement to impose on Dashboard authors that want to use the new tags and attributes.

A second suggestion was to make a special DTD. I don't like this suggestion as much, since doctypes are used for setting browser modes, and I don't want to impose a particular mode on Dashboard widget authors.

A third suggestion was to restrict these tags and attributes only to Dashboard. This seems reasonable on the surface but would be difficult to do in practice, and besides, as I stated before, we actually are submitting these extensions to WHAT-WG for review anyway. This means the intent is for them to find their way into HTML eventually.

I'll look into what it would take to implement the first suggestion. It sounds to me like people will be satisfied with such a solution. I do wonder what to do with the new values to the type attribute on the input element. Search and range are new values to an already-existing attribute, and so I'm not sure how to mollify people on this one. Breaking those out into new attributes not only makes little sense to me, since it allows for a contradictory type clash (by specifying multiple attributes), but it also would complicate the code in WebCore that routinely switches on the type of the input element.

Going forward, I'm curious what the reaction will be as WHAT-WG works to further extend HTML. Assuming that the W3C has really decreed HTML4 to be obsolete, what happens when a proposal is made by multiple browser vendors to extend it? If the W3C rejects it, should the browser vendors be forced to keep their content namespaced forever? I guess we'll cross that bridge when we come to it.

A few people have written me expressing concern over the extensions that Apple has made to HTML in order to support Safari RSS and Dashboard. I wanted to explain what we've done and hopefully clear up any confusion.

Let's start by talking about the contenteditable attribute and drag and drop. I bring these up first because what we implemented is exactly compatible with WinIE. In the case of contenteditable, we have no choice regarding syntax. We have received many bugs to support already-deployed systems that use contenteditable, and so we are constrained syntactically. Had we gone our own route, we still wouldn't work with the Web pages that use it, and it would be unrealistic to expect all of those Web sites to modify their systems simply to support Safari. This is especially true if you consider that Web sites frequently deploy systems that they didn't write in the first place, and so they wouldn't know how to modify them anyway.

Drag and drop is a similar situation. Web sites use it, and so we need to support it. We already support dozens of WinIE-invented properties, many of which are incredibly useful and well-specified, so I'm a bit confused as to why contenteditable and drag and drop are creating any stir at all. These attributes are no different from innerHTML or offsetWidth and offsetHeight or innerText or oncontextmenu or any one of the other WinIE extensions that Safari has supported since its first beta 18 months ago.

We have a phrase we like to use here on the Safari team, and that's "real-world standards compliance." What that means is that where possible we attempt to be fully compatible with the W3C standards, but we also want to support the real-world standards, i.e., extensions that for better or worse have become de facto standards. If you really do believe we should not have implemented contenteditable, then you are simply out of touch with reality.

As for the Dashboard extensions that involve changing HTML, there are exactly four of them. We've tried to keep the number to a minimum, but this functionality was required in order to build the gadgets. Let me outline them again:

(1) Slider controls. This is not only used by Dashboard but also by Safari RSS, and so this feature cannot be restricted only to the Dashboard.
(2) Search fields. Again, this feature is used by Dashboard and Safari RSS.
(3) The new composite attribute on the img tag. This feature is used only by Dashboard.
(4) The canvas tag. This feature is used only by Dashboard.

The principal complaint seems to be that we should not be polluting HTML. However, I'm not sure what we should have done instead. I can outline some of your suggestions and explain why we discarded them.

First, it was suggested that the widgets be written in XML rather than HTML and that all of the new tags and attributes be namespaced. However, this would have dramatically increased the complexity of crafting Dashboard widgets. People know how to write HTML, but most of those same people have never written an XML file, and namespaces are a point of confusion.

In addition there are technical hurdles to the use of XML. Every modern browser, including Mozilla and Safari, is much worse at XHTML than at HTML. People tend to foolishly gloss over the transition from one to the other, thinking that code you write for one will "just work" when you switch to XHTML. That simply isn't true. If you look at XHTML in both Mozilla and Safari and compare it to HTML, you'll see that it's slower, non-incremental, and generally buggier than HTML.

An example of a feature that won't "just work" when moved from HTML to XHTML is editing. The serialization model is totally different for XHTML, and HTML elements that have to be written out when you get the raw markup must know to do so using XML-style syntax in XHTML documents. Editing must be able to serialize namespaces, and ideally even preserve the namespace prefixes that were used at various points in the document as well as the use of default namespaces as set up by the author. Right off the bat I've outlined a challenging editing feature that only exists in the XHTML world. There are many more examples of these kinds of problems.

The perfect example of a widget that combines editing with HTML extensions is the Stickies widget. We simply could not have moved this widget to XHTML without doing an enormous amount of XML work.

A second complaint leveled against us was over the canvas tag, namely that it should have been done using SVG. My response to this is simple. Go to the w3c Web site and print out the SVG specification. Twenty minutes later, after you've killed a few dozen trees, then maybe you'll have an appreciation for why this wasn't practical.

Remember that SVG would have forced the use of XHTML, which had all the problems outlined above. Now add to that time the amount of work that would be required to get even a rudimentary SVG implementation going. Now factor in the time it would have taken to make that implementation perform well enough when compared with a programmatic counterpart like the canvas. Canvas only took a handful of days to implement. SVG would take months to implement.

In other words, in an ideal world where we had two years to craft Dashboard, maybe we could have used XHTML and SVG, but we aren't living in that ideal world. We can basically manage only one "huge" layout engine feature in a development cycle, and given our developer feedback the choice of HTML editing as the feature to focus on this cycle was clear. We would still love to implement SVG and XSLT and other great technologies in the future, but we simply can't do everything at once.

Finally we have submitted all of our extensions to the WHAT-WG for review. The slider in particular is already in the Web Forms draft. It is our hope that these HTML extensions will ultimately be standardized by a working group, but I wanted to emphasize that we are working with other browser vendors such as Opera and Mozilla to ensure that these extensions are implementable in those browsers and that these extensions can be standardized. We are not simply off "doing our own thing."

Another extension we made to HTML is a new element called the canvas. This element is essentially an image element that supports programmatic drawing. The way it works is that you can invoke a method called getContext on the canvas and then you have access to a whole range of 2d drawing calls. This element is how the hands of the Dashboard clock are drawn in HTML. The canvas fully participates in CSS styling too, so you can give it borders or background images while still painting the foreground content programmatically.

In addition to the canvas element, we've also introduced a new attribute onto the img element. The composite attribute allows you to control how an image gets composited. The default is source-over, which is how images are normally composited on the Web, but now we've introduced other values for more complex compositing operations. Especially in the Dashboard with its transparent windows, these other compositing operations become useful, since you can use this feature to "punch holes" to the desktop, allowing for all sorts of wild irregular shapes.

The new WebCore also supports HTML editing. You can specify editable regions in a page using the contenteditable attribute (which maps to a CSS property behind the scenes, so you can even set that property in your user stylesheets if you want to get crazy). We've also added a selection object so that you can get the current selection, and of course support for execCommand so that you can perform operations on the selection. And yes, so that you can actually tell what you're doing stylistically, getComputedStyle is now also supported.

One reader of my previous entry (in the trackbacks) expressed concern that only Dashboard widget apps could execute native code. Well, we have an answer to that as well. You see, we're working with other browser vendors to extend the plugin API to add scriptability. See this link for more information. This means you can obviously execute native code from within any WebKit application's Web pages through the use of scriptable plugins.

Todd Dominey writes about Dashboard in his blog and asks some questions that I'd like to clear up.

A Dashboard widget is a bundle that contains a principal HTML file and any supporting code that the widget requires (be it CSS, JS, images, or native code). A widget can add an optional interface to native code, written in Objective-C, that can be bound into JavaScript and made accessible from the HTML document's JS window object.

In other words, an address book widget could inject a property called "addressBook" into the JS window object of an address book widget's HTML document, and then expose methods and properties on that object that can be invoked from the JS. This effectively allows you to execute native code through the use of this special type of plugin.

The "native code as a service accessible from JS" model should be familiar to anyone who has used XPCOM with XUL. It's essentially the same idea. Extensions to the Firefox browser that contain native code can expose that native code to script as an XPCOM service, and then that object can be obtained from JS and have methods/properties invoked.

Again, when viewed from a certain perspective, this is a competitive Web browser feature that has been fused with Expose. These widgets that might otherwise have had to be inside the browser window as sidebar panels or toolbars have been set free by the brilliant idea of using Expose.

Anyway, some points about this model.
(1) The native plugin code must be owned by root. This means that in order for a Dashboard widget that contains one of these special types of plugins to execute that code, you have to enter a root account password (to chown the plugin code). This plugin code cannot execute, therefore, without the widget being "blessed" just as an application that you might install on your system must be.
(2) This plugin will not be present in Safari or other WebKit applications, and is only accessible from Dashboard.
(3) The dashboard object is also exposed on the JS window object of the HTML document and has methods for "meta-functions" that the Dashboard can execute.

As for many of the animations, fades, slides, etc in the widgets themselves., they simply look so damn cool because of Safari's rich support for CSS3 used in conjunction with DHTML. Do you know what I talked about at WWDC? Image replacement. Sliding doors. Using opacity to create fade effects. CSS3 text truncation. Web standards. All of which are being used to full effect in Dashboard widgets. Our standards support has grown so rich and our engine has become so smooth at effects that people are constantly mistaking pure JS/DHTML/CSS stuff that people are doing for something fancier. I've heard "That's HTML?!" several times in the past week.

Now it is true that we have made many extensions to WebCore, but only in places where there are holes in HTML that must be filled. And even then, we have tried to implement compatible models or to design so that our enhancements could be standardized in the future.

For example, the new WebCore supports all of WinIE's drag events, and that's how drag and drop is done in the Dashboard. So at the same time we added this rich support to WebCore, we also added support for a feature that can now be used in Web pages in a compatible fashion with WinIE. We started with a compatible base and enhanced drag and drop to allow you to dynamically set the drag image and even enhanced CSS with a new drag pseudo-class so that you could re-resolve style on the element while it's being dragged, but at the core, we made sure to pick a practical starting point.

In other examples, we added support for new slider widgets and search field widgets (wrapping NSSlider and NSSearchField respectively). HTML is missing these widgets, and so we had to add them so that Dashboard widgets could use them. But even there we did so in a way that is designed to be compatible with other browsers.

I haven't had a chance to talk about this, but I thought I'd again start by briefly clearing up a point of confusion. Safari RSS is not the name of the entire Safari browser on Tiger. It is the name of the RSS/Atom feature in Safari itself. If you pull down the About information in Safari on Tiger, you'll see that the version is 2.0.

One of the new features added to WebCore is an HTML wrapper for the Cocoa NSSearchField, which means you will be able to use this control in your Web pages and Dashboard widgets.

The syntax is:

<input type="search">

The incremental attribute can be used to control whether or not the search is performed as you type.

The placeholder attribute can be used to set the greyed placeholder text in the control.

The autosave attribute allows you to specify an autosave name for your search results in the dropdown. What's cool about the autosave is that it's global, which means that two Web sites could put up a search field with an autosave name of google and then they would share the search results. (Since the actual result values are not accessible to the page there is no privacy violation.)

The results attribute allows you to specify how many results should be displayed in the dropdown.

The search DOM event fires at the precise time the search should be performed and obeys the heuristic for the control, so you don't have to trap key and click events at all. You can use the onsearch attribute on the control to respond when the search happens.

When submitted, the search control will save the submitted result, so that the user can select it from the dropdown when returning to the page.

In addition the search control is a subclass of NSTextField and also supports all of the functionality of the text input element, so you can make it readonly, set a maxlength, etc.

The search control will also even degrade gracefully in other browsers, since they won't recognize the unknown type and so will just use a text field instead.

If you have the 1.3 preview for Panther or are playing with Tiger, you can try this feature out. It works in both.