monday, 28 april 2008

posted at 22:17

There's no easy or amusing way to say it, so I'm just going to say it. My involvement in AROS, including Traveller and the nightly builds, ends right now.

Over the last few weeks I've been doing a few different things. I played lots of Morrowind, started work on a couple of brand new projects, played lots of the new Advance Wars game that I got for my birthday, read the new Ben Elton book, and a few other things. I've enjoyed every part of it. I've been doing lots of different things, stretching my brain in different ways, and not been beholden to anyone. Since I'm happier, work has been much better, home has been much happier, everything just seems good.

The whole time though there's been a tiny nagging voice in the back of my head. Thats the one that has been telling me that I need to get on with Traveller. Only a couple of months to go. I hate that voice. I've tried a number of times to get into it, but I've only added about twenty lines of code to the loader in that time.

The fact is that I'm just over it all. Every part of AROS that I was interested in I've done enough work on to learn as much as I'm interested in. I wrote a filesystem. I wrote a graphics driver. I ported some minor apps. I hacked on some libraries. Its the same for Traveller. I got it to a point where you could browse the web. Everything else is just a bonus - in these areas, there's not really much left to take my interest.

I've been over this before. This is a major part of my reason for planning to leave after Traveller. But I really started thinking about why its so difficult for me to get motivated. The question I eventually got to was "would I be trying to finish this if there wasn't a nice prize in it for me?". And the only answer I had to that was "no".

That was a rather enlightening moment. I'm a little bit ashamed of myself actually, but not surprised. I've known since forever that money is not really a motivator for me, it never has been. I think I just got a bit dazzled by the possibilities; large amounts of spare cash don't come my way too often and there's at least one neat gadget that I've been hankering for.

So all in all, I have no compelling reason to continue. I realise I've made commitments, and I hate breaking them, but I've made other commitments in my work and personal lives, and I can't do them all, so I have to choose. Once I really looked at it seemed to be a fairly straightforward choice.

It shouldn't take long to remove myself. I've already managed to offload nightly build duties, as there were some issues and recent updates that I've been rather tardy in sorting out and so someone else offered to take the build on. I'll email TeamAROS shortly to let them know that I'm ditching the bounty. I guess I'll spend a little time during the week responding to email and that should about take care of it.

For anyone who wants to take on WebKit and make a browser, feel free to take the code I've already done. Its all available via my git repository under free licenses. Do contact me if you need help; while I'm not working on stuff and won't be paying a lot of attention the current goings on, I'm quite happy to offer support and advice on specific issues.

Finally, thanks to all the nice people in and around the AROS community. I've had a great time getting to know you and working with you. I've no doubt that we'll see each other around the internets from time to time, and I'll drop into #aros when I can too.

This blog isn't closing up shop, of course. Once I'm back from playing games and reading I'll likely be back writing about whatever I end up working on next. Current interests are DS hacking, binary decompilation and RPGs. By the time I write something they may not be interesting anymore in which case you'll get to read about something else :)

sunday, 13 april 2008

posted at 22:34

You've probably noticed that I've been fairly quiet for the last couple of weeks. Its partly related to what I wrote about last time, but not entirely. I've actually been doing quite a bit of hacking on a new project that I'm quite excited about, but I'm putting on hold for a little while so I can get back to Traveller. I'll write more about that one when I get back to it.

Other reasons are that work has been absolutely insane, so I haven't been able to send mail or IRC much during that time, and when I get home I'm tired enough that if I do want to sit at the computer I'm either playing a game or just not interested in talking to anyone.

Fortunately a big part of the last couple of weeks at work has been getting some new staff acquainted with the place and offloading some work, so next week should be much more relaxed. I still may not blog much, and I probably won't be on IRC, but I'm responding to email again at least.

My head seems to be in better shape, though I'm not taking anything for granted. Generally I found that I wasn't even having to try to enjoy myself, but on the other hand I had a couple of days like yesterday where my daughter was probably not aware just how close to death she came. So I'm hardly cured, but at least I've been reminded of what it can be like. I had a good chat with my minister who had some good advice, and I'll see my doctor this week, and I'm feeling pretty positive about the whole mess, so its good!

Thanks to everyone for your support and advice after my last post. Lots of people have told me their stories which I've really appreciated hearing. If you've sent me email and I haven't got back to you, please be patient - I will get there.

So starting tomorrow, on my bus trip, I'll be back working on the new shared object loader. I'm really going to push hard to get motivated and have something to show by the end of the week, because this bit is rather starting to hang over me. I really want to get back to the eyecandy :)

sunday, 2 march 2008

posted at 12:21

I have a little treat for the adventurous today.

AROSWebKit-preview-r30597-20080302.zip [8.1M]

Its mostly unusable, but many many people have requested a demo. Its still quite difficult to build it from the source, so here it is.

This will crash your system. No support of any kind is offered, but feedback is welcome. Send some mail or nab me in IRC :)

monday, 25 february 2008

posted at 15:34
A week later:

The major new things compared to my last post are the addition of the page title (and progress bar), the URL entry bar, and scrollbars. The last one is the thing thats been killing me for the last week, and I'm happy to finally have it over and done with.

What you don't see is that most of the details of integrating WebKit with Zune so that it can request and control native UI widgets. At its core, WebKit is a layout engine. It takes a UI description (in the form of HTML, CSS, etc), creates a bunch of objects, positions them relative to each other and then draws them. Sometimes (eg for a HTML form) rather than handling an object internally, it instead asks the host UI system to handle creating and drawing the object instead. When it does this, however, it expects to have full control over where the object is placed.

Zune allows a custom class to provide a function that will be used to position the widgets rather than using its own internal algorithms. This I have written. All it does is loops over the list of native widgets, asked WebKit what their dimensions are, and then tells Zune how it should draw them. Its the easy bit in all of this.

A typical Zune object is rendered in three parts. Upon receiving a request to render an object, Zune first asks the object about its dimensions, and receives back the minimum and maximum possible sizes it can use, and its ideal size. The object's parent object sets an appropriate size within the ranges and positions it in relation to itself, and then asks the object to do the same for its children, if it has any (most simple widgets do not). Finally, once the object knows its position and everything else is done, it is asked to draw itself in that space. This rendering process happens based on some external trigger, such as the window being opened or resized.

The complication arises from the order that things are done in this process, and when the process is triggered. Once its size is determined, a Zune object is asked to layout its children, if it has any, via MUIM_Layout. Once done, MUIM_Show is called to tell the object it is about to be displayed. Finally MUIM_Draw is called and the object is drawn.

Lets think about what really needs to happen to render a page, and how Zune conspires against us. I'll start by describing the obvious implementation of this mess, which is what I had before this week. In the beginning, we have a pristine WebKit_View object, with no drawing in it and no child widgets. Lets assume though, that WebKit has already loaded a page internally, because the initial page load has a couple of extra twists and this description is already complicated enough.

At the moment the application window appears (or the view is added to the UI in some other way), the magic begins. The view is asked for its dimensions, which are typically "as much as you can give me". Next, the view is asked to lay itself out via MUIM_Layout. This is actually a private method, and not one we're supposed to override, so we let that go through to the view's superclass, Group. It gets its list of sub-widgets, finds it empty, and so does nothing.

Next, MUIM_Show is called on the view. This is the first time the view knows the exact dimensions it has been given by the window, and so we tell WebKit the new dimensions and ask it to layout the page based on this size. Once thats done, the window calls MUIM_Draw, which sets up a cairo context over the view area of the window and tells WebKit to draw into it.

The cake is a lie.

If WebKit, during its layout phase, determines that it needs native UI widgets (form elements, scrollbars, etc), it asks the Zune to create them and add them to the view. Unfortunately, at this point the Zune object layout has already been done (we're in MUIM_Show, which runs after MUIM_Layout), so the new widgets have not been asked about their size, have not been placed on the page, etc. MUIM_Draw fires, the view asks WebKit to draw the page and then calls the supermethod to draw the widgets. These unitialised widgets all get drawn with no dimensions at the top-left of the view. This is not what's wanted.

At this point some way of forcing the entire layout process to run again is necessary. This is harder than it should be. You can't just call MUIM_Layout, even if it weren't a private method, because the new widgets have not yet been queried for their sizings. There appears to be no standard way of forcing the layout process to run. In the end I've abused a feature of the Group class to do what I want. The usual way you'd add widgets to a group is to call MUIM_Group_InitChange on the group, followed by one or more calls to OM_ADDMEMBER or OM_REMMEMBER. Once done, a call to MUIM_Group_ExitChange "commits" the changes by making the whole window relayout and redraw from scratch. To force the layout to happen, I simply call InitChange followed by ExitChange with no widgets added in between.

(Coincidentally, I used to use these methods when adding the widgets to the group in the first place, but stopped because it was causing a redraw every time. Now I simply use OM_ADDMEMBER and OM_REMMEMBER and assume that the layout and draw will be done elsewhere, which is correct conceptually).

The one chink in this method is that ExitChange eventually causes all three stages of the render process to run - sizing, layout and draw. We're already inside the layout section, and so we don't want everything to run again. Specifically, we don't want this secondary render process to cause WebKit to do another layout, and we don't want it to draw either, as that will be handled by the original render process. Some flags in the view object to record and detect this reentrancy are all that's required. So the final process becomes:

  • Render process triggered
  • (internal) setup widget dimensions
  • (MUIM_Layout) widget layout (ignored)
  • (MUIM_Show) WebKit layout
  • (MUIM_Show) force second render process
    • (internal) setup widget dimensions
    • (MUIM_Layout) widget layout
    • (MUIM_Show) WebKit layout (ignored)
    • (MUIM_Show) force second render process (ignored)
    • (MUIM_Draw) draw everything (ignored)
  • (MUIM_Draw) draw everything

Do you see what we did there? We just bent the Zune render process to our will by turning it inside out :) There's a couple of other warts thrown in to the mix to deal with a couple of edge cases, but thats basically it. You can read the dirty details in webkit_view.cpp.

Now I have no idea if this is compatible with real MUI. MUIM_Layout is actually private in MUI, but public in Zune, so I wouldn't be able to override it there, but the override could probably be done well enough in the custom layout function. I'm not overly concerned if its not compatible; I'm not developing for MUI after all, but I am curious.

This all points to what I believe is a fairly major design flaw in MUI, that being that the stages of the render process are fairly tightly coupled. There should be a direct a way to force a single object to relayout itself from scratch, and doing it without triggering a redraw. There should be a way to get dimensions recalculated. I suppose its not unreasonable that these things can't be done directly as its probably not often that an attempt is made to bolt an entirely seperate layout engine to it. I suppose it is a testament to MUI's flexability that I can twist it like this at all.

Next up is to get the scrollbars hooked up to the page. After that is the RenderTheme implementation which gives all the other widgets necessary to view pages with forms. A little input handling after that and then we'll have something usable on our hands!

monday, 18 february 2008

posted at 11:03

A couple of hours work on yesterday's effort, and we see this:

Had I known just how close I was, I probably wouldn't have even bothered posting yesterday.

The wonky text was because a stupid assumption I made in cairo's font code, which I've now fixed. The text still looks crap, mostly because of issues with the renderer, but I've been pointed at TTEngine this morning which looks much more like what I want and would let me remove some of the hacks I've had to do in cairo. I'll be looking at this further this week.

There's still a hell of a lot to do, so don't get too excited. At least now I have a way to see whether or not my changes are actually doing something or not.

I'll be posting many more screenshots as work progresses, but I won't be blogging them all. Things are moving just too fast for that. If you want to follow the screenshots, watch my screenshots set on Flickr or just subscribe to its feed.

sunday, 17 february 2008

posted at 09:46
Current progress:

This is WebKit on AROS rendering a trivial page containing a H1, a H2, a DIV with CSS styles set to force to 100x100 with a green background, and a IMG of a pirate, though thats not working yet.

The text alignment appears to be screwy because my code in cairo is not correctly calculating the baseline on tall glyphs. It works as expected from my cairo tests though, so I'll need to dig a lot deeper to figure this out. Likely I just missed some mundane detail; font metrics are actually quite difficult and I'm not help by the fact that the bullet interface doesn't provide away to get the metrics for the font as a whole, meaning I have to generate them in a rather horrible way.

There's also an issue where if a line of text is wrapped (eg if I resize that window to be really narrow), only the last line is rendered. I still haven't looked into that yet. Oh and of course there's a bunch of internal stuff that really isn't correct but won't noticeably affect the outcome just yet.

All in all, not bad progress so far. Its only going to get more difficult as I really get into the details, I think. Not to mention the many many shortcomings in services provided by AROS, which are going to need to be addressed if this thing is to look nice and not be insanely slow. I'll write more about that lot later.

monday, 14 january 2008

posted at 12:23

A year ago today I made my first commit to the AROS Subversion repository. It feels like I've been doing this forever, not only a year. I've been digging back through the history to see what I've been up to over the last year.

Here's a list of things I've done that I think are worth noting:

  • 21 January: tap.device network driver for Linux hosted AROS
  • 4 March: DOS packets and initial FAT filesystem support (minimal read-only)
  • 27 April: FAT write support
  • 7 May: Fast bitmap scaling, made Wanderer startup faster and made FFE usable in hosted and boosted it from 8 to 20FPS in native
  • 16 May: FAT notifications
  • 20 May: PA_CALL and PA_FASTCALL taskswitch and lock-free message ports for speed
  • 8 June: GetDeviceProc() and ErrorRerport() rewrite and internal DOS refactoring
  • 17 June: Pipe() DOS function
  • 21 September: hostlib.resource for calling host libraries from inside AROS
  • 2 October: Converted X11 driver to use hostlib and moved it out of ROM
  • 3 October: SDL driver
  • November (and ongoing): C/POSIX library improvements
  • 17 November: Math library upgrade
  • 3 December: thread.library
  • 13 December: ELF loader support for large objects

There's also a pile of tweaks and fixes that don't feature in this list. According to git, I've made 269 commits to the core AROS repository, adding 23182 lines and removing 12741 lines.

In addition to this, I've got plenty of work-in-progress stuff that hasn't (or won't) hit the AROS repository:

And of course, the Traveller-related work:

2008 should be a bit more focused for me, as most of the first part of the year will be working on getting Traveller out the door, and then on a few big supporting things like SysV shared object support. I don't think it'll be any less interesting as a result :)

Thanks to everyone who has helped and guided me through the many many mistakes I've made, particularly the freaks in #aros. The major reason I'm still here and wanting to work is that is fun, nothing more. Cheers lads :)

wednesday, 9 january 2008

posted at 09:17

With the help of WebKit developers I finally sorted out the crasher that plagued me over Christmas, and now I see WebKit making network requests, receiving data and calling into the graphics code to get it on screen. The next step is to begin implementing this graphics code.

As far as I can tell I need support for both blitting objects (like images) to the screen, but also need drawing primitives, both simple stuff like lines, circles and rectangles as well as complicated things like Bézier curves and arbitrary paths. It needs to be able to apply a transformation matrix to both paths and images. It needs compositing support. It also needs to be able to operate on screens of arbitrary size and depth.

AROS (and the standard Amiga API) can't support this. Some of it exists, just not enough. graphics.library has basic drawing primitives but not advanced stuff like splines and such. Its primitives don't operate reliably on TrueColor screens, which is what pretty much everything is these days. CyberGraphics provides access to higher-depth modes, but only really for blitting. And we have no support for affine transforms, compositing, or other advanced features.

To Commodore's credit, its pretty clear that they were moving in this direction. They had these concepts on the market in a time where they were barely even considered elsewhere. I'm quite sure that were they still around today we'd have these features available. Sadly, we don't, so we must find another way.

I've studied the problem in some depth, and I've decided to port the cairo graphics library to AROS. Their description sums it up well enough:

The cairo API provides operations similar to the drawing operators of PostScript and PDF. Operations in cairo including stroking and filling cubic Bézier splines, transforming and compositing translucent images, and antialiased text rendering. All drawing operations can be transformed by any affine transformation (scale, rotation, shear, etc.)

A port will be a good thing for everyone. WebKit already has a cairo graphics target, so I'd get my rendering for free. The library is extremely portable, with a great target abstraction. Indeed, I already have the thing compiling and the AROS backend stubbed.

More controversially, I think cairo could actually become the core 2D graphics system for AROS. graphics.library could be trivially implemented on top of it for compatibility, so there's nothing to worry about there. We'd implement a cairo backend that talks to a stripped-down version of our graphics HIDD layer (as much of their functionality would no longer be necessary). Once it place it would give easy support for eyecandy like real transparent windows or something like Exposé. Combine that with the plan to get 3D via Gallium, and AROS could become the shiniest thing out there.

My port will be a proper AROS-style shared library, cairo.library. Cairo's code is clean enough that I think I can do this without requiring the API to change and while still making it possible to contribute all the changes upstream without adversely affecting them.

Port repositories: cairo and pixman. These will be combined in the final library.

monday, 7 january 2008

posted at 15:41

Christmas and New Year festivities are over, and I enjoyed them thoroughly. I spent some awesome time with both sides of my family, played some cricket and soccer, played some Wii, ate way too much several times, and scored a nice pile of DVDs and t-shirts. In the long drives between various parties and dinners I've had a lot of time to ponder a WebKit problem, which I document here :)

WebCore has some functions that arrange for a simple timer to be implemented. Its very basic; there's three functions: one to set a function to call when the timer goes off, one to set an absolute time that the timer should go off, and one to disable the currently set timer. This simple interface is used by the higher-level Timer class which can be instantiated multiple times. It handles coordinating the current timers and making sure the system timer is requested at the proper interval.

I did a first implementation of this using timer.device directly, but it really didn't feel right. The interface has no provisions for initialising or finalising the timer, so I hacked it such that the first call would open the timer device if it wasn't already open. I ignored the finalisation for the time being, and started looking at how to arrange triggering the timer.

We're back to the old problem that AROS basically does not have any provisions for signals/interrupts that preempt the running process in the process context (actually, task exceptions can, but they're too low-level for our purposes and don't work properly under AROS anyway). When timer.device fires, it pushes a message onto the IO request port, which either raises a signal (MP_SIGNAL port) or calls a function directly from the scheduler context (MP_SOFTINT port). There's also MP_CALL and MP_FASTCALL ports; these are the same as MP_SOFTINT for our purposes.

Having a soft interrupt that calls the timer callback doesn't work, as it would cause us to do large amounts of work inside the scheduler which is bad for system performance. Having a signal requires the main process to Wait() for that signal and then call the timer callback. The main loop is controlled by the application and by Zune, both things we have no control over.

I confirmed via #webkit that the timer callback is indeed supposed to be called from UI main loop. Studying the MUI docs and the Zune code, it seems that it is possible to have the Zune main loop setup a timer and trigger the callback itself using MUIM_Application_AddInputHandler. This is perfect for our needs, as it removes any need for initialisation and finalisation in the shared timer code itself.

The only thing that has to be arranged then is for the shared code to get hold of the application object to setup the timer. The application object is created and controlled by the application, of course, but there is only ever supposed to be one of them per application, and I can't think of a good reason why there should ever be more than one. Its easy to get hold of this object from any Zune object inside the application, via the _app() macro, with the slight quirk that its only available when the object is actually attached to the application object. We can detect that well enough though and defer calls into WebKit until we're attached, so all that remains is to grab the application object, stow a pointer to it in a global variable, and then have the shared timer code use that variable.

This all took me a few hours to work out, and then I happily went off to do Christmas things. Over the next couple of days, the nagging seed of doubt that I had in the beginning grew into some kind of spooky pirahna flower thing. This morning while hanging clothes out to dry I finally understood the issue. Its all to do with how global variables work, and its has much greater implications for this project than just getting hold of the Zune application object.

Lets think about what happens when you load a program into memory. Forgetting about the details of the loader doing relocations, setting up space for variables, etc and the program startup making shared libraries available, effectively you just have the system allocating a chunk of memory, loading the program from disk into that memory, and then running the code within it. Space for and initial values for global variables are all held within that chunk of memory, and only the program code knows where they are and what they're for. Nothing else on the system can reasonably access them so there's nothing to worry about.

A shared library is essentially the same as this, except that it is only ever loaded into memory once. When a second program requests it, the systems checks if the library is already in memory, and if it is arranges for the program to use it. This is where things can get complicated. The big chunk of memory contains some things that are sharable because they can be considered read-only - things like program code, const data, and so on. Regular global variables are generally not sharable, as you generally don't want changes made by one process to be seen by another.

In systems that have a MMU, the usual way that this is dealt with is to make a copy of the global data somewhere else in memory, and then map it into the process address space at the appropriate location. That is, process share the read-only parts of the shared library, but have their own copies of the writable areas. (In practice its quite a bit more complicated, but this is the general idea).

AROS, like AmigaOS before it, has all processes, libraries and anything else coexisting in the same memory space. Shared libraries pretty much don't use global data. There is no support for MMUs so the kind of copying and remapping descibed above is impossible. If per-process data is required, then various techniques are employed explicitly by the shared library author - per-opener library bases, data access arbitration using semaphores, and so on. That works fine, because the author is fully aware of these limitations when he designs and implements the library.

Its worth noting that this problem is not isolated to AROS, but to every system where a MMU is not available. uClinux has had the same issue in the past and dealt with it in a couple of different ways.

Now lets look at what I'm trying to do. My goal is and has always been to make WebKit a shared library (actually a Zune custom class, though as far as the OS is concerned its the same thing). WebKit and its dependencies all make use of global variables as necessary, and assume that their globals are isolated to a single process, which is a reasonable assumption given that basically every system out there that WebKit currently runs on works this way. For AROS though, this is a huge problem.

The cheap way out is to just ignore the whole mess by producing a static libWebKit.a and requiring any applications to link it statically. This is essentially what I'm doing now. It works well enough, but currently the (non-debug) library weighs in at a touch under 18MB, and thats with barely any AROS-specifics implemented. For every WebKit-using application you have running, thats at least 18MB of duplicated code that you have to hold in memory. There's also all the usual issues with static linkage: greater disk usage, no ability to upgrade just the library and have all its users get the update, and so on.

The least favourable option would be to rewrite all the parts of WebKit and its dependencies that use global variables and either find a way to remove them or otherwise move them into a per-process context. This is horrendously difficult to do and would pretty much remove any hope of contributing the code back to its upstream sources, which I consider an imperative for this project. So lets say no more about it.

The only other option is to add support to the OS to do the appropriate remapping stuff. This is no small undertaking either, but I think as time goes on, its a very good thing for us to have. I haven't investigated it in depth, but in addition to actually implementing the stuff in the loader, its also necessary to make some changes to the way modules are held in memory and shared between users.

Currently a module can exist in memory and be used as-is by multiple users without too much effort. Because there's no global data, sharing a module is as simple as incrementing a use count, so that the module isn't purged from memory ahead of time.

When sharing an object with global data, in the absence of a MMU, its necessary to allocate new global data for each opener and do its relocations each time. This requires keeping a record of the required relocations. There's also the issue of constructing the global offset table and the procedure linkage tables, and making sure the pointer to the GOT is carried around the application appropriately. Work that will be usefel here is Staf Verhaegen's current project on library bases and preserving the %ebx register. Of course this will all have to integrate nicely with that.

Then there's also the matter of detecting when to use all this new stuff over the standard loading and linking code. I think I can make that as simple as requiring all code to be shared in this way be position-independent (ie compiled with -fPIC). Code compiled in this way is incompatible with the standard load method anyway, and for this type of shared object its far simpler to implement this whole mess if PIC is enabled. If it is, then detecting which type to use should be as simple as looking for the presence of the .got section in the object.

Thats about as far as my thinking on the matter has come. The shared timer stuff that originally provoked all this is working happily, but if WebKit is ever to be a shared object on AROS, all this will need to be revisited. Because its such a huge undertaking I'm going to leave it until after the WebKit and Traveller are in some kind of usable state. At that time I'll look at handing off care of the web browser to someone else for a little while and work on this stuff instead.

tuesday, 1 january 2008

posted at 22:53

Hi. I have lots to tell you, but haven't had time to write it all down yet. But I wanted to share this, the very first web request ever done by WebKit on AROS:

GET / HTTP/1.1
Host: 192.168.0.1:8080
Accept-Encoding: deflate, gzip
User-Agent: WebKit AROS
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5

I'll post more details sometime in the next couple of days. Happy new year :)

wednesday, 19 december 2007

posted at 22:06

This is week is insanely busy, as is typical of the week before Christmas, so I've had very little time to think about code in the last couple of days. I therefore opted for something that wouldn't steal too much of my brain, and began stubbing the Zune View class.

The interface will be typical Zune stuff. To get a web renderer into your app, you'll include a WebKitViewObject in your widget tree, and go from there.

The launcher is just a fairly standard Zune application setup. It will get a little more code before the end, mostly adding basic navigation buttons and location bar, but the basic structure won't change. This will serve as both a test program and an example of how to use WebKit in your own applications.

tuesday, 18 december 2007

posted at 11:17

Now that I've (apparently) fixed the loader, my mammoth WebKit test binary loads and runs, and so I've begun implementing the stub functions with earnest. To start my method has been to run the program until it crashes, find out where the crash happened, which is usually a NULL pointer dereference, and then provide a basic implementation of the class that that thing is supposed to be pointing to.

The current problem is a crash that occurs inside a regular method call, for no apparent reason. The offending method, in its entirety:

void DocumentLoader::setFrame(Frame* frame)
{
    if (m_frame == frame)
        return;
    ASSERT(frame && !m_frame);
    m_frame = frame;
    attachToFrame();
}

Good old printf() tracing shows that the crash occurs after m_frame = frame but before attachToFrame(). That is, that method is never called. This is highly unusual, and tedious to debug, because it means we have no choice but to drop down to assembly code, which I can muddle through well enough but can't really wrap my brain around.

Disassembling the last two lines of the method, we get this:

    mov    0x8(%ebp),%edx
    mov    0xc(%ebp),%eax
    mov    %eax,0xc(%edx)

    mov    0x8(%ebp),%eax
    mov    (%eax),%eax
    add    $0x8,%eax
    mov    (%eax),%eax
    sub    $0xc,%esp
    pushl  0x8(%ebp)
    call   *%eax
    add    $0x10,%esp

The pointer to the current object, this, is on the stack, 8 bytes in, as is the frame pointer, 12 bytes in. So we see the value of this being dereferenced through the stack and stored in %edx, and then the same for the frame pointer, being stored it in %eax. Then the location 12 bytes into the object proper is computed (which is where m_frame is stored), and %eax (the location of the frame object) is stored in it. Thus, m_frame = frame.

The next chunk, predictably, is the call to attachToFrame(). The important thing about this method is that its what C++ calls a virtual method. It wasn't until Friday that it was actually explained to me what that meant, and I found it hilarious. Consider:

    Object *o = new Object;
    o->method();

    o = new SubObject;
    o->method();

(where SubObject is a subclass of Object).

Now, if method() is a virtual function, this will do what you'd expect from most other OO languages: the first call will call Object::method(), the second calling SubObject::method(). If its not virtual, then both calls will go to Object::method, because its taken from the type of the pointer, not the type of the object itself.

I don't know if this was considered counterintuitive when it was first designed, but its certainly not the way most OO languages work these days. Usually you have to be explicit when you want to call a superclass version.

In any case, the code generated is different. In the simple non-virtual case, the call can be done via an absolute address, as the compiler can know exactly where the method() function is for the type. The virtual case is more complicated as the object itself needs to be interrogated to find out where its function is.

To do this, a table for each class that the object inherits from is placed inside the object, containing pointers to the functions that the object wants to use for its virtual methods. A virtual method call might then be rendered in C as:

    o->_vtbl_Object.method();

That is, go through the table of implementations of methods defined in the Object class to find the method, and call it.

So, getting back to our disassembly. attachToFrame() is a virtual method. The code gets this from the stack, 8 bytes in, and puts it in %eax. Then it dereferences the pointer to find the actual memory location of the object. It then adds 8 to that to get the location of the virtual method table, and dereferences that to get a pointer to the attachToFrame() function, which goes into %eax.

Then it does the usual function call setup, making room on the stack for the arguments and return address, and then calls the function at the location in %eax. It is here that the crash occurs, because %eax has 0 in it.

I was floored when I first saw this. I checked a number of times in different places, finally checking the constructor itself. And sure enough, the virtual table contains all zeroes. To me this smelt suspiciously like a relocation problem - if the the ELF loader is not correctly doing the relocations for virtual tables, then they'll point to garbage memory, causing a crash.

I'm not entirely sure how this can be, and haven't figured it out yet. I need to check the place where virtual table is normally initialised, but I don't know where that is! I can theorise by thinking about the structure of an object and the virtual table internally.

The first critical thing is that the virtual table inside the object is a pointer. That is, when the memory for the object is allocated space is not allocated for the virtual table too. A pointer needs to be to point to a valid virtual table. There's two ways this could be done: setting a pointer to some known static data that contains the data for this class, or allocating some more memory and copying the pointers from same known static data.

The former seems the more likely to me. The extra allocation and copy seems unnecessary as the table for the object will not change during the lifetime of the object. There are seperate tables for each class the object inherits from, so there's no need for a group of tables to be mixed into a single one.

So given that as a theory, we should be able to find some code somewhere around the constructor that sets up the virtual table pointer. It'll probably be the first thing after the memory allocation is done. This code might not exist in the binary itself though but may be part of a language support library (libgcc or similar). Regardless, the thing that will need to be there is the virtual table location.

I'm expecting to find that the location of the virtual table is not being relocated properly by the ELF loader. Basically, I trust GCC to produce correct code than I trust our loader to do the right thing. The problem could also be within our linker, collect-aros, but its so simple that I'm happy to rule it out initially.

Stuart, get back to work!

Update 3pm: Found it. I missed one section header table index conversion when I was updating the loader for large numbers of sections. Stupid, but it never hurts to exercise my brain on the really low level stuff.

thursday, 13 december 2007

posted at 22:04
  • mood: hobbitish

I just now have the extensions to the ELF loader implemented such that my gargantuan WebKit test binary loads. It took me a lot of reading and experimenting to figure out what was going on but I got it.

In my last post I talked about how files with large numbers of section headers store the count of headers elsewhere in the file. I'd taken care of that just fine. The other important thing that I missed is that every entry in the symbol table has a section header index that points to the section that the symbol is relative to. Of course this is a 16-bit field also, and has the same problem as the header count does.

The solution to this one is even more crazy. Basically there's an entire extra section in the file that is just an array of 32-bit values. If a symbol refers to a section with an index that is too large, you basically go fishing into that array to find the index instead. This of course means that I have to have that array loaded and available before I start doing symbol table work.

Finally, something that confused me until I put together some nice macros to deal with it was that there's a "hole" in the range of possible section header index numbers. What used to be the top 256 values (0xff00 to 0xffff) are reserved as special control codes, markers and other such things. Now that the header number is fudged into 32 bits, we get the situation where the header at index 65279 (0xfeff) corresponds to section 65279, but the header at index 65280 actually corresponds to section 65536 (0x10000). So basically, anywhere that a section number is found in any of the ELF structures, it has to be massaged into a header array index number taking the hole into account. This caused no end of issues, particularly since my file has hundreds of effectively unused sections - it was hard to even see when it was going wrong!

So now ArosLauncher loads and runs and I get some debug output before it crashes:

UNIMPLEMENTED:
(../../../WebCore/platform/aros/TemporaryLinkStubs.cpp:42 void WebCore::setFocusRingColorChangeFunction(void (*)()))
UNIMPLEMENTED:
(../../../WebCore/platform/aros/SharedTimerAros.cpp:10 void WebCore::setSharedTimerFiredFunction(void (*)()))

Before I get back into WebKit though I need to cleanup this code and commit it. I still need to merge the other two ELF loaders. As far as I can tell from a cursory glance the elf64 version is basically the same but using 64-bit definitions, macros, etc as defined by the ELF specs. The other, elf_aros, I'm not entirely sure about but its certainly much simpler. Its possible it just hasn't been looked at for a long time (the changelog certainly appears to show that). I'll continue to try to figure out what its for, but my feeling is that it can probably go, and elf64 and elf can be comfortably merged with a little restructuring of the code.

One thing that has become apparent is that our loader is incredibly slow and rather naive. As we start implementing more features (debug support, memory protection, shared objects) I don't think its going to cope well with its current structure. And its certainly got its work cut out for it - I've been reading Ulrich Drepper's paper "How To Write Shared Libraries", and it goes into a lot of detail about the kind of pain the dynamic linker has to go through to make things work. The glibc loader is something I'll have to study, I think.

wednesday, 12 december 2007

posted at 13:56
  • mood: elvish

I wrote a simple launcher for WebKit that creates a WebCore::Page, attaches it to a WebCore::Frame, then tries to load the Google homepage with it. Unsurprisingly, when I ran it it crashed, as most of my factory methods just return NULL. I fired up the debugger and figured out where the crash was coming from, and found it was in FrameLoaderClient::createDocumentLoader, one of my factory methods. Curiously, this function calls notImplemented(), and so should have printed something to the console. A little poking revealed that I had been done a release build, not a debug build, so I recompiled with --debug.

The resulting binary was almost three times the size, up around 300MB, which makes sense because its now carrying almost the entire source code for debugging as well. I had to start AROS with -m 512, to give it enough memory to actually be able to load the thing. I started AROS, opened a shell, started ArosLauncher, and then the amazing fireworks began.

On my debug console, I got a line of output:

[LoadSeg] Failed to load 'ArosLauncher'

Thats a problem - LoadSeg() is the program loader/linker. More exciting though was the line after line of pure binary appearing in my AROS shell. Do something like cat /bin/ls to see what I mean.

My first thought was that the awesome size of the binary was trampling something in memory, but a bit of poking around revealed the answer. When you type a command into the shell, it tries to load it as an executable file. If that fails, it checks if the file has the script flag enabled. If it does, it calls C:Execute with the file as an argument. Execute is the script runner, and it simply feeds the contents of the file into the shell's input buffer to be executed as though the commands were being typed.

Execute doesn't have any smarts to determine if what its being passed is really a script; that would be a useful feature for it to have. The real issue though is that the ArosLauncher binary had the script flag. I never set it, it shouldn't be.

Closer inspection revealed that the hosted filesystem driver, which maps Unix file permissions to AROS file permissions, was setting the script flag for every file without exception. That was perhaps a reasonable choice at the time it was written, as Unix does not have a script flag or anything similar it wouldn't have been immediately obvious what to map it too and it was never used in AROS anyway until recently (the shell gained support for testing for it and calling Execute a couple of weeks ago). Clearly though its not write, so I had to do something. I modified the permissions mapping code in emul.handler to map the AROS script flag to the Unix "sticky" (t) permission bit. I also implemented FSA_SET_PROTECT at the same time, so now typing protect +s file in AROS acheives the same as chmod +t file in Unix, and vice-versa.

So with that fix in hand, ArosLauncher was rerun and the far simpler error was returned:

ArosLauncher: file is not executable

So the next step was to dig into LoadSeg() and find out why it couldn't load the file.

A tiny bit of background: Any program, library or other "executable" thing under AROS (and most Unix systems) is stored in a format called ELF. It is split into a number of "sections". Each one contains some information. It might be program code, data, symbol names, debugging info, there's lots of different types. Its up to the OS loader/linker to pull all these together into a runnable program.

So, with the ELF specs in hand I started stepping through the loader code, and quickly found the problem. When you compile something with debugging information, it adds many extra sections to the binary object, containing what amounts to the entire source code for the program, so the debugger can give you the proper context and so on. Because it includes all of WebKit, ICU, cURL, libxml and SQLite, it has a lot of sections. Somewhere in the order of 75000 in fact.

The field in the ELF header that stores the count of sections is a 16-bit field, which means it can count up to ~65000. Clearly there are too many sections in the file to fit. In this case, the number of headers is marked as 0, and the loader should try to load the first header. In there is the real count, in a 32-bit field that normally is used for something else (the header size) but is borrowed just for this special case.

So I implemented this, and it works - it finds the headers correctly and does the relocations as it should. Its still not at the point where it will run ArosLauncher. It would appear that there's a symbol type that the AROS loader doesn't know about and is interpreting as being invalid, rather than handling/ignoring it. I'm not sure what's appropriate yet; I'll take more of a look on my bus ride home today.

More todo items: There are three ELF loaders in AROS currently, elf, elf64 and elf_aros. elf is the main one that I'm working on, elf64 is a copy of it taken recently with support for 64-bit symbols, and elf_aros is an old one that I have no idea of what its for or where it came from. I have no desire to make my modifications in three files, particular when I have no 64-bit system to test on, so I'm going to look at trying to merge these three files back together.

sunday, 9 december 2007

posted at 11:27
  • mood: sucks
  • music: machinea supremacy - sidology 2

I'm a little stuck. Last night I wrote a trivial startup program to make sure linking and calling into WebKit was working correctly:

#include "config.h"
#include "Logging.h"

extern "C" {

void webkit_init(void) {
    WebCore::InitializeLoggingChannelsIfNecessary();
}

}

int main(void) {
    webkit_init();
    return 0;
}

It compiled fine, but the link failed:

There are undefined symbols in 'ArosLauncher':
         U _GLOBAL_OFFSET_TABLE_

All the bits of information I need to resolve this are scattered around (if they exist at all), but what I've learnt is this. WebKit is compiled with -fPIC, which produces position-independant code. This is what you want when producing an ELF shared library. Essentially what it does is setup an offset table to hold the locations of all the global symbols in the library, and causes the generated code to access those symbols through the table instead of going direct. Later, when the library starts, the runtime linker fills in this table with the correct locations to all the symbols. This allows the OS to place the library anywhere in memory it wants, rather than at the location the library was compiled for initally. This is all great stuff than doesn't make the slightest impact on AROS, as our shared libraries don't work this way. Well, they do conceptually, but thats a topic for another time.

I'm compiling all this code into a static library, but because it was compiled with -fPIC it has lots of references to _GLOBAL_OFFSET_TABLE_. Here's where I'm unsure of what's happening. Either GCC is not setting up the offset table because our patches to make it work on AROS don't enable it (reasonable, since we don't have support ELF shared libraries), or its just implied that if you're linking with a static library you won't need the offset table and are expected to compile with -fPIC. I spent a lot of time last night believing the former, but after being completely unable to find anything in the GCC code that supports this, I'm really starting to lean towards the latter.

Which brings us to the next problem. Currently AROS WebKit is build using qmake, the build system for QT. I chose this because it was the easiest way to get a cross-build running at a time where I had no real idea what I was doing. It would seem that its currently setup to build a shared library, which I'm hacking around at the last stage to make it output a static library. I haven't found an obvious way to disable -fPIC yet.

This highlights the next issue. qmake is not going to cut it going forward. Actually, none of the existing WebKit build systems are really suited to cross-building - its all hacks so far. Before long its going to need a real build system. I'd like to use plain GNU make so that there won't be an issue with compiling the mess on AROS, but there's still going to have to be some stuff copied from the AROS build system to support setting up the Zune stubs, for example. That suggests just using mmake directly, except that I have my reservations about its long-term suitability for anything. The build system is not something I want to debate here, I've said my piece about it elsewhere and I'm deliberately not discussing it until I have time to do my own experiments.

So here I am at a bunch of apparent deadends. I'm going to spend a little more time right now trying to bend qmake to my will, but this whole mess is rapidly getting out of hand. I believe a sigh is the appropriate action at this point.

Sigh.

Update 12:53: Figured out how to turn -fPIC off, and I now get why it wasn't working. I now see logging output on the console, awesome! A better build system is still required.

saturday, 8 december 2007

posted at 18:25

Today marks a major milestone for the WebKit port. It compiles!

-rw-r--r-- 1 rob rob 24782208 2007-12-08 18:25 libWebKit.a

It doesn't do anything yet, but it compiles. I have 298 stub methods across 41 AROS-specific files. Each one calls the notImplemented() macro, which simply prints the name of the method that was called.

The plan of attack from here is to write a tiny main() that tries to create a frame object and hand control to it. That should yield several million lines of output from notImplemented(). I implement those methods, one at a time, until I get something on screen.

Once I get a decent way into that process I should start to gain some understanding of how WebKit is actually bolted together. Once I have that I can start to think about the design of the Zune API.

The real fun starts now. I'm looking forward to writing some actual code rather than just stubbing functions :)

saturday, 1 december 2007

posted at 09:04

This week I've been working on another WebCore dependency, though a little different to the previous ones. To work well, it seems that WebCore needs a threading system. AROS doesn't have one. I think WebCore can work with just its stubs, but I don't want to. I want this done properly.

I started looking at how I might implement threads, and it seemed that the interface was general enough that it could be useful to as a shared library for other things besides WebCore to use. And so thread.library was born.

Its almost ready. Threads work, mutexes work, conditions work. The only thing I'm dealing with is what to do when the main task exits while threads are still running. There's a bunch of bad things that can happen, which I don't have time to go into right now, but the best thing I can do is to simply detach the threads and allow them to continue running. See here for more details, though the description is out of date - the code is now doing option 1, and the ThreadBase issues have been dealt with. The last thing to take care of is a small memory allocation issue that is causing a crash, but once thats done I'll check it in to AROS subversion for anyone to use.

Update 2007-12-03: Code is now in AROS SVN, and will be available for use in tonight's nightly and SDK builds. Be sure to read the README before starting.

sunday, 25 november 2007

posted at 20:50

Late Friday I reached a minor milestone when I got the platform-independent part of WebCore fully compiling and linking. Next up, the tricky bit: the platform dependent stuff, otherwise known as the actual port.

I spent a couple of hours staring at various WebCore classes trying to make sense of them, and eventually I started to get a feel for the structure, though I'm a long way off really understanding it. Basically, WebCore has classes for common GUI elements, like fonts, menus, and so on. To do a port, you have to reimplement these classes to wrap the same functionality in whatever graphics system you happen to be targeting. It was around this point I realised that I know basically nothing about the AROS GUI toolkit, known as Zune.

I had a look around for examples and documentation, and I started to see what was going on, but a lot of the code is a mess and its hard to get a clear picture of what's happening in my head. The only option left to me is to write a small application using some of the Zune features that I'll need to get an idea of what makes it tick.

I thought about it a bit on Saturday and today spent a couple of hours implementing a this little app that I call fonty:

Its a font preview program. You give it a font name and point size, and it'll render some text in that font. We already have stuff like it, so its not particularly useful, but so far I've learnt about the basic structure of a Zune application, how to make a Zune custom widget class (I have a seperate FontPreview custom class), and how the Amiga font system works. It'll soon have a context menu that allows selecting different styles, and changing the text. Again, not really great in terms of usability, but lets me see how everything works. And kinda fun to write too :)

tuesday, 20 november 2007

posted at 13:45

I finally finished my dependency porting stuff, with libxml2 coming to life late last night. I haven't tested it properly yet, as its test suite requires glob(), which we of course don't have. I'll look at integrating a version of it soon so that I can run the tests. For the moment I'm totally over dependency porting, and eager to get onto WebKit proper.

Before bed I wrote the first line of AROS-specific code in WebCore. Ready? Here it is:

typedef struct BitMap *DragImageRef;

I have no idea what it does yet, but it was enough to get the relevant file (WebCore/platform/DragImage.h) to compile, and thats all I care about right now.

The build is going well. So far I'm just stubbing platform-specific classes to get the thing to compile. Once its compiled, I'll start implementing those stubs.

One thing that was missing that would have been difficult to work around inside WebCore itself was the reentrant time function localtime_r(). A bit of hacking on the bus this morning and AROS now has this function, along with its friends ctime_r(), asctime_r() and gmtime_r(). Phew.

Tonight's work is adding stubs for PlatformMenuItemDescription, whatever that is :)

sunday, 18 november 2007

posted at 21:16

Today I finished porting cURL, a library for getting things from the internets (or actually, anything with a URL). Its probably the dirtiest port I've done so far, both because the configure script is a mess (it knows enough to know that I'm cross-compiling, but then doesn't know enough about cross-compiling to do anything other than get in my way), but also because of the bsdsocket.library madness (which if you've been in #aros at all in the last couple of days you'll have heard my opinions on).

Obligatory screenshot:

Code at /git/aros/curl.git.

In other news, I committed my mlib patches this morning after a little testing and tweaking by Markus Weiss to get them working on PPC. I'm quite proud of it - it was a big, unknown thing and it came off without a hitch.

So now, armed with the 20071118 nightly (available in just a few short hours), you can (theoretically) possible to built all of the Traveller stuff done so far. If only you all had some build instructions .. :P

friday, 16 november 2007

posted at 21:53
Quick one before bed: a port of [OpenSSL](http://openssl.org/), which is needed for [cURL](http://curl.haxx.se/), which is needed for WebCore.

It was actually a pretty easy port to make. OpenSSL is ported to so many platforms already that it was pretty much just a case of copying stuff from similar platforms. Amusingly, the platform most similiar to AROS as far as OpenSSL is concerned is Netware :)

Code available at /git/aros/openssl.git.

thursday, 15 november 2007

posted at 14:09

The results are in. The browser will be called "Traveller" (that's British spelling, with two ells). I had already thought of this as a potential name before asking for ideas, and when a couple of people suggested it too I knew it was good.

The reasons I like it are threefold:

  • Its a good companion for Wanderer.
  • It carries on the tradition of giving browsers a name related to finding the unknown: Navigator, Explorer, Konqueror, Safari, etc.
  • It references an in-joke among the members of my team at work, so its just a little bit personal too.

So thanks everyone for your input. I enjoyed hearing all your ideas :)

Relatedly, Paul J. Beel asked me a bunch of questions about the project and has just posted my answers over at the AROS Show. That should pretty much cover what exactly it is I'm doing and what you can expect.

I'm looking for someone who can is savvy with graphics to produce some art for the browser - icons, throbber, about screen, etc. I have ideas, but need someone who knows how to produce art and animations to give me a hand. Contact me via email (rob@cataclysm.cx or grab me on IRC (fce2 on irc.freenode.net).

Now that all the excitement and administrivia is out of the way, time to do some actual hacking.

wednesday, 14 november 2007

posted at 16:02
  • mood: coy

Just a few quick updates.

First, thanks all for your name suggestions. I hated some of them, I loved some of them, and I've finally decided on the name. Its one that I had thought of beforehand, but a couple of people suggested it here too. I'm not revealing it yet though; Paul J. Beel of The AROS Show has sent me some questions for an interview and I've promised that I'll reveal the name there. Of course I'll post it here shortly after, but of course you all read his stuff so you won't need it :)

I've started porting the WebCore dependencies. First up is the easy one, SQLite, which I finished porting this morning. Its a horrendous port, with no file locking and hacked up path handling, but it was the cheapest and fastest I could do, and will suffice for what I want. I don't want to get bogged down on tangents; WebCore itself is going to take enough time and brain to do without being distracted.

I'm now publishing my work as I go. The repositories for WebKit and its dependencies will appear at http://cataclysm.cx/git/. Feel free to clone from them and do whatever you want with the code. I'll post some build instructions soon; its quite hairy. I've also put my AROS repository up, which is where I'll publish stuff that hasn't made it to AROS SVN yet (usually becaused its unfinished and/or broken).

Thats all for now. Heading home now :)

sunday, 11 november 2007

posted at 21:01

Ever wanted to name a web browser? Here's your chance. I need a name now so that I have a way to refer to the whole project, rather than "WebKit port" (accurate until I start work on the chrome) or "browser bounty" (duh).

I have a couple of ideas, but feel free to post a comment with names of your own devising. I'll choose the one I like the most, or if they all suck, I'll choose one of my own. Its not a democracy, you know ;)

Update: Name has been chosen. Thanks all for your suggestions :)

saturday, 10 november 2007

posted at 10:07

Its been a big week of AROS coding, with a milestone being reached last night: JavaScriptCore, the JavaScript engine inside WebKit, is now compiling and running inside AROS. As such, I'm satisfied the a full port of WebKit to AROS is feasible, and as such, I've taken on the bounty to produce a browser.

My process for building WebKit has been simple. I made minor changes to their build system to use AROS crosscompilers, and then let it build until it breaks. Then I go in, figure out what died, and fix it. Often this is easy, requiring only some platform defines and such. Sometimes its been a little harder, which is where posix_memalign() came from. The really fun thing happened at the start of the week when the build failed because a couple of math library functions were missing.

Our math library (known as mlib or arosm, depending on where you look, though every other platform calls it libm, go figure) was originally taken from a math library written at Sun way back in 1993, and released for free. We got our copy from FreeBSD in 1999, and it was updated again in 2003. Its missing a lot of stuff though, notably things from C99.

I had a look through the FreeBSD code and found the functions I needed, but on noticing just how much stuff was missing I decided it might be better to do a full refresh of libm. As is usual when I start on something, it rapidly got out of hand.

I had to make a few changes to our core headers to provide all the necessary defines and types and such to make it work. The new code also has an amount of architecture-specific code for using the FPU. Fortunately FreeBSD supports all of the architectures that we have active ports for (i386, x86_64 and ppc), so it was just a matter of getting the right code into place.

In any case, lots of tweaking and merging has been going on such that I now have about 20000 lines changes spread out over 21 patches. I haven't committed them yet as I'm waiting on some build macros from Staf to allow me to build the architecture-specific files into the library correctly. My hacked version seems to work well, and passes a couple of tests from Fred Tydeman's C99 FPCE test suite. I'll run all the tests soon, but I expect them to pass without issue.

Once the patches can compile cleanly, I'll try to get some other AROS devs to review them, as they're big and I'm scared. Once its all deemed good, they'll go in, and we'll be doing fancy math forever. Hurrah!

Anyway, after shoring up the holes in AROS, it was back to JSCore. The code is exceptionally well written, and easy to port. Apart from adding #ifdef here and there, the only actual code I had to write was stuff to help the garbage collector find the stack base, and thats two lines in kjs/collector.cpp:

    struct Task *task = FindTask(NULL);
    return task->tc_SPReg;

The JavaScript engine test program testkjs runs properly. The only issue is that the garbage collector is not fully cleaning up all the objects at script exit, which I think may be a memory management issue. I haven't fully tracked it down, but the folks in #webkit (particularly bdash) have been very helpful and I'm expecting to have it sorted out soon.

So thats my progress so far. My plan for the browser proper is to implement it in two stages. The first is the port of WebKit proper, which is a porting JavaScriptCore and WebCore, writing a trivial launcher application, and porting libraries it depends on and otherwise fixing things in AROS. Once thats done, the second stage begins, which involves integrating WebKit into AROS proper. I haven't thought this through fully yet, but I expect at this point that I'll be writing a Zune widget to allow applications to embed WebKit, and from there writing a Zune application to be the browser proper.

I'll be making my git repositories available shortly, so the brave can track my progress. And you'd better believe that only the brave need apply - you need to be willing to track AROS and WebKit SVN repositories and regularly recompile AROS, gcc and WebKit. Oh, and there's a 20-step build process for ICU as well, one of the WebKit prerequisites. Its early though, this will be made easier after I'm finished so other people can hack on this too.