The Cliffs of Inanity

Wed, 14 Jun 2006

Moving my blog

I'm moving my blog to a new location (RSS feed). With the new site I have more space (for screenshots or whatever). Also I can finally do posts while traveling, and also enable comments.

The old peakpeak.com site is dead, I won't be adding entries here any more.

posted at: 09:48 | path: | permanent link to this entry

Sun, 11 Jun 2006

Web Pages

I've been looking at moving my blog to something less lame -- right now I can't blog while I'm traveling, which is why I still haven't written about my trip to the Red Hat Summit. I've also been looking at other software for creating web sites.

While exploring I ran across the Open Source CMS site. This site lets you play with many different CMS, blog, photo gallery, etc, installations, so you can try them out before committing to one.

So far my experiences with these programs haven't been very positive. I'll write more about that later.

posted at: 11:47 | path: /personal | permanent link to this entry

Wed, 24 May 2006

A Couple libgcj Updates

I've been working steadily on replacing gcj's front end with the Java compiler from Eclipse. This is largely working now. I have a new main program for the Eclipse compiler, and I have gcc set up to invoke this (via the magic of gcc specs -- an evil little ad hoc scripting language that you should hope you never have to learn).

The new driver is a little funny. When ecj compiles a file, it writes the classes to a jar which gcj compiles. This way we don't have to have an arbitrary number of temporary files for communication, e.g. for all the inner classes. This takes advantage of java's built-in ability to make jars, and gcj's existing ability to compile a jar file all at one. I thought this was amusing, anyway... maybe I've been working too much.

I thought this would be very simple, but I should have realized that this would reveal every bizarre class file compilation bug in gcj, some of which can only be seen if you are compiling the core class library this way. For example, the bytecode verifier needed a special case to handle the constructor in Object.

In any case, I can now build all of libgcj this way. I'm debugging a few runtime failures now.

LLVM-based JIT

Aside from the whole exception handling mess -- which experts more expert than I are, hopefully, busily hacking on -- the JIT seems to be working reasonably well. I'm just about ready to clean up the API and check it in (as an experimental preview), I don't think any more changes in that area will be needed in the near future.

There are still a few lurking code generation bugs. Nothing too hard, just mishandling jsr a little.

I recently added the first bits of recompilation to the JIT. Now it will realize when it resolves a constant pool entry, and mark the method as ready for re-jitting. The idea here is that before linking, a constant pool reference requires a method call to incrementally link the class, whereas after linking, a constant pool reference is simply a constant. Another similar optimization is that when we initialize a class, we mark the method as ready for re-jitting -- the code to lower from bytecode to LLVM will check a class' state and avoid emitting initialization calls as needed.

This still hasn't seen much testing. And, to really do a good job here we have to add profiling code of some kind. I really need to read the literature here. If only there were time.

posted at: 13:06 | path: /software | permanent link to this entry

Mon, 22 May 2006

Eclipse and Teams

In Classpath we've narrowly avoided most of the team-related Eclipse problems Daniel Berrange mentions encountering.

Partly this has been because we're all mad free software hackers, and so we don't even try to ensure we all have the same tools. Instead folks doing development in Eclipse have a choice of operating systems and of virtual machines (both jamvm and cacao work nicely in this environment).

Over time Eclipse has gotten better at separating the personal from the project -- Eclipse 3 is much better than Eclipse 2 was here (e.g. now you can store the coding style for the formatter in the project). And, we do make an effort to set things up so that typical uses won't require any changes to the project metadata; that both avoids unpleasant accidents and lets people in different environments continue to get sane results. Partly we're able to do this because we have an unusual hybrid build which includes autoconf (this isn't without its bad qualities as well).

We put a fair amount of effort into making our setup turnkey. There's a nice web page on the wiki which explains, step-by-step, how to get set up for working on Classpath. This is still a lot harder than we'd like, but it can be done in thirty minutes or so if you're already familiar with Eclipse. I demoed this at FOSDEM this year; most of the time during the demo was spent waiting for the build (Classpath is over a million lines of code, so that can be forgiven). In real life you also spend some time waiting for the network -- my demo was with a local cvs server.

Also, supposedly team sets help a bit here. As I understand it a team set is a way to specify a group of projects to check out. My brief experiment here was a little un-promising; I think it picked the wrong user for CVS repositories by default (I was hoping it would know that when I used "anonymous", I wanted the end user not to have to specify a username). Someone in the Classpath community (I forgot who, sorry) did make a team set but then it never ended up on the wiki... (ping).

As a user you do have to watch out a bit. I once installed the FindBugs plugin, and then to my dismay found out that it modified the project builders. This is unacceptable; I had to remove the plugin to avoid accidentally contaminating the shared build. (Any plugin architecture suffers from problems like this, though. Allowing plugins means giving up centralized quality control.)

For what it's worth, our "initial setup" problems with Eclipse haven't been much better or worse than the problems we've had with shell-based builds. Eclipse could definitely improve here. (Once you're set up, of course, the difference is startling.)

Ideally Speaking

My ideal setup for these things would involve more integration between Eclipse, the OS, and the organization.

It would be nice if a project could specify required plugins; when checking out the project Eclipse would first launch the update manager to install them. In this ideal world, the update manager would work with RPM rather than stand perpendicular to it.

It would also be very nice to have rendezvous support or the like; the idea being that in a typical organization you would simply launch Eclipse and not have to figure out where the version control server lies. In the free world an analog would be the ability to click a link in Mozilla and have the repository automatically show up in Eclipse. Every project's home page would have a big "Hack Me" button which you would click to get a working development tree. Or, you could have Eclipse interface to irc and pick up on repositories and update sites that way... join #classpath, get a dialog asking if you want to access the classpath team set.

This latter idea gets to something that bothers me about programming. Computers are powerful communication devices, and a huge part of our jobs entails using them to communicate. However, much of this communication happens at two extremes -- there is the extremely unstructured form like email or irc; and there is the overly structured form, like a cvs commit. I'm hoping for a generation of tools that is a bit more loose; tools that notice interesting things without too much interaction or attention on my part; and also tools that let me more easily wire up communication the way I need it (my perennial example here is the amount of time I would've saved over the years if only we could simply drive the debugger over irc).

There's some work in this area. Eclipse has ECF, NetBeans has its thing, whatever it is called. There's also the Jazz project for Eclipse, from IBM. This isn't open source (it may be someday), but I hear the demo is quite cool. It sounds a little too command-and-control for my taste, perhaps, but I'd imagine it can be made a little more peer-oriented.

Naturally, none of this is available on the timescale I want, namely last week.

posted at: 01:31 | path: /software | permanent link to this entry

Sun, 21 May 2006

NCLUG

Last week I drove up to Fort Collins to give a talk about gcj at NCLUG. I thought it went pretty well... I gave an updated version of my old talk from FOSDEM 2004, but then deleted the slides by mistake when I was trying to upload them. The problem with my computer (and me!) assuming that I'm a power user is that, occasionally and unpredictably, I am not.

Afterward a bunch of us went next door for Chinese food. I talked to Evelyn from tummy.com a bit. Apparently Fedora has let them retire KRUD, a local RH-based distro. From the KRUD page it isn't clear if this is a plus or a minus, but in my mind it is a plus -- it means Fedora is successfully addressing needs that were not addressed by the old Red Hat Linux.

Evelyn also had an experience similar to mine -- and everybody's, I suppose -- when installing linux for desktop use. I can't just install Fedora, I must also download flash (mozilla makes this easy, but of course yum would be nicer), java (I didn't on my FC5 box, but partly because I'm keeping up appearances), and various sound and video things. Evelyn also needed acroread, to my surprise; but apparently only acroread can handle editing PDF forms.

Add to this the messy situation with proprietary drivers (my laptop came with the atheros wifi stuff, which I still can't get to work on FC5) and the lack of ipod support, and you'd think that Linux sucked.

I'm still hopeful though. We'll outgrow this annoying phase.

I also learned about Night Vision for Java, a planetarium program written by Brian Simpson (he was sitting across from me at dinner). Apparently this runs ok if you enable the java2d stuff in Classpath; he tried it without success during my talk but I'm told that things are all fixed in cvs (which, I hope, we'll be shipping in FC6).

Finally, I got to meet Bob Proulx. Bob does a lot of stuff in GNU-land and I had seen his name before on the automake list, but I embarrassingly failed to connect all the dots until after I had left. I hate those awkward social moments. They seem to occur more often to me than to other people.

I'll be back in Fort Collins in a couple months to talk about autoconf and automake. A little weird, since I haven't worked on these for so long.

Eclipse Plugins

I was also in Raleigh last week for a speaker training class, and I caught up with Andrew Overholt there. We talked a bit about Eclipse packaging, a hell we've both had to live in.

Whenever I think about what it was like to try to build that thing, or its various plugins, I start thinking: why bother with this at all? It's just a huge mess!

But then I remember more. Of course we have to build it. We're building the OS, which changes. We need a reliable process from start to finish so we can make and ship bug fixes. These are, btw, the same reasons that open source java is needed -- compatibility is desirable, even necessary; but it is meaningless if you have no power to fix the bugs preventing it.

As a user, it is convenient to just use the eclipse update manager to download things. (Well, sort of convenient. The update manager UI sucks and it has zero integration with the mozilla or anything else.) And I do use it for a number of plugins. But installing an OS reminded me why this approach sucks -- it is a lot friendlier to have a single way to install everything. The Eclipse approach means yet another step in setting up a machine.

I suppose one answer here is to set up a site that provides a bridge.

I've often thought about making an Eclipse meta-update site, which would mirror every plugin available. The idea here is, why bother copying those URLs to the update manager, navigating its brainless UI once again? Instead, let one person do this and let Eclipse users just point at this site. (The only problem with actually doing this is that I couldn't think of a way to make money off it. No ad revenue via the update manager :-)

Anyway, in conjunction with that I suppose you could auto-generate RPMs from binary plugins, and from there a convenient yum repository. This would solve the problem on the user end. Distros would still be screwed, of course. Annoying binary distributions are the Java standard, and Eclipse would just keep on contributing to the problem.

posted at: 01:18 | path: /software | permanent link to this entry

Fri, 19 May 2006

Happenings

So the buzz is that Sun will really actually truly free Java sometime. Details, timeline, license, etc.: TBD.

This makes me feel very weird. I assume for a moment that it is true and that it happens under acceptable conditions: it comes pretty soon, it is complete, it is under a non-crazy license. On the one hand, hallelujah! This is what we've wanted these 10 years.

On the other hand... I wonder what I'll do with myself. I suppose there are plenty of interesting things to work on. Even the Sun JDK I suppose.

But the dislocation goes far beyond my future to-do list. What does this mean about all the work I've done? Is it a waste?

I probably should've come up with answers to that back when we merged libgcj into Classpath and nuked a lot of code. Sometimes I feel bad about that process.

I do have my own answers for those questions. Everything is born, lives for a while, and dies; our programs are no different. That they die early or late doesn't render them meaningless -- only dead. And meaning itself is something we bring, in interpretation; it isn't an intrinsic quality. Of course it is one thing to think that and another to know.

Whew. Back to reality, we're still hacking away on gcj. It makes no sense to change course based on a maybe as big as this one.

Miguel's blog pointed to a nice entry on this topic.

Danese Cooper says we're too poorly organized, or at least thought of that way. I think she is using "organized" to mean "backed by IBM" or something like that. Anyway there's not much correspondence between that idea and what we've actually done.

It is true that Harmony has been a notable winner in lining up IBM and Intel behind it. I often think of Harmony as a consortium in the guise of an ASF project. I suspect our failure here was our license; but it is difficult to say whether this was really a mistake per se.

She also wonders. "I'm wondering how long it will take the various Linux distros to figure out that they can ship Harmony". We already know about Harmony. When shipping it isn't a big regression from shipping gcj, we'll probably ship it. What does that mean? It means that platform coverage and library coverage matter. Meanwhile gcj remains the best free VM on my list of metrics: platforms, performance, debuggability, and community.

gcj details

I've got the eclipse front end plugged into gcj here. It consists of a new driver for ecj and a patch to the gcj specs to invoke it. I'm debugging some .class compilation bugs that this found, but I should be able to build everything soon. (I've already built 1.5 code with it.) Next step: a branch in the gcc repository.

gnash

Last night when I couldn't sleep I became bizarrely interested in gnash and flash software. First, I found the gnash source code kind of unreadable -- pretty messy. I read a bit about SWF; what a weird setup this thing has.

A flash plugin is a classic example of what not to write in C or C++. You end up reimplementing the world. Instead, start with a library-rich language like java and it looks much simpler. I found JSwiff for SWF reading. Am I deluded when I think that this plus Java2d (and sound and I guess JMF -- yuck) plus a bit of glue would make it all happen?

posted at: 15:30 | path: /software | permanent link to this entry

Sat, 06 May 2006

Another One

Today I wrote another optimization pass for gcj. This one collapses equivalent vtable references and array length references. You'd think that GCC itself would do this, but there's no way to tell the optimizers that a given field is write-once.

Really I should fix GCC to do this... but writing a new pass is easy to do, and fixing the generic code looks daunting.

The other day I also rewrote my devirtualization pass to use the SSA propagation engine. Again, simple to do, and it improved the results a bit.

Hacking GCC these days, while still tricky in some details, is just enormously simpler than it was 5 years ago. Kudos to all the tree-ssa folks who made this happen.

ecj

I spent some time this week hooking ecj up to gcj, as threatened. I've got a new driver for the eclipse compiler that eases the argument processing a bit. This is working well enough now that I was able to successfully compile some source code using generics by running the gcj driver.

If only I had a decent place to check this in. I wonder if the SC would let me make a branch for this, even though it is in political limbo.

JIT etc.

I started writing my GCC optimizer passes because I was curious about writing a devirtualization pass for LLVM. I wrote about half of it and then thought that surely this would be just as simple for tree-ssa.

I've been thinking a bit about heuristics for when the libgcj JIT should recompile. The easy ones are things like: recompile when classes are initialized, so we can remove initialization calls from inside loops; and recompile when constant pool references are resolved, so we can replace expensive indirect accesses with cheap direct ones.

There's probably a lot of literature out there that I should be reading on other times this is worthwhile -- detecting when partial specialization is worthwhile, profile-directed runtime optimization, etc. Maybe HLVM will help.

Actually doing the recompilation is simple; LLVM provides the needed hooks. For things like constant pool references, I think I will take the simple approach of simply re-lowering from bytecode to LLVM. If this proves to be too expensive, it can always be changed, I think. But I suspect it won't be. And, anyway, it will be fun finding out.

posted at: 14:11 | path: /software | permanent link to this entry

Mon, 01 May 2006

It's that time of year

SlashDot is running a story speculating about Sun open-sourcing their Java implementation. Hey, it must be just about time for JavaOne again!

It is sad how predictable this PR campaign is. And how predictable the reaction.

posted at: 22:20 | path: /software | permanent link to this entry

Fri, 28 Apr 2006

gcj optimization pass

Today I wrote a GCC optimizer pass. The new pass is specific to gcj and does a simple form of devirtualization. The idea here is that if we have some extra information about a method call, we can turn an indirect virtual call into a direct call.

My pass does this in one particular case. If the "receiver" object of a virtual call was allocated with new in the current method, then we know its exact type, and we can devirtualize. This is conceptually trivial on the SSA form.

I wrote this pass since I had been playing with similar code for my LLVM-based JIT, and I wanted to compare LLVM and GCC here -- I'd never written a GCC optimization pass and was curious about the effort involved.

It turns out to be simple. This pass is about 200 lines of code. And, when building libgcj, there were more than 6000 cases where it triggered. I'm encouraged by this and now I'm considering writing more gcj-specific optimization passes. Some ideas:

• "Strength reduce" interface calls to ordinary virtual calls. The only difficulty here is in the bookkeeping; it could be inserted into the current pass.
• Special handling for StringBuffer and StringBuilder.
• A simple gcj-specific VRP-like pass, to handle array bounds checks, null pointer checks, redundant checkcast calls, and the like.

I guess some of these would make ok SoC projects.

posted at: 19:16 | path: /software | permanent link to this entry

Thu, 20 Apr 2006

LLVM JIT now accessible

I got a nice bug report about the LLVM JIT from Haren Visavadia the other day; his one short test case found 4 or 5 bugs.

I decided to stop hoping that sourceforge would start working well, and instead I just moved the JIT cvs repository to sourceware. It is now on sourceware.org, repository /cvs/rhug, module gcj-jit. Instructions on how to build it are included. You can also see it via cvsweb now.

If you give it a try, please drop me a line, especially if you hit a bug.

posted at: 10:52 | path: /software | permanent link to this entry

Mon, 17 Apr 2006

LLVM Update

Last night I found the buglet in the JIT preventing "hello world" from working. Now it is time to start more serious testing; first the libgcj test suite and then Mauve.

posted at: 09:19 | path: /software | permanent link to this entry

Sun, 16 Apr 2006

LLVM Thoughts

On Friday I translated my libjit-based JIT to use LLVM. This took a good part of the day; then I spent a chunk of Saturday debugging it.

LLVM has a few drawbacks, as compared to libjit. There's not really any documentation for how to use LLVM as a JIT, so I ended up reading the header files quite a bit; libjit is much better here. LLVM's API is quite a bit bulkier than libjit's, and it is more idiosyncratic. For instance, in LLVM many objects can have a name, and many classes require a name in their constructors; this seems a bit bloaty in a JIT context -- but I didn't measure it. Finally, LLVM is installed strangely; it is mostly static libraries, but with a few random object files thrown in for good measure. This is unfriendly to say the least... also, link times with LLVM are much longer than with libjit, reducing my efficiency.

Some of these I would like to see fixed -- either in LLVM or in whatever ends up, someday, in GCC. Names could perhaps be handled optionally. Other oddities in the interface could be fixed (not that I have a list or anything...). Shared libraries should be made.

All this is gas, though, in a way. LLVM is generally more functional than libjit: it has many more ports, more optimizers, and a friendlier license. It probably represents a better long-term approach.

With a little help on irc from Chris Lattner I got the LLVM JIT running a couple very simple Java programs; with the optimizers enabled it appears that LLVM correctly notices that the empty loops are empty and removes them... so, it is working. There's still a lot of debugging to do ("hello world" still crashes), but this isn't a big deal.

Naturally, exception handling remains a problem. I'm hoping to get Bryce or Andrew to solve that problem :-)

posted at: 18:45 | path: /software | permanent link to this entry

Thu, 13 Apr 2006

libjit and gcj

Last weekend I wrote a JIT for libgcj using libjit. Well, 90% of a JIT anyway.

libjit is remarkably simple to use. It took me about a day to write a functioning (if not completely debugged) JIT for java bytecode. On some microbenchmarks it was between 2 and 6 times faster than the existing bytecode interpreter.

I've checked it in to the old gcjx repository... but you won't be able to see it; I heard that sourceforge has stopped updating its anonymous CVS. Email me if you want a copy. The repository includes a patch for libgcj, the needed modifications there are very minor.

Note that exception handling doesn't work. This is somewhat hard to do, since it requires modifying the JIT and also (probably) patching libgcc. And...

Unfortunately libjit is pure GPL, so I doubt we'll be including this in libgcj, or even finishing it. Instead I think I'll investigate rewriting this JIT using LLVM instead. I've been thinking of generalizing my existing patch to libgcj to make it possible to dynamically load a JIT. That would make it easier to experiment here.

posted at: 19:05 | path: /software | permanent link to this entry

Sun, 09 Apr 2006

99 percent

According to our nightly JAPI run, we hit 99% of 1.4 the other day. Finally! We've been hovering above 98% for quite a while, and checking in the patch to make stubs disappear from the JAPI score (thus making it more accurate) didn't help.

At this point it looks like 1.4 completion is mostly about filling in a few missing pieces here and there, and finishing the HTML support.

The 1.5 scores still look like a disaster, since the trunk doesn't have generics and the generics branch doesn't have all the most recent patches. We'll be fixing this problem this year when we merge the generics branch back to the trunk.

posted at: 12:20 | path: /software | permanent link to this entry

Wed, 15 Mar 2006

FOSDEM 2006

I've been back from FOSDEM for a couple of weeks. I had a rough homecoming, though, and after that got lost in catch-up and bug fixing for a while. Now I'm finally ready to write about what I saw.

The Classpath contingent was lively again this year -- some new faces, but also, unfortunately, some folks didn't make it. I suppose we'll never achieve 100% attendance; that's ok since I always leave FOSDEM feeling as though I should have somehow fit in even more talking than I did.

I stuck to the Classpath room this year. I had really wanted to get over to see the PiTiVi talk in the Gnome room, but this overlapped with the Classpath general discussion session and so I couldn't leave. Maybe with FC5 I'll be able to easily build my own pitivi and demo it to myself.

Free Swing - Roman Kennke

The first talk I have notes for was the Free Swing talk by Roman Kennke. He gave an overview of the Swing work, contrasting November 2004 (when he started working on it) with today. The Swing hackers have made huge progress -- a year ago it couldn't really do anything, and now it can run many actual programs. There are still some missing things, but today the list looks doable.

For 2006 Roman essentially predicted completeness. His real list was styled text and HTML, 1.4 compatibility, stability, usability, and java2d.

CORBA - Audrius Meskauskas

Audrius talked about his work on the GNU Classpath CORBA implementation. He may have mentioned his work on RMI in passing -- it isn't in my notes but I remember something about this.

Audrius singlehandedly wrote our CORBA code. Some of the other free ORBs rely on un-free code -- IDL from the OMG which can't be modified (incidentally this is apparently one reason we avoided jacorb). Not us, Audrius reverse engineered it from the spec. Our implementation also passes more of the official test suite than other implementations.

JamVM - Robert Lougher

The first thing my notes say is, "send him some congratulations". So, congratulations, Rob. JamVM is an excellent program. I use it every time I work on Classpath and I'm always grateful that it is so easy to set up and use.

Normally I'd follow this with a wish-list, but as Andrew pointed out, the time lag between "that's cool" and "feature X is broken" is often much too short. So let's just leave it at congratulations :-)

Rob's talk mostly concentrated on JamVM's implementation approach. It is what I would describe as a state of the art interpreter. It is direct threaded, it does stack caching (it has 2 cache registers and hence 3 versions of each opcode -- except on x86 where this is not very useful), and does dispatch prefetching.

Apparently the stack cache yielded a 20-50% improvement, depending on the benchmark, especially in combination with the dispatch prefetching (also called interpreter pipelining or something like that).

Vmgen - Christian Thalinger, aka twisti

Twisti gave a talk on incorporating a vmgen-based interpreter into Cacao. vmgen is a cool project from Anton Ertl (read his interpreter papers, they are quite nice) which automates certain aspects of stack-based interpreter construction. The basic idea is that you write your instructions (in a kind of mixture of C and a forth-like language) and it will generate your interpreter.

The trick is that vmgen knows how to do all the nice interpreter optimizations for you, so you don't have to do them by hand. So, it can do direct threading, stack caching, dispatch prefetching (Twisti said it doesn't have this, but I know I first read about it in one of Ertl's papers... so I dunno), and both static and dynamic superinstructions.

A static superinstruction basically turns a common static instruction sequence into a single instruction. You decide what superinstructions to make, and then the code that vmgen generates to handle direct threading will automatically recognize these patterns. The idea is to cut down on dispatch overhead for common code.

Twisti said that they tried the absurd approach of having 10,000 different superinstructions, but in the end found that just having 100 is enough.

Vmgen also can do dynamic superinstructions. The idea here is similar to that of SableJIT (or qemu, I think, though I haven't looked), where you memcpy the executable instruction sequences (that GCC generates for your hand-coded VM instructions) into a big buffer, and then execute that. It is basically a cheap JIT.

Now we just need to get Ertl to hook vmgen up to LLVM :-)

Twisti thought it would take about 2 weeks to get a vmgen-based interpreter up in another free VM. I'd imagine that due to oddities in our ABI it might be harder to do it for gcj :-(.

Both Rob and Twisti had performance results. Both these interpreters beat gij pretty handily (and also the Sun and IBM interpreters), probably due to our ridiculous function call overhead. I forget which of vmgen or jamvm was fastest; I think they were reasonably close.

Knopflerfish OSGi - Philippe Laporte

Philippe Laporte gave an impromptu talk about Knopflerfish OSGi. I didn't take many notes on this, except that it is BSD licensed right now.

My Talk

I didn't take any notes on my own talk. That would have been weird. My talk was about hacking on Classpath in Eclipse. This went pretty well -- I used the native eclipse on my laptop and didn't have any major problems. The talk is basically already encapsulated in a page on the Classpath wiki.

Round Table

At the end we went around the room to talk about what we were doing and the like, led by Mark. I learned some interesting things here.

Robert Schuster got a job hacking on Classpath, and in particular Swing. His company does some kind of office application.

Wolfgang Baer talked about printing a bit. He talked about using iText for printing PDFs. I didn't really understand what he was about until I saw his patch this week that implements an IPP client in Java. I'm constantly amazed at what Classpath hackers come up with -- congratulations Wolfgang.

Our Plans

I also talked a bit about what we're doing at Red Hat. Nothing here is too surprising. We're currently working in several areas: JDWP (so you can debug from inside Eclipse), security, the library (in particular Swing and java2d), and 1.5 stuff (compiler, runtime, and library).

We tend to do application-driven development, where we look at the applications that we want to have working, and then finish the things needed for those.

We've got a pretty long list of interesting applications at this point: jonas (which is driving some of our performance work), the RHDB tools, Fedora Directory Server (this drove some of the Swing work), Megamek (found lots of AWT bugs), FOP/Batik (driving Graphics2D work), gcjwebplugin/netx (driving security work). Of these I would say that gcjwebplugin is probably the most interesting.

Everybody got a chance to speak during this part, but I didn't write down most of it. There was a film crew here, too, and at the end they asked some questions, which were ok but a bit on the bland side.

posted at: 16:46 | path: /software | permanent link to this entry

Wed, 15 Feb 2006

FOSDEM

I'm leaving soon for FOSDEM and getting all my stuff in order. (Unfortunately due to my lame setup I won't be writing any blog entries while I'm away... I really need to fix this. I've been eyeing bluehost, since that is what Elyn uses.)

At FOSDEM I'll be giving a demo on using Eclipse to develop Classpath. Not super deep, but something I'm passionate about. This switch gave me a a major productivity boost, and I think this is something that most Classpath developers can benefit from.

In fact this boost is one reason I'm interested in changing g++ to be better -- moving from the Java world back to the C/C++ world is amazingly painful, and not for any good reason. One way that GCC's relative dominance is a bad thing in that there's no competition to force it to improve quickly. That is, there is competition on the back end from things like icc, but nobody is carrying the flag for developer productivity and front end improvements.

Medi8

I've done a little Medi8 hacking lately. I have thumbnails working better and it can now turn its internal model into Westley XML. This means we're a big step closer to, you know, actually doing useful things like playing or exporting the resulting movie.

posted at: 14:30 | path: /software | permanent link to this entry

Mon, 06 Feb 2006

Miscellaneous Stuff

libjit looks pretty nice; pity Mono didn't take this route. I'm not sure why you'd use this rather than LLVM though.

I've been using mock lately. This is a nice tool that makes it easy to build an SRPM in a particular environment. I'm trying to set up an auto-builder which will build gcj from svn, then use the result to build all the java-based RPMs in Fedora. The idea here is, continuous testing as close to upstream as possible will let us avoid ugly regressions late in the release cycle.

Aspect-Oriented C++. Madness! But, I suppose, one more reason that one might want an API to the compiler's internals.

I didn't realize this before, but Chandler also has a copy of libgcj in it. I suppose this is for PyLucene... funny. Their plan for integrating into distros mentions using the system libraries, python, etc... I wonder if they know about the BC ABI.

I've been interested in Deming's 14 points lately. I'm particularly interested in number 8, "drive out fear", since it has a personal application, but really all of them are worth a read and a think.

posted at: 15:30 | path: /software | permanent link to this entry

Sun, 29 Jan 2006

gcjx non-status

This week I proposed killing gcjx and replacing it with the Eclipse compiler. Per had looked into this before, but this proposal was triggered by a comment that Andrew made on irc that same morning. I surprised myself by taking to it with enthusiasm.

Since then I've done some more investigation. This project seems very practical, and I think will let us have a 1.5 gcj much quicker. There are a couple potential optimization regressions by going this route, but these are fixable in the compiler without too much work.

I also spent a little time hacking on the eclipse compiler's driver, trying to get it in shape for an experiment to test this plan. That turned out to be easy.

While doing this though I finally felt the sadness I knew would eventually arrive. The trigger was something very minor -- I was looking at the eclipse compiler driver, and realized that on a lexical level it is pretty ugly code. There aren't many comments, and the ones that are there aren't very good; the class I was hacking on didn't have a very layout or even consistent indentation style. And so I took a quick look at the corresponding code in gcjx... we're definitely losing something in this exchange. (But to be fair, the driver is not exactly a core part of the compiler. I doubt it gets much love.)

We're not losing much though, and I still think this is the best way forward. Plus, and this also surprised me, I seem to have gotten whatever emotional fix I was looking for from writing gcjx. I started it at a kind of professional local minima, and writing it helped remind me that I'm reasonably competent at this programming thing. Now I'm on to feeling inadequate at a higher level.

Future compilers

I think some aspects of gcjx should be emulated in all future GCC front ends. For one thing, front ends should have their own representation, derived from the language being compiled -- they should treat GCC trees as a target format, not a high-level representation. Trees aren't statically typed, and they carry too much other baggage as well.

Second, front ends ought to be written as libraries. These days it isn't enough to write a traditional batch compiler -- you really want to look ahead a bit and consider IDE indexing, incremental compilation, and other uses of the parser and semantic analyzer.

More recently I've been interested in applying this treatment to the C++ compiler. Recently I've been surprising myself quite a bit; I was never interested in C++ compilation at all until the last few months.

posted at: 13:27 | path: /software/gcjx | permanent link to this entry

Fri, 27 Jan 2006

Frysk

I was looking around today and noticed that Frysk has a redesigned web page. Looking good!

The "questions" page seems to be fleshed out a bit more, at least as compared to what I remember. For one thing it is a lot clearer about the relative strengths of frysk and gdb. Nice job guys.

posted at: 17:22 | path: /software | permanent link to this entry

Context-threading

Graydon pointed me at a paper on context threading, a somewhat different way to organize an interpreter. It is a nice trick -- subroutine threading with a twist, or a baby step toward JITting. It is a way to avoid trashing the pipeline, which happens in an ordinary interpreter due to all the unpredictable branching. Unfortunately I don't think this would work well for gij, due to the way we handle exceptions; a problem that also affects most of the JITs we've considered embedding. Andrew has talked occasionally of changing our exception handling approach, that is looking more and more necessary.

Java Language Future

Here's the presentation on the future of the Java Language that I saw part of while I was in Brazil. It is interesting for the most part. The XML bits are fairly weird.

posted at: 10:07 | path: | permanent link to this entry

Wed, 18 Jan 2006

Trying Chandler

I downloaded and installed Chandler yesterday, and copied over my (few) appointments to try it out. I only use evolution for its calendar, and I've left it running as well so that I can more easily compare the two side-by-side.

Chandler installed easily -- the FC2 RPM installed without hitch on my FC4 box. There's no .desktop file for it, so I ended up making one of my own... in general its Gnome desktop integration is weak at the moment. The install is big, but then they do package their own copy of Python in there -- a typical (and sane) ISV approach, but still a little unfortunate.

Chandler is definitely prettier than evolution. It shows my events in color with a nice (but by now almost standard) gradient effect. The icon for Chandler is also beautiful, though again in a somewhat standard way. You can see this stuff on the main Chandler page.

Chandler does fix one evolution bug that has bothered me. So far they are doing well on the little details like these, especially when you consider that this is a pre-1.0 release. Also, they have been very responsive on my bug reports -- big plus to them!

It seems more memory-hungry than Evolution, that's a minus.

I didn't try any of the sharing stuff. Unfortunately for Chandler my calendaring needs are quite modest at the moment.

I frequently wonder about the rationale behind writing Chandler. It is nicer than Evolution in some ways, but surely not enough to justify writing an entirely new application. Is it really about the cross-platform-ness of it? As in, success looks like Mozilla, having users across the platform spectrum? I haven't found a really straight answer to this, and I missed a chance to ask it directly at OSCON.

Cosmo

The OSAF is also working on Cosmo, a calendar server. This is an area I've never really looked into; I seem to remember hearing a lot of complaints about the lack of this in the past, but now there seem to be a few of these around. Maybe someone out there knows what they are and how to compare them?

posted at: 17:26 | path: /software | permanent link to this entry

Tue, 17 Jan 2006

Netx RPM

I've packaged Netx for FC5: x86 RPM, SRPM. Thanks to Anthony this version will install itself as the handler for JNLP files. WARNING: I've disabled the security code in this version, so use at your own risk.

Install it and you will have a new javaws command. Now you can try a couple fun things. First, listen to the radio:

javaws -headless -jnlp http://irate.sourceforge.net/webstart/stable/irate-client-swt.jnlp

(This one must be run headless due to an unusual bug... either be patient while it downloads, or run it normally and then kill it after the download and then re-run with -headless.)

Then you can run a fun game:

javaws -jnlp http://rockz.co.uk/preview/plugin/rockz.jnlp

I've been playing this lately but I'm really bad at it.

Unfortunately netx doesn't play too well with gcj 4.0.x, so this package won't really work on your FC4 box.

Duh, I just realized that I should just give out real JNLP links: here's iRate Radio and Rockz. I still haven't tried this part, let me know if it works :-)

posted at: 18:11 | path: /software | permanent link to this entry

Rockz

I went through a number of the applications on the connect and work site, trying them with netx and gij. Only a few really worked, mostly due to bugs in our still-new swing implementation.

However, Rockz is a fun game that actually works great.

Last night I made a netx RPM; I'll post it once I dig up a rawhide box to test it on.

NetBeans -vs- gcj

The NetBeans FAQ mentions gcj. We must have officially arrived. Dalibor thinks we'll run NetBeans this year. I'm a bit less sure, simply because it uses a number of internal APIs, which are occasionally a pain to work around.

posted at: 10:51 | path: /software | permanent link to this entry

Fri, 13 Jan 2006

Absurd JNLP Site

I ran across this absurd JNLP site the other day. It has 2,927 web-startable applications on it. I need to find a minion to test each one of these with gcj and netx. Please email me to volunteer.

Voight-Kampff

Politically relevant Voight-Kampff test.

posted at: 13:21 | path: /software | permanent link to this entry

Wed, 11 Jan 2006

Java versus C# Delegates

This white paper explains why Java doesn't have C#-style delegates. It should be read with care; the "application footprint" section is pretty bogus, contrasting the existing reality of delegates with a hypothetical (but easily possible) future world where anonymous classes have less metadata.

What I found most interesting about this paper, though, is its bigger mistake. Much of its argument is actually quite reasonable. However, it completely misunderstands the psychology of programming language adoption by developers, and in particular of those developers most likely to imprint on a language and argue (usually inanely) about its details. People in this mode don't want to hear reasons why not, they want notational convenience. I'm there myself when I complain about missing UI interconnects and missing Gnome features... I don't care about the cost, I just want the darn feature.

Usually languages handle this by using their bureaucracy to thwart some kinds of additions -- which is usually a good thing. Java is no exception here, but the situation is somewhat unusual because C# occupies such a similar niche but has a very different design philosophy.

C++/CLI

I read a draft of the proposed ECMA C++/CLI language specification. This thing is so bad that it is hard to know where to start critiquing it. How about, context-sensitive keywords? Or, my personal favorite, keywords with embedded whitespace? This latter is scarily close to Dalibor's joke that some future Java will have keywords in color... "oops, you want the salmon 'if' there".

(This will actually be an important development for two reasons. First it will allow the long-promised but never-quite-actualized crossover between programming and interior decoration. Second, it will give a new and richer meaning to the traditional Emacs haiku:
M-x font-lock-mode:
Your buffers should resemble
Angry fruit salad)

But back to the subject... a language with both templates and generics. Yeah! A language that adds "^" and "%" operators to go alongside "*" and "&"! Rock! This language is like an ordinary programming language that fell into a huge vat at a chemical factory, and emerged as a supervillain. With tentacles, laser beam eyes, and the ability to change sex at will.

Clearly, though, what we really need is Objective C++/CLI. That way we can combine all the bad and overlapping features, all the punctuation, and all the insanity into one nice package. Which we can then detonate.

posted at: 16:54 | path: /software | permanent link to this entry

Interconnections

I saw that a new Chandler release came out, and that reminded me of a missing interconnection.

I read a number of mailing lists via the excellent Gmane news gateway. This is great because I can open a group and reply to a message.

Sometimes, though, I'll run across a posting to some mailing list via the list's online archives. Replying to this is much harder -- really, absurdly hard. Mailing list archives ought to have associated read-only imap servers, so I can drag a URL from Mozilla to my mailer and have it magically start browsing the list in a reply-able way.

I suppose archives could add mailto: links all over. That would work reasonably well for the particular case of replying.

posted at: 12:49 | path: /software | permanent link to this entry

Tue, 10 Jan 2006

A couple Frysk notes

Frysk remains a bit opaque, unfortunately -- there's a steady stream of cvs commits but little action on the mailing list and little marketing. That's too bad... sometime soon I'll try to find out more about what is going on and talk about it here.

One thing that did float by on the mailing list looked cool: scripting frysk with python. (This is kind of old but I've been wanting to mention it.)

More JNLP stuff

I thought this hack was cute. Interconnection is awesome.

posted at: 18:52 | path: /software | permanent link to this entry

Sun, 08 Jan 2006

Classpath Update

Classpath hit 98% of 1.4 today, and I think we'll see another big bump tomorrow since the XMLEncoder patch went in today. I think we're in the 90s against 1.5, though it is hard to say since there hasn't been a branch merge in quite a while. Of course, japi isn't the final word on how we're doing, which is why Mark is going to talk about this at FOSDEM.

The coming year looks as though it will be a very good one for gcj and Classpath. I think we'll finish 1.5.

Netx Hacking

I've been doing a little hacking on netx lately. netx is a java web start implementation; this is a way of bundling a java application so that you can click on a link in the browser and have the application automatically download and execute. I set out to make netx a bit friendlier for Linux use.

My first goal is to make it integrate more nicely with the panel and the desktop. My idea was to make it so the user can drag a JNLP link from Mozilla to the panel, and netx would invisibly download the application, install it, and make a .desktop file for future launches. The reason to download immediately is that this lets you disconnect your laptop from the network and still have the application work.

I've got a patch to let netx download the application and make a desktop file. This works great. I'm not sure how to do the next step, namely integrate with Mozilla and the panel; something to research.

My test app, by the way, is iRATE radio, which is pretty cool and worth a look. Thanks to Anthony for pointing it out to me.

The next idea I had was to make netx a bit friendlier if the network isn't up -- instead of trying to update an application and timing out, it should use DBus to discover whether the network is up, and, if not, skip the check. I looked at the Java DBus bindings and I don't see any big problem here, except that I'm still running DBus 0.50 and the bindings want 0.60.

My final thought for netx was to have it gcj-compile jar files as it downloads them. This isn't super-friendly though... what I would really prefer to see here is a push forward on gcc+llvm, so we could have both AOT and JIT compilation for gcj. More on this later.

posted at: 21:29 | path: /software | permanent link to this entry

Wed, 04 Jan 2006

Nice gdb change

Michael Snyder's gdb checkpoint patch was checked in today. Yay! This will make it simpler to use watchpoints even on machines that do address space randomization.

Now he's apparently going to start work on reverse execution -- excellent.

posted at: 15:34 | path: /software | permanent link to this entry

Tue, 03 Jan 2006

Trying VEP

I spent yesterday hacking on medi8 a little. I got thumbnails working again as well as they ever did -- meaning that we show some image that is somehow related to your movie, just not the right ones.

I also rewrote the monitor widget using VEP, the visual editor plugin for Eclipse.

VEP has a couple of unusual features. First, it is not specific to a particular widget set. It comes set up to handle AWT, Swing, and SWT, and supposedly you can add your own toolkits to it as well.

Second, VEP uses the Java DOM that is built by Eclipse. All GUI editors face a fundamental problem: how to represent the generated UI in the user's code. These days the consensus approach seems to be to create an XML file describing the UI, and then provide a library that turns this into a widget hierarchy. VEP's approach is to understand your source code; you can edit the VEP-generated code and it will preserve your changes even while you modify the GUI.

So, this is cool and high tech. I'm not in a very good position to say how well it works in practice; my edits were limited to adding event handlers to buttons. It was able to recognize my own home-grown widget as a bean with no fuss, and I was able to build a little monitor widget with VCR-like controls pretty quickly, and without knowing anything useful about SWT. ... but that is about what I'd expect.

However, VEP still has a long way to go. The handling of widget layout leaves a lot to be desired, I had a lot of trouble figuring out how to make it work. You have to know that the "layout" property of a bean is editable in this general properties view, and then muck with the settings there; sometimes it pulls up a dialog for you to mess with. VEP could take a huge usability lesson from Matisse here. Ideally I barely want to deal with layout at all.

Also VEP has an unusual implementation in another way -- it runs a second JVM to display your widget tree as you edit it, and then it (as I understand) screen scrapes this to show you what you are editing. This doesn't perform very well and at least on my machine results in a VEP window permanently hiding near the bottom of the screen. Eww.

posted at: 16:54 | path: /software | permanent link to this entry

Really Big Birds

This morning I noticed that whenever you see a movie or read a book including really big birds, I mean birds that are so big that you can saddle them and ride them, the birds are always raptors of some kind. But why not gigantic ducks? Or enormous flamingos with 30 foot legs?

posted at: 09:50 | path: /misc | permanent link to this entry

Mon, 02 Jan 2006

The Gift of Therapy

I just finished The Gift of Therapy, which Yalom describes as an open letter to the next generation of therapists. I thought this book was fascinating and Elyn and I have had a number of conversations about it.

I also recommend reading Love's Executioner or Momma and the Meaning of Life, both of which are collections of stories about therapy and don't assume as much knowledge of what therapy is about.

posted at: 11:13 | path: /books | permanent link to this entry

Sat, 31 Dec 2005

Consumer Linux

This Christmas I bought an iPod Nano for Elyn and ran into a lot of software ugliness.

First, her computer runs Windows 98, and, apparently, cannot be upgraded to some more recent version of Windows. That sucks, but is apparently unfixable. Naturally, iTunes doesn't support 98. Some sites on the web suggest that, perhaps, you can apply some hacks, attach an iPod to the machine and use some software to copy music over -- but even then iTunes won't work.

Of course there isn't a Linux port of iTunes (too bad for us it isn't written in Java...).

I tried both Wine and CrossOver Office -- the latter can actually run iTunes, but unfortunately still does not support connecting it to your iPod. It might be possible to do what I want with some combination of iTunes, a program to remove the DRM from the downloaded songs, and some Linux-based iPod program like gtkpod... but this seems awful, even assuming it would work.

So, upgrading Elyn's machine to Linux (if it is even possible) wouldn't help. The lack of TurboTax would also be a problem -- they have an online version but, from what I've read, it isn't as featureful. The Linux worlds needs to make inroads with ISVs in order to get users like Elyn; I really couldn't recommend Linux to her today.

At this point it looks like buying a new machine is the only answer. I hate computers.

posted at: 16:50 | path: /software | permanent link to this entry

Sat, 24 Dec 2005

Medi8 Thoughts

There's been a fair amount of activity in the open source video editing space lately, and unfortunately not nearly enough on medi8 (in fact we haven't even updated our web site so there is nothing to see there, aside from the source).

Instead of going through the other projects I thought I'd finally post my reasons for writing medi8 as an Eclipse plugin. I think at first blush this seems like a strange idea -- certainly I've gotten negative reactions from a few people I've talked to about it. However, really, it makes a lot of sense, as I will try to show; and not everybody disagrees (I seem to be following Doug Schaefer around a little lately. And likewise I sometimes feel that Per has been everyplace interesting about 3 years ahead of me...).

Note that in particular we're writing Eclipse IDE plugins, and not an RCP application. For RCP the choice is mostly about technology: what is the simplest way to get an application up and running? What framework provides the platform support and ease of development that you need? Etc. But, that's not what we're talking about here.

The basic observation is that editing a movie has a lot in common with developing a program. It can be a large project, taking many months and involving many people (back when I was seriously working on my free software movie, on Vicki's advice I planned to take 6 months off work in order to edit). A "real" movie can involve a large number of parts, some of which could be custom software (e.g., a custom titling plugin to make cool credits); data management is a serious issue; etc.

Once you draw this parallel the reasons for developing Eclipse plugins become more clear. Using Eclipse as a framework gives us many things for free.

We don't have to worry about project structuring or management, Eclipse has this. Likewise for file management, and Eclipse even provides features that, were we to write a video editor from scratch, we would not include -- e.g., the option of integrated source control (I saw a feature request for CVS support in Kino, and felt vindicated). Eclipse also has infrastructure for help, for manipulation of editors and views, for task management (to-do lists, markers, and errors), and builders.

Really all we need to concentrate on is the mechanics of movie editing, how our model works, and things like that. We don't spend any time writing the same application infrastructure that has been written so many times before.

Of course this isn't without drawbacks. Eclipse's GUI tends toward the cluttered and it could benefit greatly from a UI designer (I'm not very concerned about this, though; cleanups are inevitable). There's also the possibility that Eclipse will somehow be incompatible with what we want to deliver; but the evidence is that this is merely a theoretical worry.

The overall vision for this tool is pretty straightforward: a full-featured sound and video editor built as a set of Eclipse plugins. Part of the big goal is using Eclipse as a framework to unify all the parts of film editing; so for instance I picture being able to build effects plugins of various kinds as Eclipse projects themselves, and have the final film project depend on the effects written for it.

posted at: 15:13 | path: /software/medi8 | permanent link to this entry

Sun, 18 Dec 2005

Aeon Flux

My memories of the original Aeon Flux are, perhaps appropriately, fragmentary. Back in the day I didn't get MTV and in the end I think I only saw two episodes all the way through. (Remember when MTV was interesting?) In any case, I remember liking what I saw, so I was looking forward to the film, although with some caution. After hearing a bit about it, I entered the theater expecting only disappointment.

The film itself is an unusual mix. It definitely has its bad points, which I feel compelled to enumerate. Some of the dialog is laughable. Stylistically it seems like it can't decide between a certain kind of Star Trekian futurism (people in subtly matching outfits walking easily through crowded outdoor markets), smooth concrete abstractionism (Flux's apartment, seemingly located in a scrubbed out sewer), something more baroque (the library, the Relical), or something hyper-realistic (close-ups lit to show skin texture, the unusual posters of the dictator).

Likewise it can't decide what kind of an action film it wants to be. The cutting is very choppy and the action scenes are consequently difficult to follow. At times the settings seem to indicate a more visually ambitious film is buried in the sub-liminal -- the locations chosen generally impressed me.

Despite its flaws this film did capture elements of the original. The world is mysterious, post-Picardian Kafka (nicely captured in the phrase "industrial virus"). The technology is alien and magical and as viewers we often do not know whether some gadget is taken for granted in this world where any object can have the function of any other, where form and function are completely divorced. Perhaps we are viewing ordinary things. Or perhaps Flux knows as little as we.

It brings to mind the Codex Seraphinianus, which in its own world is most likely a perfectly ordinary book, no more exotic than an issue of National Geographic.

The ending was a mild let-down, as were the moments when the film devolved to mere action and the typical tropes of the genre.

I enjoyed this film quite a bit more than I expected to. In an artistic sense I found it inspiring, similar to the way that reading a story by Delany invariably makes me want to write.

posted at: 16:52 | path: /movies | permanent link to this entry

Fri, 16 Dec 2005

C++ compilation

I've been spending a lot of time thinking about C and C++ development lately. In particular I've been thinking along the lines of, "how can we make C++ development as easy as Java development?". This turns out to touch a lot of different things, and I'm going to blog more fully about it later. Today I just wanted to share some thought about one particular problem: building gcjx.

I wrote about this earlier, noting the incredible performance difference between g++ and jikes. Today I went a little bit deeper and built gcjx with the -ftime-report flag. Then I wrote a little perl script to summarize the results (I'm only showing the top 4 here, the rest are all 3% or less):

Pass	User Time	% Total
parser	113.62	44.6
name lookup	34.34	13.5
symout	26.59	10.4
tree gimplify	18.95	7.4

I think the underlying problem here is that each compilation of a gcjx source file re-scans many of the gcjx headers and also a fair chunk of libstdc++, which hugely inflates the amount of work done. For instance, mangle.cc is a mere 386 lines of code, but run it through cpp and out comes a whopping 51,997 lines. Like most sane programs, gcjx doesn't play odd cpp tricks like including files multiple times; so much of this work is simply redundant.

The fix here is to move away from purely textual preprocessing and move to a more sophisticated model, one that eliminates redundant work. This is actually not as hard as you might first believe. For the most part it is an application of better data structures to the problem, coupled with the observation that, since typically header files do not interfere with each other via macro tricks, you can gain efficiency by caching lookup contexts during preprocessing. Perhaps I ought to write this up more fully; that is pretty sketchy.

I know of two attempts at this already. One is the gcc compiler server branch. Unfortunately this project seems to have been canceled by Apple, as there has not been progress on it in quite some time. There was little documentation for this project and, I think, nobody outside Apple really understood what it was about back when it was active.

The other project in this area is Doug Schaefer's PDOM work in the Eclipse CDT. Unfortunately this is for indexing only -- as far as I know there is no code generation planned. (Remedying this is one of my major prescriptions for improving C++ tools... but you have to wait for the bigger entry on this topic.)

posted at: 15:00 | path: /software | permanent link to this entry

Mon, 12 Dec 2005

gcjx generics milestone

Last week I got gcjx to successfully parse and analyze the Classpath generics branch. I only needed a couple of hacks to get there :-). More recently I fixed some of the remaining problems with 1.5 code generation -- I added code to generate bridge methods and handle enum-typed switches. Now I'm going to switch back to the tree back end, with an eye toward merging gcjx to the trunk.

posted at: 16:49 | path: /software/gcjx | permanent link to this entry

Fri, 02 Dec 2005

NetBeans

I'm going to go out of order a bit here and describe two different NetBeans talks together, though they happened on different days. Then I'll use this as a place to discuss some interesting NetBeans-related conversations and explorations.

Charlie Hunt gave a talk titled, "NetBeans: Pushing Java Productivity". (There's that word again.) This talk was a run through of all the new NetBeans features and some future plans. Tim Boudreau gave a talk about Matisse and other things -- this talk was largely demos of some of the new things. I freely intermix stuff the two of them said here.

The new NetBeans release includes Matisse (a Swing GUI builder), new editor features for productivity, newly rewritten CVS support (no SVN yet, but planned for the near future). There is now support for JSF, Struts, and JBoss in the IDE -- Eclipse probably lags a bit here.

This revision of NetBeans looks a lot like Eclipse. Astoundingly so in some cases, for instance the editor window has a small vertical pane on the right which represents errors and warnings, exactly the way Eclipse does. I wondered whether NetBeans was copying Eclipse, or whether both of the are copying some standard-but-unknown-to-me IDE approach.

Bad News

First the bad news for NetBeans. In some important ways Eclipse remains superior. NetBeans does not have an integrated compiler. Instead it invokes ant. It does parse and understand a Java source file when you open it -- but this means that, unless you do a full compilation, you won't have a global view of all the errors in your project. When you go to run your project, NetBeans does a real compilation. Contrast this with Eclipse, where it compiles in the background, has global knowledge of your program (and can do things like mark up the project view to show which files have errors in them -- before the file is opened), and where running a test case doesn't mean that you have to wait for a rebuild. This is core stuff, directly impacting your workflow, and Eclipse is just better.

I'm told a future NetBeans will integrate with the compiler better (there is some JSR for the compiler API. This stuff scares me given what it would imply for gcjx...).

Eclipse's CVS support also looked better offhand, but I didn't play with NetBeans' myself.

Tim and I had a long talk about Classpath, NetBeans, Eclipse, Swing, and SWT. While I'm not really a fan of SWT's implementation, the simple fact is that it is open source and readily runnable with gcj. On the other hand Swing requires a major development effort, which we're still undertaking. And while we definitely plan to be able to run NetBeans, we're not there yet. And in short -- feature comparisons aside (features are in constant flux anyway) -- this is why I'm an Eclipse user and promoter.

I think it would be perfectly fine -- cool, even -- to set up Classpath and friends to build with NetBeans the same way I set them up to build with Eclipse. I gave a quick try to this at the airport, using a NetBeans CD conveniently handed out at the conference, but ran aground on a quirk (NetBeans doesn't like your source to appear in . -- which is where Classpath puts it). Bummer! (I found a number of other oddities too, like the lack of a "Save As" action -- maybe I'll be spending some quality time with their bugzilla soon.

Also it strikes me that in some ways Emacs is still the state of the art. For instance people gush endlessly about the "new" Mozilla search "non-dialog" -- umm, that is Emacs isearch. So, now NetBeans has abbrevs... yay, color me unenthused that the IDEs are catching up on basic editing skills. Ahh, I'm being excessively snarky.

Matisse

The bad news is pretty bad. However, the good news is also pretty damn good. In fact the good things in NetBeans are just amazingly awesome.

By which I mean, Matisse gives a really awesome demo. Matisse is a Swing GUI builder that plugs in to NetBeans. It purports to let you build cross-platform applications correctly. It comes with a new layout manager called GroupLayout (which, it is claimed, is open source, but I have not verified this) which handles all the details, for instance this is where information about platform-specific spacing differences is kept. It also has some nice tricks like aligning font baselines even across different widget types.

Matisse tries hard to make it easy to build sensible GUIs, but doesn't actually force you to. For instance, in the demo, as Tim dragged UI elements around, guidelines would appear showing suggested drop points. This kind of thing is very handy, he was able to create new composite widgets with proper spacing and proper resize behavior in just a couple of minutes.

I still think that Swing kind of looks like crap on Linux. It should be indistinguishable from any other application, and unfortunately that is not the case yet. This is something that Classpath can, and should, solve.

Fun quote: "you can't spend an hour on Matisse". Meaning, it is too easy to use to take up that much time.

Sharing

On stage Tim demoed something that I thought was really cool and revolutionary. NetBeans has a built-in IM client, and Tim was talking to a co-developer in the Czech Republic. Tim chose "share this project" and, over IM, the code was transferred to his colleague, who then proceeded to edit it -- and the changes showed up instantly in Tim's workspace.

This is so damn cool. I've been wanting something like this for quite a while... not every change is worth sending to version control, and sometimes you really want to use the pair-programming style. (Also it would be super great to be able to share a debug session, but I don't know if NetBeans can do that yet.) Anyway I've thought about having this in Eclipse, but NetBeans is actually already there. Amazing.

Currently you need an account on share.java.net to get this functionality. That seems smart for them, in a building-a-portal way -- and sourceforge or savannah should offer IM service like this. In combination with the Eclipse Mylar plugin this would be quite cool; for instance open a project and be available for talking with other developers from that project. They plan to look at a jxta-based implementation soon, meaning the sharing could really be P2P. I think this would be friendlier for users.

Eclipse is working on similar functionality as part of the ECF project. This is now definitely on my must-examine list, not only for shared hacking on Classpath, but also for integration with Medi8. Shared movie editing seems like a cool idea.

Other NetBeans stuff

My notes say that Charlie's talk was full of unsubtle anti-Eclipse jabs. Which it was. Come on guys, rise above.

NetBeans has an ant debugger (nice idea, didn't see a demo). It has a PDE-like thing for writing NetBeans plugins (the demo here was about the same as Bob's, making me suspect that they both chose the only easy thing). You can make rich client applications using the NetBeans base (does Gnome have this yet? I think these days I would probably pick Eclipse RCP as the base for a traditional-style application). NetBeans makes it really easy to download and install the latest LookingGlass. It seems to me that we could do this for all of Fedora, by providing a way to say "I want package X", and then fetch the SRPM and turn it into an Eclipse project. Well, except for the fact that most SRPMs are probably not really suitable for hacking inside today's Eclipse :-(

You can set background wallpaper in NetBeans. This is useless but they did ask, "Can Eclipse do this?". Seriously, come on guys.

By the way, LookingGlass gives a really nice demo. Things whirl and slide around, and pulse organically. But does it help my productivity? Or just make me reach for the motion sickness pills?

Tim hinted at scriptable refactoring coming in NetBeans. Hmm, sounds like something I wanted.

There is a J2ME plugin for NetBeans that allows debugging on the actual device you are using -- they've found the device emulators to be somewhat flaky and so debugging on a live device is more useful. This sounds pretty hand for folks in the J2ME space. Maybe I misunderstood this though, since Michael Koch tells me that not many devices actually support this.

posted at: 11:18 | path: /software/brazil | permanent link to this entry

Wed, 30 Nov 2005

Muito Sou Java

Geir talked again about Apache, Harmony, and Geronimo. It was again basically the same talk, so I didn't note much. He did say that Intel's contribution of the JCE framework (no providers, they use GNU Crypto, Jessie, and Bouncy Castle for that) included 3000 test cases -- definitely need to investigate that for Mauve.

He also said that "Apache is not out to compete with anyone else". I don't recall if this came from the Harmony part or the Geronimo part of the talk. It seems to apply equally well to either. I'm not really sure what this means, as in a way some kinds of competition are inevitable and not contingent on our intentions. In the Classpath world we've (for the most part) handled this by recognizing that, while the VMs do compete in some ways, in general our interests -- in a solid, complete class library and in free Java implementations in general -- outweigh our differences. Start with that, work together for a while, have a few beers and late night chats together, and one day you wake up with real trust and a real community. (I know Geir understands this in a deep way. This is for any journalists out there looking for a controversy :-)

One way not to compete would be to work harder to cooperate, i.e., move license harmony back to the front burner instead of the current situation, which is that the majority of folks on both sides (me included) are just hoping it will magically fix itself. Magical fixes just don't happen.

Max Rydahl Andersen, JBoss

Max is a Dane living in Switzerland. Sometime I wonder whether giving these personal details of the speakers crosses some line of etiquette. I do it just to personalize these writeups in a small way.

This was about Hibernate 3.1 -- what's next. My first note is that his powerpoint presentation was cool, in that he could highlight parts of a slide with a yellow highlighter (software) pen, which marked up the slide as if it were an image (i.e., not restricted to textual selections). I wonder what program that is.

So what is new? JBoss is writing NHibernate, for one thing. I wonder whether stuff like this makes Sun nervous. Maybe not -- the crowd at this event was pretty rah-rah-Java; and just like at the Denver JUG I didn't find anybody planning to move development to .NET, which rather surprised me (e.g., I talked to a guy from a Delphi shop that is moving to Java -- not C#).

I don't know diddly about Hibernate and I didn't take many notes on it as a result. The Hibernate eclipse plugin that he demoed looks nice, though I don't recall why. And, he said on the Hibernate web site there is a "help wanted" search you can do, to see how you can help out. This struck me as a nice idea and I think for Classpath we should either have an automated "FIXME finder", or perhaps "stub finder". Or, change the cpbot to assign you a "FIXME" to fix on request. In case you're, like, bored or taskless or something. Ha, ha.

Sun's View of Open Source

Simon Phipps gave a nice presentation on Sun's view of open source. This covered a lot of ground, with a lot of ideas, and unfortunately my notes on it are rather fragmentary since I was sleep deprived. Bah.

Early in the talk he made a prediction, which is that due to Peter Jackson's engagement with the fans (the parallel to an open source company is obvious -- breaking down barriers between supplier and customer, having a conversation instead of an ivory tower), King Kong is going to make money. Hmm. I will make a counter prediction, which is that since monster movies are intrinsically boring, and that since we're all deadly tired of remakes, it will not do very well. This film, sorry Peter, is already on my must-miss list.

Simon talked about one important aspect of standards, which is that standards mean substitutability. The idea here is that if there is a governing standard then you can switch implementations with little cost.

However, according to Simon, neither open source nor open standards is enough. And, dammit, my notes are so vague here as to be useless.

My recollection is that this was a somewhat pro-Java (there are standards) and anti-Linux (LSB being what it is) approach. He didn't say what standards cover Open Solaris though :-). My notes say that he did tout the Debian/Solaris project. That was interesting to me since I thought there would be a licensing smackdown here. Maybe that will come from the FSF instead. Simon also talked a lot about the CIO's point of view, which is super important if you're delivering to enterprise customers. And, he made the point that the needs of development are not the same as the needs of deployment (I think meaning something like, open source alone is great for development, but freedom in deployment requires the substitutability delivered by standards).

I felt bad that I didn't take better notes. This was an important talk. I disagreed with a lot of it; as I remember it many of the assertions were a bit too binary, whereas in reality there is a continuum of, say, compatibility even if standards are in force -- a quick survey of C++ compilers is instructive (Simon, I suspect, would argue, correctly, that C++'s approach is a poor one and that Java-style test compatibility kits are more useful). Anyway, it is hard to express one's disagreement cogently when one doesn't even remember the material. Sigh.

ObjectWeb Consortium

François Letellier gave a talk. He represents the ObjectWeb Consortium, the makers of JoNAS. His talk was not very technical but instead more economically focused. He was the only person at the whole conference to mention the 4 Freedoms -- bravo! And, any slide about free software that mentions Nash equilibria and societal externalities is cool by me. (Yes. Any slide.)

Once again though I fell down on the note-taking. He described ObjectWeb's approach as "open source with governance". I think that is pretty smart, as in general I prefer explicitness and transparency. The GNU style (cf GCC) is more anarchic -- there is governance but it is often implicit, or largely embodied in tradition, or simply not discussed much as if it were my crazy aunt who I keep locked in the attic.

He said the software industry can best be viewed as competing networks of firms. Good observation.

Update

Graydon pointed out I inverted the sense of an important clause. Fixed.

posted at: 19:13 | path: /software/brazil | permanent link to this entry

Tue, 29 Nov 2005

Floyd Marinescu

Floyd is a friendly guy, from Canada, formerly of theserverside.com, who is going to launch his own site soon. It sounds quite good; I'm looking forward to it and I'll post about it when it is live.

Unfortunately I missed the first part of Floyd's talk about trends in Java, since I got locked in a bathroom. You'd think this would be hilarious, but really it was just boring and mildly irritating. One of the other guys called Bruno on his cell phone and after a short wait maintenance physically removed the lock.

Floyd had many examples of AOP to present. I dislike AOP quite a bit, since I think it violates the heuristic that says that it is better to have local constructs be lexically signified. (There. I put that in the most obscure and highfalutin way I could think of. Am I pomo yet?)

I didn't write down the examples but one idea was architectural enforcement -- where the AOP insertions can check that your code is following some rule supplied by the framework. But see what I mean here? It would be better to enforce these rules statically, using compiler-level tools (perhaps in addition to dynamic checks if that is really needed -- though these too should, ideally, be done in a way akin to verification).

About 15% of the audience planned to use AOP, and Floyd said this was about what he found globally. Floyd is a gold mine of this sort of information.

Dependency injection is a must-learn trend according to him. So are annotations, everybody likes them. (C#'s influence on the language is easily felt, but I suppose the web services guys would have pushed for this regardless -- look at xdoclet to see why.) I'm probably going to use annotations for the next generation of GDirect, for what it's worth, as it is a natural fit. Floyd had a nice list of known best practices for annotation use -- most of these can be thought up pretty easily, but it is smart to just distribute the information like this, I think, to try to get good penetration of the ideas early on.

He said that many shops are moving back to OO design and POJO applications. I'm not really a J2EE type guy but I gather that this is a change from how web applications are written today. The new buzzword here is "domain driven design", which really just means "OO". Buzzword proliferation is a disease and the UN ought to combat it.

Scripting is a good "new" trend, with Groovy soaking up most of the press gravy lately. Both NetBeans and Eclipse folks are investigating scripting the IDE. Got Emacs? Though one nice thing is that since the JRE provides a consistent platform, the scripting language choices matter a lot less; you can pretty much write scripting goo in any language you prefer.

"Domain specific languages" is also a new trend, just like it has been for the last 10 years. "Software engineering" is the endless renaming of existing ideas, and the history of it is largely the story of IBM introducing the same ideas that it likes, over and over, with different names each time, until they finally take hold.

Floyd also talked about Ruby on Rails (as did many here, mostly to argue against it), and AJAX. (Ean pointed out that Project Blackwood, which got little uptake, is really just AJAX for Java applets. This seems like a cool thing to revive.) Running out of time he mentioned "SOA" (service oriented architecture). I didn't know what this meant, but this explanation seemed relatively clear.

To sum up he hit on something that was repeated a couple of times, namely the importance of open source Java for emerging economies like Brazil. "Emerging" is the official phrase -- I heard it 2-3 times. I think "developing" is reserved for poorer places.

I was glad to hear this from someone! There's a quote I like which goes something like: "Philosophy is the middle ground between science and religion, open to attack from both sides". This describes work on gcj perfectly -- the free software world and the Java world are similarly unenchanted. The Brazilian Java community, though, really gets what we're about, and why we're important, and this is why I was so happy to come here to talk. The point here is, we have a lot of free infrastructure below the JVM, and we have a lot of free applications above the JVM -- but that crucial middle layer is a proprietary zone with all the attendant pitfalls. Our vision is to change this, and we're actually quite close to that.

posted at: 18:47 | path: /software/brazil | permanent link to this entry