You are viewing hughsient

Technical Blog of Richard Hughes - GNOME using waf?

Richard Hughes
Date: 2007-01-04 21:03
Subject: GNOME using waf?
Security: Public
First, apologies about the long blog.

The programs autoconf, automake are not the easiest friends to get to know. Scripted in odd languages, and with seemingly many ways to do the same thing they are confusing. Most developers I know simply copy/paste configure.in code from one project to another with little understanding of how it works. I call it auto-foo.

People working on GNOME have looked at many replacements over the years, and for gnome-power-manager I've even looked at using scons as a build system in the past. Scons is great for building a program, but not suitable for managing a project. Something bigger and better is required.


waf example

I've also been following the progress of a project called waf quite closely for the last few months, and even getting involved with providing gnome example code and filing bugs. All the helper modules are python, so it's quite easy to extend and hack. As a build system it's looking quite promising, and I have even got a local copy of gnome-power-manager built using it. It takes about half the time to configure and build compared to ./configure && make and also has pretty output formatting.

I have found a few architectural design choices in waf that in my mind make it less suitable as a general build system:
  • It creates project cache files in ~/.wafcache, which is obviously not okay in an automated build machine. In my opinion it should create the configure cache in the project directory under .waf/_cache_.
  • There is a full tree of wafadmin (waf python source code) in every project, which for managing many projects is a no-no. That's like having a copy of the m4 interpreter in every module (okay, it's pretty small, but still a bad idea)
  • Waf has no stable branch, and the API is not stable yet. I keep having to change my wscripts to keep up with svn head.
  • You have to set the WAFADMIN path for each user if you have installed a shared copy in /usr/local/share - no other paths are permitted. waf's ability to work with a shared wafadmin path is poor.
  • By default waf outputs details in console colour text mode - and the yellow text is unreadable.
  • Checking for headers is not simple, requiring a few lines of python for each include, where autoconf could do this in one line.
  • You have to decompress a semi-binary waf-light file to waf before you can start. This also smatters files into ~/.waf-version/wafadmin/. This is confusing on so many levels. If waf installed a single system shared copy then we wouldn't need all this madness.
  • You can have lots of different input files, like wscript, wscript_build, wscript_configure etc. I don't think the extra suffix saves anything at all except a layer of indentation, and it certainly confuses people.
  • Handling of the .po, .xml and .in files for GNOME is basic at best (mainly my fault).
waf does do a lot of stuff right in my opinion:
  • All generated files are put into a _build_ directory rather than clutter up the source tree.
  • Configure checking is really quick and cached for the next run
  • Building is fast due to parallel building
  • Dependency resolution is really quick, REALLY quick.
  • Can change the prefix and destpath for rpms and debs.
  • Lots of support for lots of languages (qt, c++, c, java, tex and more) and modules ready to go
  • Written in 100% python so no messing about with custom scripting languages - it also means you can do some funky python stuff in the wscript.
This is simple stuff should just work. An application developer shouldn't have to be playing with manually setting WAFDIR or debugging m4.
In my ideal world (with my GNOME "just works" hat on), waf would install the common python files in /usr/share/wafadmin and there would be one main file waf in /usr/bin. waf would just run the top level wscript in each project and then follow subdirs looking for other wscripts. The author isn't keen on this, and would prefer one complete copy of waf in each project. In my opinion there's a fair bit that needs to be changed before it can be used with GNOME software.
What can we do about waf? It's BSD licensed, so we could fork the project and make it more suitable for GNOME (and then try to stay in sync or contribute the new stuff back to waf). We could just re-write the frontend (waf-gnome?) so that it works better with stuff like jhbuild and is easier to use. Or we could just start pestering the maintainer to modify the way development is going.
So, the purpose of this blog. What do you think about the way forward? Do we keep limping on with auto-foo? Is it just me who doesn't understand all this m4 stuff? Is waf a viable system to try? Has anybody else got any ideas for a new build system? Am I asking too many questions?

Thanks.
Post A Comment | 18 Comments | Add to Memories | Share | Link



User: (Anonymous)
Date: 2007-01-05 19:57 (UTC)
Subject: Waf ideas
I hate Gnome and I hate Python even more. But I say this not to start a flame war, but to help you understand just how happy I am that Gnome could one day be using Waf.

XMMS2, the media player project I work on, has recently switched to Waf from SCons. I don't know a lot about Waf and I had only just started to catch on to SCons when we switched over, but I've seen a lot of Waf bugs get fixed just from being the XMMS2 build system. If you're on IRC or you browse around the Waf bugtracker you've probably seen tru, one of our head devs who has been contributing a lot back to the Waf project based on our experience. So to see a big project like Gnome use it would mean that it would likely get enough contributions to propel it into the mainstream and supplant autohell on modern systems.

I do have some gripes, though, most of which are in line with yours:
- needs a config file--with sane defaults (COLORS OFF!)
- local wafadmin stuff is insane
- standalone zipped waf script is cute, but ultimately makes it harder to stay up-to-date with HEAD (needs manual updating periodically)
- build dirs are annoying, ex. _build_/default/ is a PITA to keep typing in a shell and the default directory seems extraneous; try using .waf/build, .waf/cache, etc. (Remember I don't know a whole lot about Waf still, so there could be a good reason for this, but probably not.)
- prefix and destpath still need work; things like java and ruby install directories are not easy to prefix

and as for praise:
- lightweight and fast, much nicer than SCons, comparable to autohell
- does more in the core and handles components better than SCons (for XMMS2, we had to make our own classes to abstract away components of the build, much like how wscripts work by default)
- built-in caching, eliminates the need for things like ccache and works at the buildsystem level
- configure step is easy and permanent, e.g. CC will stay set throughout the build steps rather than having to export it or specify it during every step
Reply | Thread | Link



browse
my journal
April 2008