Eigenmumbles

I mentioned gmllua before – this is the little tool that I wrote to translate GML files into Lua data structures for further processing. I’ve been pretty happy with it. Alas, text formats like GML aren’t the most compact things, so I’ve taken to compressing the GML files on my drive that are associated with large networks. It’s easy to read and write gzip files using zlib, which has an interface that looks a lot like the standard C I/O interface. So, I thought, why not glue zlib into gmllua so that I can read either plain GML files or gzipped GML files with the same code?

This should have been a simple task. It probably would have been a simple task if flex were designed a little differently. But flex is what it is, and what it is seems particularly ill suited to easily switching between multiple I/O libraries. It didn’t take long for me to figure this out, and to decide that I didn’t really want to hack the generated code (though that probably would have been the fastest thing to do). This is not the first time that I’ve been frustrated by the organization of flex and bison, so I decided that this was a good excuse to try something different: a parser generated using Lemon and a lexer generated with re2c. It took about half a day to figure it out and make the port, despite the terse documentation.

I considered doing something with ANTLR as well, since it can supposedly generate C output. But I actually like LALR(1), and I really liked how neatly the stack associated with the LR parse matched the stack-oriented Lua interface. And anyhow, it was pretty clear how to wire up re2c to read from different types of streams (since re2c doesn’t actually care how it gets the data). Huzzah for small, sharp tools!