Feep! » Blog » Post

Second thoughts about redo

Despite my previous enthusiasm about redo, I’ve just finished tearing the last of my .do files out and replacing them with shell scripts again. I don’t think the problems I had were redo’s fault, really; I was just asking it to do things it wasn’t really designed for.

The biggest problems I had were with trying to set up hard links to share data between my development and production environments (to save disk space—but that’s a post for another time). The two main showstoppers with Appenwarr redo were:

These two issues are with the way Apenwarr redo in particular handles file changes (and, again, I don’t think this is a problem with it—this design makes a lot of sense for a general software build tool, where you want to be as careful as possible about making sure build artifacts aren’t out of sync with the source code. It only falls down in my situation because I need to avoid running scripts unnecessarily and am willing to sacrifice some correctness guarantees to achieve that.)

However, there are also some more general problems with the design of redo that mean switching to another implementation probably isn’t worth it for me.

In light of all this, I decided it was time for yet another rewrite. (I think this is the fifth? I keep thinking “this time I’ve got the right abstraction!”, but this problem is proving bizzarrely difficult to tackle for some reason. At least it’s small enough that it only takes a few hours to write a new system from scratch with the strategy du jour.)

I briefly considered writing my own redo(-ish) implementation, but concluded that keeping it as simple as possible was the best policy for the moment. I therefore have gone back to shell scripts, though the new ones are more carefully written to automatically keep track of changes that will affect downstream data, which should cut down on the confusion about stale data that I experienced last time I was using shell scripts for this.

Given the previous history of this particular problem, I don’t expect this to be the last time I write about this, but hopefully I’ve at least got something that will tide me over for a while. There are also some interesting-looking patterns for dependency management that are revealing themselves, so if these scripts hold up in the long run I may also revisit them later and write another blog post about how they’re structured.