30 minutes with Aider Chat

I said in my last post that I was going to try out OpenHands; but then somebody showed me Aider Chat and I decided to try that instead, on the strength of a personal recommendation. The results were very good, and confirmed my suspicions that having the AI agent as a collaborator rather than attempting to one-shot it seems like a good idea. I got a fix I’m happy with in one go, with only a little bit of hinting and none of the fiddling with prompts that I had to do to keep SWE-agent from going off the rails.

I set up aider using the installation instructions and, like last time, gave it an Anthropic API key. Then I started it on the highlight.js repository, giving it the files that I thought were relevant:

aider src/languages/bash.js test/markup/bash/not-comments.txt test/markup/bash/not-comments.expect.txt

I pointed it at the GitHub issue for it to download, prompted it with the contents of the relevant commit, and gave a vague handwavy question to start things off:

/web https://github.com/highlightjs/highlight.js/issues/4234
/run git show d78749a
/ask thoughts on how to investigate and fix this bug?

It asked me to look at another file I hadn’t included in the original list, made a couple of false starts that it noticed problems with without intervention, and then came up with the same fairly obvious fix that SWE-agent had also attempted:

const COMMENT = hljs.inherit(
    hljs.HASH_COMMENT_MODE,
    {
        begin: /(^|\s)#/  // Only match # if it's at the start of a line or after whitespace
    }
);

So I pointed out that that wasn't correct:

/ask Won't that fix mean that the whitespace also gets marked as "comment"?

It agreed, made another few false starts, and then produced a solution which had the opposite problem, leaving out the # character instead. Pointing this out didn’t help, but reading through the previous attempts, I noticed that it had come up with a fairly straightforward sounding solution early on:

begin: /(?:^|\s)#/,  // Look-behind assertion: # after start of line or whitespace

and then immediately rejected it:

But wait, that won't work because JavaScript regex doesn't support lookbehind in all browsers.

I didn’t know that JavaScript regexes supported lookbehind assertions at all, since the last time I really sat down and read through the documentation for regexes was probably in the Internet Explorer era. I did some research and discovered that the LLM was also behind the times, though somewhat less than I. The highlight.js maintainers had discussed this question recently and decided that the relevant browsers were old enough now that they could start using lookbehinds as soon as they bumped the library to a new major version.

So, I told it that actually using lookbehind was fine, at which point it produced exactly the right fix, and then implemented it including test cases:

/ask We've made the decision to allow using lookbehind; all new browsers support it now.

Great! That simplifies our solution significantly. With lookbehind support, we can create a clean solution:

const COMMENT = hljs.COMMENT(
    /(?<=^|\s)#/,  // begin: match # only if preceded by start of line or whitespace
    /$/,           // end: end of line
);

I cleaned this commit up a bit (the comments probably helped it to think through the solution, but the rest of the library is generally written with the assumption that the reader understands the regexes without help), and then wrote up and submitted a pull request with this fix, along with a note about needing lookbeind support.

This hasn't been merged yet (it seems like it often takes the maintainers a while to review submissions), so I don't know what they want to do about bumping the major version, but I am confident that it’s a correct fix, and if the problem were more urgent for me I could install a patched copy of the package myself in order to get this bug fixed without waiting for it to merge.

This was a much more helpful tool than the last one I tried, and I'm not sure why I didn’t find it when I was doing my (admittedly cursory) research before selecting SWE-agent in the last post. As I said, it confirms my suspicion that being able to guide the model as it’s working is kind of necessary. It was helpful that it pointed out that a lookbehind regex could be used here, but it clearly wouldn’t have decided to actually use one on its own, since I had to do the research and tell it that lookbehinds are actually fine in this situation. On the whole, though, I think that there are definitely some tasks it can make a lot easier, and I’m looking forward to using Aider (or maybe trying some other tools) more in the future.