“Frontier 10: Being Involved (Long and Rambly)”

After Frontier's source was released, I wrote about my thoughts and feelings on the subject. I couldn't afford to get involved with it but I was happy it was done, my huge (emotional and professional) investments in Frontier, etc., etc., etc.

"Blah blah blah." That's what it amounted to.

In late September (edit: this said December. Duh.), I decided to join (and lurk on) the developer mailing list. The list is quiet, and for awhile there wasn't much to catch my attention. That changed when they started discussing tests and optimizations. (I'm freaky-addicted to both of those things. I love writing test scripts, and I can lose myself in code optimizations for days at a time.) Andre wants to optimize Frontier's handling of large tables, and needed a test script to look at the current performance, so he'd have some hard numbers for comparing with whatever he does to the kernel. I wrote the test code, and a few revisions based on group feedback (especially his).

Don't worry, I wasn't just feeding my addiction. Any significant improvement to performance could and should have a direct affect on Conversant.

In November, Andre asked if anyone was interested in working on some of Frontier's string verbs. He described how many of them could be significantly optimized. Shoot, there's that word again. I resisted at first, but eventually the itch needed to be scratched, and I took a stab at it.

This was the first C coding I'd done in a couple years. Andre helped a lot with my first task, string.mid(). Next up was string.delete(), and I needed a little help from him and a little from Brian Andresen because I was confused over operator precedence. (Which comes first: pointer dereferencing or pointer addition? Ack! Solution: use parens.) Finally, I optimized string.nthField(), string.nthWord(), and string.firstWord(), and by then I didn't need any help at all. (Note: links point to messages describing the before-and-after test results.)

A day of testing Conversant with these improvements (plus a switch to the newer PCRE verbs instead of the old regex DLL) revealed a very nice boost in Conversant's performance. Depending on a number of factors like server load, page size, and page complexity, I saw improvements of 10% to 50%! In other words, if rendering and serving a page took 1.0 seconds before the optimization, it would take somewhere between 0.5 and 0.9 seconds afterwards.

This was too good to pass up, so after some more stability tests I upgraded all of Macrobyte's production servers. Lots of people have commented on the speed boost, so I'm glad I did it.

A bug in Frontier's dialog boxes on the Mac (any dialog with a text field, like the Find/Replace window, or the window opened by dialog.ask()) finally annoyed me enough that I fixed it. Now, when a dialog is opened, the contents of the text field are pre-selected. (This was a very small bug.)

The upgrade on the servers introduced a strange bug that went unnoticed for days. Eventually, I realized that there was something wrong with the PCRE verbs: they were unable to insert just a single backslash into the result via the replacement pattern.

What, I'm speaking greek? Maybe this will help. This worked:

local ( s = "ab" );
regex.subst( "(a)(b)", "\\1\\\\\\2", @s );
dialog.notify( s )

The notification window would show "a\b". However, this didn't work:

local ( s = "ab" );
s = re.replace( re.compile( "(a)(b)", "\\1\\\\\\2", s );
dialog.notify( s )

That way, the dialog would show "a\\b" (two backslashes).

I surprised myself by finding and fixing this one in just a few hours, too. The function in the kernel that scans the replacement string is supposed to treat two backslashes as a single escaped (literal) backslash. Instead, if the backslash wasn't escaping a named or numbered pattern group reference (like \1 or \g<foo>), each backslash was treated as a literal. With that fixed, the weird bug "in Conversant" was gone, too.

Anyway, I've decided that contributing to the kernel in ways that will help Conversant (or any of my clients) is a very good use of my time. (Just need to remember to keep it balanced! It would be easy to "play" with the kernel all day...)