Truer Words - A Journal

What About Conversant?

seth@macrobyte.net — Thu, 18 Oct 2007 12:19:22 GMT

With all the work I do on BBEdit for Bare Bones these days, some friends and clients have asked me, "What about Conversant?"

Conversant is still loved, don't worry. In fact, here are the three things I'm working on, and hope to have ready by the end of the year:

Tagging — Enough with the annoying ‘category checkboxes.’ Other sites and systems have proven a million times over that people prefer free form tagging. (That is, "categorize" your messages by typing words or quoted phrases, rather than selecting from a predefined list of checkboxes.) Not that the old way will go away. They may even work together.
Tweeting - Sites (conversations), zones and servers will be able to tweet. Because it's easy, that's why.
MySQL - This one has been on hold, but I can't let it wait forever. We'll be able to move individual sites over to a SQL database. This may or may not be done by the end of the year.

Working On Tools

seth@macrobyte.net — Tue, 31 Jul 2007 22:17:03 GMT

Immediately after "retiring" from the Prototype Core Team, I became active (for the first time!) on the group and finally did what I was there to do in the first place. The next version of Prototype (1.6) will have custom events. The custom events code in 1.6 doesn't look much like the code I described in my essay a year ago, but it's built on the same idea: piggyback custom events on one of the browser's built-in events. (The custom events code in 1.6 was written by a number of people, not just me.)

Anyway, the real point here is that I use Prototype for nearly all of my web projects now, and I contribute to its development. That's working on my own tools.

Plus, immediately after finishing my side of Prototype's new events code, I realized that the next version of Prototype didn't look quite right in BBEdit's function popup. (Some objects were listed as [anonymous] when they should have had names, and some class methods were listed as though they weren't contained by anything.)

So, I updated BBEdit's JavaScript module to fix that problem.

I'm rather proud of the JavaScript support in BBEdit, but (again) the real point here is that I love being able to work on my own tools! (See the BBEdit Disclaimer...)

The same is true for Conversant, which currently runs on Frontier, and which runs my site (and lots of others).

Being a tool-builder makes me feel like a real craftsman.

MySQL in Frontier, and Conversant!

seth@macrobyte.net — Fri, 15 Jun 2007 21:04:04 GMT

MySQL is now supported by Frontier (the open source version). Took a little longer to settle down than was first expected, but it looks like it's ready for testing in teh real world now.

What's that mean for, uh, me? Well, Conversant is still written on top of Frontier. We'd like to change that. In fact, there's a third party that has expressed some interest in Conversant, but that absolutely requires a migration plan. In plain (but still somewhat techie) English: the first step of any migration plan for moving Conversant from Frontier to another scripting language (Python or Ruby) is to move all of the data to MySQL.

Conversant already has support for non-Frontier databases, but there hasn't been a good connection between Frontier and MySQL until now so there hasn't been any good way to store message and user data in a relational database. That has now changed.

So my next step is to merge the MySQL code from the main Frontier branch into the Conversant branch. Then I'll finish two Twine MySQL modules: one for users, one for messages. When those are done... well, then it's time to start testing Conversant running with MySQL databases. That's exciting!!!

(To be clear: I had nothing to do with Frontier gaining MySQL support. That was done by David Gewirtz.)

Steve Yegge on Living Software, Reminds me of Something...

seth@macrobyte.net — Thu, 25 Jan 2007 19:05:33 GMT

Steve Yegge has written another fantastic rant, this time about designing for living software.

He starts out by reiterating something I've said before: all software is crap. (I usually say it all sucks, and then follow it up with a comment about all email software sucking infinitely.) He goes on to talk about the traits that make some software suck the least.

I got a small thrill as I read it, and I think some of my friends will, also, if they take the time to read it (specifically, those friends who spent time working on Conversant, or a lot of time using it).

We did the UI all wrong (something which I hope to remedy), but this article has reminded me that the design principles we followed in building Conversant's core — and many of its plugins — were pretty good.

Moving right along, world-class software systems always have an extension language and a plug-in system — a way for programmers to extend the base functionality of the application. Sometimes plugins are called "mods". It's a way for your users to grow the system in ways the designer didn't anticipate.

…

The very best plug-in systems are powerful enough to build the entire application in its own plug-in system. This has been the core philosophy behind both Emacs and Eclipse. There's a minimal bootstrap layer, which as we will see functions as the system's hardware, and the rest of the system, to the greatest extent possible (as dictated by performance, usually), is written in the extension language.

Certainly, there are aspects of Conversant's core we should have done better, or at lesat should have re-done by now. Also, there are a couple of things on his checklist which we barely even considered. Still... go read his rant, and see if it doesn't sound familiar to you.

How to Serve Compressed Data with mod_gzip and Apache 1.3 on Mac OS X

seth@macrobyte.net — Fri, 17 Nov 2006 02:04:09 GMT

Some of us have recently been discussing the size of the Prototype library, my preferred library for DHTML/AJAX). Proponents of some of the other libraries play up their smaller file sizes, and it's true that this is a real issue for some people.

This little essay/how-to explains the basic ideas (the what, how, and why), and then walks you through setting up Apache on Mac OS X, to enable mod_gzip and serve compressed content. If you skip the editorial content and just follow the steps I've outlined, you should have everything up and running in fifteen minutes or less.

Googlepinging

seth@macrobyte.net — Thu, 05 Oct 2006 22:55:11 GMT

That gibberish word is there so I can test Google's new "Blog Search Ping" service. If this works correctly, Google will be immediately notified of this update to my site, and so will know to add the new page to its index.

I'm actually not completely sure it will work. Theirs is the first ping service (that I know of) to only support the extendedPing method, so I'm not sure how Conversant and Weblog II will handle it.

...

And a minute later, here's the evidence that it worked correctly:

<weblogUpdates version="2" updated="Thu, 05 Oct 2006 22:55:58 GMT" count="1160088958">
  <weblog name="Official Google Blog" url="http://feeds.feedburner.com/frat/feed" rssUrl="http://feeds.feedburner.com/frat/feed" when="18"/>
  <weblog name="Truer Words - A Journal" url="http://www.truerwords.net/index" rssUrl="http://www.truerwords.net/index/rss" when="42"/>
  <weblog name="" url="http://frazer.rice.edu/~erkan/blog/" rssUrl="" when="65"/>
  <weblog name="" url="http://prekinderbristol.blogspot.com/" rssUrl="" when="112"/>
</weblogUpdates>

(That's Google's list of pings received in the last 120 seconds, as of October 5, 2006 at 6:55:58 PM (Eastern US Time).

Frontier Developer Preview Release 2

seth@macrobyte.net — Fri, 26 May 2006 20:20:34 GMT

The Frontier developer's group has produced the next Developer Preview release of Frontier. Actually, Thomas Creedon deserves most of the credit for the release itself: he assembled everything and put an awful lot of time into the browser-based installer.

Lots of new features and bug fixes in this version, and it's even a Universal Binary for the Mac OS X!

(The Conversant branch of Frontier is kept in lock-step with the main 'trunk' of Frontier. I believe most of the Conversant servers 'out there' are running the Conversant-equivalent of this Developer Preview Release 2.)

New Character Set Features in Frontier (Done!)

seth@macrobyte.net — Wed, 26 Apr 2006 00:46:07 GMT

As I've hinted at a few times, I spent a lot of time in the last few weeks working on text encoding issues. Specifically, in the Frontier kernel, I hooked up the OS API's for converting text between character sets (macintosh, iso-8859-1, utf-8, utf-16, shift_jis, etc., etc., etc.)

Thanks to some help from a few people — especially Jim Correia, someone with some Hard Core experience in this area — everything is working on both platforms. Excellent!

On Windows, I had to actually learn a little about COM programming. (Really, I'm not kidding.) Frontier is now a real COM client (at the kernel level), just so that it can ask the OS for a list of the character sets available on the machine.

I don't really enjoy windows programming, it makes me feel greasy and out of my element, but I must admit that I *did* enjoy seeing it work the first time. It's especially gratifying to see the feature working on both platforms. Also in this case I didn't really have a choice: we needed this for Conversant, and most of the Conversant servers run on Windows.

Character Sets "Oops"

seth@macrobyte.net — Fri, 21 Apr 2006 14:00:04 GMT

After my last post about switching to UTF-8 for all of our content in Conversant, people wrote to say that my XML feed had problems since we made the change.

Heh. That's ironic.

Anyway, some wrote to me privately, as if a bug is something I'd be embarrassed about. "Oh no! A bug in the software! Say it isn't so!" All this time I thought my software was bug-free, like everybody else's. (What matters is how we deal with them. They're inevitable.)

Well, now the bug should be fixed. All "above ascii" text in the XML output was being automatically converted to numerical entities (like &234;), but that's no longer necessary when the output is UTF-8.

In other words, my “curly quotes” — and long dashes — should all look fine now. Feel free to give me a shout if something looks wrong, though.

Character Sets and Conversant (and the Eudora Problem)

seth@macrobyte.net — Thu, 20 Apr 2006 17:14:42 GMT

We recently decided to 'standardize' all Conversant text (everything from templates to messages) on UTF-8. I've been quite happy with this decision, as it made it possible (well, easier... it was always possible) to host truly international and multi-lingual sites, and made it a lot easier to deal with content coming in from a variety of sources like Microsoft Word. For example, we no longer bat an eye at ‘fancy’ characters like “curly quotes” or — for another example — long dashes.

This hasn't come without some pain on our end, though. We have to figure out what character set was used for the text being sent when a new message is created. That's supposed to be pretty easy: email, for example, generally includes a special header called "Content-Type" which lists the character set.

The problem is when the software that sent the email lies to us. This is where I'm stumped at the moment.

One of my clients uses Qualcomm's Eudora for all of his mail. He sends HTML messages to his Conversant site, and the messages always contain curly quotes.

Here's the problem: Eudora claims the message's character set is us-ascii. This is the simplest character set in use today, and converting it to UTF-8 should not be a problem... but those of you who have any experience with character sets already know what I'm going to say, right?

US-ASCII (a.k.a. ASCII) doesn't have curly quotes. Eudora must be lying about the character set, right? US-ASCII is a seven-bit set, and the character codes for the curly quotes are all in the 8-bit range, so it seems that it must be lying to me.

This has plagued me for days, and the client is beginning to think I'm being lazy by blaming his Eudora. (Surely a big company would never do anything so obviously wrong with 'established' software, right?)

Anybody have any suggestions? I haven't had much luck looking for answers in Google. What character set is Eudora really using when it sends "above ascii" text but claims it's all ASCII? Is it the platform-native character set, like windows-latin-1 or iso-8859-1, or something else entirely?

Greg's Conversant-Ruby Scripts

seth@macrobyte.net — Fri, 14 Apr 2006 21:25:20 GMT

Greg has produced a set of Ruby scripts for editing your Conversant site's templates, javascripts, and stylesheets *locally*. It downloads them all via xml-rpc. You edit them in your local editor, then run the upload script and it sends back whatever has been modified.

Very handy when you're doing a lot of work on a site... such as when you're trying to perfect your design for your next entry in the patterns contest!

Conversant Patterns Contest is Live!

seth@macrobyte.net — Mon, 10 Apr 2006 15:27:08 GMT

At long last, the Conversant patterns contest is live!

Prizes include a 60 GB video iPod with a $50 iTunes gift certificate, a 30 GB iPod, and one year of Macrobyte's "Domain Hosting" package!

This contest was originally the idea of Terry Frazier. Terry, Clark Venable, and Macrobyte are the sponsors.

We've been planning this for months. :-)

A New Conversant Branch?

seth@macrobyte.net — Sat, 01 Apr 2006 20:55:23 GMT

The following is a copy of a letter I just sent to the Frontier-kernel developers' mailing list.

(This is no April Fools Day joke, I promise!)

Hi all,

I've been maintaining my own local branch of the kernel with the changes we need for Conversant.

The only significant difference between the trunk and my copy is one verb group (builtins.conv) with a single verb (compileString). The other differences are all to resources, either to support the extra verb group or to change various static strings in the app (like the name and version number).

More parts of Conversant can/should/will be kernelized, so the number of verbs in that group will grow with time, but that's all there is for now.

It's been Macrobyte's plan, from the very beginning, to release Conversant as open source. Now I'd like to follow through with that by setting up a conversant branch on the sourceforge-hosted svn repository, and maintaining the source code there rather than locally. This code would be available under the same license as the rest of the Frontier kernel, of course.

Does anyone have any significant objections, questions, or concerns before I proceed with this plan?

Now the FAR bigger problem for me is actually the Conversant-related .root files. This is a totally HUGE amount of data, much bigger than everything in Frontier.root and MainResponder.root. These should all go into the repository also (probably as another branch, like Thomas did with Frontier's .root files), but I haven't yet figured out how to handle root updates with a cvsSync-based system.

(Perhaps I'll actually do everything twice: check all changes into a root updates server, AND check the changes in with cvsSync. Honestly, I haven't spent much time investigating this yet.)

Anyway, I hope and assume that my plan is ok with everybody, but please speak up if I'm mistaken.

Thanks,

Seth

We'll see what happens. I'm not anticipating any problems, but who knows? Not knowing is why I asked in the first place (and is why I probably shouldn't hav ended the letter by saying that I "assume" the plan is ok with everybody, since asking shows I wasn't assuming that at all!)

Mike Black's New Design, Kiwi

seth@macrobyte.net — Mon, 27 Feb 2006 22:28:53 GMT

I've spent some time over the last week helping Mike Black with questions about various parts of Conversant. He already had a very pretty site, but he's started over from scratch to create not just a prettier site, but one which really makes use of a lot of Conversant's features. Check it out, and don't miss the Technology and Travel links at the top of the page.

Even better is that he's maintaining a new pattern file called Kiwi, based on his new design. He even includes documentation in PDF form!

Patterns

seth@macrobyte.net — Mon, 16 Jan 2006 20:43:38 GMT

Conversant's patterns system is finally being tested and talked about as I hoped it would have been last year.

(Way better late than never.)

This reminds me of something I've known for a long time but sometimes forget. You can't motivate people by asking them to be motivated, and you can't form or maintain an active community (of any type, online or off) by asking people to share. Someone has to lead by example, and the desired outcome needs to follow naturally.

In this case, I finally put together some decent pattern files for Conversant, and the support site -- Conversant's community center -- has been like a busy little beehive all weekend. Very cool!

Weblog II is Finally "Out There"

seth@macrobyte.net — Thu, 01 Dec 2005 17:19:31 GMT

After years of private beta (man that's embarrassing to admit out loud), Macrobyte has finally released Weblog II, our 'next generation' weblogging system for Conversant.

Big, smart, easy to use.. and that just describes the lead developer behind it! Weblog II is even better than that!

(No, seriously, Weblog II is fully buzzword compliant. There's even a super-simple wizard for converting your old Conversant weblog to a Weblog II page.)

Greg said, "It's about time!" I agree. It's been like having a lazy, 30 year old kid still living at home with his parents. "Get out there and earn your keep, lazy bones!"

Updated Page of URL-Detecting Regular Expressions

seth@macrobyte.net — Wed, 30 Nov 2005 01:52:07 GMT

Three years ago I posted a description of the regular expressions we use in Conversant for detecting URLs in text (like message bodies), so that they can be turned into links.

That page has been fairly popular (based on the number of hits, and considering that it's one of the most boring topics in the universe). It's in the top 10% of my pages, anyway.

Brian and I were complaining about a URL-detecting bug in Adium this afternoon, and I'm constantly annoyed by a larger URL-detection bug in Mailsmith, so I decided to update that page with all of the changes we've made in the last three years. Maybe someone will see it, learn something from it, and the world will benefit in some small way. (Yes! World peace through regular expressions!)

I don't know of any bugs in these regular expressions (that doesn't mean there aren't any). They've improved a lot over the years.

Finally in Beta!

seth@macrobyte.net — Fri, 01 Jul 2005 01:15:20 GMT

I've referred to this a few times over the last couple months, without really explaining myself.

A couple months ago I presented an idea to a few of Macrobyte's clients. The idea was for a set of new features in Conversant, to make it extremely easy to share all of the important parts of a Conversant site, the parts that define the site's "look and feel," as well as most of it's functionality. Things like templates, stylesheets, javascripts, resources, page configuration settings, etc.

All three clients agreed to co-sponsor the development of these features. That's the first time this has happened, and I was quite excited.

No part of what I was proposing would be difficult. I know Conversant as well as it is pogssible to know any software's code, and that made me confident that what I was suggesting could be done.

Unfortunately, I'm too close to Conversant. I used my familiarity with it as an excuse to avoid the hard work of writing down everything that would be involved in the project and doing an honest assessment of the time it would take.

I told those three clients it would take two weeks. What a fool. Fully two months later, the last significant piece of the puzzle is finally in beta.

It's not that anything about the project was difficult. I was right about that much. I simply made the classic mistake of adding 1 and 1 and 1 and 1 and 1 to get... 1. As in, "that part will be easy, and that part will be easy, and that part too, and that one, and that one... yep, the whole thing will be really quick and easy." Completely forgot that one hundred easy pieces is still one hundred pieces, and a programming task with one hundred pieces is NOT quick and easy. Period.

I've told a few people privately, and now I'll admit it publicly: this was the biggest mistake and the most severe underestimation of my entire career (hopefully, both "to date" and forever).

The clients have been patient with me (Thank God), and for the last couple of weeks I've been able to use some parts of this project in other projects for another client. Still, I've been very hard on myself about this. I even joked with my dad that if an employee had put Macrobyte in this situation I'd have had to fire or execute him. In fact, i'd probably be better off just riding my bike all day, and pay someone else to do my work for me.

Now I can calm down a little, and hopefully start remembering why I love this business so much.

(I hope it's not another mistake to admit that I make them, sometimes!)

Why Is Weblogs.com Ignoring Us?

seth@macrobyte.net — Tue, 10 May 2005 20:30:37 GMT

It seems that since sometime in March or very early April, weblogs.com stopped listing updates to my site, and lots of the sites in my buddy list. Specifically, the sites that run on Conversant.

weblogs.com is a sort of "weblog tracking service." When your site is updated, you send a "ping" (usually automatically, but you can do it manually, too) to the weblogs.com server. The ping just tells it the name of the site that has been updated, and the site's URL. weblogs.com then checks to make sure it really has been updated, and then adds a link to the long list of weblogs updated during that hour. There are two versions of the update list: one for people to read in html format, and one for applications (like blogshares) to read in XML format.

It even maintains archives, so you can go back to see what sites were updated, and when.

I've debugged the pings we're sending, repeatedly. The server always returns the "thanks for the ping" response that indicates the ping was received successfully. Yet, our sites are never listed. The same is true for pings I've sent manually. It appears to be accepting the pings and then ignoring or forgetting them.

The service is run by Dave Winer. I wrote to Dave on May 6th to find out if he knew about the problem. No response. I wrote again on May 8th. Still no response. I wrote again this morning. Still no response.

This afternoon I mentioned the problem to Greg, so he ran test of his own. As expected, his site wasn't listed on weblogs.com, even an hour later.

Since I know the pings are going through correctly, and Dave is ignoring my email, I'm concluding that we're being intentionally ignored. Why, though? I haven't a clue. Is it something personal against me? Something technical about Conversant that he doesn't like? (Then why didn't he inform me?) Is it some business consideration? (I can't imagine what that could be, since Dave isn't in business right now.)

Here's at least a partial list of the sites that are being ignored:

Being ignored by weblogs.com isn't the end of the world, but there are a number of blog tracking and categorization systems (including Blogshares) that using its changes.xml files in the heart of their applications. Not being listed means we basically fall off the radar.

Unfortunately, I'm at a loss for how to handle this. I tried to handle it privately, three times, to no avail. What should I do next?

(Compare this situation with four years ago. When weblogs.com was rewritten for performance reasons, Truer Words was one of Dave's favorite sites. He included this site in a special group of favorite sites that weblogs.com continued to check until such time as we started sending automated pings to the server.)

Sometimes the Users Scare Me

seth@macrobyte.net — Sun, 01 May 2005 01:46:22 GMT

A Conversant user recently asked me to custom-configure one of his sites, because he wants to do something that isn't available via the web-based configuration.

Why not? Because we never saw anybody wanting to do this. We've never done it, never tested it.

I set it up for him, but I'm still experiencing heart palpitations.

In fact, I actually told this client that if he doesn't report every little problem to me, I won't be held responsible if it hoses his entire site, and he'll have to pay for my time to fix it. (As long as he reports every little problem, I should be able to stop him before he does anything fatal. That's the plan, anyway.)

(My previous post actually started out as this one, believe it or not. It morphed, big time, and then I realized that this one still needed to be written!)

Where I've Been Busy

seth@macrobyte.net — Fri, 29 Apr 2005 18:13:55 GMT

I was surprised to see how little I've written here on [tw] this week, since I feel as thought I've done nothing but write (and ride) all week.

Truth is, most of my writing has been code, not journal posts. New features for Conversant -- including an improved cookie model, and better RSS rendering. Plus the related announcements and documentation so users can figure out how to use them.

With that little update, I'm now headed out for what will probably be the last ride of the month, on which I should cross the 1,000 mile marker. Last year, I didn't hit 1,000 miles until July 3!

(Actually, my 1,000 miles this year is from January 1, although the season didn't really start for me until March 18. See what I mean?)

Server Retirement

seth@macrobyte.net — Mon, 28 Mar 2005 15:54:07 GMT

Macrobyte's original Free-Conversant server was retired from the ISP a couple of weeks ago, and replaced with a much faster machine (which was another of our servers, whose position was filled by a brand new crazy-fast machine).

This is a machine I built from the ground up, in late 1999 (I think my brother-in-law Art helped some, but I can't remember for sure...).

Until yesterday, the plan was to sell it on eBay for whatever we could get. Instead, it's now in service as my brother's eBay slave. I guess eBay has some software (turbo lister) that only runs on Windows, so Jed's slummin'.

This is a sort of trade, though. He's going to list some of my remaining machines for me. I have way too many functional-but-unused computers around here that need to go.

New Security Feature in Conversant

seth@macrobyte.net — Wed, 23 Mar 2005 20:51:18 GMT

I meant to say something about this on Monday but completely forgot about it and Greg beat me to it.

Macrobyte rolled out new security features in Conversant today. For the average non-technical Joe, I'm sure blogger, Live Journal, etc, are fine -- but I pity the geek blogging in anything but Conversant. You don't know what you're missing. Incredible flexibility.

He goes on to thank Clark for sponsoring the feature, which is also something I forgot to do when I first announced it on Conversant's support site. (Which was very quickly remedied.)

In spite of what Greg said there, these new security features aren't directly related to weblogs (though I'm currently in the process of extending the feature to cover channels in Weblog II.)

What Happened to HTMLArea?

seth@macrobyte.net — Thu, 10 Mar 2005 20:52:06 GMT

Interactive Tools, former providors of HTMLArea (a popular WYSIWYG editor that we adapted for use in Conversant), have announced HTMLArea's demise.

I'm not sure how they can do that, since it was open source, and the lack of explanation was maddening. Here's the entirety of their comments on the matter:

htmlArea.com has been re-launched as a resource directory of browser-based WYSIWYG ("what you see is what you get") editors with over 50 listed so far

In case you didn't know, htmlArea was a free open-source wysiwyg editor component that we made available for developers building web applications.

If you build web applications or you're looking for a wysiwyg editor component, check out the new htmlarea.com and let us know what you think:

Bah! Since we use HTMLArea in Conversant, I've been keeping up with it over the years. Once they released it under the BSD License, the worst they could really do was screw up the web site... others are still free to use and/or distribute their own versions. (They don't claim otherwise, thankfully.)

dynarch.com had taken over HTMLArea and rewritten it from scratch for version 3. Here's what they've had to say on the matter:

Update, March 8 2005. Some time ago, InteractiveTools expressed the will to take over the project. We provided some fixes that we made and were not in the CVS version and a RC2 was released at htmlarea.com; however, soon thereafter InteractiveTools announced the project closed and forums discontinued. Bang!

Our position on this is that the editor will keep going; we are actually making quite some progress in its development, but only in house at this time. We are still planning to release version 3.0, quite possibly under a different name (so it might actually be a 1.0) but still free, at least for the core editor--some plugins might be released under a commercial license. We can't provide explicit deadlines, so please bear with us.

I'm betting that the problem is that dynarch.com took way, way to long to release HTMLArea... they took over years ago, and there really hasn't been any progress in a very long time. This looks like interactivetools.com's way of delivering a spanking.

Updates to HTMLArea (and all of the other x-platform WYSIWYG editors) are going to be required soon, in order to be compatible with Safari 2.0's editor features.

Finally, though I don't like their tactics, I'll admit that the directory of WYSIWYG editors is quite well done. There certainly are a lot of them.

(Conversant users needn't worry about any this. We're still maintaining our version, and will be re-releasing the source to it soon. If dynarch.com ever does move beyind 3.0-rc1, we may roll their changes in with our own.)

Creative Commons, Trackback, HTML Comments, and Embedded RDF

seth@macrobyte.net — Mon, 07 Mar 2005 23:57:15 GMT

How many years do I have to work with HTML before I stop discovering important technical points of which I should have been aware all along?

Today's rather embarrassing example regards the format of HTML comments. This completely took me by surprise.

HTML comments have the following syntax:
    
    
White space is not permitted between the markup declaration open delimiter("<!") and the comment open delimiter ("--"), but is permitted between the comment close delimiter ("--") and the markup declaration close delimiter (">"). A common error is to include a string of hyphens ("---") within a comment. Authors should avoid putting two or more adjacent hyphens inside comments.

Information that appears between comments has no special meaning (e.g., character references are not interpreted as such).

Note that comments are markup.

In other words, comments don't end with "-->". They end with "--" followed eventually by an occurence of '>'.

This explains problems I've seen for years but never bothered to dig into deeply enough. I thought that the HTML comment delimiters were simply a "balanced pair", similar to the < and > that mark the start and end of a tag. A "".

Practical implications? There are a few. The most obvious (in my world) is that one can't reliably "comment out" the results of some Conversant macros. For example, if the macro returns a user-defined string from a database (such as a message subject), that string might include a "--". If it does, then the very next '>' will close the comment.

Trackback "autodiscovery" data is RDF embedded in HTML comments. It looks something like this:

<!--
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
    xmlns:trackback="http://madskills.com/public/xml/rss/module/trackback/">
<rdf:Description
    rdf:about="http://www.example.com/example/page.html"
    dc:identifier="http://www.example.com/example/page.html"
    dc:title="HTML Comments -- They Don't Work How You Thunk"
    trackback:ping="http://www.example.com/4594/trackback" />
</rdf:RDF>
-->

See the problem? The dc:title attribute of the Description element contains a "--", and so the comment is closed by the very next '>'. That leaves the </rdf:RDF> and the --> outside of the comment, and in fact Firefox displays the --> on the web page!

Creative Commons Licenses work in a similar way: they embed licensing information in the HTML via comments. They're not bitten by this comment syntax problem, though, because they have more control over the attribute values of the tags, and can intentionally avoid the double-hyphen problem.

See Kendall Grant Clark's Creative Comments: On the Uses and Abuses of Markup for a discussion of the semantic problems with this approach, and Phil Ringnalda's throrough review of the trackback problem (years old).

I didn't see any mention, in my brief research, of using a technorati-style link to solve the problem. Instead of embedding the RDF in the html, we could link to an RDF autodiscovery file with an invisible-to-people link like <a href="http://www.example.com/mt-trackback.cgi?type=autodiscover&id=4594" rel="autodiscovery"></a>. Any reason that wouldn't work?

Comments appreciated.