twit haiku hacking

[trochee]

For those of you not following along on the Twitter thing, you may not know that I've built a bot:
Ohaihaiku¹ is a twitterbot: a 'robot' (automated script) that identifies and retweets english language haikus. some of them are pretty good!:
Spring frock in warmth yesterday /
to tuque and snow today /
Oh Calgary

Here's how it works.
From time to time (at the moment, whenever I remember; I haven't automated this part completely yet) it looks at the twitterers it's following, and looks over all their tweets. Each tweet is cleaned up slightly, and then checked to see whether it's 17 syllables. If it is, they are reformatted as nearly as possible into 5/7/5, and retweeted.

Somewhat less often, ohaihaiku checks the twitter public timeline and pulls in a couple hundred tweets and puts them through the same process. If a haiku is found, it retweets it and "follows" that user.

Lastly, if another user 'follows' ohaihaiku, it will reciprocate² -- and if a user 'unfollows' it, it will unfollow in return. I try to make it be polite like that.

challenges:

• There are a few challenges here still. It's not fully-automated -- I am not quite ready to put it on a server and leave it alone overnight, even though it runs all its checks with a one-word command.
• I'd like to build out its English dictionary. It's currently based off marnanel's excellent Lingua::EN::Phoneme, which itself is built on the CMU dictionary, and when it cannot find a word's pronunciation there it falls back to some more robust hyphenation code for syllable counts. But there are lots of words (e.g. twittering) that these approaches are less than ideal for.
• I'd like to look into some of the easier number and acronym normalizations: "lol" should be one syllable, but surely "OMG" should be three?
• I'd like to put in some language-id. This is challenging for two reasons: (1) tweets are very short and (2) the usual approach -- character trigrams³ -- will be very sparse on such short tweets, but it will help with the occasional accidental processing of Polish or (more problematic) German. I could probably set a threshold for the percentage of words that are not in the dictionary, but I'd rather do this the Right Way with character trigrams.

Check it out! enjoy. let me know if you see things. Anybody wants to peek at the code, I can point you to some of my git repositories; several of the modules I built to support this will become CPAN distributions of their own in time.

¹get it? it's like "O Hai - Haiku!"... well, I thought it was clever at the time, but the fact is, ohaihaiku does crap with mis-spellings.
²I reserve the right to un-follow and/or block spammers or people who post too many non-words.
³Character trigrams should probably be called tri-glyphs but what can you do?

Comments

[q-pheevr.livejournal.com]

How many syllables are in "^__^" anyway?

Ten.

I considered constraining it to 5/7/5 strictly, but that cuts out about three-quarters of the haikus it's detecting.

I definitely think you were right to be more lenient about the line breaks—it would be a shame to miss out on gems like the Calgary haiku (which fulfils the seasonal requirement) just for the sake of following strictly a rule that is, after all, an adaptation. (Cole and Miyashita (2006) claim that traditional Japanese verse is actually based on eight-mora lines—there's one silent mora at the end of the seven-mora lines, and three at the end of the five-mora lines—and note that there's some room for variation: you can substitute one extra pronounced mora for a silent mora.)

[trochee.livejournal.com]

any argument for three silent morae seems a little contrived, unless they're actually recording pronunciations. Is that the argument?

and your hyper-joke is awesome.

[q-pheevr.livejournal.com]

Yep. They recorded native speakers reading tanka, and measured the durations of lines and pauses:

	5-mora line	7-mora line
mean duration of spoken portion:	801 ms	1060 ms
mean total duration including final pause:	1216 ms	1195 ms

So the five- and seven-mora lines are padded with silence to achieve essentially equal duration.

[trochee.livejournal.com]

neat! hey look everybody! this is what it means when most of us say 'empirical'.