– 🕓 5 min read

Book Review: Dataclysm

If you've ever read OkCupid's popular OkTrends blog, you're already familiar with the work of Christian Rudder. As OkCupid's resident mathematician and one of its founders, he has been writing for years about statistical trends related to dating and attraction, leveraging OkCupid's vast database of traits about, and interactions between, its millions and millions of users. OkTrends produced numerous interesting and surprising results, and it developed a significant following before going quiet for the better part of three years beginning in 2011.

Now, Rudder is back with Dataclysm: Who We Are When We Think No One's Looking, which expands on OkTrends by drawing from a wider variety of sources (including Facebook, Google, and Twitter, in addition to OkCupid and other dating sites) and discussing a more diverse set of topics. Organized into three themed sections which focus on, respectively, unity, division, and individuality, the book explores what data can tell us about gender, sexuality, race, identity, community, and other sociopolitical questions. The discussion is consistently balanced and neutral, which is important when dealing with such potentially charged subjects, but Rudder also manages to inject enough of his personality and characteristic style to keep the book enjoyable throughout.

– 🕓 11 min read

Game Review: The Elder Scrolls Online

[Updated March 17, 2015: The Elder Scrolls Online: Tamriel Unlimited was officially released today, meaning that ESO no longer requires a subscription fee to play. In addition, there have been six major content updates since the game's original release, with dozens upon dozens of improvements. I'd encourage you to read my new post on the subject or just go straight to Zenimax Online Studios' own announcement.]

It has been about a month and a half now since The Elder Scrolls Online was released, and it's received generally mixed reviews. Despite some undeniable technical issues and a handful of design flaws, it's a great game and more than a bit underrated in my opinion; if you're a fan of The Elder Scrolls or an MMO aficionado, you owe it to yourself to give ESO a try. I'll explain why.

First, by way of qualifications, I should mention my history with the Elder Scrolls series. I didn't play Arena or Daggerfall, so I haven't been following the series since its inception, but Morrowind captured hundreds of hours worth of my attention and made a huge impression on my taste in games—it was unlike anything I'd ever played at the time, and not even other Elder Scrolls games have quite managed to recapture that magic for me. Oblivion was, at first, quite awe-inspiring, but once the captivating new graphics wore thin, it ultimately seemed shallow and uninspired compared to Morrowind. Skyrim was a return to form, with better and more varied environments, a more interesting story, and improvements to game mechanics all around—in addition to, once again, remarkable visuals. In short, I've been an Elder Scrolls fan for more than 12 years now, so although I don't go all the way back, I have a good basis for comparison.

The Elder Scrolls Online came as a surprise, being announced less than a year after Todd Howard indicated that Bethesda had no interest in an Elder Scrolls MMO. In fact, at that point ESO had been in development for four years, but Howard may not have known and certainly would not have been at liberty to reveal that. At any rate, my initial reaction was extreme skepticism; I doubted that the best aspects of The Elder Scrolls could be preserved when introduced to the MMO formula. As more details were revealed about the game's design and mechanics, I grew more interested, but I after playing the beta I was again left with lukewarm impressions. Still, something about it kept nagging at me, and I couldn't stop watching Twitch streams of ESO as launch drew nearer. Eventually, I decided to buy it just in time for early launch. Since then, I've put in nearly 150 hours, and that number would doubtless be much higher were it not for the obligations of the real world.

– 🕓 11 min read

Road Trip! Part II

Five months ago, I left off my last post having just left Canyonlands National Park, heading further south to visit Bryce Canyon and Zion National Parks. I had booked a motel for the evening in Panguitch, Utah, which is a three or four hour drive from Moab. The drive there was uneventful, with relatively little in the way of scenery. I mostly followed the interstate, until about an hour outside of Panguitch, where I exited onto state route 89.

Route 89 winds through scrub desert, farmland, and several very small towns nestled between modest mountains. I arrived in Panguitch and discovered that it, too, is very small. In contrast to Moab, though, it didn't seem to have much of a tourist draw despite its proximity to Bryce Canyon. As a foreigner, I have to admit that I felt some apprehension at this. Although everyone I'd met in Utah so far had been very friendly and welcoming, a small, isolated town like Panguitch is the kind of place where I imagined that I could encounter a Top Gear-esque redneck mob. Granted, I wasn't driving around with a purposefully offensive slogan on my car, nor do I really have a discernible accent, but I still felt like a pretty obvious outsider with my Washington plates and conspicuous blue car with absurd spoiler. I was relieved, then, when I checked in at the New Western Motel and was greeted at the front desk by an older Indian man (who I presume is also the owner); it was good to see that the population of Panguitch isn't completely homogeneous.

If you read the previous post you might recall that I had only slept for a few hours the previous night, as I was up around 3:30am to get pictures of Delicate Arch at night. That being the case, I was exhausted by the time I checked into my room, so I decided to nap for a few hours before heading to Bryce Canyon. It might have just been because I was so tired, but I distinctly recall thinking that the bed in my motel room was among the most comfortable I had ever slept in.

I woke up around five or six in the evening and went to grab dinner from "The Pizza Place" in the nearby town of Tropic. Yes, it's literally just called "The Pizza Place." It's a nice little family-run business, and the pizza was pretty good. I couldn't help but notice that they had several copies of the Book of Mormon available for guests to peruse. That didn't surprise me, given that I was in Utah, but I did find it strange that most if not all of the books seemed to be written in Danish.

I made it to the Bryce Amphitheater in Bryce Canyon National Park just in time for sunset, and took the opportunity to walk the Rim Trail and take some pictures. Unfortunately, as it was quickly becoming dark, I didn't have time to go on any of the hiking trails or explore the rest of the park. I contemplated spending another day in Panguitch and going on a horseback tour of Bryce Canyon, but ultimately opted to leave for Zion National Park the next morning so that I'd have more time there.

– 🕓 12 min read

Road Trip! Part I

I love visiting new places. I've lived in the United States for just over a year now, and one of my favourite things about this country is how vast it is and how many opportunities it presents for exploration. Canada is just slightly larger than the United States, but it's also very sparsely populated north of, say, Edmonton, so practically speaking I think one can explore much more of the States while still maintaining access to the comforts of civilization.

To that end, I try to go on at least one long road trip every year. I'm also one of those people who actually enjoy driving, even for absurdly long stretches of time; anything up to 12 hours per day on the road is fairly comfortable for me, and even beyond that I don't find long trips all that tiring. I'd much, much rather drive 12 hours in my car than take a two hour plane trip.

My new car was also begging to be broken in on a long road trip. Although I had a bit of a rocky start with it, all of the issues have now been sorted out (or at least as sorted out as they'll ever be), so it offered another compelling reason to take a long trip. And so it was that I booked the 5th to 12th of July off of work and embarked on a 3,378 mile (5,436km) drive from Redmond, Washington through Oregon, Idaho, and Nevada to Utah, and then back home through Arizona, Nevada, California and Oregon.

– 🕓 37 min read

The Subaru Saga

Or "buying used cars is the worst and I will never do it ever again."

I recently purchased a new-to-me 2011 Subaru WRX STI sedan. I had been thinking about buying a new car for a long time—since I graduated, more or less. Until I bought my Subaru, I owned an ageing Honda Civic (1997, to be precise). While it was a very capable and extremely reliable car throughout the many years I owned it for, it was old enough that it lacked some of the conveniences one expects in a car these days, such as power locks and windows, and most importantly, air conditioning. The lack of air conditioning in particular made it less than ideal for summer road trips, and more importantly, it was in need of some moderately expensive maintenance, and it didn't seem to make sense to me to spend much money on it given that the overall value of the car was fairly low.

I didn't go on many test drives before I purchased my Subaru, but I didn't feel the need to; I had already spent months watching reviews, reading forums, comparing specs and window shopping on AutoTrader. I knew I wanted a sporty car with plenty of horsepower for under $40,000, and all-wheel drive was a must as I go to the mountains nearly every weekend in the winter and almost as often in the summer. Besides the STI, the Evo, MazdaSpeed3, and the Golf R were the best contenders, but each had significant drawbacks with regard to my personal preferences. The Evo has a notoriously lacklustre interior (not that the STI is a lot better, to be fair); I don't like the way the MazdaSpeed3 looks, and the Golf comes with the reputation of higher maintenance costs for German cars.

I also really like the Subaru community. /r/subaru is the largest manufacturer-specific subreddit, and in addition to forums like NASIOC, it comprises a vibrant community of enthusiastic Subaru owners. Of course other brands have significant followings as well—particularly VW—but my impression of the Subaru community is that it's less elitist and, in contrast to the others, puts as much emphasis on what one can do with the cars as on the cars themselves. To "go Subaruing" is a popular term for going out into the mud or the snow or the sand or the bush and getting the car wet and dirty, and having a ton of fun in the process.

The STI, then, was the clear winner in my eyes. Truth be told, a WRX probably would have been more than sufficient for my needs, but I generally prefer to put down the extra money and be sure that I'm getting the best product. And hey, one only has so long to exercise the opportunity to be young and irresponsible!

– 🕓 7 min read

Photo App Progress Update

In September I wrote about my intention to develop a web application to share my photos. I made good progress on it throughout September and early October, but for various reasons I haven't been able to work on it much in the past month or so. Last weekend, though, I was able to solve some blocking issues which were preventing me from doing an alpha release. Those being dealt with, I've now got a few of my albums up at; take a look and leave a comment if you have any feedback.

I haven't released the source code yet as there's still a great deal of work to be done, and I'm generally of the belief that dumping a bunch of unfinished, messy code on Github with the hope that the community will sort it out is of little benefit to anyone. My goal at this point is to do a beta release in a two to three months or so and release the code at that point. That timeline is anything but firm, though. This project is turning out to be much more work than I expected (as projects are wont to do), and I expect to be fairly busy over the next couple months.

Anyway, with the alpha version up and running, this seems like a good opportunity to reflect on some of the issues I've faced that complicated the project, and on what remains to be done.

– 🕓 7 min read

Switching to Octopress

I've been using Wordpress on various sites that I've maintained for probably the better part of a decade now. By most metrics, it's still very good blogging software; I'm not aware of any self-hostable alternative that can match Wordpress in terms of the union of ease of use, feature set and flexibility that it provides. That said, lately I've been feeling that perhaps Wordpress is no longer the right choice for my specific needs. These are my problems with it:

  • It's not very good for sharing source code. There are a multitude of plugins available for syntax highlighting; the best of them seems to be Alex Gorbatchev's Syntax Highlighter, which is used by and is also what I was using on my blog. But I've had problems with the Wordpress converting some of the characters in my code to HTML entities (e.g. & for & ) which are then printed literally by the plugin.
  • I find that more often than not Wordpress' visual editor is insufficient for the formatting I want in my posts. That forces me to use the raw HTML editor. That would be fine, except that Wordpress has a strange sort of off-spec way of storing post HTML. Paragraph and line break tags are omitted in the editor and the database, and are added when the page is actually rendered to the client. This is okay, I guess, and I think Wordpress is smart enough to not insert extra paragraph tags if you do wrap your own paragraphs with them. Still, it leads to some unpredictability with regard to how a post with hand-written HTML will actually be rendered.
  • A similar but worse problem occurs when you try to insert extra line breaks. Even if you're using the raw HTML editor to insert <br /> tags, Wordpress will just get rid of them unless you insert additional non-breaking spaces (&nbsp;) so that it thinks the lines are non-empty. But! If you switch back to the visual editor for any reason, even your &nbsp;<br /> lines will be wiped out. Even for "average" users I can't think of any situation in which this "feature" would be desirable. If I put a bunch of extra newlines into something I'm writing, it is because I want exactly that many newlines. I tend to not have much patience for software that thinks it's smarter than I am; that is why I still write most of my code in Vim and use hand-written makefiles in an age of IDEs.
  • Never-ending updates. It seems like every time I log in there's a new update to download. Wordpress has updates down to a one-click install process, so this wouldn't be much of a problem except for the fact that every time I have to update Wordpress warns me that I should back up my database first. Why? Is there a significant possibility that the update might break everything? It's probably just a COA, and I back up my database daily anyway, but this whole aspect of Wordpress makes me uneasy and I don't want to have to deal with it any more. Even though I'm covered for backups, recovering from a botched update would undoubtedly be a nightmare.
  • Vulnerabilities. One of the big reasons that Wordpress is updated so frequently is that it has a long history of security vulnerabilities, and more are still being discovered and patched. This also means that if I don't log in frequently enough to download new security patches, I'm at risk of having my site compromised, loaded up with malware and blacklisted by Google.
  • It's written in PHP. PHP sucks. (I can't link to that enough.) I don't want to be dogmatic about technology choices; good software is good software regardless of what language it's written in, and my primary desire is always to use whatever is best for the task at hand. But lately I've been pretty vocal, both here and in person with friends and colleagues, about how bad PHP is. Continuing to rely on it undermines my credibility on the issue and makes me a bit of a hypocrite.

And while this might seem trivial or pedantic, it's actually one of the root causes of almost all of the issues above; the weird HTML handling, inappropriate entity conversion, and security issues are all due, at least in part, to bad design patterns popularized and perpetuated by PHP. Wordpress still uses raw non-parameterized SQL queries; blogging software built on a modern web framework with a proper ORM would not be plagued by SQL injection exploits as Wordpress has been historically and probably continues to be. - Wordpress supports themes so that you can customize the apperance of your site, but editing them kind of sucks, because they're all written in PHP instead of a proper templating language. Years ago I actually wrote a lot of PHP, but I still find Wordpress templates pretty ugly and difficult to understand; they seem more verbose than should really be necessary for what they accomplish, and have, on average, more <?php ?> sections than a keyboard has keys. (That analogy sucked. Sorry, it's late. What I'm trying to say is that there are a lot of them.)

– 🕓 5 min read

strlen without conditionals

I'm not usually much enamoured with interview-style programming puzzles because I find that a lot of them are actually more akin to math problems, trivial to implement once you figure out the salient mathematical property.  I think I have a decent intuition for math, and I certainly took enough math courses in high school and college to give me a solid foundation in the fundamentals of algebra, geometry, statistics, calculus, etc., but I'm not confident enough in my math skills to be entirely comfortable being judged by my ability to exercise them.

There are some programming puzzles I really enjoy, though.  This evening I happened across one such puzzle, via Eevee's Twitter: "implement a strlen() function in C that, when compiled, would not contain any conditional branches."  (The page contains solutions, so don't read the orange text if you want to try this yourself.)  This is exactly the kind of puzzle I like; it's fun to think about, reasonably challenging, and requires knowledge of language features combined with creative thinking.

My solution is below, but I'd recommend that you go give this a try yourself before you read on.

– 🕓 5 min read

The state of Python imaging

(With apologies to Eevee, of whom I am a great fan.)

So, I'm working on a photo album app written in Python.  I'm using Python because, well, I love it.  I think it's the perfect language for web development; it's simple and very readable, and ample syntactic sugar and just the right mix of procedural and functional features almost completely eliminate tedious boilerplate.  As a trivial example for those who have never used Python, consider:

db.session.query(db.MetadataTag).filter_by(source = 'exif')])

What I'm doing here is grabbing all of the EXIF tags I know about out of my database with SQLAlchemy (also awesome, by the way), and creating a dictionary which maps each tag's name to the corresponding ORM object; I can find the ORM object simply with db_tags[tag_name], which lets me easily and efficiently insert new tag values as I read EXIF data from a photo.  This isn't particularly difficult to do in any other language; as I said, it's a trivial example.  But consider how much more verbose the code would be: PHP, which is still the most popular language for web applications despite being an incorrigible pile of garbage, doesn't have list comprehensions.  I don't even want to think about how many lines of code this would take in Java; additionally, while I'm unfamiliar with them in general, I suspect Java ORMs require the use of generics to a painful extent (i.e. more than not at all).  In Python it only takes one SLOC, without sacrificing any readability.12

So, Python's pretty great. But like all languages, it does have a few problems. The one I have had the most difficulty with in my current project is that third party library support is somewhat lacking in some important areas; specifically, there aren't many good options for reading and manipulating images. The most popular library seems to be the Python Imaging Library, PIL. It's easy enough to install and provides a passable set of core features, but doesn't go much beyond that.  For instance, none of the various camera raw formats seem to be supported, which I consider a fairly important feature for my photo album; I'd very much like be able to upload original raws and have the application automatically convert them to JPEGs for me.  Additionally, it doesn't provide much in the way of convenience methods for cropping and resizing images.  ImageMagick has a very nice means of specifying cropping and resizing geometry with various aspect ratio-preserving (or not) behaviour, and I wanted to be able to leverage something similar for my app so that I could quickly prototype with various size options.

– 🕓 4 min read

Social networks and content ownership

I'm always wary about pre-announcing things I'm working on; I have more ambition than spare time, which leads to a tendency to start projects and not finish them.  But if there's to be a point in maintaining this website, I need to actually write things once in a while, so I may as well write about what I've been hacking on lately.  At the moment, I'm building a web-based photo album in Python.  My primary reason for doing this is that I'd like to own more of my information.  Apps and social networks have some significant advantages, particularly for the less technologically inclined, but I'm ambivalent about the trend away from an open Internet and towards a set of non-interoperable, monolithic walled-garden services.