On Clarity

So we’re replacing a nice, succinct one line regular expression with ten lines of objects, methods, and named enumerations. This is progress?

(via Coding Horror)

The problem with this kind of future primitive style of programming is that while perhaps hardcoding SQL and RE’s inside of your tiny web application makes sense, the system breaks down when you need to scale the codebase to do much more than it was originally intended, or if more than 1 programmer ever works on it, or if you yourself work on it for more than a week. For example:

“SELECT * from Customers WHERE referrer_id in (0, 1, 3, 4, 7) ORDER BY CompanyName”

This is a great statement to type into a MySQL prompt. But how do I re-use something like this query from elsewhere in my application? The obvious (and sadly frequent) answer is to copy and paste the source code. Now a couple of months go by, my application has scaled to 65MM customers, and 15% of the customers were referred by source 0 (let’s say this represents our own web site). Most other referrer_ids are spare, representing another customer who refers a couple of people at best. You’ll find pretty quickly that your query time skyrockets for queries which contain referrer #0, even with the best of indexing and such — 15% of 65MM rows takes hella time to sort and return. To avoid live site issues, you have to trace down and expunge all of the queries which could possibly request referrer ID #0, which means putting some custom application code[1] in front of all of those copy and paste queries, plus any minor variants that may or may not textually match the original problem query depending on whether or not your engineers have been consistent with whitespace and capitalization[2].

Admittedly the SQL-wrapping API showcased in the blog post looks pretty poorly designed, but I’d like to point out that even with this obviously shitty API designed by rodents, you can still perform structured transformations on your code using OO techniques, e.g., if you find some horrible performance bug in your CustomerCollection it becomes a fairly trivial fix to modify the Where() function to get the performance you want or patch the security hole you need or whatever the case may be. Let’s review: a problem with a whole class of queries can be solved if you’ve built a wrapper API with somewhere between 10 to 100 lines of Java (or 1 to 5 lines of Python) after finding the one location in the codebase where that table is accessed. With hardcoded queries you not only have to find all of the places in your codebase, you have to patch them all (and they’re probably all slightly different). You can imagine a similar workload imbalance with regression testing.

With each hardcoded query, that’s one more bit of flexibility that some future programmer doesn’t have when facing all the challenging problems you’ve created. OOP has some great ideas behind it, especially in the arena of applying either general code changes to a large class of problems, or in applying a really precise code change to a single or very tiny class of problems.

CustomerCollection c = new CustomerCollection();
c.Where(Customer.Columns.Country, “USA”);
c.OrderByAsc(Customer.Columns.CompanyName);
c.Load();

Unfortunately, it’s not easy to model a relational database query langauge well in a generic way, and so we see a metric ton of really, really, really bad APIs like this[3]. API design is pretty hard, most people can’t do it because it requires a good degree of experience and foresight, so the common conclusion is to not write a bad API and to instead hardcode a bunch of SQL queries. This is perhaps actually worse in that now a) you’ve already admitted that you can’t program well enough to design an API and b) now you’re condemning other people who actually can program well enough to design an API to live with the nightmare you’ve created, thank-you-very-much.

Typically those who can program well enough to design an API leave to go take some ultraboring enterprise job where the work is as interesting as stale oatmeal in Noe Valley, but they think “well, at least I don’t heave to deal with wackos hardcoding a bunch of shit anymore” and the industry suffers. The people left behind are stuck in a neverending hell of constantly applying patches to a crappy codebase, the continued application of which eventually corrupts the purity of the WYSIWYG hardcoded SQL query anyway.

At some point most programmers mature to the point where you understand that the total pain-in-the-ass cost of every line of code is actually multiplied by the lifetime of the line of code. So while I agree with:

There’s nothing but code that looks like exactly what it does. And that’s a beautiful thing.

… in this particular case of hardcoding shit from foreign languages into your application, you actually just totally fucked some future version of yourself that now needs to find and unpack n instances of your hardcoded shit. And that’s only a beautiful thing if you’re a masochist.

1. One constant in the world of modern software development is that your application is never entirely written in SQL.
2. Ha!
3. I think the Daily WTF is predominantly comprised of this and funny Windows error messages developed in Southeast Asia.