Clojure: All grown up

Tuesday, 5 March 2013

I want to convince you of one thing: You should adopt Clojure. It will simplify your coding life, speed up product development. It will clarify how you think about structure and complexity. And–if you like avoiding unnecessary frustration and boilerplate–it will make you happy. So throw away the plans you had to use Rails or Django or Play to build your next business. Today. And never look back. Clojure is ready for prime time.

I’m serious.

Clojure is the most pleasant language I’ve ever worked with, and that’s after 8 years of Ruby. It brings a different paradigm to the table, changes the way you think about code. In fact, telling you everything I’ve learned from Clojure would take a book and then some, and I can’t possibly tell it all here. But I do hope to at least whet your appetite.

What is Clojure, and why should I use it?

Clojure is one of the newer kids on the block, and I like to think it’s shaking things up. Though the language rests on top of the JVM, a benevolent host, it looks and feels nothing like Java. Clojure takes advantage of the years of work that have gone into optimizing the JVM. But it kicks up the level of abstraction you can work in by about 10. (That’s 10 arbitrary units, if you’re keeping score at home.)

At a high level, Clojure makes life simpler and clearer – whether you’re building a Web API, a machine learning algorithm, or the ultimate music synthesizer. Let me tell you about some of my favorite parts.

  • Functional programming is standard. Functional code tends to be clean and easy to reason about. Instead of thinking in terms of computer-based limitations from decades ago, you think in terms of what you want to get done. You write it in the most natural way possible, and the language optimizes for you. You can use mutable state if you think it’s absolutely necessary. But most of the time, you’re better off without it. Studies have shown that reading this last sentences can cause light-headedness, fever, and feelings of mild indigestion. Nevertheless, encapsulating mutable state in one place in your code is something that, once you start doing, you will love. Functional and object-oriented programs are Turing-equivalent. That means that even though you might be thinking “How can I possibly do X without state?”, there is always a way (and it’s often easier). Studies have shown that understanding this truth tends to cause euphoria, hallucinations, and delusions of grandeur. Once you move to a mostly immutable world, you’ll wonder how you ever survived a world filled with so much state floating around, running into other bits of state – just like you probably wonder why you ever put up with pointer arithmetic and memory deallocation.

  • Clojure is built for the real world. Clojure likes – but doesn’t enforce – purity. For the few cases that it’s convenient to model data using state (see: databases), Clojure gives you safe ways to handle that mutability. And get this: You don’t need a PhD in Monads. Huzzah! (Haskell is a great language, by the way – it just isn’t a great first functional language for Web programming.) As an engineer, I sometimes feel like Clojure is the forgiving parent, and I’m the semi-responsible teenager. It will try to help me live a good life when it can … but when I sneak into the beer cabinet, it looks the other way. (After drinking it, by the way, I immediately feel ashamed. Because really … Smirnoff Ice?)

  • Clojure redefines the fundamental units of code. Instead of thinking in terms of loop boilerplate, guard clauses, “what if this isn’t initialized yet?”, off-by-one errors, design patterns, serialization, or semicolons, you get to think about data transformations. About defaults that are ultimately flexible. Rubyists know that their language effectively got rid of for loops. In the same way, Clojure gets rid of imperative iteration in favor of declaration. Your thoughts shift away from place-oriented constructs like memory and gravitate to data structures and functions like map, reduce and filter. Your “class hierarchy” turns out to be a type system that happens to also lock away well-meaning functions into dark dungeons (more on that in another article), and getting away from that is freeing. Transitioning to an open, abstract world means the intention of your code isn’t obscured by artifacts of computing, by the fact that your code is using and reusing finite memory. Your code is modeled as data, too, by the way. That means you can manipulate it using the same functions you use to manipulate strings and lists and maps.

  • Clojure code tends to be insanely beautiful. More importantly, though, beautiful code happens to be both readable and efficient. (If you’re skeptical, you should read up on persistent data structures.) Speaking of beauty, in Clojure, the physical shape of your code is a nice indicator of how clean it is. For example, when you use side effects in code (like writing to disk, logging, throwing exceptions), it’s immediately obvious to anyone who knows anything about the language. And the number of parentheses at the end of your function declaration tells you how much you should probably break it up into smaller functions. So when you’re looking to clean up code, a quick visual scan can often tell you where to find candidates for refactoring.

  • Clojure is primed for parallelism. Functional code is easy to reason about, regardless of how many threads you’re working with. For non-functional concerns like database manipulation, you have Clojure’s software transactional memory (STM) at your disposal – it will help you coordinate state change in a wide range of scenarios. So when you do have your Twitter moment and need to scale, you have loads of options to do so, from lightweight to industrial. You won’t have to change stacks or even the way you think about code. Maybe you’ll convert a (map ...) call to (pmap ...) to make it parallel. Or maybe you’ll decide to use agents or refs coordinated by STM. What you won’t be doing is spending weeks scratching your head over semaphores and mutexes.

  • You can forget context. Why? Because there is no context to learn. Most of your code takes input and produces output, and the pieces of code that don’t follow that pattern tend to stick out. On top of that, each file states its own dependencies, so you’re never left guessing where that strange symbol came from. (“Did one of my libraries monkey-patch my code? Is that a local var or a method name? Did I inherit that, or is it in a mixin?”) Given this explicitness, you can move between abstraction layers when its appropriate, but you’re rarely forced to. Each layer also stands on its own. This means it’s relatively easy to ramp up new engineers on one part of your codebase at a time. Lack of context also means that any change you might want to make to a library is as easy as wrapping a function from the library you’re importing. Never again will you have to bite your nails as you monkey-patch a monkey-patch to try to tweak that opinionated library ever so slightly to meet your needs. Just swap out a context-free function!

  • The community values simplicity. Rich Hickey, Clojure’s inventor, is focused on disentangling unrelated concepts. He talks a lot about the benefits that gives to a codebase one, two, even five years out. So naturally, simplicity is one of the community’s key values. You’ll often hear the term “complect” when you’re talking with Clojure engineers. The word is an ancient one and was revived when Rich used it to describe code that ties unrelated concerns together. (His examples included the class construct, stateful variables, and conditional statements.) Because Clojurists are bent on decomplecting, most libraries compose with each other. This means you can build up a dependency tree that matches your domain requirements instead of basing that decision on which libraries are meant to play nice with each other. Switching out a templating engine or database layer should not be hard at all, because at the end of the day, if a library’s API has been designed correctly, functions are the actors, the heroes. And what’s more composable than a function?

Clojure has a lot more going for it too. Maybe I’ll be able to tell you more about its Java interop, concurrency primitives and macro systems some other time. For now, rest assured: this language is awesome.

Why now?

So the question is, why is now the right time to build your product using Clojure? You were planning on building on Rails or Django. “Can I even hire for Clojure?”, you might be thinking.

These are fair questions. Clojure has been around for over five years now, and I have only felt like I could recommend it in the last couple of years. For a while, the tooling situation was iffy, the language was in flux, and the community had not solidified. But all of that has changed:

  • In Chas Emerick’s annual State of Clojure survey, it became obvious that Clojure adoption is happening. Plenty of people love Clojure and are looking for a job. More are trying it out and sticking around.

  • Community projects like ClojureDocs, Clojure Toolbox and ClojureWerkz make library discovery easy

  • Leiningen and Cake, joined forces to become an all-powerful build tool. Then Leiningen reached version 2. (And let me tell you, Leiningen 2 alone makes Clojure worth using.) On the one hand, Leiningen makes it possible to reap the benefits of Maven without knowing anything about it. On the other, its tasks system will automate anything you want using Clojure.

  • Vim and Sublime are just as viable as Emacs for Clojure coding.

  • Midje fixes problems with clojure.test and makes top-down testing possible and fun.

  • Datomic solves the versioned database problem everyone seems to be facing right now in today’s “Big Data” world.

  • Immutant lets you deploy your Clojure Web app to a JBoss server and take advantage of JBoss’s scalability without any XML configuration. You get a mature enterprise-ready Java server without the pain of Java or of configuration. Feels almost like cheating, doesn’t it?

If it matters to you, by the way, Clojure is officially in Thoughtworks’ technology adoption ring, which means that the firm recommends it for production use right now.

Oh, and big names have already adopted Clojure. There is a running list of companies using Clojure, and it includes companies like the Climate Corporation, Akamai, Flightcaster and BackType (now owned by Twitter). Prismatic (née Woven) is a Clojure team, and according to Quora, Amazon is on board with the language, too.

(For one other take-on-the-world company that’s decided to hop on the Clojure train, check out the end of this article.)

What you’re thinking

Okay, so you’ve just heard the high-level benefits of Clojure, and you know that successful companies are using it right now. But you’re skeptical. You might be thinking:

I don’t need to analyze big data; I don’t need to scale yet

Clojure is not a language dedicated to math or big data. While it happens to be better at dealing with math and physics and machine learning better than most other languages (especially the imperative ones), it is a general purpose language. It’s does everything that Ruby does in a scalable way. (The one exception is creating Unix scripts – Ruby is better for this because the JVM takes more than 500ms to warm up. I still write all my scripts in Bash and Ruby).

So even though scalability and mathematical prowess are things Clojure excels at, don’t think of it as “a language for number crunching and statistics”. The real benefit of Clojure is its tendency towards functional purity. The way it makes you want to get rid of mutable state, or at least contain it in a box. The way it gets you to think of almost all code, even frontend code, in terms of data flow. The higher-level fundamental units it adds to your toolbelt. All of these let you squash bugs before they happen, create parallel code without agonizing over locks, and ramp up developers right away on a new codebase.

Functional programming is just a fad

Another objection I’ve heard is that functional programming is a fad and will go away.

Functional programming is not new and it is not a fad. While functional is having a resurgence, it has been around at least since the days of Scheme and is a proven model, having more than 60 years of research behind it. Being so well thought out, the functional philosophy tends to irreversibly change your brain, change the way you think about data flow. It’s one of those things you adopt and never want to leave.

So if you’re thinking to yourself, “I don’t want to use the latest language or library, I want old, boring technology”, remember that you’re dealing with the confluence of the oldest (Lisp) and most boring (Java) technologies available.

Starting down the rabbit hole

Even though this article is far from comprehensive, I hope I’ve coaxed you into thinking that you would be a happier, more productive engineer if you were using Clojure. Or at least that you should find out more.

If you do want to know more about the philosophy of Clojure and simplicity, a good place to start is with two of Rich Hickey’s best talks (in my opinion), Simple Made Easy and The Value of Values. No one explains these concepts like Rich. Moving on from there, I’d recommend reading one of three books. They are, affiliate links included, the following (feel free to remove the affiliate code):

  • Clojure Programming: This is a comprehensive book on Clojure. It starts with the basics, including rationale but does not stop there. If you want a one-stop book that explains everything from the reduce function functional coding to database connections and Web programming, this is the book for you. It also happens to be the most recently published book, so it’s got the most current information. Also, like all O’Reilly books, it has great typography. (Get the printed version!)

  • Clojure in Action: Clojure in Action is a great book about how to use Clojure day-to-day, for example, how to interface with Postgres, RabbitMQ and Map/Reduce.

  • The Joy of Clojure: The Joy of Clojure is my favorite book on Clojure. It is deeply philosophical but also approachable. It covers the most advanced material of all three. Focuses less on the practical than on the theoretical, this book aims to explore what it means to code in Clojure in the twenty-first century. I highly recommended it, but maybe not as your introduction to the language.

While you’re reading the book, you should check out the mailing list and follow as many Clojure contributors as possible. Maybe subscribe to the Clojure Gazette or def newsletter or follow Planet Clojure on Twitter.

Finally, feel free to stop by here once in a while to say hey. I’m hoping to pick up writing more about functional experiments and code architecture now that I have more time to think about these things, and I hope to see you again … which brings me to my last point …

One more thing

At Minerva, we have decided to implement our platform in Clojure. So saying “you should use Clojure” isn’t something I’m saying lightly. Rather than being a hindrance to hiring, we’ve seen it boost interest from people who were otherwise not looking for new opportunities. (Psst … This is how you fight Google for talent.)

We’ve been using a Clojure stack for our production code, and have seen that the codebase is already cleaner, leaner, and more extensible than any we’ve worked with in the past. Being able to tailor abstractions to our domain means that we can move quickly and focus on features instead of low-level thinking. If you’re interested in finding out more, I’d love to talk to you. And make sure to say hi at Clojure/West!