Programming

Clojure: All grown up

by

I want to convince you of one thing: You should take a look at Clojure. It will simplify your coding life, speed up product development. It will clarify how you think about structure and complexity. And–if you like avoiding unnecessary frustration and boilerplate–it will make you happy. So reconsider the plans you had to use Rails or Django or Play to build your next business, and don’t look back. Clojure is ready for prime time.

I’m serious.

Clojure is the most pleasant language I’ve ever worked with, and that’s after 8+ years of Ruby. It brings a different paradigm to the table, changes the way you think about code. In fact, telling you everything I’ve learned from Clojure would take a book and then some, and I can’t possibly tell it all here. But I do hope to at least whet your appetite.

What is Clojure, and why should I use it?

Clojure is one of the newer kids on the block, and I like to think it’s shaking things up. Though the language rests on top of the JVM, a benevolent host, it looks and feels nothing like Java. Clojure takes advantage of the years of work that have gone into optimizing the JVM. But it kicks up the level of abstraction you can work in by about 10. (That’s 10 arbitrary units, if you’re keeping score at home.)

At a high level, Clojure makes life simpler and clearer–whether you’re building a Web API, a machine learning algorithm, or the ultimate music synthesizer. Let me tell you about some of my favorite parts.

  • Functional programming is standard. Functional code tends to be clean and easy to reason about. Instead of thinking in low-level constructs (like imperative for loops), you think in terms of what you want to get done. You write it in a natural way, and the language optimizes for you. You can use mutable state if you think it’s absolutely necessary. But most of the time, you’ll want to contain that in just a few places, and Clojure encourages you to do that. Studies have shown that reading this last sentences can cause light-headedness, fever, and feelings of mild indigestion. Nevertheless, once you move to a mostly immutable world, you’ll wonder how you ever survived with so much state floating around, running into other bits of state–just like you probably wonder why you ever put up with pointer arithmetic and memory deallocation.

  • Clojure is built for the real world. Clojure likes–but doesn’t enforce–functional purity. For the cases where it’s necessary to model data using state (see: databases), Clojure gives you safe ways to handle it. And get this: You don’t need a PhD in Monads. (Huzzah!) Now, don’t get me wrong–I love Haskell. But as an engineer, I sometimes feel like Clojure is the forgiving parent, and I’m the semi-responsible teenager. It will try to help me live a good life when it can … but when I sneak into the beer cabinet, it looks the other way. (After drinking it, by the way, I immediately feel ashamed. Because really … Smirnoff Ice?)

  • Clojure redefines the fundamental units of code. Instead of thinking in terms of loop boilerplate, guard clauses, “what if this isn’t initialized yet?”, off-by-one errors, design patterns, serialization, or semicolons, you get to think about data transformations. About defaults that are ultimately flexible. Rubyists know that their language effectively got rid of low-level for loops. In the same way, Clojure gets rid of imperative iteration in favor of declaration. Your thoughts shift away from place-oriented ideas like memory addresses and gravitate to data structures and functions like map, reduce and filter. Your “class hierarchy” turns out to be a type system that happens to also lock away well-meaning functions into dark dungeons (more on that in another article), and getting away from that is freeing. Transitioning to an open, abstract world means the intention of your code isn’t obscured by artifacts of computing, by the fact that your code is using and reusing finite memory. Your code is modeled as data, too, by the way. That means you can manipulate it using the same functions you use to manipulate strings and lists and maps. This shift dramatically simplifies and levels-up the tools available for solving new problems.

  • Clojure code tends to be insanely beautiful. More importantly, though, beautiful code happens to be both readable and efficient (after you learn to love or ignore the parentheses). (If you’re skeptical, you should read up on persistent data structures.) Speaking of beauty, in Clojure, the physical shape of your code is a nice indicator of how clean it is. For example, when you use side effects in code (like writing to disk, logging, throwing exceptions), it’s becomes pretty obvious from indentation and ! s in method names. And an excess of parentheses at the end of a function call tells you that you may want to break it up into smaller functions. So when you’re looking to clean up code, a quick visual scan can often tell you where to find candidates for refactoring.

  • Clojure is primed for parallelism. Functional code is easy to reason about, regardless of how many threads you’re working with. For non-functional concerns like database manipulation, Clojure has brought several innovative concepts from academia into the mainstream. (In fact, several of these have seen adoption in other languages, too.) For example, you have Clojure’s software transactional memory (STM) at your disposal–it will help you coordinate state change in a wide range of scenarios. So when you do have your Twitter moment and need to scale, you have loads of options to do so, from lightweight to industrial. You won’t have to change stacks or even the way you think about code. Maybe you’ll convert a (map ...) call to (pmap ...) to make it parallel. Or maybe you’ll decide to use agents or refs coordinated by STM. What you won’t be doing is spending weeks scratching your head over semaphores and mutexes.

  • You can forget context. Why? Because there is no context to learn. Most of your code takes input and produces output, and the pieces of code that don’t follow that pattern tend to stick out. On top of that, each file states its own dependencies, so you’re never left guessing where that strange symbol came from. (“Did one of my libraries monkey-patch my code? Is that a local var or a method name? Did I inherit that, or is it in a mixin?”) Given this explicitness, you can move between abstraction layers when its appropriate, but you’re rarely forced to. Each layer also stands on its own. This means it’s relatively easy to ramp up new engineers on one part of your codebase at a time. Lack of context also means that any change you might want to make to a library is as easy as wrapping a function from the library you’re importing. Never again will you have to bite your nails as you monkey-patch a monkey-patch to try to tweak that opinionated library ever so slightly to meet your needs. Just swap out a context-free function!

  • The community values simplicity. Rich Hickey, Clojure’s inventor, is focused on disentangling unrelated concepts. He talks a lot about the benefits that gives to a codebase one, two, even five years out. So naturally, simplicity is one of the community’s key values. You’ll often hear the term “complect” when you’re talking with Clojure engineers. The word is an ancient one and was revived when Rich used it to describe code that ties unrelated concerns together. (His examples included the class construct, stateful variables, and conditional statements.) Because Clojurists are bent on decomplecting, most libraries compose with each other. This means you can build up a dependency tree that matches your domain requirements instead of basing that decision on which libraries are meant to play nice with each other. Switching out a templating engine or database layer should not be hard at all, because at the end of the day, if a library’s API has been designed correctly, functions are the actors, the heroes. And what’s more composable than a function?

Clojure has a lot more going for it too. Maybe I’ll be able to tell you more about its Java interop, concurrency primitives and macro systems some other time. For now, rest assured: this language is awesome.

What’s it missing?

Now, every language decision is a trade-off, and Clojure has its own set of trade-offs, too. While at the language level, I think most people like the trade-offs that Clojure makes, some common concerns are:

  • Error messages . While Clojure is beautifully designed, it still rests on the JVM and has some pretty ugly stack traces and error messages. This is steadily improving over time, but hopefully this will become a top priority soon.
  • Debugging . Clojure has never had a great debugger. Some say that once you are comfortable with Clojure, you won’t need one. (They recommend you use a REPL to execute functions that you’ve constructed over predictable data.) However, in my experience, most engineers are used to having full-context debuggers at their disposal, and I don’t blame them.
  • Startup time . This isn’t a big deal except when writing one-off scripts, but the JVM takes a good second or two to launch, which means that writing instant scripts or CLI tools in Clojure isn’t a great choice right now.
  • Ecosystem . The Clojure ecosystem is vibrant and evolving. However, given that it’s only a few years old, there is still a gap between what we have now and what Rails and Django offer.

All of these are valid concerns, but in my experience, they’re the kind of thing that you quickly learn to work with and are worth it to have such a powerful language and set of libraries (JVM) at your disposal. Of course, your team will have to weigh these trade-offs for yourself.

Why now?

So the question is, why is now the right time to build your product using Clojure? You were planning on building on Rails or Django. “Can I even hire for Clojure?”, you might be thinking.

These are fair questions. Clojure has been around for over five years now, and I have only felt like I could recommend it in the last couple of years. For a while, the tooling situation was iffy, the language was in flux, and the community had not solidified. But all of that has changed:

  • In Chas Emerick’s annual State of Clojure survey, it became obvious that Clojure adoption is happening. Plenty of people love Clojure and are looking for a job. More are trying it out and sticking around.

  • Community projects like ClojureDocs, Clojure Toolbox and ClojureWerkz make library discovery easy

  • Leiningen and Cake joined forces to become an all-powerful build tool. Then Leiningen reached version 2. (And let me tell you, Leiningen 2 alone makes Clojure worth using.) On the one hand, Leiningen makes it possible to reap the benefits of Maven without knowing anything about it. On the other, its tasks system will automate anything you want using Clojure.

  • Vim and Sublime are as viable as Emacs for Clojure coding.

  • Midje fixes problems with clojure.test and makes top-down testing possible and fun.

  • Datomic solves the versioned database problem everyone seems to be facing right now in today’s “Big Data” world.

  • Immutant lets you deploy your Clojure Web app to a JBoss server and take advantage of JBoss’s scalability without any XML configuration. You get a mature enterprise-ready Java server without the pain of Java or of configuration. Feels almost like cheating, doesn’t it?

If it matters to you, by the way, Clojure is officially in Thoughtworks’ technology adoption ring, which means that the firm recommends it for production use right now.

Oh, and big names have already adopted Clojure. There is a running list of companies using Clojure, and it includes companies like the Climate Corporation, Akamai, Flightcaster and BackType (now owned by Twitter). Prismatic (née Woven) is a Clojure team, and according to Quora, Amazon is on board with the language, too.

(For one other take-on-the-world company that’s decided to hop on the Clojure train, check out the end of this article.)

What you’re thinking

Okay, so you’ve just heard the high-level benefits of Clojure, and you know that successful companies are using it right now. But you’re skeptical. You might be thinking:

I don’t need to analyze lots of data; I don’t need to scale yet

Clojure is not a language dedicated to math or big data. While it happens to be better at dealing with math and physics and machine learning better than most other languages (especially the imperative ones), it is a general purpose language. It’s does everything that Ruby does in a scalable way.

So even though scalability and mathematical prowess are things Clojure excels at, don’t think of it as “a language for number crunching and statistics”. The real benefit of Clojure is its tendency towards functional purity. The way it makes you want to get rid of mutable state, or at least contain it in a box. The way it gets you to think of almost all code, even frontend code, in terms of data flow. The higher-level fundamental units it adds to your toolbelt. All of these let you squash bugs before they happen, create parallel code without agonizing over locks, and ramp up developers right away on a new codebase.

Functional programming is a fad

Another objection I’ve heard is that functional programming is a fad and isn’t a way to build a long-term business.

Functional programming may be having a resurgence, but it is not new and it is not a fad. It has been around at least since the days of Scheme and is a proven model, having more than 60 years of research behind it. Being so well thought out, the functional philosophy tends to irreversibly change your brain, change the way you think about data flow. It’s one of those things you adopt and never want to leave. And given the increased need for concurrency, it’s unlikely that the functional style will go away any time in the near future.

So if you’re thinking to yourself, “I don’t want to use the latest language or library, I want old, boring technology”, remember that you’re dealing with the confluence of the oldest (Lisp) and most boring (Java) technologies available.

The ecosystem is small

This is one objection I’ve heard, and I do think there’s merit to it. Though I think we’re past the early adopter phase of the adoption cycle, the Clojure ecosystem isn’t big enough to have worked out all possible kinks or problems that you might find. If your team doesn’t have a tolerance for solving pain points on their own (for example, training your engineers on this new way of thinking or spending time to make editing code awesome), Clojure might not be the best fit. However, if you’re willing to eat the up-front cost, I can tell you that what you find on the other side will almost certainly be worth it. (And finding people who are passionate about Clojure shouldn’t be hard, either.)

Starting down the rabbit hole

Even though this article is far from comprehensive, I hope I’ve coaxed you into thinking that you would be a happier, more productive engineer if you were using Clojure. Or at least that you should find out more.

If you do want to know more about the philosophy of Clojure and simplicity, a good place to start is with two of Rich Hickey’s best talks (in my opinion), Simple Made Easy and The Value of Values. No one explains these concepts like Rich. Moving on from there, I’d recommend reading one of three books. They are, affiliate links included, the following (feel free to remove the affiliate code):

  • Clojure Programming: This is a comprehensive book on Clojure. It starts with the basics, including rationale but does not stop there. If you want a one-stop book that explains everything from the reduce function functional coding to database connections and Web programming, this is the book for you. It also happens to be the most recently published book, so it’s got the most current information. Also, like all O’Reilly books, it has great typography. (Get the printed version!)

  • Clojure in Action: Clojure in Action is a great book about how to use Clojure day-to-day, for example, how to interface with Postgres, RabbitMQ and Map/Reduce.

  • The Joy of ClojureThe Joy of Clojure:The Joy of Clojure is my favorite book on Clojure. It is deeply philosophical but also approachable. It covers the most advanced material of all three. Focuses less on the practical than on the theoretical, this book aims to explore what it means to code in Clojure in the twenty-first century. I highly recommended it, but maybe not as your introduction to the language.

While you’re reading the book, you should check out the mailing list and follow as many Clojure contributors as possible. Maybe subscribe to the Clojure Gazette or def newsletter or follow Planet Clojure on Twitter.

Finally, feel free to stop by here once in a while to say hey. I’m hoping to pick up writing more about functional experiments and code architecture now that I have more time to think about these things, and I hope to see you again … which brings me to my last point …

One more thing

At Minerva, we have decided to implement our platform in Clojure. So saying “you should use Clojure” isn’t something I’m saying lightly. Rather than being a hinderance to hiring, we’ve seen it boost interest from people who were otherwise not looking for new opportunities. (Psst … This is how you fight the big cos. for talent.)

We’ve been using a Clojure stack for our production code, and have seen that the codebase is already cleaner, leaner, and more extensible than any we’ve worked with in the past. Being able to tailor abstractions to our domain means that we can move quickly and focus on features instead of low-level thinking. If you’re interested in finding out more, I’d love to talk to you. And make sure to say hi at Clojure/West!