Why I love Common Lisp and hate Java, part III – macros

Macros are for lazy people. If you are a relentless data-monkey who diligently punches away and like to keep yourself mechanically occupied while scoffing down your lunch behind your desk — and you don’t mind it — that’s great, keep it up; macros are probably not for you. A powerful tool, once mastered might leave you with too much time on your hands — and you don’t want to seem idle.

If you are like me — lazy, likes short-cuts and maximizing the output-to-effort ratio — read on. I don’t like wasting time. Coding in Java feels like a waste of time, oftentimes. Therefore, I don’t like Java. Common Lisp equips you with an arsenal of tools that enables you to graduate to a whole new level of efficiency. I have pointed out the conciseness and the almost divine REPL experience in Part I, the elegance of non-verbose programming with functions as first-class citizens in Part II; this post will drive home the point with Lisp’s powerful meta-programming macros.

There are plenty of good, books, articles and weblogs out there that cover the larger-than-you-think topic of Common Lisp macros. I only scratched the surface myself, but I’ve experimented enough to see the extent of its powerful potential — I will share with you these few moments of clarity.

One aspect of the Common Lisp macro is that it can save you a lot of time and effort when you have to write a lot of similarly patterned code. A great example is HTML — Java’s cousin of the Verbose family. Here (and Part II) is an excellent showcase of exploiting Lisp macros in a way to make building websites fun and engaging, rather than tedious, meticulous and did I mention verbose? Scroll down to “Macros: Fighting the evils of code duplication” on Adam Petersen’s article to get straight to the point.

The very inception of this blog was due to a eureka moment while I was writing unit tests for Open-VRP. An oft-procrastinated task of frustratingly repetitive code-monkeying became an intellectually challenging optimization task. Unit testing is a great way to spot deep bugs while you are developing, but often dismissed due to the irksome procedure of actually writing them out. One by one. After I wrote a few, I noticed a pattern and wrote a simple macro to save me some typing.

Let’s say the following defines a general unit test in FiveAM (which in itself relies on macros) that checks if the algorithm finishes without errors:

(test "ts-1" (is (solve-prob test-vrp (make-instance 'tabu-search)))

‘Test’ defines a unit test; the first argument defines the name and `solve-prob` is a call to solve a VRP-testcase with Tabu Search. The ‘is’ is part of FiveAM and basically checks if the call successfully returns (non-NIL).

If we have a few more test-cases we want to test the algorithm on, we define (with copy-paste):

(test "ts-1" (is (solve-prob test-vrp-1 (make-instance 'tabu-search)))
(test "ts-2" (is (solve-prob test-vrp-2 (make-instance 'tabu-search)))
(test "ts-3" (is (solve-prob test-vrp-3 (make-instance 'tabu-search)))
(test "ts-4" (is (solve-prob test-vrp-4 (make-instance 'tabu-search)))
(test "ts-5" (is (solve-prob test-vrp-5 (make-instance 'tabu-search)))

There is quite a clear pattern. Let’s abstract that away with a simple macro:

(defmacro one-testcase (algo test-case)
   `(test ,(symb algo test-case) 
          (is (solve-prob ,test-case (make-instance ,algo)))))

Assume we have defined symb as a simple function that concatenates symbols, which we use to create a unique name for the unit test. Now we can define the above 5 unit tests more concisely with:

(one-testcase 'tabu-search test-vrp-1)
(one-testcase 'tabu-search test-vrp-2)
(one-testcase 'tabu-search test-vrp-3)
(one-testcase 'tabu-search test-vrp-4)
(one-testcase 'tabu-search test-vrp-5)

Do I see another pattern? Great! Let’s write another macro!

(defmacro on-all-testcases (algo)
`(progn
   (one-testcase ,algo test-vrp-1)
   (one-testcase ,algo test-vrp-2)
   (one-testcase ,algo test-vrp-3)
   (one-testcase ,algo test-vrp-4)
   (one-testcase ,algo test-vrp-5)))

Now we can define 5 test-cases in one line, which is awesome, because we have more algorithms to test:

(on-all-testcases 'tabu-search)
(on-all-testcases 'genetic-algorithm)
(on-all-testcases 'ant-colony-optimization)
(on-all-testcases 'branch-and-cut)

When we have more test-cases, just add them once to the on-all-testcases macro. What would I do without macros? I would copy-paste blocks of code, use find-replace to change snippets and leave it at that. A quick fix far from elegance — “productive” indeed, since you’ll have 20 screen-filling lines of code. This is just a small simple example. Such efficiency hacks can come a long way, further down the road.

Great. So macros just save me some typing and copy-pasting? In this example, yes. Let’s continue with another example where macros are in-expendable — to write something that is simply impossible to write in Java (or other languages without a Lisp macro system). Quite a bold claim indeed. I’ll take my chances.

Metaheuristics are often used to optimize NP-hard problems. They are very useful and practical to quickly get a near-optimum solution to an otherwise extremely large problem. An inevitable balancing-act is required to explore the search-space wide enough, whilst keeping your search algorithm effective. Multi-start heuristics are often adapted to widen the search space by initializing the search with multiple different starting points.

Here is a task: Implement a multi-start heuristic for any algorithm and collect the results in a list/array. The implementation should not have any assumptions about the algorithm, except that it returns a solution.

Take a second to think this through and I’m sure there are wonderful hackers out there that have a pretty solution to this, perhaps through interfaces and abstract classes.

Now behold what Lisp allows you to do:

(defmacro multi-run (times &body algo-call)
   `(loop for ,(gensym) below ,times collect (progn ,@algo-call)))

One line?! *gasp* How? What about.. all the classes.. files.. and the colourful boilerplate code from Eclipse? What is my boss going to think about one feeble line of code?

*gulp*

The reason why you cannot do this as a function is multiple evaluation. When you pass arguments to a function, all arguments will be evaluated before the body of code is called (with the exception of special operators, such as if). The macro expands the algo-call — which could be anything, no assumptions necessary — into the loop form and evaluates it multiple times. A 50 times multi-start tabu search is achieved simply by calling the algo:

(multi-run 50 (solve-prob test-case (make-instance 'tabu-search)))

During my Java times I couldn’t even dream up such an elegant solution. Learning Common Lisp made me realize that there is a lot more to programming. Complete ways of coding I was oblivious to. A chest full of unexposed tools and tricks that keep me wired in. The journey into the realm of Common Lisp has only begun for me. Hopefully this series will attract like-minded people to discover this world alongside with me.

These are just two simple examples where I have personally come in contact with macros that are different from the common examples you see in books. There are plenty of other useful things you can do with macros, such as extending the language with your own special operators. Here is an anaphoric macro that I particularly like to use from Paul Graham’s On Lisp, chapter 14:

(defmacro aif (test-form then-form &optional else-form)
   `(let ((it ,test-form))
      (if it ,then-form ,else-form))) 

Because I often write the following:

(let ((x (some-test)))
   (if x
       (do-something x)
       (something-else x)))

which can now be written as:

(aif (some-test)
     (do-something it)
     (something-else it))

Pretty sweet. I never fell in love with Java’s syntax, but I couldn’t change it. I would have to wait another year for a newer version of Java to be released and hope that this time things will be better. Even then, I was spoon-fed the syntax and constructs that Java deemed appropriate. In the paradigm of Common Lisp, there is no spoon.

To conclude, I love Common Lisp and hate Java for the same reason I love Nassim N. Taleb and not Thomas L. Friedman. ‘The Black Swan’ would be a monolith if Friedman were to write it. On the other hand, ‘The World is Flat’ could be a densely written two-part essay. Just click on their website and you’ll see who cares more about appearance.


For further reading:

About these ads

11 thoughts on “Why I love Common Lisp and hate Java, part III – macros

  1. loop in Java 1.5, as part of JSR-201. Notice what a difference macros make. A Lisp programmer who notices a common pattern in their code can write a macro to give themselves a source-level abstraction of that pattern. A Java programmer who notices the same pattern has to convince Sun that this particular abstraction is worth adding to the language. Then Sun has to publish a JSR and convene an industry-wide “expert group” to hash everything out. That process—according to Sun—takes an average of 18 months. After that, the compiler writers all have to go upgrade their compilers to support the new feature. And even once the Java programmer’s favorite compiler supports the new version of Java, they probably still can’t use the new feature until they’re allowed to break source compatibility with older versions of Java. So an annoyance that Common Lisp programmers can resolve for themselves within five minutes plagues Java programmers for years.

  2. Interestingly, Tcl provides a very similar ability to your Lisp macros:

    proc aif {test_form then_form {optional “”} {
    if { [ uplevel 1 $test_form ] } {
    uplevel 1 $then_form
    } else {
    uplevel 1 $optional
    }
    }

    Call as:

    aif some-test {do-something it} {something-else it}

  3. I think magical IT variable is not a good idea. I’d put the binding visible:

    (ifb (var (some-test))
    (do-something var)
    (something-else var))

    But ALEXANDRIA library has IF-LET which can bind several variables.

    • I agree that it is not a good idea to create a free variable in a macro — it may cause a bug once in a blue moon, if you decide to use the ‘it’ as another variable. Your suggestion to explicitly create your own binding is a good one, thanks.

    • test is a macro and expects the first argument to be a name (not a form). symb is a simple function that concatenates symbols, so we can create a unique name for the test-case. Please let me know how — replacing it with a function would be great.

      • (defun symb (a b)
        (intern (format nil “~a-~a” (symbol-name a) (symbol-name b)))

        (defun one-testcase (algo test-case)
        (test (symb algo test-case)
        (is (solve-prob test-case (make-instance algo)))))

        (one-testcase ‘tabu-search ‘test-vrp-1)

          • Thanks Foo for your input. I like your symb function. Much nicer than my solution here. I’ll change it to yours if you don’t mind.

            I tried your approach, but it didn’t work. I think because of the nature of FiveAM’s test macro. What do you think? Is there another way?

            ps: feel free to fork ;-)

What say you?

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s