Wrapping code using compiler macros

Macros are probably my favourite thing about Lisp. They were the reason I started using Common Lisp and I do a lot of work with them. I guess I like manipulating programs, but I don't want to write new compilers.

Recently, I've been looking for ways to take an existing function definition and replace each occurrence of it in some source code that wraps the call to set up a specific environment. Furthermore, I wanted to avoid the need to actually change the source code. I thought it would be nice to share the way I went about doing this. (Hey, that's what blogs are for, right?) Plus, it gives a nice demonstration of what compiler macros are and how they work.

The problem

For demonstration purposes, here's a simple function definition that we'll use throughout.

(defun example (x y) (* x x y y))

The goal is to wrap each occurrence of example in the source code with code that will keep a history of the second argument. That is, throughout the entire program, we want to remember all the values that were passed as the second argument.

There are certainly other ways to do this (using around methods, for example), but for the sake of illustration, we'll ignore them.

Ultimately, we want to turn this

(example x y)

into something like this

;; You only want to evaluate the second argument once.
(let ((tmp y))
  (add-value-to-history tmp)
  (example x tmp))

A solution using compiler macros

Recall that compiler macros are macros that are applied only during compilation. Although it is optional for an implementation to apply them, I'm going to assume that the implementation does.

One very important property of compiler macros is that if the same form to be expanded is returned, no expansion is done. This is in contrast to regular macros in that they must always provide an expansion that is not the same as the form to expand lest you end up in an infinite loop.

How can we take advantage of this? The idiom for this is to insert a macro into the enclosing lexical environment and decline to expand the form if the macro is present in the environment.

(defparameter *value-history* nil)

(defun add-value-to-history (val)
  (push val *value-history*))

(defun expands-in-environment-p (form env)
  (second (multiple-value-list (macroexpand-1 form env))))

(define-compiler-macro example (&whole form x y &environment env)
  (if (expands-in-environment-p '(%foo%) env)
      form
      (let ((tmp (gensym)))
        `(macrolet ((%foo% () t))
           (let ((,tmp ,y))
             (add-value-to-history ,tmp)
             (example ,x ,tmp))))))

We can try it out to see if it works as advertised.

CL-USER> (funcall (compiler-macro-function 'example)
                  '(example 1 2)
                  nil)
(MACROLET ((%FOO% NIL T))
  (LET ((#:G177 2)) (ADD-VALUE-TO-HISTORY #:G177) (EXAMPLE 1 #:G177)))
CL-USER> (macrolet ((%foo% () t))
           (macrolet ((check (&environment env)
                        `(quote ,(funcall
                                   (compiler-macro-function 'example)
                                   '(example 1 2)
                                   env))))
             (check)))
(EXAMPLE 1 2)

Note that the second example sets up the environment so that the compiler macro doesn't expand the form. We use the check macro so that we can get an environment object to pass to the compiler macro function.

We're not quite done yet. The next step is to recompile the code that uses example. Remember, compiler macros are only expanded by the compiler, which means simply running code that uses example isn't enough.

(defun foo ()
  (loop
     repeat 10
     for i = (random 100) and j = (random 100)
     collect (example i j)))

Now we'll test to see if everything is working.

CL-USER> (foo)
(10890000 5793649 14561856 7617600 327184 33918976 980100 3418801
 467856 2509056)
CL-USER> *value-history*
NIL
CL-USER> (compile 'foo)
FOO
NIL
NIL
CL-USER> (foo)
(944784 2446096 659344 49702500 0 792100 13924 19377604 15023376 290521)
CL-USER> *value-history*
(77 51 62 59 10 0 94 28 17 12)

Excellent!

It turns out there is an even easier way to prevent recursive expansion of compiler macros. According to the HyperSpec, if a function name is declared as notinline and the call appears within the scope of this declaration, then the compiler macro is not applied. Our compiler macro gets a little simpler.

(define-compiler-macro example (&whole form x y)
  (declare (ignore form))
  (let ((tmp (gensym)))
    `(let ((,tmp ,y))
       (declare (notinline example))
       (add-value-to-history ,tmp)
       (example ,x ,tmp))))

Subtleties

Above I used a call to compile to compile foo, but that only worked because foo wasn't a compiled function yet. If you redefine the compiler macro, you have to recompile everything. Unfortunately, there isn't a way to recompile an already compiled function without re-evaluating its definition. Thus, my presentation of compilation is simplistic. Since I do most of my Lisp coding in an IDE of some form (Slime, Allegro, LispWorks), recompilation isn't usually a big deal. In the future, I may look into a way to use asdf to make it a little easier.

Technorati Tags: lisp, macros