Wednesday, November 21, 2007

Code walking caveats

As Richard C. Waters has pointed out, it often handy to have a program that can walk Lisp code to be able to analyse it. In my case, I've tried to avoid the need for a code-walker, but the time has come in my work to make use of one. (Compiler macros can only take you so far.)

There are plenty of code walking programs out there, so I'm not overly concerned with implementing it. What I'm more concerned with is idiosyncrasies across implementations.

The first one I've found is in SBCL (1.0.11.22). When you walk a DEFUN form and expand the macros, you get something that fools some code-walkers. For example,

(defun example (x) (+ x x))

eventually expands into

(PROGN
 (EVAL-WHEN (:COMPILE-TOPLEVEL)
   (SB-C:%COMPILER-DEFUN 'EXAMPLE 'NIL T))
 (EVAL-WHEN (:LOAD-TOPLEVEL :EXECUTE)
   (SB-IMPL::%DEFUN
    'EXAMPLE
    (FUNCTION
     (SB-INT:NAMED-LAMBDA EXAMPLE (X) (BLOCK EXAMPLE (+ X X))))
    NIL 'NIL (SB-C:SOURCE-LOCATION))))

The key here is the NAMED-LAMBDA part. Note that it is found within a FUNCTION form. FUNCTION is a special operator which takes either a function name or a lambda expression. The SB-INT:NAMED-LAMBDA form doesn't qualify as either, so I've got myself a classic special case particular to an implementation. (Although it is clearly a special kind of lambda form.)

The reason this is important is that I want to get at the BLOCK part which holds the body of the function. The arnesi walker (the one I've been using) doesn't walk the NAMED-LAMBDA part unless the form is a LAMBDA. On the other hand, the PCL walker does (as far as I can tell), which is what SBCL's is based on. Allegro CL's walker behaves similar to arnesi's.

So it seems that with choosing a code-walker (or writing one), I'll have to make it handle things that don't quite mesh with the HyperSpec.

7 comments:

Pascal Costanza said...

ANSI CL actually allows Common Lisp implementations to add new special forms, and I guess that's what named-lambda is in this example. However, ANSI CL requires new special forms to have corresponding macro definitions, and that should allow you to write a portable code walker that handles such cases.

Geoff Wozniak said...

What I didn't mention was that NAMED-LAMBDA is a macro, but that it expands into itself. It's expansion returns itself wrapped in a FUNCTION form.

I probably should have mentioned that...

Anonymous said...

Why should't (NAMED-LAMBDA ...) be a function name? (setf foo) is.

* #'(setf car)

#<FUNCTION (SETF CAR)>

* #'(SB-INT:NAMED-LAMBDA EXAMPLE (X) (BLOCK EXAMPLE (+ X X)))

#<FUNCTION EXAMPLE {1002729859}>

Geoff Wozniak said...

@anonymous: A function name, as defined by the HyperSpec, is either a symbol or a list of the form (SETF sym).

David Jones said...

I've always regarded this as a bit of a weakness in the spec. Macro operators are required to have expansions (even if implemented as special forms) so that expanders can operate on them, but the expansion is allowed to be non-portable!

What we need is for implementations to provide an expansion that is portable, at least in the cases where that's easy / sensible.

Anonymous said...

Sure, it's non-standard, but it's still a function name. (It's obviously not a special operator.)

Read the glossary entry for function block name:

"An implementation which supports additional kinds of function names [...]"

FWIW, Allegro also has non-standard function names.

http://www.franz.com/support/documentation/current/doc/operators/excl/function-name-p.htm

Geoff Wozniak said...

@anonymous: This is true, but it doesn't really help me. :)

NAMED-LAMBDA is treated as a special form by the SBCL code walker, but is not reported as one by SPECIAL-OPERATOR-P. When you walk the example code with SBCL's code-walker, it does not expand the NAMED-LAMBDA macro. (See the code in src/pcl/walk.lisp in the SBCL source tree. Look for the DEFINE-WALKER-TEMPLATE form that defines it, or look for WALK-NAMED-LAMBDA.)

Also, the NAMED-LAMBDA form is not reported as a legal function name by SB-INT:LEGAL-FUN-NAME-P.