Problems with Scheme, and comments on Common Lisp#

I have a deep love for the Lisp class of languages. It is my favorite by far. My experience started with Interlisp in 1980 and has now included various other dialects including Franz Lisp, Common Lisp, Scheme, and a few others. I used Scheme in a commercial environment for many years. I chose it over Common Lisp at the time in part because of my preference for a Lisp1 (functions and variables share a single namespace) language. After many years of using Scheme, I now regret that decision and feel Common Lisp would have been a better choice. I still strongly prefer a Lisp1 environment but prefer Common Lisp for other reasons as detail in this page.

Contradiction in separation of NIL and #f#

Lisp has long used NIL as "false" or #f. This was very convenient since lisp is a list processing language, traversing through a list is very often done. Having NIL and #f equal means one could use:

    (while  lst   ...
        or
    (if  lst  ...

With the separation of NIL and #f in scheme one is forced into:

    (while  (not (null? lst))   ...
        or
    (if (not (null? lst))   ...

The argument for separating NIL and false was in the name of clarity as opposed to depending on a coincidence. It would be much more clear (to a beginner?) what was going on. (Although, if you really want to be clear, perhaps we should all program in ADA. ADA is a language in which virtually every line of code has to do with clarity and virtually none has to do with solving the problem at hand!)

The problem is that scheme has eliminated, rather arbitrarily, one instance of dependence on a trick but left so many others that it renders the NIL / false separation no more than a constant annoyance. For example, if NIL and #f are kept apart, why allow anything except #t equal true? Yes, I know it is very convenient (just like #f and NIL being equal), but it is also "unclear". Having:

    (if  "abc"  6  7)

return 6 is absolute nonsense. Scheme has chosen clarity and correctness in one instance and convenience in another. It is very inconsistent. If you wanted to be truly consistent:

    (+  "abc"  6)

returns a type error. So:

    (if  "abc"  6  7)

should also return a type error. The same is true for scheme's:

    and
    or
    not
    cond
    etc.

Look, ADA already has the market on clarity and correctness. Lisp and scheme have the elegant, expressiveness, and power market. Why arbitrarily go down the clarity path with the separation of NIL and #f - it is a mistake. The only other "consistent" choice is to separate #t from all else, make all logic operations require #t or #f, and make scheme the ADA of lisp.

Unspecified returns#

Scheme has many functions which have "unspecified" returns. This is somewhat arbitrary, and runs contrary to the whole s-expression style of programming. For example, if you have two variables you want to set to the same value it would be nice to say:

    (set! abc (set! def 55))

like you can in Common Lisp. But in Scheme set! doesn't return a defined result so you are forced into:

    (set! abc 55)
    (set! def abc)

In Scheme you would have to write:

    (set! abc 55)
    (if (> def abc)
        ......

instead of:

    (if (> def (set! abc 55))
        .....

If you think it's no big deal then why the heck does "cond" and "if" return a result? It doesn't in most other languages. The reason "if" returns a result is to take advantage of the nested s-expression style of programming that lends itself to lisp style languages rather than a line by line style of programming like Fortran. If Scheme were consistent then either "if" would return no result or set! would.

Worse yet, there are many function Scheme which return valid responses most of the time and "unspecified" values at other times. You don't know whether you are coming or going.

What do you do if a "case" falls off and returns "unspecified"? Wouldn't nil be a little more predictable?

The same is true for eqv?, eq?, equal?, set-cdr!, set-car!, memq, assq, string-set!, pair?, display, etc......

The bottom line is that describing the return of a function as "unspecified", especially only under certain circumstances, leaves programs which ultimately run in "unspecified" ways. All functions should either return a specified, predictable result or they should signal an error, period.

I would further add that Scheme should define argument evaluation order. Compilers can be written to support a particular order just as well as another order. This is no argument for unspecified argument evaluations order. Having the argument evaluation order unspecified is a huge and totally unnecessary opportunity for porting bugs.

The issues here are consistency, predictability, and convenience.

Macro System#

My complaint here is that the Scheme macro system is so complicated and difficult to use that most of the power of lisp macros is lost. I also contend that it has important limitations.

Scheme's macro system is theoretically pure in some sense related to capturing of variables. But for that advantage you end up with:

1. While the Scheme macro system is simple and easy to use for simple macros it quickly becomes inordinately complicated for more complex macros. I believe that a very large percentage of Common Lisp programmers are able to create and understand moderately complex macros with a lot of ease. On the other hand, I think the number of people in the world who understand complex macros in Scheme are extremely rare. Fully understanding Scheme's macro facility basically requires a Phd.

You can pick up virtually any language, learn a "Hello World" program and then pick up various pieces as you begin to understand the language incrementally. Each particular piece typically takes a couple of minutes to understand. Because of their abstraction, macros might take a bit longer. You could probably get a pretty good working knowledge of lisp macros in a day. Scheme macros, on the other hand, in all except the simplest cases, are unbelievably complex. I doubt you could get a good handle on it in less than a week. Scheme macros are too complex.

If you doubt my "complexity" arguments try taking a look at the portable Scheme macro system available on the net. I haven't seen much that was more complicated than that in my life.

2. I understand why you don't want variable capture - usually. Sometimes you do. Until R6 Scheme had no standard way of capturing variables. It seems like they've added this capability in R6. One problem is that it is unbelievably complicated to use. You really have to understand what is going on under the covers. Common Lisp makes this sort of thing easy and straight forward.

3. I am under the impression that there are levels of complexity in macros that the Scheme macro system flat out can't handle - at least not in a portable way. In my own experience trying to use the Scheme macro system I often and quickly hit a wall in which the complexity of the solution exceeds my understanding. In nearly every case I asked an expert and I almost always get one of two answers. They either tell me to use some implementation specific facility or they tell me it can't be done with it. I've never experienced this with Common Lisp macros.

4. Understand that Lisp macros suffer from variable capture. It's amazing how we've lived with this for 30+ years now. Having to deal with variable capture is a small price to pay for the power, ease of use, and flexibility of Common Lisp macros.

The bottom line is that the Scheme macro system is more of an experiment in complexity than a useful tool.

OO and Modules#

The first is Scheme's lack of an object oriented system (OO). In today's world OO programming is as fundamental to programming as "while" loops are instead of goto statements. OO concepts are probably the biggest and most important addition to computer languages since structured programming (multithreading is next!!). Not having a standard OO facility renders Scheme somewhat archaic.

And, if/when one is added it has got to be powerful - like CLOS. It has got to have multiple inheritance. It has got to have a full MOP.

I started to write a CLOS like OO system in very portable Scheme. I am about 80% done but haven't worked on it in a long time. I get discouraged about adding a good and portable OO system to Scheme because of Scheme's many other issues.

The second issue has to do with a standard module system. I know R6 defines a library system. That is fine. The problem is that virtually none of the Scheme implementations has implemented it and virtually all of the Scheme implementers has publicly stated that they will not support R6RS! Where exactly does that leave us?

Conclusion#

In conclusion, Scheme has, IMO, many important shortcomings. I understand that I don't set the standards and these are just basically my opinions. Although I believe these shortcomings are relatively easy to fix, the real problem is the political element. No matter what you think there is always someone out there who thinks the opposite. The net result is that language development ends up unnecessarily slow.

Common Lisp, again IMO, did a lot of the right things. There are, however, two significant problems with Common Lisp as follows.

One big technical error (IMO) was the LISPn thing. I know a bunch of you guys want to jump on me about this with all kinds of technical BS trying to rationalize your view. The truth of the matter is that the highest and most respected experts on Common Lisp are utterly unclear about the issue. So how does that make you an expert? You are more of a fan club than a technical expert (again read the articles from the authors of Common Lisp). The real truth is that the number one reason Common Lisp is LISPn rather than LISP1 is historical and for purposes of backward compatibility (all of which is hardly applicable anymore). It isn't technically better. The experts were able to point out trade-offs with either design. They even stated that LISP1 encouraged more elegant programming style.

The second mistake of Common Lisp is not requiring support for tail recursion without taking up stack space. This causes the use of inelegant loops when the size of the data is large or unknown in order not to fill the stack. I know some Common Lisps support this, but unless it is part of the standard and something that can be relied upon, it shouldn't be depended on.

It is easy to create a Common Lisp package that merges function and variable name spaces without incurring runtime efficiency (I've done it). Fixing the recursion issue is a significant problem. I'd love to see a new ANSI Common Lisp that just addresses the recursion issue. Changing the LISPn issue would be such a radical change that Common Lisp would be rendered a new language. Using a package to merge the function and variable namespaces is sufficient for those who want it.

Just some opinions.