Why I love Lisp

This post was extracted from a small talk I gave at Simplificator, where I work, titled “Why I love Smalltalk and Lisp”. There’s another post titled “Why I love Smalltalk” published before this one.

Desert by Guilherme Jófili

Lisp is an old language. Very old. Today there are many Lisps and no single language is called Lisp today. Actually, there are as many Lisps as Lisp programmers. That’s because you become a Lisp programmer when you go alone in the desert and write an interpreter for your flavor of lisp with a stick on the sand.

There are two main Lisps these days: Common Lisp and Scheme, both standards with many implementations. The various Common Lisps are more or less the same, the various Schemes are the same at the basic level but then they differ, sometimes quite significantly. They are both interesting but I personally failed to make a practical use of any of those. Both bother me in different ways, and of all the other Lisps, my favorite is Clojure. I’m not going to dig into that, it’s a personal matter and it’ll take me a long time.

Clojure, like any other Lisp, has a REPL (Read Eval Print Loop) where we can write code and get it to run immediately. For example:

;=> 5

"Hello world"
;=> "Hello world"

Normally you get a prompt, like user>, but here I’m using the joyful Clojure example code convention. You can give this REPL thing a try and run any code from this post in Try Clojure.

We can call a function like this:

(println "Hello World")
; Hello World
;=> nil

It printed “Hello World” and returned nil. I know the parenthesis look misplaced but there’s a reason for that and you’ll notice it’s not that different from Javaish snippet:

println("Hello World")

except that Clojure uses the parenthesis in that way for all operations:

(+ 1 2)
;=> 3

In Clojure we also have vectors:

[1 2 3 4]
;=> [1 2 3 4]


;=> symbol

The reason for the quote is that symbols are treated as variables. Without the quote, Clojure would try to find its value. Same for lists:

'(li st)
;=> (li st)

and nested lists

'(l (i s) t)
;=> (l (i s) t)

Here’s how defining a variable and using it looks like

(def hello-world "Hello world")
;=> #'user/hello-world

;=> "Hello world"

I’m going very fast, skipping lots of details and maybe some things are not totally correct. Bear with me, I want to get to the good stuff.

In Clojure you create functions like this:

(fn [n] (* n 2))
;=> #<user$eval1$fn__2 user$eval1$fn__2@175bc6c8>

That ugly long thing is how a compiled function is printed out. Don’t worry, it’s not something you see often. That’s a function, as created by the operator fn, of one argument, called n, that multiplies the argument by two and returns the result. In Clojure as in all Lisps, the value of the last expression of a function is returned.

If you look at how a function is called:

(println "Hello World")

you’ll notice the pattern is, open parens, function, arguments, close parens. Or saying it in another way, a list where the first item is the operator and the rest are the arguments.

Let’s call that function:

((fn [n] (* n 2)) 10)
;=> 20

What I’m doing there is defining an anonymous function and applying it immediately. Let’s give that function a name:

(def twice (fn [n] (* n 2)))
;=> #'user/twice

and then we can apply it by name:

(twice 32)
;=> 64

As you can see, functions are stored in variables like any other piece of data. Since that’s something that’s done very often, there’s a shortcut:

(defn twice [n] (* 2 n))
;=> #'user/twice

(twice 32)
;=> 64

Let’s make the function have a maximum of 100 by using an if:

(defn twice [n] (if (> n 50) 100 (* n 2))))

The if operator has three arguments, the predicate, the expresion to evaluate when the predicate is true and the one when it’s false. Maybe like this it’s easier to read:

(defn twice [n]
  (if (> n 50)
      (* n 2)))

Enough basic stuff, let’s move to the fun stuff.

Let’s say you want to write Lisp backwards. The operator at the last position, like this:

(4 5 +)

Let’s call this language Psil (that’s Lisp backwards… I’m so smart). Obviously if you just try to run that it won’t work:

(4 5 +)
;=> java.lang.ClassCastException: java.lang.Integer cannot be cast to clojure.lang.IFn (NO_SOURCE_FILE:0)

That’s Clojure telling you that 4 is not a function (an object implementing the interface clojure.lang.IFn).

It’s easy enough to write a function that converts from Psil to Lisp:

(defn psil [exp]
  (reverse exp))

The problem is that when I try to use it, like this:

(psil (4 5 +))
;=> java.lang.ClassCastException: java.lang.Integer cannot be cast to clojure.lang.IFn (NO_SOURCE_FILE:0)

I obviously get an error, because before psil is called, Clojure tries to evaluate the argument, that is, (4 5 +) and that fails. We can call it explicitly turning the argument into a list, like this:

(psil '(4 5 +))
;=> (+ 5 4)

but that didn’t evaluate it, it just reversed it. Evaluating it is not that hard though:

(eval (psil '(4 5 +)))
;=> 9

You can start to see the power of Lisp. The fact that the code is just a bunch of nested lists allows you to easily generate running programs out of pieces of data.

If you don’t see it, just try doing it in your favorite language. Start with an array containing two numbers and a plus and end up with the result of adding them. You probably end up concatenating strings or doing other nasty stuff.

This way of programming is so common on Lisp that it was abstracted away in a reusable thing call macros. Macros are functions that receive the unevaluated arguments and the result is then evaluated as Lisp.

Let’s turn psil into a macro:

(defmacro psil [exp]
  (reverse exp))

The only difference is that I’m now calling defmacro instead of defn. This is quite remarkable:

(psil (4 5 +))
;=> 9

Note how the argument is not valid Clojure yet I didn’t get any error. That’s because it’s not evaluated until psil processes it. The psil macro is getting the argument as data. When you hear people say that in Lisp code is data, this is what they are talking about. It’s data you can manipulate to generate other programs. This is what allows you to invent your own programming language on top of Lisp and have any semantics you need.

There’s an operator on Clojure called macroexpand which makes a macro skip the evaluation part so you can see what’s the code that’s going to be evaluated:

(macroexpand '(psil (4 5 +)))
;=> (+ 5 4)

You can think of a macro as a function that runs at compile time. The truth is, in Lisp, compile time and run time are all mixed and you are constantly switching between the two. We can make our psil macro very verbose to see what’s going on, but before, I have to show you do.

do is a very simple operator, it takes a list of expressions and runs them one after the other but they are all grouped into one single expression that you can pass around, for example:

(do (println "Hello") (println "world"))
; Hello
; world
;=> nil

With do, we can make the macro return more than one expression and to make it verbose:

(defmacro psil [exp]
  (println "compile time")
  `(do (println "run time")
       ~(reverse exp)))

That new macro prints “compile time” and returns a do that prints
“run time” and runs exp backwards. The back-tick, ` is like the quote ' except that allows you to unquote inside it by using the tilde, ~. Don’t worry if you don’t understand that yet, let’s just run it:

(psil (4 5 +))
; compile time
; run time
;=> 9

As expected, compile time happens before runtime. If we use macroexpand things will get more clear:

(macroexpand '(psil (4 5 +)))
; compile time
;=> (do (clojure.core/println "run time") (+ 5 4))

You can see that the compile phase already happened and we got an expression that will print “run time” and then evaluate (+ 5 4). It also expanded println into its full form, clojure.core/println, but you can ignore that. When that code is evaluated at run time.

The result of the macro is essentially:

(do (println "run time")
    (+ 5 4))

and in the macro it was written like this:

`(do (println "run time")
     ~(reverse exp))

The back-tick essentially created a kind of template where the tilde marked parts for evaluating ((reverse exp)) while the rest was left at is.

There are even more surprises behind macros, but for now, it’s enough hocus pocus.

The power of this technique may not be totally apparent yet. Following my Why I love Smalltalk post, let’s imagine that Clojure didn’t come with an if, only cond. It’s not the best example in this case, but it’s simple enough.

cond is like a switch or case in other languages:

(cond (= x 0) "It's zero"
      (= x 1) "It's one"
      :else "It's something else")

Around cond we can create a function my-if straightforward enough:

(defn my-if [predicate if-true if-false]
  (cond predicate if-true
        :else if-false))

and at first it seems to work:

(my-if (= 0 0) "equals" "not-equals")
;=> "equals"
(my-if (= 0 1) "equals" "not-equals")
;=> "not-equals"

but there’s a problem. Can you spot it? my-if is evaluating all its arguments, so if we do something like this, the result is not as expected:

(my-if (= 0 0) (println "equals") (println "not-equals"))
; equals
; not-equals
;=> nil

Converting my-if into a macro:

(defmacro my-if [predicate if-true if-false]
  `(cond ~predicate ~if-true
         :else ~if-false))

solves the problem:

(my-if (= 0 0) (println "equals") (println "not-equals"))
; equals
;=> nil

This is just a glimpse into the power of macros. One very interesting case was when object oriented programming was invented (Lisp is older than that) and Lisp programmers wanted to use it.

C programmers had to invent new languages, C++ and Objective C, with their compilers. Lisp programmers created a bunch of macros, like defclass, defmethod, etc. All thanks to macros. Revolutions, in Lisp, tend to just be evolutions.

Thanks to Gonzalo Fernández, Alessandro Di Maria, Vladimir Filipović for reading drafts of this.

Croation translation on http://science.webhostinggeeks.com/zasto-volim-lisp

The joyful Clojure example code convention

I started reading The Joy of Clojure, seems like a great book and I like its example code convention. I’m documenting it here because I’ll adopt it on my blog and I want to be able to link to it. I’ll call it the joyful Clojure example code convention.

A simple piece of Clojure code looks like this:

(* 2 10)

If you run it on the REPL it looks like this:

user> (* 2 10)

The first example, it’s just the code, with no return value, the second one, shows the return value, but when you have several lines it becomes cumbersome to copy and paste:

user> (* 2 10)
user> (* 2 11)
user> (* 2 12)

The Joy of Clojure code convention solves both problems by removing the prompt and leaving return values in comments:

(* 2 10)
;=> 20
(* 2 11)
;=> 22
(* 2 12)
;=> 24

Now you can copy and paste and still have the results.

If the snippet prints something, it will also be displayed but without the arrow, like this:

(println "Hello world")
; Hello world
;=> nil

Why I love Smalltalk

Smalltalk logo

This post was extracted from a small talk I gave at Simplificator, where I work, titled “Why I love Smalltalk and Lisp”. There should be another post, “Why I love Lisp” following this one.

After I learned my basic coding skill in more or less traditional languages, like C, C++, Python, there were four languages that really taught me something new. Those languages changed my way of thinking and even if I never use them, they were worth learning. They are:

  • Smalltalk
  • Lisp
  • Erlang
  • Haskell

You can probably add Prolog to that list, but I never learned Prolog. This post is about Smalltalk.

My goal is not to teach Smalltalk but to show things that you can do with Smalltalk that you can’t do with any other language (disclaimer: surely other languages can do it, and we’ll call them Smalltalk dialects). Nevertheless I need to show you some basics of the language to be able to show you the good stuff, so here we go, a first program:

1 + 1

That of course, evaluates to 2. If we want to store it in a variable:

m := 1 + 1

Statements are finished by a period, like this:

m := 1.
m := m + 1

In Squeak, a Smalltalk implementation, there’s an object called Transcript and you can send messages to it to be displayed on the screen. It’s more or less like a log window. It works like this:

Transcript show: 'Hello world'

and it looks like this:

Squeak transcript showing the result of Transcript show: 'Hello World'

The syntax is quite unique to Smalltalk. The message, otherwise known as “method call” in other languages, is called show: (including the colon) and it takes an argument. We can run it 10 times in a row with the following snippet:

10 timesRepeat: [
  Transcript show: 'Hello world'

There you can start to see how Smalltalk is special. I’m sending the message timesRepeat: to the object 10, an Integer. Doing something N times repeatedly is handled by the Integer class, which if you think about it, makes sense.

The second interesting part, is the block. The part inside squared brackets. You might thing that’s the equivalent of other language’s block syntax, like in this Java example:

for(int i=1; i<11; i++) {
  System.out.println("Hello world");

but Smalltalk version’s is a bit more powerful. It’s a real closure. Look at this:

t := [
  Transcript show: 'Hello world'

Now I have a variable named t, of type BlockClosure, and I can do anything I want with that variable. If I send it the class message it’ll return its class:

t class

and if I sed it the value message, it’ll execute and leave a “Hello World” in the transcript:

t value

Let’s see some more code. A message without any arguments:

10 printString

a message with one argument:

10 printStringBase: 2

and a message with two arguments:

10 printStringBase: 2 nDigits: 10

Isn’t it cute? That method is called printStringBase:nDigits:. I never seen that syntax anywhere else; well, except in Objective-C, which copied it from Smalltalk.

Enough toying around, let’s start building serious stuff. Let’s create a class:

Object subclass: #MyClass
       instanceVariableNames: ''
       classVariableNames: ''
       poolDictionaries: ''
       category: 'Pupeno'

Notice that a class is created by sending a message to another class telling it to subclass itself with the name and a few other arguments. It’s a message, a method call like any other. Object is a class, classes are objects. The object model of Smalltalk is a beauty but that’s a subject for another post.

Now that we have a class, let’s create a method called greet: in that class.

greet: name
  "Greets the user named name"

  | message |

  message := 'Hello ', name.
  Transcript show: message.

In that method definition first we have a comment for the method, then the list of local variables within pipes (“|”), and then the implementation, which sets the variable message to contain “Hello ” and the comma concatenates name to it. Then we just send it to the transcript.

It looks like this:

MyClass greet method

Ok, let’s use it:

m := MyClass new.
m greet: 'Pupeno'

To create an object of class MyClass, we send the new message to that class. There’s no new keyword like in Java. new is just a method. You can read its code, override it, etc. Don’t mess with it unless you really know what you are doing.

Actually, if you think about it, we haven’t seen a single keyword. Look all the code we wrote without having to memorize any keywords! What’s even more important is that by now you essentially know Smalltalk. That’s all there is, but like LEGO bricks, this simple and small building blocks allow you to build whatever you want.

Yes, that’s it, that’s all there is to it. We already saw that Smalltalk doesn’t need loops, it has integers and that class implements the timesRepeat: message which allows you to do something N times. There are many other looping methods here and there.

What about the if keyword you ask? Surely Smalltalk has an if? Well, no, it doesn’t. What you can recognize as an if is actually implemented in Smalltalk using the same mechanism of classes and message passing you saw already. Just for fun let’s re-implement it.

We starte by creating the class PBoolean and then two classes inheriting from it, PTrue and PFalse.

Object subclass: #PBoolean
       instanceVariableNames: ''
       classVariableNames: ''
       poolDictionaries: ''
       category: 'Pupeno'

PBoolean subclass: #PTrue
       instanceVariableNames: ''
       classVariableNames: ''
       poolDictionaries: ''
       category: 'Pupeno'

PBoolean subclass: #PFalse
       instanceVariableNames: ''
       classVariableNames: ''
       poolDictionaries: ''
       category: 'Pupeno'

For the class we created before, MyClass, we define a equals: method that will return either true or false, or rather, PTrue or PFalse.

equals: other
  ^ PTrue new

The little hat, ^, means return. For now, just a hardcoded true. Now we can do this in the workspace:

m1 := MyClass new.
m2 := MyClass new.
m1 equals: m2

and get true, that is PTrue, as a result. We are getting close but no if yet. How should if look like? It’ll look like something like this:

m1 := MyClass new.
m2 := MyClass new.
(m1 equals: m2) ifTrue: [
  Transcript show: 'They are equal'; cr
] else: [
  Transcript show: 'They are false'; cr

and you can start to imagine how to implement it. In PTrue we add the method:

ifTrue: do else: notdo
  ^ do value

That method basically takes two parameters, evaluates the first one and ignores the second one. For PFalse we create the oposite:

ifTrue: notdo else: do
  ^ do value

and that’s it. A working if! If you ask me, I think this is truly amazing. And if you check Squeak itself, you’ll find the if is actually implemented this way:

True's ifTrue:ifFalse:

If your programming language allows you to create something as basic as the if conditional, then it allows you to create anything you want.

Getting rid of RubyGems deprecation warnings

A recent update to RubyGems is causing a lot of deprecation warnings like these:

NOTE: Gem::Specification#default_executable= is deprecated with no replacement. It will be removed on or after 2011-10-01.
Gem::Specification#default_executable= called from /usr/lib/ruby/gems/1.8/specifications/rubygems-update-1.4.1.gemspec:11.
NOTE: Gem::Specification#default_executable= is deprecated with no replacement. It will be removed on or after 2011-10-01.
Gem::Specification#default_executable= called from /usr/lib/ruby/gems/1.8/specifications/bundler-1.0.7.gemspec:10.
NOTE: Gem::Specification#default_executable= is deprecated with no replacement. It will be removed on or after 2011-10-01.
Gem::Specification#default_executable= called from /usr/lib/ruby/gems/1.8/specifications/file-tail-1.0.5.gemspec:10.

I generally like software to move forward and the way to do that is deprecate and then after a while, make backwards incompatible changes. It’s painful but there’s no other way.

I do have a problem with all the cron jobs of my web apps like Keep on Posting or DNSk9 flooding my inbox with those warnings. Thankfully, that’s not hard to fix. Where I was doing:

rake pre_calculate_thingies > /dev/null

now I’ll be doing:

rake pre_calculate_thingies 2>&1 >/dev/null | grep -v default_executable

Metaprogramming Ruby

There are thousands of books that will take you from illiterate to novice in any programming language. But finding those that will take you from novice or intermediate to expert is hard. I remember reading Effective Java some years ago and wishing I had something like that for Python. I’ve never found one.

Metaprogramming Ruby is a great book full of very interesting knowledge, full of those things that separate a Ruby programmer and an export Ruby programmer. Before finishing the book I’ve already put to use some of the lessons and it saved me a lot of time. The book payed for itself before I’ve finished reading and I really recommend it to anyone who is serious about coding in Ruby.

Are dynamic languages just a temporary workaround?

This can unleash so much hate mail, but here it goes, my inbox is ready!

Are dynamic languages just a temporary workaround? I’m not sure! I’m switching between the two types of languages all the time: Java, Python, C#, JavaScript. I’ll try to make the long story short.

Statically typed languages, like Java and C#, are nice because when you do


you know that blah’s class has a bleh method, at compile time. But better than that, when you typed “blah.” you get a list of methods, and you already know whether there’s a bleh method or not, and if you typed bleh and it doesn’t exist, the IDE lets you know, no need to wait for the compiler. Also you can do very deterministic refactoring, renaming all “bleh” for “bluh” for example.

Statically typed languages are not nice because they are very verbose and require a lot of boilerplate (if you’ve used Haskell, just bear with me for now please), so you end up with things like:

List[Car] cars = new List[Cars]();
foreach (Car car in cars) {

How many “cars” do you read there? And that’s a nice example. There are worse. So come dynamically typed languages and you can write:

cars = []
for car in cars:

You have less cars, and less (no) lists. That means you are more productive. You start chunking out code faster without having to stop and think “What type of object will this or that method return?”. But crash() can crash your application instead of just the car because you can’t know if it exists until run-time. That might be OK or not, testing and whatnot.

Then comes C# 3.0 where you can do:

var cars = new List[Car]();
foreach (var car in cars) {

And you can see that syntactically it got closer to Python, which is what gives you the productivity. Don’t know the type? type “var”. But semantically, it’s still statically typed, like the previous statically typed example. You know that car is going to be of some class that has a crash method. You can actually know car’s class at compile time, no need to run it.

That’s called type inference. You don’t have to specify the type where the compiler is capable of inferring it for you. C# type inference system is still very limited (but better than Java’s). Let’s see an example in another language

cars = []
map crash cars

That means, create a list called cars, call the function crash on each car. Would you say that that is a statically typed language? or a dynamic one? I’d say it is dynamic, but it is static. Very static. It’s Haskell. Haskell’s compiler will infer all the types for you. It’s amazing, you’ll write code as robust as with C#, but as terse as Python’s (Monads will then kill your productivity, but that’s another story).

In Python 3 you can define types for arguments. They are mostly useless, but it’s an interesting direction. I think the best it can do is that when a program crashes it’ll tell you: “function blah expected an int, but got a float, not sure if that was the problem, but you might want to look into that”.

Now, my question is, are dynamically typed languages just a temporary workaround? As our compilers get better, our computers faster, will statically typed languages keep giving us as many or more reassurances about our code and utilities while at the same time they become as simple and terse as dynamically typed languages? Or will dynamically typed languages start to gain types and over time be more static without the programmers that use them ever noticing?

My question is, will we in the future, 50 or 100 years, look back and said “Dynamically typed languages where a temporary workaround to statically languages being painful to use when human beings and their toy computers were so primitive?” in the same way we can say today that “non-lexical scope was a limitation we had and have due to the limitations of computer hardware 30 years ago”.

Reviewed by Daniel Magliola. Thank you!

Formating strings in C#, like in Python

I like Python’s way to format strings because you can use it everywhere, it’s part of the strings. You can do

print("Welcome %s!" % user.name)

as you can do

Console.Writeln("Welcome {0}!", user.Name)

But then in Python you can also do

randomMethodThatTakesAString("Welcome %s!" % user.name)

In C# it gets trickier, because the call to Console.Writeln takes extra arguments for the string arguments, while RandomMethodThatTakesAString doesn’t. It just takes a string. So the only way to go is

RandomMethodThatTakesAString(String.Format("Welcome {0}!", user.Name))

which is ugly.

Thanfully C# 3.0 has extension methods, so I quickly wrote this method:

blah blah

and now I can write:

RandomMethodThatTakesAString("Welcome {0}".Args(user.Name))

which is more verbose that Python’s version, but equally nice in my eyes.

If you can understand why allowing the language to be extendable in one aspect was a win here, then you can understand why so many, me included, love Lisp that is extendable in every possible way.

Reviewed by Daniel Magliola. Thank you!

Update 2009-09-17: For those complaining that I didn’t show how I did it, here is the Args method:

public static class StringExtensions {
    public static string Args(this string str, params object[] args) {
        return String.Format(str, args);