I’m not really going to talk about the problem with Lisp, but only a problem. And I’m not really going to talk about Lisp, but Common Lisp. So this post should be called “A problem with Common Lisp”, but the current title is more catchy, isn’t it?
You are still with me? nice. Because I believe I have something important to say. A problem with Common Lisp is that you can’t make programs with it. That may have been another nice catchy title: “You can’t make programs with Lisp”.
You can make libraries just fine, and package them, and distribute them, and re-use them, but not programs. To give meaning to these statements I’ll have to define what a program is. A program is something that you can run from a shell (like bash, csh, tcsh, etc) by typing its name and pressing enter . Or, if you want another definition, something that you can run from a C program using the function exec.
I’ve asked this question several times on #lisp on Freenode and generally the answer is this: SBCL can generate an runable image. Although it may be nice in some cases: like shipping an alpha version of a program to someone who doesn’t have a clue what Lisp is, it is not a good way to ship a program expecting an OS like Debian or Ubuntu to include them: “25MB for a hello-world? you are crazy”. The size is not the only problem, upgrading the libraries get harder, because you have to upgrade the libraries in the image and not only the modules in the file system that the OS can easily upgrade with tools like aptitude or rpm. They are just monolithic. I can imagine ls and cp being written in Lisp, so I can’t imagine them being huge monolithic images. Any other language that I know doesn’t have this problem, be it Python, C, C++, Java, Objective-C, C#, Haskell, Chicken Scheme, Erlang (this one has its problems but I’ve already solved some of them), Perl, PHP, any other?
Other Lispers claim that you can just ship a library and the user would just run a function on it; like (run-checkers) to play checkers. This has various drawbacks. First, not every user is a Lisper. Most computer users don’t even know what Lisp, so they wouldn’t know how to run a lisp. They also wouldn’t be able to have an icon in their desktops to run the program, they’d have to open a terminal. For people that can do all that, finding the right function to run in one of those so-called programs can be a challenge. My wife wouldn’t have even tried to run any of those programs. Yes, I want to write software for my wife, in Lisp. And last, even when they can do all that, they may still search for an alternative that just puts an icon somewhere and they click it.
Yes, long rant. I hope I could convince you of the need to produce programs. It has been said many times that Lisp needs a killer application, like ejabberd for Erlang or darcs for Haskell. Both awesome applications that get people to approach the language (indeed, what made me learn Erlang was ejabberd). Without being able to make programs, you can’t make killer programs.
After pushing this issue a bit on comp.lang.lisp I’ve got pointed to cl-launch by Alex Mizrahi (I actually had to say that I was going to do something about it for someone to say “don’t re-invent the wheel”). I knew there had to be something out there! I couldn’t believe I was the first experiencing this. There’s hope!.
Unfortunately, not a single cl-launch powered Lisp program seems to be out there. Maybe because there’s no build-tool integration for it anywhere. Let’s see if I can do something about it.
Update (2007-09-05): Two possible solutions I posted: Solving Lisp’s problem: a simplistic solution with make, Another simplistic solution, with SCons.
Comments at the original blog
- Pupeno Says:
Just when I’m about to publish this document, John Q. Splittist covers the same topic on: http://blog.splittist.com/2007/08/25/lisp-faq-how-can-i-generate-a-standalone-executable/We have different points of views, which makes it really interesting.
Note: my article was not inspired by John Q. Splittist’s, I’ve been working on this article for a couple of weeks while I’ve only read John’s today.
-
Pupeno Says:
I’ve submitted this article to Reddit: http://programming.reddit.com/info/2ivn2/comments and it seems most Redditers took it as not wanting to ship or depend on a VM or environment.
That’s not what I’m saying. What I’m saying is that shipping a monolithic binary, with VM and libraries that doesn’t use the system shared VM and libraries is a problem.
Also shipping a program that depends on one and only one VM in a world of multiple VMs is a problem.
And last but not least, I don’t express this as “let’s not use Lisp” but as “let’s fix it” and a “it won’t be hard to fix”. -
Pupeno Says:
One simplistic solution: http://pupeno.com/2007/08/26/solving-lisps-problem-a-simplistic-solution-with-make/
-
Vitor Says:
“Also shipping a program that depends on one and only one VM in a world of multiple VMs is a problem.”
What problems are you REALLY running into?
You keep making these apples to oranges comparisons with the other languages. They have your odd “VM issue” as well:
see Python/Jython/Stackless/IronPython or Ruby/JRuby/Ruby.Net or Java/Beanshell/Groovy/Scala. The people that use these languages generally favor and deploy on one VM of the choices they have–why should it be different for Lisp?People have made standalone applications with Lisp with little trouble. It takes very little effort to Google the issue and find several answers.
Feel free to delete this comment like you have deleted others, but realize that you are creating problems that don’t exist.
-
Anonymous Says:
“Every other language that I know doesn’t have this problem, be it Python, C, C++, Java, Objective-C, C#, Haskell, Chicken Scheme, Erlang (this one has its problems but I’ve already solved some of them), Perl, PHP, any other?”
Smalltalk
-
Pupeno Says:
Vitor,
The difference is that in many of those cases there’s actually only one VM/environment. That’s why I believe it is better to just implement a language, like in Python’s case, than have a standard and many implementations. In the case of Java and .Net, the way to build programs is quite easy, due to “java” standard command line flags or .Net standard way to build programs (you can compile with mono and run on the other one and viceversa).
Most Lisp standalone applications suffer of some problem or another, I have already looked it most free ones I could find and disregard the way they do it because there has to be and there is a better way. -
Pupeno Says:
About Smalltalk, I’d say that actually, due to its heavy use of the “image” concept, it suffers from the same, or worst problem than Lisp itself.
You may be able to write a program to GNU Smalltalk and ship it sanely, but GNU Smalltalk never ends up looking like a nice implementation for me (and obviously, don’t even dream about cross-implementation programs, but that was never intended, unlike with Lisp). -
Juanjo Says:
Si podés hacer librerías en Common Lisp.. pero no programas.. en que programas usas esas librerías?
Saludos!!
-
Pupeno Says:
Juanjo: hasta ahora, yo diria que en ninguno; en Common Lisp solo hay bibliotecas, no programas.
-
The Regex Coach Says:
I had no problems when I installed The Regex Coach http://weitz.de/regex-coach/#install
-
Pupeno Says:
About The Regex Coach, that is proprietary software, it probably is a monolithic thing that includes its own Lisp and its own set of every library. Now imagine you have 100 of those programs installed and there’s a bug in a library. You’ll have to upgrade 100 programs with 100 copies of Lisp and of every library instead of just upgrading a library.
In the free world we like re-usability, so we separate libraries which can easily be upgraded by tools like apt.
Please, read carefully what I wrote, I already acknowledge the monolithic way of doing things which is actually easy with SBCL, and I already explain why it’s not viable in a OS like Ubuntu or Debian. Please, please read and understand that, I’ve answered this question/complain/issue a hundred times already in different places. -
Nathan Says:
And so we wind up with the current situation, which is that you have the Lisp implementation and lots of separate loadable text files…just like Python, Perl, Ruby, etc. etc. Problem solved. What’s wrong with this again?
-
Pupeno Says:
Nathan,
In Python, Perl or Ruby, you just put a she-bang at the beginning, like #!/usr/bin/env python, and you have a Python program. That’s not possible in Common Lisp. It might be possible in some implementations, in some cases needing customizations that are not widely available.
If you need further proof of this, check any distribution and you’ll find thousands of programs that use this method made in Python, Perl, Ruby and other interpreted languages. You’ll find absolutely none (well, maybe one) written in Common Lisp. -
ken Says:
This is a nonproblem. Or at least, it’s no worse than any of the other languages you’ve mentioned.
The Python programs on my system (not written by me) are run via a tiny shell script, which starts the Python VM pointed at some source code.
I can do exactly the same with Lisp: put some source code in a folder, write a shell script, and point it at them. Presto, a tiny ‘ls’.
Now, in addition to that, I have other options: I can compile my Lisp code beforehand, I can distribute an image, I can use SBCL’s shebang feature to run a Lisp file directly, and so on. But I can do what all my Python apps do, too.
For people shipping big serious apps on Windows and Mac, a 25MB app doesn’t look so scary. It *really* doesn’t look scary to the programmer because it means he doesn’t have to implement that stuff himself! 25MB hello world, 26MB Excel.
Yeah, it might be nice to have a Lisp with compilation facilities more like C. Would I ever need them? Probably not. If you want to help Lisp, go ahead and try to solve this, but don’t blow it out of proportion. We’ve got much bigger fish to fry.
-
Pupeno Says:
Ken,
Here I go again repeating myself. With Python you can put “#!/usr/bin/env python” in top of the source file and you already have a program. You can’t do the same in Common Lisp. For two reason, one is syntactically, and can be solved easily, the other is that Common Lisp is a standard and there are many implementations and you don’t know which implementation you are going to find on a system.
I’ve said, from the very beginning, that I don’t care about Windows and Mac. In those platforms they have huge programs because they don’t ever share anything. In pretty much the rest of the world we have thousands of small libraries that are independent of each other. So if there’s a bug in one, you only update that one. If we have 100 applications all being 25MB applications, first we need 2500MB just to store, but if there’s a bug in a 2KB library somewhere in there, an aptitude update will have to download 2.5GB instead of 2KB (not counting the huge amount of work that is needed by the packagers to recompile all that). This doesn’t happen in Python or Perl because your program is only your program and the rest is on the system.Now, most important, the problem has been solved 90%, so my blog post is more about pointing where the solution is. I posted two other blog posts walking the other 10% (OK, you could say I walked 9%). It was easy, simple and guess what. Nobody cared! Everyone comes here like you, complaining that I complain about a problem without even noticing that I complain about a problem and I immediately point the to the solution. And nobody is using or caring about the existence of a solution: cl-launch. I think that’s because most lispers live in a live of Lisp and they forget about the users, they only care about libraries and not programs. Pick up any Linux distribution and compare the amount of libraries vs the amount of applications and you’ll see it doesn’t make any sense. Specially if you compare it with other languages, even those with ugly syntax or imposing standards or very unpopular (more unpopular than Lisp, that is, with smaller communities) and you’ll see less libraries, but more applications (Haskell, OCaml, Erlang, etc).
-
TheGZeus Says:
…couldn’t you just set your favourite Lisp as a dependency to the package?
Non problem as far as I can see.Just keep the code as something.lisp and have a shell script pointing to clisp, cmucl, lush, whatever. I prefer clisp, mainly because it works nicely with SLIME.
You could install all the implementations in the Ubuntu repositories and not feel a twinge on today’s hard drives.
They’re not THAT big, and it’s not nearly as bad as the first time a KDE user installs a GNOME program!Why package it into a binary if it’s going to be problematic (size or something else) unless you’re not giving up the code?
