Related
I have started experimenting with Clojure, and I am impressed by what it can do. However, I don't think I have really 'got' functional programming design. My background is in Smalltalk, and that colours how I think about design. I would like to really understand how FP differs from OO, and one approach would be to have a good side-by-side comparison of a solution.
I don't think the problem needs to be that complex. Maybe it is would be enough to show how FP would implement the typical bank account problem?
Does anyone know of a suitable resource?
I think that most books on Clojure will have something like a simple bank account program, which I view as a little database in which states get updated. For example, Halloway and Bedra's Programming Clojure has a program that keeps track of music recordings. When you read through a description of that program, you'll immediately see how to map it into an OO solution. However, I don't think you're going to "get" FP from such a simple example. Programming Clojure also has functional snake game. You could compare that to an OO snake game. (I'm not promoting Programming Clojure; it's a worthwhile book but I think it has a number of flaws. It just happens to be the Clojure book that I know best. A lot of people seem to like it, though.)
I'd suggest another strategy, maybe as a supplement to yours, for grokking FP: Read something introductory that immerses you in an FP worldview. My favorite is Friedman and Felleisen's The Little Schemer. It uses Scheme, not Clojure, but the ideas translate. There is a Clojure website devoted to it, but I recommend working through the original book in the way that it recommends, covering parts of the page as you go. I think the book that gave me an understanding of parts of FP that go beyond The Little Schemer, extending to much of what you find in the Clojure world, was Bird's Introduction to Functional Programming using Haskell. I feel funny suggesting a Haskell book, though. Why not a Clojure book? Well, maybe one of the Clojure books will have the same effect of giving you a pure baptism in the FP Way of Thinking, but the Clojure books I've read (P.C, The Joy of Clojure), though very good books, are somehow less immersive--maybe because they're more practical. On the other hand, P.C (and probably some other Clojure books) is (are) largely targeted toward OO programmers, so that might actually be what you want.
A few random observations on Clojure from a Smalltalker's perspective.
The Clojure data structures will remind you of Smalltalk's collection classes, but immutability/persistence takes a bit of getting used to.
You already use do and its relatives to abbreviate control structure. You'll find that the sequence library (map, reduce, and the rest), with laziness offer you another level of power and concision.
You're already used to writing your code in little pieces. The pieces will get smaller and there will be more of them.
But
Clojure is less pure than Smalltalk: not everything is an object.
Nothing like the wonderful Smalltalk browser is standard.
If you like Smalltalk, you'll love Clojure.
Now I'm generally in Java/C# (love both of them, can't really say I'm dedicated to one).
And I've recently been discussing the differences between F# and C# with a friend, when he surprised me saying: "So.. F# sounds a lot like lisp, but with way less 'Swiss-army knife' feel to it."
Now, I was partly ashamed of saying this but I have no idea what lisp was.
After some searching, I saw that lisp is very interesting, but got stumped by the multiple dialects and running environments.
Here is what I know:
I know of 3 dialects:
Common Lisp (I have the Practical Common Lisp book in my bookmarks.
Scheme (a more "theoretical" version of CL)
Clojure. Seems to be a version of CL that runs on JVM.
The basic idea of lisp seems to be about using code as data.
What I want to know:
What is the running environment for different dialects? How do they work/get installed (by this I mean is it a runtime like Java Virtual Machine, or if it requires something else, or if it's supported generally by the OS (as in compiled)). And how to get them (if something is to be gotten)
What is the better dialect to learn (I want the dialect not to be a "learning language" but one you can fully use afterwards without regret of not learning some other one, for example one should first learn C++ before trying out Visual C++, if you know what I mean)
What are the main advantages of lisp in general (I've seen many pages about that saying it's faster in development and execution, but they were all pretty vague about the details)
Can it be generally used for general purpose, or is it concentrated on AI? (By this I mean if, for example, one could make a full console app with it, and then implement OpenGL just as easily and make a game. Learning a language specialized on something precise is worthwhile, but not at the moment for me)
I would also be very happy about any additional details you guys can give me! (Links are appreciated too! E-Books and whatnot.)
Edit: all of the answers here were very useful. As such, I gave them all a +1 to rep, but chose the more concrete one as best. Thank you all.
I also learnt Java and C# intensively before coming to Lisp so hopefully can share some useful perspectives.
Firstly, all Lisps are great and you should definitely consider learning one. There's a famous quote by Eric Raymond:
"Lisp is worth learning for the profound enlightenment experience you
will have when you finally get it; that experience will make you a
better programmer for the rest of your days, even if you never
actually use Lisp itself a lot."
Reasons that Lisps are particularly interesting and powerful are:
Homoiconicity - in Lisp "code is data" - the language itself is written in Lisp data structures. In itself this is interesting, but where it gets really powerful is when you start using this for code generation and advanced macros. Some believe that this features is a key reason why Lisp can help you be more productive than anyone else (short Paul Graham essay)
Interactice development at the REPL - a few other languages also have this, but it is particularly idiomatic and deep-rooted in Lisp culture. It's remarkably productive and liberating to develop while altering a live running program. Recent examples that caught my eye include music hacking with overtone and editing a live game simulation.
Dynamic typing - opinion is more divide on whether this is an advantage or not (I'm personally neutral) but many people thing that dynamically typed langauges give you a productivity advantage, at least in terms of building things quickly. YMMV.
My personal recommendation for a Lisp to learn nowadays would be Clojure. Clojure has a few distinct advantages that make it stand out:
Modern language design - Clojure "refines" Lisp in a number of ways. For example, Clojure adds some new syntax for vectors [] and hashmaps {} in addition to lists (). Purists may disapprove, but I personally believe these find of innovations make the language much nicer to use and read.
Functional first and foremost - all the Lisps are good as functional languages, however Clojure takes it much further. All the standard library is written in terms of pure functions. All data structures are immutable. Mutable state is strictly limited. Lazy sequences (including infinite sequences) are supported. In some senses it feels a bit more like Haskell than the other Lisps.
Concurrency - Clojure has a unique approach to managing concurrency, supported by a very good STM implementation. Worth watching this excellent video for a much deeper explanation.
Runs on the JVM - whatever you think of Java, the JVM is a great platform with extremely good GC, JIT compilation, cross platform portability etc. This can be a barrier to entry for some, but anyone used to Java or C# should quickly feel at home.
Library ecosystem - since Clojure runs on the JVM, it can use Java libraries extremely easily. Calling a Java API from Clojure is trivial - it's just like any other function call with a syntax of (.methodName someObject arg1 arg2). With the availability of the huge Java library ecosystem (mostly open source) Clojure basically leapfrogs all the "niche" languages in terms of practical usefulness
In terms of applications, Clojure is designed to be a fully general purpose langauge so can be used in any field - certainly not limited to AI. I know of people using it in startups, using it for big data processing, even writing games.
Finally on the performance point: you are basically always going to pay a slight performance penalty for using higher level language constructs. However Clojure in my experience is "close enough" to Java or C# that you won't notice the difference for general purpose development. It helps that Clojure is always compiled and that you can use optional type hints to get the performance benefits of static typing.
The flawed benchmarks (as of early 2012) put Clojure within a factor of 2-3 of the speed of statically typed languages like Java, Scala and C#, a little bit behind Common Lisp and a little bit ahead of Scheme (Racket).
Lisp, as you've discovered, is not one language; it's a family of languages that have certain features in common.
There are two primary dialects of Lisp: Common Lisp and Scheme. Each of those two dialects has many implementations, each with their own features. However, both Common Lisp and Scheme are standardized, and the standards define a certain baseline of features which you can expect any implementation to have.
Scheme is a minimalistic language with a very small standard library. It is used primarily by students and theoreticians. Common Lisp has many more language features and a much larger standard library, including a powerful object system, and has been used in large production systems.
Clojure is another minor, more recent dialect. If you want to understand Lisp, you're better off first learning either Common Lisp or Scheme.
My recommendation is to learn Scheme first; it's a purer expression of the ideas that Lisp is made of, and will help you understand the essence of the language. In many ways, Lisp is completely different from Java and other imperative languages; however, what you learn from it will make you a better programmer in those languages. You can easily learn Common Lisp after you know Scheme.
The advantage of Lisp is, simply put, that it's more powerful than other languages. All Lisp code is Lisp data and can be manipulated as such; this allows you to do really cool things with metaprogramming that simply can't be done in other languages, because they don't give you direct access to the data structures that comprise your code. (The reason Lisp can do this and they can't is intimately related to its strange-looking syntax. Every compiler or interpreter, after reading the source code, must translate it into abstract syntax trees. Unlike other languages, Lisp's syntax is a direct representation of the ASTs that Lisp code is translated into, so you know what those trees look like and can manipulate them directly.) The most commonly used metaprogramming feature is macros; Lisp macros can literally translate a bit of source code into anything you can program. You can't do that with, say, C macros.
The "faster in development and execution" thing may have been a reference to one specific feature which most Lisp implementations provide: the read-eval-print loop. You can type an expression into a prompt and the interpreter will evaluate it and print the result. This is wonderful both for learning the language and for debugging or otherwise investigating code.
Lisp is dynamically typed (though statically typed flavors do exist). Most implementations of Lisp run on their own virtual machine; however, many can also be compiled to machine code. Clojure was written specifically to target the JVM; it can also target .NET and JavaScript.
Though originally created for AI research, Lisp is by no means exclusively for AI. The main reason why it's not more popular in mainstream production environments (apart from the self-perpetuating dominance of Java and C#) is library support. Common Lisp has many good libraries out there (Scheme less so), but it pales in comparison to the vast amount of library support available for Java or Python.
If you want to get started, I recommend downloading Racket, a highly popular implementation of Scheme. It has everything you need, including a simple-but-very-powerful IDE with a read-eval-print loop, right out of the box. Though originally developed as a teaching language, it comes with a very large standard library more characteristic of Common Lisp than of Scheme. As a result, it's seeing use in real production environments.
Runtime Environments
Common Lisp and Scheme generally have their own unique runtime environments. There are some variants of Scheme (Chicken and Gambit) which can be translated to C and then linked with their environments so as to be able to be deployed as stand alone executable programs. Clojure runs in the JVM, and there is also a CLR port, but its not clear to me that the CLR port is current with the JVM. Clojure also has Clojurescript, which targets a Javascript runtime.
Which is Better to Learn First
I don't think that question has a good answer. Its up to you. Although if you have experience with the JVM, Clojure might be a bit smoother to start with.
What is Better about Lisp
That's a question liable to start a flame war. I don't have much lisp experience. I started learning Clojure a few months ago in earnest, have looked at Common Lisp and Scheme on and off over the years.
What I like is their dynamic natures. You need to change a function at runtime while your program is running? No problem! Like any power tool, you have to be careful not to chop your bits off when using this.
The power and expressiveness is addicting too. I am able to do some things with little effort that I know I could not achieve in Java, or I know would require a lot more work. Specifically, I was able to put together a description of a data structure - and though the use of macros, delay evaluation of parts of the data until the right time. If I had done that in Java, I would not have been able to nest the declarations like I did because they would have evaluated in the wrong order. Pain would have ensued.
I also like Clojure's view of functional programming, although I have to say it requires work to adjust.
Is Lisp General Purpose
Yes.
--
Mark Volkman has a really good article on Clojure. Many basics are there. One thing that I did in the beginning was to just fire up a repl and experiment when I needed to figure something out programmatically. e.g. explore an API or do some calculations. After a short period of time with that I started working on more building up levels of effort, and I have a project that I'm working on right now that involves Clojure.
There isn't a bad book about Clojure that has been written. The Stuart Sierra book is being updated; and the Oreilly book is about to come out soon, so you might want to wait. The Joy of Clojure is good, but I don't think its a good starter book.
For Common Lisp, I highly recommend the Land of Lisp.
For Scheme, there are several classics including The Little Schemer and SICP.
Oh, and this: http://www.infoq.com/presentations/Are-We-There-Yet-Rich-Hickey (maybe one of the most important talks you'll ever watch), and this http://www.infoq.com/presentations/hickey-clojure (IIRC, really good intro to Clojure).
common lisp
Common Lisp is both compiled and interpreted. Deployments (in Windows) can be done by an exe with DLLs. Or by a precompiled bytecode. Or by installing a Lisp system on the target device and executing the source against it.
Common Lisp is a fully usable industrial language with an active community and libraries for many different tasks.
Lisps are generally faster for development and due to the abstraction capabilities, better at developing higher level concepts. It's hard to explain. Ruby vs. C is an example of this sort of thing. All Lisps carry this capacity IMO.
Common Lisp is a general purpose language. I don't know offhand if modern Common Lisp implementations directly support executing assembly, so it may be difficult to write drivers or use compiler-unsupported CPU instructions.
I like Common Lisp, but Clojure and Racket are not to be sneezed at either. Clojure in particular represents a very interesting track, in my opinion.
For e-books, you can get On Lisp by Graham and Gentle Introduction to Symbolic Computation. Possibly others but those are the ones I can recall.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
I am learning Clojure and I really am loving some of its features. The time is coming to think of some real "pet projects" and I realize I'm not sure how to actually use Clojure.
I see many web and templating frameworks (e.g. Compojure), but somehow I'm in doubt on whether it's worth it. I feel that in the long run it can't serve the needs of real world applications which you address with Spring, Hibernate and some pieces of the Java EE stack.
On the other hand, I see great potential in the concurrency features but I'm short on ideas on how to really use them.
Enough background, my questions are:
What are the feasible applications of Clojure and functional programming? What idea for a pet project can you suggest which wouldn't be rewriting the stuff I did with OO/Java EE into different syntax? I'm looking for something what really exploits Clojure's potential and leads to a solution which feels a lot better (not just in syntax) than OO/structural programming.
Is it common, or at least reasonable, to mix Clojure and Java? I mean either of using Clojure for tiny libraries in 95% Java projects, or building Java apps on top of the core written in Clojure.
Edit: Thanks for all the great answers. They're all really inspiring. So if you have anything else to add, go ahead and don't be put off by the fact that one has been accepted.
In answer to the "background" part of the question:
I think you should read Jörg W. Mittag's answer to an SO question entitled "Real world Haskell programming". He makes a number of excellent points. Read on for my take on the FP in the real world issue; scroll past the horizontal line for answers to the two actual questions.
There's a number of FP-centric companies which seem to be really good at what they're doing; for some examples, google Jane Street (OCaml), Galois (Haskell), FlightCaster (Clojure for backend heavy lifting; I seem to remember reading that their frontend is currently done in Rails). Supposedly automated trading strategies are often coded in FP-oriented languages; that would indeed make perfect sense, although I have no inside data to confirm this. For additional examples to do with Clojure, see the list of companies on the success stories page.
Some people seem to be enjoying a degree of success in addressing the needs of real world applications in Rails, Django etc. It would appear that they feel no need to touch J2EE & friends. Not that these have much to do with FP, but they are like FP in that they're nothing like the "Enterprise Languages" of the present.
As for the two actual questions:
Why not just pick up whatever it is you've last been thinking to do and do it in Clojure? Obviously anything can be done in Java (and most things probably have been), but a leaner language might make the product cleaner, the experience more pleasant and less time consuming etc.
About mixing Clojure and Java -- I've seen a decent amount of Clojure code using a couple of classes coded directly in Java (for whatever reason). I've tried going the other way around myself and it's a bit of a pain in that it's much simpler to work with interface inheritance than class inheritance in Clojure, unexpected coupling in the Java code can seriously interfere with the ability of the Clojure code to do things in the most natural way etc. Still, it's entirely possible to extend a Java programme in Clojure and it seems like a particularly safe & sane way of experimenting with it for the worried Java developer.
Functional programming can be applied to almost any task. Web applications, scientific applications, games, you name it.
It is very common to mix Clojure and Java, since Clojure does not have many dedicated libraries for things like I/O or networking.
Organizations that already have a lot of Java code can use Clojure for small subsections of their Java projects.
For new projects, it is usually more effective to use Clojure as the high-level driver language, calling Java libraries where necessary.
I have been working on a small web application using Clojure, and while there is nothing special about the application that could not have been done in a different language, the experience of writing it has been completely different. I have written web apps using ASP.net and moving to Clojure was less about learning the different syntax and more about learning a different way to think and program. Having to learn a different way to think will occur regardless of the project you choose to work on, so I would worry less about finding the perfect functional project and more about finding something you just want to work on.
I think the answer to this has a great deal to do with the context your project is embedded in, and the constraints that imposes on you. Absent social factors I think Clojure is likely at least as "good" a language as Java is for any problem, with the possible exception of cases where you need the last bit of performance. And even in those cases things are not nearly as simple as they seem. For one thing some future version of Clojure can probably, in the theoretical limit, be compiled to bytecode that is as "fast" as what Java is compiled into (assuming a bit more work from the programmer at bottlenecks.) More importantly, optimization is a multi-factorial problem, and one in which programmer productivity and the flexibility of code factors heavily. So while there is a sense in which it would be accurate to say that Clojure is slower than Java, that sense might not be the important one when discussing the performance of a particular application.
So I'd say that if you disregard social factors Clojure's use cases are close to a superset of Java's. I wouldn't try to write a Linux kernel module in clojure though...
Of course, it's true that not all problems have equally natural solutions in functional languages. But people have come up with some interesting ways of dealing with some of the cases where FP seems to map badly to the domain, and anyway Clojure actually offers you enough escape hatches from pure FP that if you really feel the need to write part of your program in an imperative style you can (though of course you give up some of the benefits of Clojure in that case.) In the worst case you could use Clojure to drive the Java library in much the same way that you would in Java... it's hard to imagine a case where that would be a good idea, but in most cases that would not be markedly inferior to just using Java, and in many it might be better.
I'm still a neophyte at Clojure, though I've been programming in CL and scheme for a long time, and I spent about five years writing Java for a living. But I would probably prefer Clojure to Java for just about anything even without knowing it quite as well, as long as there were no social factors involved.
It would be a mistake to dismiss social factors though. I've been a Lisp programmer long enough to have a finely honed instinct for how well a Lisp will work in a given context. I've introduced Lisp to commercial settings where it has been a big win, and I've introduced it to settings where it really wasn't. I'd think long and hard about staking your career on successfully transitioning a team of programmers to any Lisp, Clojure included, particularly if they are not too keen on the idea.
Just to give you an idea of what I think Clojure might be useful for, I am currently writing a lot of poker-related code in Clojure. Some of it is pretty simple stuff (finding the best five card hand you can make from seven cards) and some of it is a bit more interesting (looking at someone's playing history and extracting meaningful trends from it using a few heuristics and some basic statistics.) None of it requires much in the way of Clojure's sophisticated concurrency mechanisms, but it is still much nicer (for me at least) in Clojure than it would be in, say, Java.
There are certainly some other cases that someone might describe where Clojure wins big because of its sophisticated mechanisms for managing concurrency, etc. I am aiming at something more modest- I am just pointing out that even if you don't need those mechanisms you might find Clojure a very congenial language for general purpose programming, albeit one that requires you to rethink how you abstract things if you're coming from an imperative/OO background. And hey, if you need the concurrency mechanisms (as you might, the way things are going), at least you already know Clojure.
I like writing game programs when I learn a new language.
I am in the process of learning Clojure and started writing a Spider solitaire player. If you have never played Spider, don't start; it is very additive :-). See http://www.spidersolitaire.org/.
In writing this game, I am getting to use several things that I want to learn: functional programming, concurrency, Java-interop (for Swing), etc.
I have also started writing a Bejeweled player (http://www.popcap.com/games/free/bejeweled2), but have run into a problem finding the definitive rules for scoring the game.
I find myself attached to a project to integerate an interpreter into an existing application. The language to be interpreted is a derivative of Lisp, with application-specific builtins. Individual 'programs' will be run batch-style in the application.
I'm surprised that over the years I've written a couple of compilers, and several data-language translators/parsers, but I've never actually written an interpreter before. The prototype is pretty far along, implemented as a syntax tree walker, in C++. I can probably influence the architecture beyond the prototype, but not the implementation language (C++). So, constraints:
implementation will be in C++
parsing will probably be handled with a yacc/bison grammar (it is now)
suggestions of full VM/Interpreter ecologies like NekoVM and LLVM are probably not practical for this project. Self-contained is better, even if this sounds like NIH.
What I'm really looking for is reading material on the fundamentals of implementing interpreters. I did some browsing of SO, and another site known as Lambda the Ultimate, though they are more oriented toward programming language theory.
Some of the tidbits I've gathered so far:
Lisp in Small Pieces, by Christian Queinnec. The person recommending it said it "goes from the trivial interpreter to more advanced techniques and finishes presenting bytecode and 'Scheme to C' compilers."
NekoVM. As I've mentioned above, I doubt that we'd be allowed to incorporate an entire VM framework to support this project.
Structure and Interpretation of Computer Programs. Originally I suggested that this might be overkill, but having worked through a healthy chunk, I agree with #JBF. Very informative, and mind-expanding.
On Lisp by Paul Graham. I've read this, and while it is an informative introduction to Lisp principles, is not enough to jump-start constructing an interpreter.
Parrot Implementation. This seems like a fun read. Not sure it will provide me with the fundamentals.
Scheme from Scratch. Peter Michaux is attacking various implementations of Scheme, from a quick-and-dirty Scheme interpreter written in C (for use as a bootstrap in later projects) to compiled Scheme code. Very interesting so far.
Language Implementation Patterns: Create Your Own Domain-Specific and General Programming Languages, recommended in the comment thread for Books On Creating Interpreted Languages. The book contains two chapters devoted to the practice of building interpreters, so I'm adding it to my reading queue.
New (and yet Old, i.e. 1979): Writing Interactive Compilers and Interpreters by P. J. Brown. This is long out of print, but is interesting in providing an outline of the various tasks associated with the implementation of a Basic interpreter. I've seen mixed reviews for this one but as it is cheap (I have it on order used for around $3.50) I'll give it a spin.
So how about it? Is there a good book that takes the neophyte by the hand and shows how to build an interpreter in C/C++ for a Lisp-like language? Do you have a preference for syntax-tree walkers or bytecode interpreters?
To answer #JBF:
the current prototype is an interpreter, and it makes sense to me as we're accepting a path to an arbitrary code file and executing it in our application environment. The builtins are used to affect our in-memory data representation.
it should not be hideously slow. The current tree walker seems acceptable.
The language is based on Lisp, but is not Lisp, so no standards compliance required.
As mentioned above, it's unlikely that we'll be allowed to add a full external VM/interpreter project to solve this problem.
To the other posters, I'll be checking out your citations as well. Thanks, all!
Short answer:
The fundamental reading list for a lisp interpreter is SICP. I would not at all call it overkill, if you feel you are overqualified for the first parts of the book jump to chapter 4 and start interpreting away (although I feel this would be a loss since chapters 1-3 really are that good!).
Add LISP in Small Pieces (LISP from now on), chapters 1-3. Especially chapter 3 if you need to implement any non-trivial control forms.
See this post by Jens Axel Søgaard on a minimal self-hosting Scheme: http://www.scheme.dk/blog/2006/12/self-evaluating-evaluator.html .
A slightly longer answer:
It is hard to give advice without knowing what you require from your interpreter.
does it really really need to be an interpreter, or do you actually need to be able to execute lisp code?
does it need to be fast?
does it need standards compliance? Common Lisp? R5RS? R6RS? Any SFRIs you need?
If you need anything more fancy than a simple syntax tree walker I would strongly recommend embedding a fast scheme subsystem. Gambit scheme comes to mind: http://dynamo.iro.umontreal.ca/~gambit/wiki/index.php/Main_Page .
If that is not an option chapter 5 in SICP and chapters 5-- in LISP target compilation for faster execution.
For faster interpretation I would take a look at the most recent JavaScript interpreters/compilers. There seem to be a lot of thought going into fast JavaScript execution, and you can probably learn from them. V8 cites two important papers: http://code.google.com/apis/v8/design.html and squirrelfish cites a couple: http://webkit.org/blog/189/announcing-squirrelfish/ .
There is also the canonical scheme papers: http://library.readscheme.org/page1.html for the RABBIT compiler.
If I engage in a bit of premature speculation, memory management might be the tough nut to crack. Nils M Holm has published a book "Scheme 9 from empty space" http://www.t3x.org/s9fes/ which includes a simple stop-the-world mark and sweep garbage collector. Source included.
John Rose (of newer JVM fame) has written a paper on integrating Scheme to C: http://library.readscheme.org/servlets/cite.ss?pattern=AcmDL-Ros-92 .
Yes on SICP.
I've done this task several times and here's what I'd do if I were you:
Design your memory model first. You'll want a GC system of some kind. It's WAAAAY easier to do this first than to bolt it on later.
Design your data structures. In my implementations, I've had a basic cons box with a number of base types: atom, string, number, list, bool, primitive-function.
Design your VM and be sure to keep the API clean. My last implementation had this as a top-level API (forgive the formatting - SO is pooching my preview)
ConsBoxFactory &GetConsBoxFactory() { return mConsFactory; }
AtomFactory &GetAtomFactory() { return mAtomFactory; }
Environment &GetEnvironment() { return mEnvironment; }
t_ConsBox *Read(iostream &stm);
t_ConsBox *Eval(t_ConsBox *box);
void Print(basic_ostream<char> &stm, t_ConsBox *box);
void RunProgram(char *program);
void RunProgram(iostream &stm);
RunProgram isn't needed - it's implemented in terms of Read, Eval, and Print. REPL is a common pattern for interpreters, especially LISP.
A ConsBoxFactory is available to make new cons boxes and to operate on them. An AtomFactory is used so that equivalent symbolic atoms map to exactly one object. An Environment is used to maintain the binding of symbols to cons boxes.
Most of your work should go into these three steps. Then you will find that your client code and support code starts to look very much like LISP too:
t_ConsBox *ConsBoxFactory::Cadr(t_ConsBox *list)
{
return Car(Cdr(list));
}
You can write the parser in yacc/lex, but why bother? Lisp is an incredibly simple grammar and scanner/recursive-descent parser pair for it is about two hours of work. The worst part is writing predicates to identify the tokens (ie, IsString, IsNumber, IsQuotedExpr, etc) and then writing routines to convert the tokens into cons boxes.
Make it easy to write glue into and out of C code and make it easy to debug issues when things go wrong.
The Kamin Interpreters from Samuel Kamin's book Programming Languages, An Interpreter-Based Approach, translated to C++ by Timothy Budd. I'm not sure how useful the bare source code will be, as it was meant to go with the book, but it's a fine book that covers the basics of implementing Lisp in a lower-level language, including garbage collection, etc. (That's not the focus of the book, which is programming languages in general, but it is covered.)
Lisp in Small Pieces goes into more depth, but that's both good and bad for your case. There's a lot of material on compiling and such that won't be relevant to you, and its simpler interpreters are in Scheme, not C++.
SICP is good, definitely. Not overkill, but of course writing interpreters is only a small fraction of the book.
The JScheme suggestion is a good one, too (and it incorporates some code by me), but won't help you with things like GC.
I might flesh this out with more suggestions later.
Edit: A few people have said they learned from my awklisp. This is admittedly kind of a weird suggestion, but it's very small, readable, actually usable, and unlike other tiny-yet-readable toy Lisps it implements its own garbage collector and data representation instead of relying on an underlying high-level implementation language to provide them.
Check out JScheme from Peter Norvig. I found this amazingly simple to understand and port to C++. Uh, dunno about using scheme as a scripting language though - teaching it to jnrs is cumbersome and feels dated (helloooo 1980's).
I would like to extend my recommendation for Programming Languages: Application and Interpretation. If you want to write an interpreter, that book takes you there in a very short path. If you read through writing the code you read and doing the exercise you end up with a bunch of similar interpreters but different (one is eager, the other is lazy, one is dynamic, the other has some typing, one has dynamic scope, the other has lexical scope, etc).
I have been learning C++ for a while now, I find it very powerful. But, the problem is the the level of abstraction is not much and I have to do memory management myself.
What are the languages that I can use which uses a higher level of abstraction.
Java, C#, Ruby, Python and JavaScript are probably the big choices before you.
Java and C# are not hugely different languages. This big difference you'll find from C++ is memory management (i.e. objects are automatically freed when they are no longer referenced). You would chose these if you were interested in desktop style applications, or keen on static typing (and you'd probably choose between them based on how you feel towards Microsoft and the Windows platform). In both cases you'll find much richer standard libraries than you'll be used to from C++.
Python and Ruby take a step away from static typing, into a world where you can call and method on any object (and fail at runtime if it's not there). That is both a blessing (a lot less boilerplate code) and a curse (the compiler can't catch those errors for you anymore). Once again, you'll find they have richer standard libraries, and are higer level again than Java / C#. Performance is the main downfall, with Python being somewhat faster than Ruby as I understand it. To choose between them, you'd probably choose Ruby if you're interesting in web development for the Ruby on Rails framework community, and otherwise go with Python.
JavaScript is even more different from C++ in that it does away with classes entirely. Objects are simply cloned from other objects and can have methods and properties added to them at runtime. Very flexible, but also very easy to make into a total mess. JavaScript is the only real choice if you're interested in running applications in a browser, which is really coming into its own as a platform. You'll find the standard libraries available rather limited if you're not doing a lot with the browser, but there are quite a few good frameworks which fill in some of the gaps.
Some other interesting, though more niche choices are
Smalltalk - More or less in the Ruby and Python camp, and significantly faster as I understand it. Be careful though _ I've seen lots of good engineers learn Smalltalk and never come back ;)
Objective-C - When C went object oriented, C++ went one way (static typing), and Objective-C went the other (dynamic typing). It's quite Smalltalk inspired, and has a good standard library if you're in Mac / iPhone land. In terms of memory management, unlike everything else I've listed, it's not garbage collected (though that's now an option on Mac OS X 10.5), but it does have a reference counting scheme which makes life significantly simpler than managing memory by hand.
Lisp - I've never learnt it myself beyond what I needed for minor Emacs hacking. As I understand it, the libraries were nice in their day, but though the language remains supremely elegant, they've fallen a little behind the times.
Haskel - If you wanted a complete break from objects and classes, Haskel and it's functional approach is an interesting way to go (or Lisp as above, or F# if you are in .Net land). Basically, you're giving up loops and variables in favour of doing everything recursively. Takes some time to wrap your mind around, and probably isn't practical for most real world applications, but it's a good one to learn.
Eiffel - I love it - Very clean syntax, and designed for serious engineering type systems. Statically types like C# and Java, and with a weaker standard library, but it will make you really think about language and class library design.
ActionScript and Flex - The programming interface to Flash, which is based on what seems to be a statically typed version of JavaScript. I've played with it a bit, and it's quite slick if you're interested in developing media based applications. You can also push beyond the browser with Flex and into the Air platform to build real desktop apps.
I would say that from your question you probably haven't finished learning about C++. If you're still doing your own memory managment then you still have a long way to go my friend!
Check out the auto_ptr and shared_ptr - check out the Boost libraries.
Similarly with abstraction - what are you specifically complaining about? AFAIK there's not much you can't do with C++ that is present in other strongly-typed languages.
I know this doesn't answer your question - you want to move forwards, but C++ is one of those things where you never really stop learning. If you get bored, take a brief foray into templates and template meta-programming...
I see a lot of excellent suggestions so far. However, I think there's something missing, assembler.
Why learn assembly language?
It's not as difficult as you may think. Assembly language is a lot smaller in scope than many modern languages, there are a few tricks you need to understand for it to make sense, but it's not that complicated.
It broadens your knowledge base. Knowing the fundamentals is almost always beneficial, even when working at a high level.
It can be extremely useful when debugging. Especially debugging native code without the source, the knowledge you gain from learning assembler enhances your ability to debug in these situations by leaps and bounds.
It gives you more options. When the rare circumstance comes up where assembly code is needed you won't be helpless.
It's good for your resume. It shows that you learn beyond just the bare minimum needed to keep your current job, it shows a curiosity about fundamentals, and it puts you in a different class of programmers, and that class tends to be more experienced and more capable.
It's just plain cool.
Some assembly language resources:
Sandpile.org (assembly language / processor architecture reference)
Gavin's Guide to 80x86 Assembly (a decent online tutorial)
Assembly Language for Intel-Based Computers (5e) (a decent textbook for x86 assembly)
Trying something really foreign like Haskell will allow you to think in different ways. It also helps you to think recursively. C++ has recursion but it infiltrates many more parts of functional languages.
ditto Lisp,.. or scheme
Even if you don't ever use it, it's handy. I only really got template programming after learning it.
Another one is prolog. it puts you in a non sequential mindset.
If you're comfortable with C++ syntax and style, you might find D to be an interesting language. Or if you want to branch out, any of Python, C#, Java, Ruby would be excellent choices.
C# if you're in the Microsoft ecosystem.
Python and Ruby seem to have the most traction in the Linux/Unix/etc space.
ObjectiveC is dominant on the Macintosh and iPhone. The most recent MacOS implements garbage collection for a subset of the frameworks, but to use the rest you'd have to do resource management yourself.
You could learn Java, as it does garbage collection as well, but the number of frameworks you'd need to become familiar with to be a productive Java developer is daunting.
Well if you're looking for a very high level of abstraction and memory management then I'd say lisp would be an ideal candidate. I'm learning it now, slowly, and it's the most fun I've had with a new language.
Having said that Python or Ruby may be a better compromise between expressiveness and popularity. Python's Django framework is one of the better RAD frameworks if you're looking for web application stuff.
I'd say it depends on the kind of programming you want to try. If you want to stay on the OOP side, learn Python or Ruby, both languages provide an easy way to create bindings to use your C++ code from a script (for efficiency reasons).
If you need another approach to programming, learn a "functional" language like Lisp or Haskell.
And if you need to include a fast and small scripting language inside your C++ application, try Lua.
Last but not least, if you know Java and hate it, you can try Scala, a language where you can mix your Java classes with your Scala code, very interesting.
Scheme.
The Little Schemer and Structure and Interpretation of Computer Program will stretch your mind in strange and wonderful ways.
DrScheme is a good IDE for beginners. The Scheme Programming Language makes a good, free reference.
try c# much :)
if you want to abstract memory management, Java comes to my mind instantly.
I suggest learning database design and a query language such as SQL.
You can start with a desktop tool like Microsoft Access or use the free SQL Server Express or Postgre or MySQL.
Well I think there is no predefined route in learning programming languages. You may learn your next lang based on your job needs, academic research, just for fun, etc. There are many options.
In you feel comfortable in C++, you can go down and learn some assembly. It's a dark art but you'll be glad when you encounter some hard debugging session.
In terms of more abstraction, Smalltalk is extremely fun, OOP-pure and 100% dynamic (debugging is a pleasant thing to do, which is not in static-typed languages). Dolphin Smalltalk is a good implementation for Windows, even the free community edition gives enough to play with. In multiplatform Smalltalk VMs, go for Visualworks or Squeak. Visualworks is extremely stable and comes with a lot of documentation.
Python is used today in many, many fields. I don't know anything about Python excepting the basic syntax and semantics, but it's required today for many jobs.
Java it's, well Java. It's interesting that Java never catch on me. You may get interested on Java, altough. Ask here for advantages of using it over C++ or other OOP languages.
For Web development go for Javascript, specially considering the AJAX wave. It's getting interesting those days. We've talked about Smalltalk, all right, Seaside is an amazing framework for web development. It works (at least I tried on) Squeak /Visualworks... it's beatiful.
Well, there are a lot of more to get your hands on: Scheme, LISP, Ruby, Lua, Bash (!), Perl (ugh), Haskell... Try them all and have fun!
Qt
Why not learn Qt? Its a great application development framework available on all platforms and even mobile devices!
Clojure is well worth exploring as it meets both of your criteria:
It has a strong emphasis on programming with higher level abstractions. see e.g. this video: Clojure: The Art of Abstraction
It has automatic memory management / garbage collection (via the JVM, which has some of the world's best GC implementations)
I'll give some examples using just one abstraction: in Clojure you can manipulate pretty much any data structure via the sequence abstraction.
;; treat a vector as a sequence and reverse it
(reverse [1 2 3 4 5])
=> (5 4 3 2 1)
;; Take 10 items from a infinite sequence
(take 10 (range))
=> (0 1 2 3 4 5 6 7 8 9)
;; Treat a String as a sequence of characters, calculate the frequencies
(frequencies "abracadabra")
=> {\a 5, \b 2, \r 2, \c 1, \d 1}
;; Define an infinite lazy sequence of fibonacci numbers, take the first 10
(def fibs (concat [0 1] (lazy-seq (map + fibs (rest fibs)))))
(take 10 fibs)
=> (0 1 1 2 3 5 8 13 21 34)
Since you are already into C++, next step would be to learn .Net through managed C++ or managed extensions for C++..this will get you a step in the big world of .Net framework..Once you understand the framework, makes it more comfortable to learn other .Net languages like C#, VB.Net etc.
One of the areas that MC++ excels in, and is in fact unique in amongst the .NET languages, is the ability to take an existing unmanaged (C++) application, recompile it with the /clr switch, have it generate MSIL and then run under the CLR. This extraordinary feat is aptly termed "It Just Works (IJW)!" There are some limitations, but for the most part, the application will just run. The C++ code can consist of old-fashioned printf statements, MFC, ATL, or even templates!
I recommend python as it's not only a sexy language, but also very widely used and easy to integrate with C++ through Boost.Python.
But as Thomi said, there's lot to explore in C++ and with the help of Boost libraries it's becoming really easy to develop in.
Rather than suggest a specific language, I would recommend you pick any language or languages that offer the following 4 features:
Automatic Memory Management
Reflection/Introspection
Declarative/Functional constructs(e.g. lambda functions)
Duck Typing
The idea here is to expand your programming perspective to include concepts that the C++ language does not offer you out of the box.
It depends on what you want to do. If you have some specific tasks that you are interested in accomplishing then look at languages that are best for those types of tasks. The best way to learn a language is to actually use it.
I'd say get started with Python. It has a higher level of abstraction and it teaches you the importance of indenting and making "pretty" code. Not that "pretty" is very important, but it will make the future maintainer of your code a lot happier :)
There's a lot of example code out there, and if you are into Linux there are various distributions out there who have all (or most) of their tools based on the language. If you like digging into how managing an operating systems works (something most programmers do) it's a good start. Before I get the flames I said managing, not the actual kernel stuff for that you mostly need C and you should have that covered.
On the other hand it might be nice to dive into the C side of things, ignore the OO stuff and learn functional programming. If you head down that road I also suggest to start with basic assembly language like one of the upper posts suggested. Maybe HLA (High-Level Assembly by Randall Hyde, he wrote a great book called Art of Assembly Language Programming) is a good start. You'll either learn to love memory management or hate it for the rest of your live. Good to know in case you want to start a career in programming :)
However if you're looking to make a job out of programming, Java and J2EE is an easy money maker if you know what you're doing. IMHO it gets boring really quick though.
Personally, I have been programming in Java, Python, C/++ and my favorite has to be python. Although C++ can do everything Python can do and more, I wrote a Python program with about 10 lines that would take about 50 in C++. So, moral of the story, use python.
If you haven't already, try out a scripting language. It should change the way you work & think. Hopefully, in a good way :)
I've got to put up a separate answer for Perl. While Python is roughly equivalent in functionality and considered more clean and modern, Perl has an elegance all of its own - the elegance of pure pragmatism. It also boasts a truly great library support. Take a look at Perl to expand your brain in the direction opposite to Haskel :) (although Perl aficionados claim that it can be used for functional programming).
Rust
Syntactically similar to C++
Designed for performance and safety, especially safe concurrency