Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
As the title says, what's the appeal of LISP?
I've seen it raised to high heavens and back and then put on a pedestal over an over. I've seen it done by by people here, by actual programmers that I know, by university professors that try to look like the kind of enlightened guys who have obviously transcended the plebeian language-war and would rather die than commit the heresy of saying that one language is better than another.
I tried to learn it, I looked at a few tutorials, at the "learn x in y minutes" page for both lisp and clojure(since from what I understand it's lisp running on the jvm with more FP thrown in), and I understand the syntax and how to do stuff in it. And I went back to tutorials about it several times over the last few months but every time the syntax made me want to look for a high-school kid and yell at him for messy code because looks like a C-style if-condition gone wild with all the nested parentheses and simple things seem to become a chore. What lisp seems to do in a few lines, other FP languages like Haskell or F# or even non FP languages like C# who just discovered that map and filter are a thing seem to do in much less in a much more expressive way.
Am I looking at this wrong?
Is there some click that needs to happen?
Did I only find crappy resources?(if yes please give me some good resources for learning lisp/clojure, preferably with some exercises)
What in the ... is the appeal of lisp? Honest to God question here.
Thank you in advance!
When I first heard about Clojure in 2009, I wrote a blog post titled "Clojure: Where's the Elegance?" I was a big fan of Python's style of elegance at the time ("executable pseudo-code"), and Lisp was deeply unfamiliar. I continued wrestling with Clojure, though, because "list processing" seemed like an important programming paradigm to have in my toolset. I used a lot of JavaScript, Ruby, and C# for a while, and on my own time played with ClojureScript. When I finally came back to Python in 2017 to do some data-munging, I discovered I was writing a lot of Clojure-inspired functions to help. Finally, I just started writing the data-processing in Clojure, and I wouldn't go back. So while you've asked a question that may not be answerable, perhaps I can shed some light on what accounts for the difference between your perspective and the enthusiasm of the Lisp community.
First, while Lisp is very alien coming from the rich grammars of C, Python, and Ruby, the lightness of Lisp's syntax turns out to be a tremendous blessing. It relieves a lot of cognitive burden. Many kinds of syntax errors completely disappear. But by far the most good I get out of Lisp's consistent grammar is how easily it can be transformed. What I miss most when working in other languages is the ability to rearrange the AST without having to fiddle with the actual text of the code. Working with Lisp is far more fluid than any other language I've experienced.
Second, it's trivial to control how code is evaluated. A lot has been written about the glory of macros, but to me the fundamental superpower is that you can prevent code from being evaluated, evaluate it in a specific order, or evaluate it multiple times, all without the syntactic overhead of wrapping that code in a function. That said, I don't write macros very much, but when I do, a function is a poor substitute.
Third, most languages don't have very good support for live coding. I typically don't use the REPL itself, but rely on an editor integration to evaluate code. Clojure's tooling is good enough that I can mold an entire codebase with dozens of namespaces without ever having to restart the JVM process or split a single big code file into separate modules.
Fourth, Lisp code tends to compose extremely well. Between everything being an expression that returns a value and the language being built on a relatively small set of data structures, you can mix and match language constructs with great liberty. That opens up a lot of possibilities where other languages put up walls.
The common factor of all these is that the language does its best to get out of your way. When I was a novice programmer, the complex grammars of C and Python and Ruby was an aid to programming. It kept me on the straight and narrow. The language constructs were like landmarks to navigate by. Lisp, without those landmarks, can feel like a foreign country. But as you get more familiar with Lisp, other languages feel artificially constraining and hard to move around in, like driving on roads when you could be flying. Lisp is very freeing, but from my personal experience, I can tell you that, like flying, it takes practice before the strangeness of it turns from fear to freedom.
I have been reading about Immutable data structures and understood that change detection has been made easy . And quite often, I hear that it makes the application maintenance simpler and provides an easy to understand programming model.
I need help to understand the way it simplifies the job.
The Clojure community has embraced immutability and it is an eye opener. The best I can do is send you to the source: Rich Hickey's essay on State and his talk The Value of Values. Rich explains how separating the concept of a variable into three distinct concepts: identity, state, and value helps you model your system and reason about it.
It boils down to this: in your programming model, you should only allow things to change if they change in the system you are trying to model. Otherwise you are adding moving parts (mutable variables and objects) to a model that doesn't need them. This makes it harder to understand the model (specially as time evolves) but has little or no benefit.
Even though reading helps, the only way to grok this is to program in a language that takes immutability as a default until you realize how most of the systems you model actually have only a handful of things that change instead of pages and pages of mutable variables.
Immutability is certainly more embraced in functional languages than in imperative ones, even if you can have a Java programming style that limits mutability (see this for immutability in Java). That said, I will just comment on [functional/immutability] and [object/mutability].
I'm Clojure fan and find functional programming really powerful, but...
May be I spent too much time with C++ & Java and not enough with Lisp & Clojure, but I reckon that the simpler maintenance argument has yet to be proven by facts. I'm not sure there are reliable surveys on the actual cost of maintenance in big production systems with data on the technology used and associated costs.
Certainly, in terms of LOC, language like Clojure are really more focused and concise than Java. Hence you can say that less code leads to less maintenance, but I think functional style gives really more compact code that needs a very focused attention to fully understand what a function is doing comparing to imperative style which is more verbose but kind of straightforward. One big advantage of functional programming associated with immutability, is the ability to isolate a function and experiment with it without the need to drag a heavy context of satellite objects or build a bunch of mocks, which is very often the case with OO languages. Putting aside the experimentation, a pure function won't modify its arguments, which ease the fear to break unintentionally some piece of code outside the scope of the function.
But, putting aside the merits of functional/immutability over oop/mutability, in terms of maintenance, my experience leads me to think that it's not the technology which is the main issue, but the design, code quality and evolution of this code over time even when the initial one was of good quality. By "good", I mean that the code is respectful of style conventions (like basic naming), managed complexity, and has a sensible test harness, in a continuous (or at least automated) build environment.
Then, the question becomes: is there a paradigm (functional/immutability, object-oriented/mutability) that enforced a better design and better code. My feeling is that functional languages are the land of computer science passionates, OTOH OOP is more mainstream. Isn't it because OOP is easier to apprehend or is ot just a matter of education? but then, in order to maintain a system in the long run, should one go for a "clever" functional environment with few people able to tackle it, or some mainstream OO technology - with its unsafeness or permissiveness - but lots of people having some knowledge in it?
Certainly the solution is to choose the right technologies (plural) with the right, motivated people...
A coworker and I are Clojure newbies. We started a project a couple months back, but quickly found that we had a tough time dealing with our code base -- by 500 LOC we basically had no idea where to start with the debugging, when things went wrong (which was often). Instead of pairs, functions were getting lists, or numbers, or what-have-you.
Now we're starting a new but related project and migrating a lot of the old code over. But we're again hitting a wall.
We're wondering, how do we effectively manage a Clojure project, especially as we make changes to existing code?
What we've come up with:
liberal use of unit-tests
liberal use of pre-, post-conditions
informal type declarations in function comments
use defrecord/defstruct/defprotocol to implement a data model, which would really simplify testing
But post-, pre-conditions seem not to be used very often. Unit-testing + comments will only help so much. And it seems like Clojure programmers don't typically implement formal data models.
Do we just not get Clojure? How do Clojure programmers know that their code is robust and correct?
I think this is actually an evolving area - Clojure hasn't really been around long enough for all of the best practices and associated tools for managing a large code base to be developed yet.
Some suggestions from my experience:
Structure your code in a "bottom up" way - in general, the way you want to structure you code will have the "utility" code at the top of the file (or imported from another namespace) and the "business logic" code that uses these utility functions towards the end of the file. If this seems difficult to do, then it's probably a hint that your code needs some refactoring.
Tests as examples - Test code in clojure works very well both to sanity check your code but also as documentation (e.g. "what kind of parameter is this function expecting?"). If you hit a bug, refer to your tests to check your assumptions and write a couple of new tests to flush out what is going wrong.
Keep functions simple and compose them - Kind of an extension of the "single responsibility principle" to functional programming. I consider more than 5-10 lines in a Clojure function as a major code smell (if this seems extreme, just remember that you can probably achieve as much in 5-10 lines of Clojure as you could with 50-100 lines of Java/C#)
Watch out for "imperative habits" - when I first started using Clojure, I wrote a lot of pseudo-imperative code in Clojure. An example would be emulating a for loop with "dotimes" and accumulating some result within an atom. This can be painful - it's not idiomatic, it's confusing and usually there is a much smarter, simpler and less error-prone functional way of doing it. This takes practice, but it is worth it in the long run...
Debug at the REPL - usually when I hit an issue, coding at the REPL is the easiest way to flush it out. Generally this means running some specific parts of the larger algorithm to check assumptions etc.
Refactor common utility functions out - you'll probably find a bunch of common or structure repeated in many functions. Well worth pulling this out into a function or macro that you can re-use in other places or projects - that way you can test it much more rigorously and have the benefits in multiple places. Bonus points if you can get it all the way upstream into Clojure itself! If you do this well enough, then your main code base will be extremely succinct and therefore easy to manage, containing nothing but the genuinely domain-specific code.
simple composable abstractions
"It is better to have 100 functions operate on one data structure than to have 10 functions operate on 10 data structures." - Alan J. Perlis
For me its all about composing simple functions. Try to break every function down into the smallest units you can and then have another function that composes them to do the work your need. You know you are in good shape is every function can be tested independently. If you go too heavy on the macroes then it can make this step harder because macroes compose differently.
D.R.Y, Seriously, just don't repeat yourself
starting with well decomposed functions in a a bunch of namespaces; every time I need one of the composable parts somewhere else I "hoist" that function up to a library included by both namespaces. This way your commonly used abstractions sort of evolve over the course of the project into "just enough framework". It is very difficult to do this unless you really have discrete composable abstractions.
Sorry to dig up this old question, the answers by mikera and Arthur are excellent, but it's something I've also wondered about as I've been learning Clojure, and thought I'd mention how we organise files.
In a similar vein to ensuring each function has a single job, we group related functions into namespaces to make it easier to navigate the code. So we might have a namespace for functions providing access to a particular database, or providing a collection of HTTP-related utilities. This keeps each file relatively small, and makes tests easier to find. It also makes refactoring much more straightforward. This is hardly anything new, but it's worth bearing in mind.
Does anyone know any sites/books/articles covering best practices or theory around design patterns in high performance applications? It seems a lot of the patterns use indirection/abstraction/encapsulation in a way that may affect performance in computationally intensive code. Head First Design Patterns and even GoF mention possibility of performance hits with many of the patterns but without more concrete advice on how to deal with it.
I’m surprised we aren’t asking what performance problems you are having!
In my experience, performance problems are usually tied to specific conditions and situations. Design patterns, on the other hand, are solutions to more general and abstract problems. It would seem a bit awkward to approach both in the same text: what of possibly many "non-patterned" solutions should the author compare the performance of a design pattern against? When the performance problem is general, there certainly already are patterns to solve them: the Flyweight is a good example.
The penalties imposed by the use of a design pattern are of a finite, very small set: introduction of virtual calls, added latency due to delegation, extra memory consumption due to the proliferation of objects and so on. If, after profiling, you notice that these are the cause of your woes, there are known ways to minimize them.
Knowing the patterns might be useful to solve performance issues, too. First, someone already mentioned that patterns break down a problem in smaller bits: this might ease pinpointing the source of the issue and isolating ugly but performant code. They also create a framework of reasoning and expectations for developers. If you must introduce a deviation for performance reasons, it will be obvious: “Except here, where we forego X and do Y to improve performance, this is a Chain of Responsibility.” They are rules to be broken when needed.
(Alas, there is one very good pattern for getting good performance: measure, pinpoint, fix.)
Design patterns exist to help you come to grips with how to design software or improve its flexibility. How you implement the pattern determines what kind of performance penalty (or benefit) you will see from its use.
Some patterns do exist because that overall way of structuring things generally does lead to faster programs. But unlike algorithms there is no good way to really formally analyze a pattern to decide on how slow or fast it is.
My advice would be to use a pattern if it helps you figure out how to design a particular piece of code, or if you need to refactor to make code more flexible or clear. If you then have performance issues, use standard profiling techniques to find them.
If you're refactoring when you encounter performance issues, maybe the cost isn't worth the refactor, or maybe there's a way to mitigate it. If you're designing new code, maybe there's a way to mutate things to fix the performance issue if it truly lies in the necessary indirection for the pattern to work.
The most concrete advice is: profile it in your application and see how much of an impact it really makes.
Any other advice is going to be considerably more general and may not necessarily apply well to how you have implemented a given pattern in your application with your compiler on your platform.
Design pattern is really focusing on how you structure the code and define the class abstraction and interaction. Performance of your computational performance will really be mostly effected by the way you write the actual code implementation (body of the method).
For C++ I definitely suggest reading Scott Meyers book on Effective C++ and More Effective C++ series of books which in itself really reveals many idioms on writing high performance code.
You can read Herb Sutter's entries under "Effective Concurrency" for things involving multi-threading and concurrency patterns and how they affect performance.
http://herbsutter.com/
Design patterns are mostly ways of breaking your program into smaller pieces, which are easier to reuse, compose, design, and test. Several design patterns will result in code that performs worse than a simpler design, but they have a significant advantage when you consider the 80/20 rule.
The 80/20 rule says that 80 percent of your program's execution time will be spent executing 20 percent of it's code. When your program is nice and modular, it's easy to throw it in a profiler and see exactly which component could be tuned/optimized, or where it makes sense to go with a less flexible design in order to improve performance. Having the design that far separated initially though makes it easier to find performance hot spots.
One term that may help you get better hits is 'pattern language'. It's a collection of patterns that go together for some purpose. If you have a more specific goal that high performance someone may have plotted out a path through patterns for your domain, for example: pattern language for parallel software. Here's another nice collection of parallel programming patterns from UIUC, a hotbed of patterns work.
The ACE/TAO guys have a lot of papers about high performance network patterns using C++
Remember the old saying "You can have it good, fast and cheap, pick two"
Design patterns address the good. A good foundation is needed so the code can be accurate, and maintainable.
If performance is an issue, then benchmark and then optimize the sections that give you problems. Many times performance is just a question of picking a proper algorithm., but it may mean you need to break-out into some horrifically optimized code for that 10% that takes up 90% of the time. Just make sure you comment the S^^T out of it.
GoF design patterns are about using proven patterns to solve common problems with elegant, maintainable code. They don't target performance.
If you want patterns for performance, you may need to look at system architecture patterns, algorithms, data structures, etc.
What does your app do?
If your application is in C++, and is written sensibly, the chances are your code will run blindingly fast on modern hardware, until it has to wait for I/O. The exception would be something like real time image analysis that is very processor intensive.
If performance is an issue, do you really mean I/O performance? (disk, DB, network etc.)
There are 'patterns' that allow your application to perform even while frequently waiting for I/O (asynchronous callbacks etc.)
If you are dealing with an uneven load, whereby the peak load may be much higher than average load, a commonly employed architecture pattern is to de-couple system components with message queues.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
I am learning Clojure and I really am loving some of its features. The time is coming to think of some real "pet projects" and I realize I'm not sure how to actually use Clojure.
I see many web and templating frameworks (e.g. Compojure), but somehow I'm in doubt on whether it's worth it. I feel that in the long run it can't serve the needs of real world applications which you address with Spring, Hibernate and some pieces of the Java EE stack.
On the other hand, I see great potential in the concurrency features but I'm short on ideas on how to really use them.
Enough background, my questions are:
What are the feasible applications of Clojure and functional programming? What idea for a pet project can you suggest which wouldn't be rewriting the stuff I did with OO/Java EE into different syntax? I'm looking for something what really exploits Clojure's potential and leads to a solution which feels a lot better (not just in syntax) than OO/structural programming.
Is it common, or at least reasonable, to mix Clojure and Java? I mean either of using Clojure for tiny libraries in 95% Java projects, or building Java apps on top of the core written in Clojure.
Edit: Thanks for all the great answers. They're all really inspiring. So if you have anything else to add, go ahead and don't be put off by the fact that one has been accepted.
In answer to the "background" part of the question:
I think you should read Jörg W. Mittag's answer to an SO question entitled "Real world Haskell programming". He makes a number of excellent points. Read on for my take on the FP in the real world issue; scroll past the horizontal line for answers to the two actual questions.
There's a number of FP-centric companies which seem to be really good at what they're doing; for some examples, google Jane Street (OCaml), Galois (Haskell), FlightCaster (Clojure for backend heavy lifting; I seem to remember reading that their frontend is currently done in Rails). Supposedly automated trading strategies are often coded in FP-oriented languages; that would indeed make perfect sense, although I have no inside data to confirm this. For additional examples to do with Clojure, see the list of companies on the success stories page.
Some people seem to be enjoying a degree of success in addressing the needs of real world applications in Rails, Django etc. It would appear that they feel no need to touch J2EE & friends. Not that these have much to do with FP, but they are like FP in that they're nothing like the "Enterprise Languages" of the present.
As for the two actual questions:
Why not just pick up whatever it is you've last been thinking to do and do it in Clojure? Obviously anything can be done in Java (and most things probably have been), but a leaner language might make the product cleaner, the experience more pleasant and less time consuming etc.
About mixing Clojure and Java -- I've seen a decent amount of Clojure code using a couple of classes coded directly in Java (for whatever reason). I've tried going the other way around myself and it's a bit of a pain in that it's much simpler to work with interface inheritance than class inheritance in Clojure, unexpected coupling in the Java code can seriously interfere with the ability of the Clojure code to do things in the most natural way etc. Still, it's entirely possible to extend a Java programme in Clojure and it seems like a particularly safe & sane way of experimenting with it for the worried Java developer.
Functional programming can be applied to almost any task. Web applications, scientific applications, games, you name it.
It is very common to mix Clojure and Java, since Clojure does not have many dedicated libraries for things like I/O or networking.
Organizations that already have a lot of Java code can use Clojure for small subsections of their Java projects.
For new projects, it is usually more effective to use Clojure as the high-level driver language, calling Java libraries where necessary.
I have been working on a small web application using Clojure, and while there is nothing special about the application that could not have been done in a different language, the experience of writing it has been completely different. I have written web apps using ASP.net and moving to Clojure was less about learning the different syntax and more about learning a different way to think and program. Having to learn a different way to think will occur regardless of the project you choose to work on, so I would worry less about finding the perfect functional project and more about finding something you just want to work on.
I think the answer to this has a great deal to do with the context your project is embedded in, and the constraints that imposes on you. Absent social factors I think Clojure is likely at least as "good" a language as Java is for any problem, with the possible exception of cases where you need the last bit of performance. And even in those cases things are not nearly as simple as they seem. For one thing some future version of Clojure can probably, in the theoretical limit, be compiled to bytecode that is as "fast" as what Java is compiled into (assuming a bit more work from the programmer at bottlenecks.) More importantly, optimization is a multi-factorial problem, and one in which programmer productivity and the flexibility of code factors heavily. So while there is a sense in which it would be accurate to say that Clojure is slower than Java, that sense might not be the important one when discussing the performance of a particular application.
So I'd say that if you disregard social factors Clojure's use cases are close to a superset of Java's. I wouldn't try to write a Linux kernel module in clojure though...
Of course, it's true that not all problems have equally natural solutions in functional languages. But people have come up with some interesting ways of dealing with some of the cases where FP seems to map badly to the domain, and anyway Clojure actually offers you enough escape hatches from pure FP that if you really feel the need to write part of your program in an imperative style you can (though of course you give up some of the benefits of Clojure in that case.) In the worst case you could use Clojure to drive the Java library in much the same way that you would in Java... it's hard to imagine a case where that would be a good idea, but in most cases that would not be markedly inferior to just using Java, and in many it might be better.
I'm still a neophyte at Clojure, though I've been programming in CL and scheme for a long time, and I spent about five years writing Java for a living. But I would probably prefer Clojure to Java for just about anything even without knowing it quite as well, as long as there were no social factors involved.
It would be a mistake to dismiss social factors though. I've been a Lisp programmer long enough to have a finely honed instinct for how well a Lisp will work in a given context. I've introduced Lisp to commercial settings where it has been a big win, and I've introduced it to settings where it really wasn't. I'd think long and hard about staking your career on successfully transitioning a team of programmers to any Lisp, Clojure included, particularly if they are not too keen on the idea.
Just to give you an idea of what I think Clojure might be useful for, I am currently writing a lot of poker-related code in Clojure. Some of it is pretty simple stuff (finding the best five card hand you can make from seven cards) and some of it is a bit more interesting (looking at someone's playing history and extracting meaningful trends from it using a few heuristics and some basic statistics.) None of it requires much in the way of Clojure's sophisticated concurrency mechanisms, but it is still much nicer (for me at least) in Clojure than it would be in, say, Java.
There are certainly some other cases that someone might describe where Clojure wins big because of its sophisticated mechanisms for managing concurrency, etc. I am aiming at something more modest- I am just pointing out that even if you don't need those mechanisms you might find Clojure a very congenial language for general purpose programming, albeit one that requires you to rethink how you abstract things if you're coming from an imperative/OO background. And hey, if you need the concurrency mechanisms (as you might, the way things are going), at least you already know Clojure.
I like writing game programs when I learn a new language.
I am in the process of learning Clojure and started writing a Spider solitaire player. If you have never played Spider, don't start; it is very additive :-). See http://www.spidersolitaire.org/.
In writing this game, I am getting to use several things that I want to learn: functional programming, concurrency, Java-interop (for Swing), etc.
I have also started writing a Bejeweled player (http://www.popcap.com/games/free/bejeweled2), but have run into a problem finding the definitive rules for scoring the game.