My understanding is Clojure's homoiconicity exists so as to make writing macros easier.
Based on this stackoverflow thread, it looks like Macros are used sparingly, except for DSLs in which higher-order functions are not to be used.
Could someone share some examples of how macros are used in real-life?
It's correct that homoiconicity makes writing Clojure macros very easy. Basically they enable you to write code that builds whatever code you want, exploiting the "code is data" philosophy of Lisp.
Macros in a homoiconic language are also extremely powerful. There's a fun example presentation I found where they implement a LINQ-like query syntax in just three lines of Clojure.
In general, Clojure macros are potentially useful for many reasons:
Control structures - it's possible to create certain control structures using macros that can never be represented as functions. For example, you can't write if as a function, because if it was a function then it would have to evaluate all three arguments, whereas with a macro you can make it only evaluate two (the condition value and either the true or false expression)
Compile time optimisation - sometimes you want to optimise your code based on a constant that is known or can be computed at compile time. For example, you could create a "logging" function that logs only if the code was compiled in debug mode, but creates zero overhead in the production application.
Code generation / boilerplate elimination - if you need to produce a lot of very similar code with similar structure, then you can use macros to automatically generate these from a few parameters. If you hate boilerplate, then macros are your friends.
Creating new syntax - if you see the need for a particular piece of syntax that would be useful (perhaps encapsulating a common pattern) then you can create a macro to implement this. Some DSLs for example can be simplified with additional syntax.
Creating a new language with entirely new semantics (Credits to SK-Logic!) theoretically you could even go so far to create a new language using macros, which would effectively compile your new language down into Clojure. The new langauge would not even have to be Lisp-like: it could parse and compile arbitrary strings for example.
One important piece of advice is only use macros if you need them and functions won't work. Most problems can be solved with functions. Apart for the fact that macros are overkill for simple cases, functions have some intrinsic advantages: they are more flexible, can be stored in data structures, can be passed as parameters to higher order functions, are a bit easier for people to understand etc.
Related
I have been trying to think more about what abstraction actually means in functional programming. The very best post I have found that talks the type of language I can understand is the following 4 abstractions. However since I'm a wannabe Clojure programmer I'm wondering what sort of abstractions macros provide. It seems that they fit in stage 2 together with HOF but at the same time they are more then a HOF. I find stage 3 to be related to the Expression problem and would be protocol and multi methods in Clojure. So my question is:
When implementing a macro in a Lisp language what would you say you are abstracting over?
What would stage 3 and 4 be in a Lisp language?
I don't really view macros as an abstraction, but more as a compiler hook.
Most languages implement what is known as an Abstract Syntax Tree (or AST). This is a representation of the code of a program in a sort of data structure. Lisp macros expose parts of this AST as data that can be transformed via a macro function. But since lisp programs are themselves data structures, macros tend to be a bit cleaner in lisp programs then they would be in Rust or Scala.
So one could say that macros are simply abstractions of language semantics...but I don't know that I agree with that. One could say that macros are extensions of the lisp compiler, but that's not exactly true either.
As it turns out, macros are quite limited. They can only see a small subsection of the code being compiled. In other words, a macro can't see up the tree, only down. In addition while macros that perform deep inspection of children in the AST are possible (known as deep walking macros) these macros tend to be complex and error prone (just look at the guts of core.async's go or the contents of midje to see how complex these can get). So I hesitate to call them abstractions, perhaps they are, perhaps they are just very limited abstractions.
So I see macros as a weird mix between the more powerful Fexprs (http://en.wikipedia.org/wiki/Fexpr) and the more complete compiler code transforms found in projects like LLVM. They provide a very limited controlled way to transform code at compile time, that's about it.
And in the end it all comes down to the lisp mantra that "code is data is code". If your code is data it makes sense to provide ways to transform it at compile time.
For example pattern matching is a programming language feature that can be added to the clojure language through macros: http://www.brool.com/index.php/pattern-matching-in-clojure
What other language features can be added to the language?
Off the top of my hat I have two examples, but I'm sure there are more.
Contracts programming: https://github.com/fogus/trammel
Declarative logic: https://github.com/jduey/mini-kanren
I think its a stupid question to ask what can be added, what you should ask is what you cant add. Macros allow you to hook into the compiler that mean you can do almost anything.
At the moment you cant add your own syntax to the language. Clojure does not have a user extenseble reader, this means you don't have any reader-macros (http://dorophone.blogspot.com/2008/03/common-lisp-reader-macros-simple.html). This is not because of a technical problem but more a decition by Rich Hickey (the Clojure creator).
What you can not do is implement features that need virtual maschine support like add tail call semantics or goto.
If you want to see some stuff that has been done: Are there any Clojure DSLs?
Note that this list is not 100% up to date.
Edit:
Since you seem you took pattern matching as an example (it is a really good example for the power of macros) you should really look at the match library. Its probebly the best fastest pattern matching library in Clojure. http://vimeo.com/27860102
You can effectively add any language features you like.
This follows from the ability of macros to construct arbitrary code at compile time: as long as you can figure out what code you need to generate in order to implement your language features, it can be achieved with macros.
Some examples I've seen:
Query languages (Korma)
Logic programming (core.logic)
Image synthesis DSL (clisk)
Infix notation for arithmetic
Algebraic manipulation
Declarative definition of realtime data flows (Storm, Aleph)
Music programming (Overtone, Music As Data)
There are a few caveats:
If the feature isn't supported directly by the JVM (e.g. tail call optimisation in the mutually recursive case) then you'll have to emulate it. Not a big deal, but may have some performance impact.
If the feature requires a syntax not supported by the Clojure reader, you'll need to provide your own reader (since Clojure lacks an extensible reader at present). As a result, it's much easier if you stick to Clojure syntax/forms.
If you do anything too unusual / unidiomatic, it probably won't get picked up by others. There is a lot of value in sticking to standard Clojure conventions.
Beware of using macros where they are not needed. Often, just using normal functions (perhaps higher order functions) is sufficient to implement many new language features. The general rule is: "don't use macros unless you absolutely need to".
I want to ask what sort of type safety languages constructs are there on Clojure?
I've read 'Practical Clojure' from Luke VanderHart and Stuart Sierra several times now, but i still have the distinct impression that Clojure (like other lisps) don't take compilation-time validation checking very seriously. Type safety is just but one (very popular) strategy for doing compilation-time checking of correct semantics
I'm asking this question because i'm aching to be proven wrong; what sort of design patterns are there available on clojure to validate (at compilation-time, not at run-time) that a function that expects a string doesn't get called with, say, a list of integers?
Also, i've read very smart people like Paul Graham openly advocate about lisp allowing to implement everything from lower-level languages on top of it (most would say that the language themselves are being reimplemented on top of it), so if that assertion would be true, then trivially stuff like type checking should be a piece of cake. So do you feel that there exist type systems (or the ability to implement such type systems) in clojure or other lisps, that give the programmer the ability to offset validation checking from run-time to compile-time, or even better, design-time?
Compilation units in Clojure are very small - a single function. Lispers tend to change small portions of running programs while they develop. Introducing static type checking into this style of development is problematic - for a deeper discussion why I recommend the post Types are Anti-Modular by Gilad Bracha. Thus Clojure's prefers pre/post-conditions which jive better with Lisp's highly REPL-oriented development.
That said, it's certainly desirable and possible to build an a la carte type system for Clojure. This trail has been blazed by Qi/Shen, and Typed Racket. This functionality could be easily provided as a library. I'm hoping to build something like that in the future with core.logic - https://github.com/clojure/core.logic.
Since Clojure is a dynamic language the whole idea is not to check the types (or much of anything) at compile time.
Even when you add type hints to your function they do not get checked at compile-time.
Since Clojure is a Lisp you can do whatever you want at compile-time with macros and macros are powerful enough that you can write your own type systems. Some people have made type systems for lisps Typed Racket and Qi. These Type systems can be just as powerful as any Type system in a "normal" language.
Ok, we now know that it is possible but does Clojure has such a optional type system? The answer is currently no but there is a logic engine (core.logic) that could be used to implement a typesystem but the author has not worked (yet) in that direction.
There is a library that adds an optional type system to Clojure,
http://typedclojure.org/
Rationale
Static typing has well known benefits. For example, statically typed languages catch many common programming errors at the earliest time possible: compile time. Types also serve as an excellent form of (machine checkable) documentation that almost always augment existing hand-written documentation.
Languages without static type checking (dynamically typed) bring other benefits. Without the strict rigidity of mandatory static typing, they can provide more flexible and forgiving idioms that can help in rapid prototyping. Often the benefits of static type checking are desired as the program grows.
This work adds static type checking (and some of its benefits) to Clojure, a dynamically typed language, while still preserving idioms that characterise the language. It allows static and dynamically typed code to be mixed so the programmer can use whichever is more appropriate.
I wondered if there are any simpler or more powerful syntax for C or C++. I have already come across SPECS. That is an alternative syntax for C++. But are there any others and what about C?
It could also be a sort of code generator so that things like functors could be defined less verbosely. I imagine it could be made as a code generator that compiles to C or C++ code which is very similar to the code you wrote in the alternative syntax.
Mirah is an example of doing this for Java.
Ideally I would want to write C in Go like syntax. I like how they fixed switch-case, and in general made everything much less verbose.
#define BEGIN {
#define END }
No! Just say NO!
The only general-purpose tool that I'm aware of is Lazy C++, which lets you create a single .lzz source file from which it can generate the .h and .cpp files.
There are also numerous approaches to doing code generation for C++. (For examples, see Cog, Pump, or Wikipedia's list.) These aren't full-fledged alternate syntaxes, but they can help with particular categories of syntax (such as automatically generating templates taking 1 to N arguments, to work around the lack of variadic templates).
Instead of a change in syntax, consider a change in abstraction: Increase your abstraction with a custom-defined DSL. Tool support would be necessary to reach optimal productivity.
If your goal is simplification, a lightweight modeling approach, either text-based (like XText), graph-based (like MetaEdit+) or tree-based (like AtomWeaver) would remove some complexity on the project by simplifying the solution.
If it is only a syntax you're after, why can't you define your own, as a trivial preprocessor->parser->C-pretty-printer chain? It will be no more than a semantically reach preprocessor, something of a CamlP4 style, but for C. No one but you knows what kind of syntax you'd find suitable, so its implementation is entirely up to you.
It doesn't look to me like SPECS is really C++ anymore, I certainly would have a hard time reading such code (at least initially).
You should pick a language based on your needs, not pick a specific language and then modify it to fit what you want to do.
If you want to program Go, then program in Go, don't try to write C in a Go-like syntax as that'll just make it hard for anyone who actually knows C to read your code.
I've written a handful of basic 2D shooter games, and they work great, as far as they go. To build upon my programming knowledge, I've decided that I would like to extend my game using a simple scripting language to control some objects. The purpose is more about the general process of design of writing a script parser / executer than the actual control of random objects.
So, my current line of thought is to make use of a container of lambda expressions (probably a map). As the parser reads each line, it will determine the type of expression. Then, once it has decided the type of instruction and discovered whatever values it has to work with, it will then open the map to the kind of expression and pass it any values it needs to work.
A more-or-less pseudo code example would be like this:
//We have determined somehow or another that this is an assignment operator
someContainerOfFunctions["assignment"](whatever_variable_we_want);
So, what do you guys think of a design like this?
Not to discourage you, but I think you would get more out of embedding something like Squirrel or Lua into your project and learning to use the API and the language itself. The upside of this is that you'll have good performance without having to think about the implementation.
Implementing scripting languages (even basic ones) from scratch is quite a task, especially when you haven't done one before.
To be honest: I don't think it's a good idea as you described, but does have potential.
This limits you with an 'annoying' burden of C++'s static number of arguments, which is may or may not what you want in your language.
Imagine this - you want to represent a function:
VM::allFunctions["functionName"](variable1);
But that function takes two arguments! How do we define a dynamic-args function? With "..." - that means stdargs.h and va_list. unfortunately, va_list has disadvantages - you have to supply an extra variable that will somehow be of an information to you of how many variables are there, so we change our fictional function call to:
VM::allFunctions["functionName"](1, variable1);
VM::allFunctions["functionWithtwoArgs"](2, variable1, variable2);
That brings you to a new problem - During runtime, there is no way to pass multiple arguments! so we will have to combine those arguments into something that can be defined and used during runtime, let's define it (hypothetically) as
typedef std::vector<Variable* > VariableList;
And our call is now:
VM::allFunctions["functionName"](varList);
VM::allFunctions["functionWithtwoArgs"](varList);
Now we get into 'scopes' - You cannot 'execute' a function without a scope - especially in embedded scripting languages where you can have several virtual machines (sandboxing, etc...), so we'll have to have a Scope type, and that changes the hypothetical call to:
currentVM->allFunctions["functionName_twoArgs"].call(varList, currentVM->currentScope);
I could continue on and on, but I think you get the point of my answer - C++ doesn't like dynamic languages, and it would most likely not change to fit it, as it will most likely change the ABI as well.
Hopefully this will take you to the right direction.
You might find value in Greg Rosenblatt's series of articles of at GameDev.net on creating a scripting engine in C++ ( http://www.gamedev.net/reference/articles/article1633.asp ).
The approach he takes seems to err on the side of minimalism and thus may be either a close fit or a good source of implementation ideas.