The quasiquoted list `(1 ,#2 3) is invalid because 2 is not a list. However, `(1 2 ,#3) is valid and will return a dotted list: (1 2 . 3). I observe this result in Common Lisp and Scheme. Why is it possible to use unquote-splicing for non-lists at the end of a quasiquoted list? Why is the result a dotted list?
The expression `(1 2 ,#3) is not valid in either Scheme or Common Lisp.
Scheme
In R6RS Scheme (and similarly for R5RS), the behavior is not specified for operating on a non-list with unquote-splicing. The R6RS Scheme Standard requires (11.17 Quasiquotation):
If an (unquote-splicing <expression> ...) form appears inside a <qq
template>, then the <expression>s must evaluate to lists....
Common Lisp
The Common Lisp HyperSpec says first that (2.4.6 Backquote):
If a comma is immediately followed by an at-sign, then the form
following the at-sign is evaluated to produce a list of objects.
These objects are then "spliced" into place in the template.
In the sub-expression ,#3, 3 does not evaluate to a list. This seems a pretty strong argument that the expression is not valid. Even if the 3 were magically placed in a list before splicing, this would not result in a dotted list. The HyperSpec goes on to provide a formal summary of backquote syntax. The part of interest is:
`(x1 x2 x3 ... xn . atom) may be interpreted to mean
(append [ x1] [ x2] [ x3] ... [ xn] (quote atom))
where the brackets are used to indicate a transformation of an xj as follows:
-- [form] is interpreted as (list `form), which contains a backquoted form that must then be further interpreted.
-- [,form] is interpreted as (list form).
-- [,#form] is interpreted as form.
So in Common Lisp the original expression, which is equivalent to `(1 2 ,#3 . nil), can be interpreted as:
(append (list `1) (list `2) 3 (quote nil))
But, this is not a valid call to append, which requires proper lists for all arguments except the last one. So, there seems to be no support for the idea that the original expression was valid.
The fact that it worked for the OP in both Scheme and Common Lisp probably comes down to similar definitions for the backquote macro across different implementations. These definitions all seem to expect that the form following ,# will evaluate to a list; when that is not the case, the observed behavior (production of a dotted list) can't be relied upon, according to the standard. That said, I tested Chez Scheme, Guile Scheme, MIT Scheme, SBCL, CCL, and CLisp: all of them exhibited the same behavior reported by OP.
An Interesting Case
I also tested against an implementation of the backquote macro by Guy Steele and published in CLTL2. This case is more interesting. This backquote implementation in CLTL2 is meant for exploring the behavior of backquote expressions and has an optional code simplification phase. Here $ corresponds to a backquote, and %# corresponds to ,#. Without code simplification, the result of expanding the original expression is:
CL-USER> (setf *bq-simplify* nil)
NIL
CL-USER> (try '("$(1 2 %#3)"))
`(1 2 ,#3) = (APPEND (LIST '1) (LIST '2) 3 'NIL)
This corresponds to the expression which was expected above from reading the description in the HyperSpec. But note that this expression will not compile:
CL-USER> (append (list 1) (list 2) 3 nil)
The value
3
is not of type
LIST
[Condition of type TYPE-ERROR]
But, when code simplification is turned on:
CL-USER> (setf *bq-simplify* t)
T
CL-USER> (try '("$(1 2 %#3)"))
`(1 2 ,#3) = (LIST* '1 '2 3)
This "simplified" expression is valid, and evaluates to a dotted list:
CL-USER> (list* 1 2 3)
(1 2 . 3)
My conclusion is that the expression following ,# must be a list per the Common Lisp standard, but some common implementations either do some form of code simplification similar to what is shown in CLTL2 or otherwise expand the backquote form in such a way that it appears that a non-list form can follow ,#. Don't rely on this, as it is difficult to say when it won't work.
Proper lists
Other answers detail this part already, let me rephrase it quickly: lists are either proper lists or improper lists; improper lists are either ciruclar lists or dotted lists.
In particular, a dotted-list is a non-circular list whose last cons cell has a non-list cdr slot.
Some functions are expected to work only when given proper lists, other have no such restrictions, typically because checking for this property is not without costs.
Writing (cons 1 2) creates such a list. Now, if the splicing syntax expands as (cons x y) where y is a non-list, you'll have a dotted-list too.
Slicing
Chapter 2.4.6 Backquote states (emphasis mine):
If a comma is immediately followed by an at-sign, then the form following the at-sign is evaluated to produce a list of objects.
I see no other note indicating that splicing a non-list value might be part of a conforming program, even if it results in practice to an improper list when splicing occurs at the end of the list.
The section also states that:
`((,a b) ,c ,#d)
will be interpreted as if it were
(append (list (append (list a) (list 'b) 'nil)) (list c) d 'nil)
but it could also be legitimately interpreted to mean any of the following:
(append (list (append (list a) (list 'b))) (list c) d)
(append (list (append (list a) '(b))) (list c) d)
(list* (cons a '(b)) c d)
(list* (cons a (list 'b)) c d)
(append (list (cons a '(b))) (list c) d)
(list* (cons a '(b)) c (copy-list d))
Expanding as calls to standard functions like list* or append make that corner case naturally produce improper lists, but notice that the first and last example do not allow splicing for the last element. For instance:
(list* (cons a '(b)) c (copy-list d))
The above form complains when d is not a list, because COPY-LIST only works with lists (either proper or dotted, but here it would be given a non-list value, e.g. a number).
It is thus my understanding that this expression:
`(1 2 ,#3)
is in fact invalid in Common Lisp and happens to works only by accident. Writing it could expose you to portability problems.
Conclusion
Why is it possible to use unquote-splicing for non-lists at the end of a quasiquoted list?
It happens by chance in your implementation because backquotes, unquotes and splicing are rewritten/expanded as list building functions that may produce dotted lists in that corner case, without signaling an error.
Why is the result a dotted list?
Because the code probably is interpreted as a call to append where the last item is a non-list value, which results in a dotted list. It does not work when splicing in the middle of a form, because append expects proper lists for all but the last arguments.
Bonus
Today I learned about ,. which is like ,# except is may use nconc instead of append
To construct a list of 3 variables with quasiquoting, one could write
`(,x ,y ,z)
This can be desugared to
(cons x (cons y (cons z nil)))
What if z is a list, and we want to splice its contents into the resulting list? Then instead of creating a new cons for it, we simply put it at the tail. When we write
`(,x ,y ,#z)
this desugars to
(cons x (cons y z))
If z happens to not actually be a list, then the result is a legal value, although not a proper list. For example, (1 2 . 3), as you observed. You will see the same result if you write the desugared form explicitly, as (cons 1 (cons 2 3)). If this also troubles you, you may want to investigate the concept of improper lists in general, or dotted pair notation.
'(1 2 3) is actually (cons 1 (cons 2 (cons 3 nil))).
So, the last element in '(1 2 3) is not a 'non-list', but a (cons 3 nil) i.e. (list 3), and thus can be spliced.
`a ``a and ```a gives the following
user=> `a
user/a
user=> ``a
(quote user/a)
user=> ```a
(clojure.core/seq (clojure.core/concat (clojure.core/list (quote quote)) (clojure.core/list (quote user/a))))
Since `a gives user/a, I'd anticipate ``a is the same as `user/a, which is user/a.
You are missing the evaluation step.
Let's first take a look at the simple quote:
user=> 'a
a
user=> ''a
(quote a)
user=> '''a
(quote (quote a))
user=> ''''a
(quote (quote (quote a)))
and so on. This is quite simple because it doesn't need to process the quoted form in any way except read it.
'a is read as (quote a), which evaluates to the symbol a.
''a is read as '(quote a), which is read as (quote (quote a)), which evaluates to the list (quote a), which is a list of the two symbols quote and a.
And so on.
Now, what does ` do that ' doesn't? It is more or less a template mechanism that
automatically qualifies symbols
automatically handles auto-gensym
evaluates things marked by ~ (or ~#) and re-inserts the results.
This means that it cannot just produce another quote form containing the following form, but has to produce an actual function call form at the call site (except in trivial cases). This form will mostly consist of sequence operations (list, cons, concat etc.) to assemble the template parts with the evaluated bits. If you then again quote that, you get to see those function call forms.
What does an underscore within the binding vector of a let do, so if i write (let [a blabla _ (println a)] etc....
what is this underline doing at the place of the keyword?
The underscore is a valid symbol identifier, as you can see by this sample code:
(let [_ 1]
(println _))
=> 1
By convention when you have an identifier you won't be using you may use _, but it's not mandatory.
Happens both for side effect situations as in your sample (in your case the println line returns nil so you won't be binding its result to a symbol)
Happens also on destructuring situations where you don't need some of the values.
(let [[a _ _ d] [1 2 3 4]]
(println a))
=> 1
In this case you're not interested in 2nd and 3rd values, so the identifier _ is idiomatic for saying you don't care.
In regular Clojure the underscore _ is treated like any other symbol (e.g. "junk"). However it has a special meaning in some libraries. For example, in Datomic the _ is treated like a "wildcard" that prevents binding/unification. In clojure.core.match, the _ is also treated as a wildcard.
See:
Datomic Query
clojure.core.match
The underscore is the universal ignore symbol.
There has to be a return value for each sexp in the let form, and println returns nil, so you can set it to _ to tell people you are ignoring the return and just doing debug at that point. e.g.
(let [foo (+ 1 2)
_ (println "debug! foo is" foo)
_ (println "more debug!" (+ 1 foo)]
foo)
each evaluation of the sexp sets the _ to the value returned, but it's not required, so just read it as such.
I am attempting to write a macro that converts "generic" expressions into expressions evaluated for a particular given value.
Let us say that I have a vector of unevaluated expressions of the form,
'[(map? %)
(-> % :value string?)
(fn[x] (apply #(pos? %) (- x 2))]
I have a gensymed value v, that I wish to "pass" to the above expression at runtime. So, assume that v will be the name of the symbol used at compile time. I also wish to assert the converted expression. Basically, for the example above, I wish to convert the above vector expression into the form below. Note that I am passing the value v to % in the first two expressions, and to x in the third expression. The code below isn't correct, but only illustrates what I want to happen. I have no idea how to actually get v into any of the below expressions assertions.
'[(assert (map? %)) ;; % is equal to v
(assert (-> % :value string?)) ;; % is equal to v
(assert (fn[x] (apply #(pos? %) (- x 2)))] ;; x is equal to v
As a guiding example, I wish to be able to do what the :post part of a prepost-map? does on a defn expression. Any help is appreciated.
Don't do this with expressions. Instead, use functions. Then it's easy to just call each of the functions with whatever value you want.
I defined an unless macro as follows:
user=> (defmacro unless [expr body] (list 'if expr nil body))
#'user/unless
user=> (unless (= 1 2) (println "Yo"))
Yo
As you can see it works fine.
Now, in Clojure a list can be defined in two ways:
; create a list
(list 1 2 3)
; shorter notation
'(1 2 3)
This means that the unless macro can be written without the list keyword. However, this results in a Java exception being thrown:
user=> (unless (= 1 2) (println "Yo"))
java.lang.Exception: Unable to resolve symbol: expr in this context
Can someone explain why this fails?
'(foo bar baz) is not a shortcut for (list foo bar baz), it's a shortcut for (quote (foo bar baz)). While the list version will return a list containing the values of the variables foo, bar and baz, the version with ' will return a list containing the symbols foo, bar and baz. (In other words '(if expr nil body) is the same as (list 'if 'expr 'nil 'body).
This leads to an error because with the quoted version the macro expands to (if expr nil body) instead of (if (= 1 2) nil (println "Yo")) (because instead of substituting the macro's arguments for expr and body, it just returns the name expr and body (which are then seen as non-existent variables in the expanded code).
A shortcut that's useful in macro definitions is using `. ` works like ' (i.e. it quotes the expression following it), but it allows you to evaluate some subexpressions unquoted by using ~. For example your macro could be rewritten as (defmacro unless [expr body] `(if ~expr nil ~body)). The important thing here is that expr and body are unquoted with ~. This way the expansion will contain their values instead of literally containing the names expr and body.