I was curious so I checked source code of swap! function on clojure repository, and it was like this:
(defn swap!
"Atomically swaps the value of atom to be:
(apply f current-value-of-atom args). Note that f may be called
multiple times, and thus should be free of side effects. Returns
the value that was swapped in."
{:added "1.0"
:static true}
([^clojure.lang.Atom atom f] (.swap atom f))
([^clojure.lang.Atom atom f x] (.swap atom f x))
([^clojure.lang.Atom atom f x y] (.swap atom f x y))
([^clojure.lang.Atom atom f x y & args] (.swap atom f x y args)))
And I don't know what the ".swap" function is doing? I tried to search for it but it's not defined in the same file, so can't find definition of it. Or is this another special thing that is actually not a function at all?
A lot of things in Clojure are actually implemented in Java, like reference types (atom, ref, var, agent), data structures (map, vector, list, set), namespaces (the actual Namespace class) and other stuff. When reading the source code for Clojure functions in clojure.core it's not rare to find an interop call to a Java method.
It is my understanding that there's a long term objective to implement these things in Clojure itself (search here for Clojure-in-Clojure), but for now these things are implemented in Java, which is really not so bad since the whole language is open source and you can check any implementation detail online in the github repo (already linked in a comment): Clojure (note that there's a jvm and a clj folder indicating in what language the code inside is implemented).
In Clojure, an expression (.x object a b c) results in a method call, which in Java would be expressed as object.x(a, b, c). For more details see Java Interop.
In this particular case, the swap! function calls an appropriately overloaded swap method of the clojure.lang.Atom instance passed as the first argument. The swap method contains the actual logic performing the swap.
Simply put, an atom is sth. that is under control of transactional memory. (Read about STM, please.) It is inmutable for its observers (dereferencing an atom returns inmutable state as of the instant of deref), but can be mutated transactionally. This is exactly what swap! does. It transactionally swaps the old value for the new value. Please note that it is very different from assignment. Assignment is not safe for concurrency.
In other words, atom works like a cell in a database table. When you query it, you will have a value, not an exception, even if at the same instant another query is updating it.
Cheers -
Related
Macros do not evaluate their arguments until explicitly told to do so, however functions do. In the following code:
(defmacro foo [xs]
(println xs (type xs)) ;; unquoted list
(blah xs))
(defn blah [xs] ;; xs is unquoted list, yet not evaluated
(println xs)
xs)
(foo (+ 1 2 3))
It seems that blah does not evaluate xs, since we still have the entire list: (+ 1 2 3) bound to xs in the body of blah.
I have basically just memorized this interaction between helper functions within macros and their evaluation of arguments, but to be honest it goes against what my instincts are (that xs would be evaluated before entering the body since function arguments are always evaluated).
My thinking was basically: "ok, in this macro body I have xs as the unevaluated list, but if I call a function with xs from within the macro it should evaluate that list".
Clearly I have an embarassingly fundamental misunderstanding here of how things work. What am I missing in my interpretation? How does the evaluation actually occur?
EDIT
I've thought on this a bit more and it seems to me that maybe viewing macro arguments as "implicitly quoted" would solve some confusion on my part.
I think I just got mixed up in the various terminologies, but given that quoted forms are synonymous with unevaluated forms, and given macro arguments are unevaluated, they are implicitly quoted.
So in my above examples, saying xs is unquoted is somewhat misleading. For example, this macro:
(defmacro bluh [xs]
`(+ 1 2 ~xs))
Is basically the same as the below macro (excluding namespacing on the symbols). Resolving xs in the call to list gives back an unevaluated (quoted?) list.
(defmacro bleh [xs]
(list '+ '1 '2 xs)) ;; xs resolves to a quoted list (or effectively quoted)
Calling bleh (or bluh) is the same as saying:
(list '+ '1 '2 '(+ 1 2 3))
;; => (+ 1 2 (+ 1 2 3))
If xs did not resolve to a quoted list, then we would end up with:
(list '+ '1 '2 (+ 1 2 3))
;; => (+ 1 2 6)
So, in short, macro arguments are quoted.
I thnk part of my confusion came from thinking about the syntax quoted forms as templates with slots filled in e.g. (+ 1 2 ~xs) I would mentally expand to (+ 1 2 (+ 1 2 3)), and seeing that (+ 1 2 3) was not quoted in that expansion, I found it confusing that function calls using xs (in the first example above blah) would not evalute immediately to 6.
The template metaphor is helpful, but if I instead look at it as a
shortcut for (list '+ '1 '2 xs) it becomes obvious that xs must be a quoted list otherwise the expansion would include 6 and not the entire list.
I'm not sure why I found this so confusing... have I got this right or did I just go down the wrong path entirely?
[This answer is an attempt to explain why macros and functions which don't evaluate their arguments are different things. I believe this applies to macros in Clojure but I am not an expert on Clojure. It's also much too long, sorry.]
I think you are confused between what Lisp calls macros and a construct which modern Lisps don't have but which used to be called FEXPRs.
There are two interesting, different, things you might want:
functions which, when called, do not immediately evaluate their arguments;
syntax transformers, which are called macros in Lisp.
I'll deal with them in order.
Functions which do not immediately evaluate their arguments
In a conventional Lisp, a form like (f x y ...), where f is a function, will:
establish that f is a function and not some special thing;
get the function corresponding to f and evaluate x, y, & the rest of the arguments in some order specified by the language (which may be 'in an unspecified order');
call f with the results of evaluating the arguments.
Step (1) is needed initially because f might be a special thing (like, say if, or quote), and it might be that the function definition is retrieved in (1) as well: all of this, as well as the order that things happen in in (2) is something the language needs to define (or, in the case of Scheme say, leave explicitly undefined).
This ordering, and in particular the ordering of (2) & (3) is known as applicative order or eager evaluation (I'll call it applicative order below).
But there are other possibilities. One such is that the arguments are not evaluated: the function is called, and only when the values of the arguments are needed are they evaluated. There are two approaches to doing this.
The first approach is to define the language so that all functions work this way. This is called lazy evaluation or normal order evaluation (I'll call it normal order below). In a normal order language function arguments are evaluated, by magic, at the point they are needed. If they are never needed then they may never be evaluated at all. So in such a language (I am inventing the syntax for function definition here so as not to commit CL or Clojure or anything else):
(def foo (x y z)
(if x y z))
Only one of y or z will be evaluated in a call to foo.
In a normal order language you don't need to explicitly care about when things get evaluated: the language makes sure that they are evaluated by the time they're needed.
Normal order languages seem like they'd be an obvious win, but they tend to be quite hard to work with, I think. There are two problems, one obvious and one less so:
side-effects happen in a less predictable order than they do in applicative order languages and may not happen at all, so people used to writing in an imperative style (which is most people) find them hard to deal with;
even side-effect-free code can behave differently than in an applicative order language.
The side-effect problem could be treated as a non-problem: we all know that code with side-effects is bad, right, so who cares about that? But even without side-effects things are different. For instance here's a definition of the Y combinator in a normal order language (this is kind of a very austere, normal order subset of Scheme):
(define Y
((λ (y)
(λ (f)
(f ((y y) f))))
(λ (y)
(λ (f)
(f ((y y) f))))))
If you try to use this version of Y in an applicative order language -- like ordinary Scheme -- it will loop for ever. Here's the applicative order version of Y:
(define Y
((λ (y)
(λ (f)
(f (λ (x)
(((y y) f) x)))))
(λ (y)
(λ (f)
(f (λ (x)
(((y y) f) x)))))))
You can see it's kind of the same, but there are extra λs in there which essentially 'lazify' the evaluation to stop it looping.
The second approach to normal order evaluation is to have a language which is mostly applicative order but in which there is some special mechanism for defining functions which don't evaluate their arguments. In this case there often would need to be some special mechanism for saying, in the body of the function, 'now I want the value of this argument'. Historically such things were called FEXPRs, and they existed in some very old Lisp implementations: Lisp 1.5 had them, and I think that both MACLISP & InterLisp had them as well.
In an applicative order language with FEXPRs, you need somehow to be able to say 'now I want to evaluate this thing', and I think this is the problem are running up against: at what point does the thing decide to evaluate the arguments? Well, in a really old Lisp which is purely dynamically scoped there's a disgusting hack to do this: when defining a FEXPR you can just pass in the source of the argument and then, when you want its value, you just call EVAL on it. That's just a terrible implementation because it means that FEXPRs can never really be compiled properly, and you have to use dynamic scope so variables can never really be compiled away. But this is how some (all?) early implementations did it.
But this implementation of FEXPRs allows an amazing hack: if you have a FEXPR which has been given the source of its arguments, and you know that this is how FEXPRs work, then, well, it can manipulate that source before calling EVAL on it: it can call EVAL on something derived from the source instead. And, in fact, the 'source' it gets given doesn't even need to be strictly legal Lisp at all: it can be something which the FEXPR knows how to manipulate to make something that is. That means you can, all of a sudden, extend the syntax of the language in pretty general ways. But the cost of being able to do that is that you can't compile any of this: the syntax you construct has to be interpreted at runtime, and the transformation happens each time the FEXPR is called.
Syntax transformers: macros
So, rather than use FEXPRs, you can do something else: you could change the way that evaluation works so that, before anything else happens, there is a stage during which the code is walked over and possibly transformed into some other code (simpler code, perhaps). And this need happen only once: once the code has been transformed, then the resulting thing can be stashed somewhere, and the transformation doesn't need to happen again. So the process now looks like this:
code is read in and structure built from it;
this initial structure is possibly transformed into other structure;
(the resulting structure is possibly compiled);
the resulting structure, or the result of compiling it is evaluated, probably many times.
So now the process of evaluation is divided into several 'times', which don't overlap (or don't overlap for a particular definition):
read time is when the initial structure is built;
macroexpansion time is when it is transformed;
compile time (which may not happen) is when the resulting thing is compiled;
evaluation time is when it is evaluated.
Well, compilers for all languages probably do something like this: before actually turning your source code into something that the machine understands they will do all sorts of source-to-source transformations. But these things are in the guts of the compiler and are operating on some representation of the source which is idiosyncratic to that compiler and not defined by the language.
Lisp opens this process to users. The language has two features which make this possible:
the structure that is created from source code once it has been read is defined by the language and the language has a rich set of tools for manipulating this structure;
the structure created is rather 'low commitment' or austere -- it does not particularly predispose you to any interpretation in many cases.
As an example of the second point, consider (in "my.file"): that's a function call of a function called in, right? Well, may be: (with-open-file (in "my.file") ...) almost certainly is not a function call, but binding in to a filehandle.
Because of these two features of the language (and in fact some others I won't go into) Lisp can do a wonderful thing: it can let users of the language write these syntax-transforming functions -- macros -- in portable Lisp.
The only thing that remains is to decide how these macros should be notated in source code. And the answer is the same way as functions are: when you define some macro m you use it just as (m ...) (some Lisps support more general things, such as CL's symbol macros. At macroexpansion time -- after the program is read but before it is (compiled and) run -- the system walks over the structure of the program looking for things which have macro definitions: when it finds them it calls the function corresponding to the macro with the source code specified by its arguments, and the macro returns some other chunk of source code, which gets walked in turn until there are no macros left (and yes, macros can expand to code involving other macros, and even to code involving themselves). Once this process is complete then the resulting code can be (compiled and) run.
So although macro look like function calls in the code, they are not just functions which don't evaluate their arguments, like FEXPRs were: instead they are functions which take a bit of Lisp source code and return another bit of Lisp source code: they're syntax transformers, or function which operate on source code (syntax) and return other source code. Macros run at macroexpansion time which is properly before evaluation time (see above).
So, in fact macros are functions, written in Lisp, and the functions they call evaluate their arguments perfectly conventionally: everything is perfectly ordinary. But the arguments to macros are programs (or the syntax of programs represented as Lisp objects of some kind) and their results are (the syntax of) other programs. Macros are functions at the meta-level, if you like. So a macro if a function which computes (parts of) programs: those programs may later themselves be run (perhaps much later, perhaps never) at which point the evaluation rules will be applied to them. But at the point a macro is called what it's dealing with is just the syntax of programs, not evaluating parts of that syntax.
So, I think your mental model is that macros are something like FEXPRs in which case the 'how does the argument get evaluated' question is an obvious thing to ask. But they're not: they're functions which compute programs, and they run properly before the program they compute is run.
Sorry this answer has been so long and rambling.
What happened to FEXPRs?
FEXPRs were always pretty problematic. For instance what should (apply f ...) do? Since f might be a FEXPR, but this can't generally be known until runtime it's quite hard to know what the right thing to do is.
So I think that two things happened:
in the cases where people really wanted normal order languages, they implemented those, and for those languages the evaluation rules dealt with the problems FEXPRs were trying to deal with;
in applicative order languages then if you want to not evaluate some argument you now do it by explicitly saying that using constructs such as delay to construct a 'promise' and force to force evaluation of a promise -- because the semantics of the languages improved it became possible to implement promises entirely in the language (CL does not have promises, but implementing them is essentially trivial).
Is the history I've described correct?
I don't know: I think it may be but it may also be a rational reconstruction. I certainly, in very old programs in very old Lisps, have seen FEXPRs being used the way I describe. I think Kent Pitman's paper, Special Forms in Lisp may have some of the history: I've read it in the past but had forgotten about it until just now.
A macro definition is a definition of a function that transforms code. The input for the macro function are the forms in the macro call. The return value of the macro function will be treated as code inserted where the macro form was. Clojure code is made of Clojure data structures (mostly lists, vectors, and maps).
In your foo macro, you define the macro function to return whatever blah did to your code. Since blah is (almost) the identity function, it just returns whatever was its input.
What is happening in your case is the following:
The string "(foo (+ 1 2 3))" is read, producing a nested list with two symbols and three integers: (foo (+ 1 2 3)).
The foo symbol is resolved to the macro foo.
The macro function foo is invoked with its argument xs bound to the list (+ 1 2 3).
The macro function (prints and then) calls the function blah with the list.
blah (prints and then) returns that list.
The macro function returns the list.
The macro is thus “expanded” to (+ 1 2 3).
The symbol + is resolved to the addition function.
The addition function is called with three arguments.
The addition function returns their sum.
If you wanted the macro foo to expand to a call to blah, you need to return such a form. Clojure provides a templating convenience syntax using backquote, so that you do not have to use list etc. to build the code:
(defmacro foo [xs]
`(blah ~xs))
which is like:
(defmacro foo [xs]
(list 'blah xs))
Clojure and Scala both have composing functions/methods (comp and .compose respectively). Scala has a method .andThen which works the same way as .compose, except it nests the function application in the opposite order. Does an analog of this function exist in Clojure?
I could write it easily enough myself for example like this:
(defn and-then [& fns] (apply comp (reverse fns))), however it seems basic enough that it must already exist, especially since in Clojure it has natural application in converting threading-macro expressions into point-free functions, e.g. the following two expressions would be equivalent in the unary case: #(->> % a b c) and (and-then a b c)
I don't think there's anything built-in, likely because clojure does not have an infix syntax (i.e. (and-then f g) is not as clear as f andThen g). As you already recognized -> and ->> are what's generally used in it's place.
No, there is no such built-in functionality in Clojure. The code example you provided is good enough I believe.
I'm just getting started with Clojure and I have no fp experience but the first thing that I've noticed is a heavy emphasis on immutability. I'm a bit confused by the emphasis, however. It looks like you can re-def global variables easily, essentially giving you a way to change state. The most significant difference that I can see is that function arguments are passed by value and can't be re-def(ined) within the function. Here's a repl snippet that shows what I mean:
towers.core=> (def a "The initial string")
#'towers.core/a
towers.core=> a
"The initial string"
towers.core=> (defn mod_a [aStr]
#_=> (prn aStr)
#_=> (prn a)
#_=> (def aStr "A different string")
#_=> (def a "A More Different string")
#_=> (prn aStr)
#_=> (prn a))
#'towers.core/mod_a
towers.core=> a
"The initial string"
towers.core=> (mod_a a)
"The initial string"
"The initial string"
"The initial string"
"A More Different string"
nil
towers.core=> a
"A More Different string"
If I begin my understanding of immutability in clojure by thinking of it as pass-by-value, what am I missing?
Call-by-value and immutability are two entirely distinct concepts. Indeed, one of the advantages of variable immutability is that such variables could be passed by name or reference without any effect on programme behaviour.
In short: don't think of them as linked.
generally very little is "def"d in a clojure script/class, it's mostly used for generating values that are used outside of the class. instead values are created in let bindings as you need them in your methods.
def is used to define vars, as stated in Clojure Programming:
top level functions and values are all stored in vars, which are
defined within the current namespace using the def special form or one
of its derivatives.
Your use of def inside a function isn't making a local variable, it's creating a new global var, and you're effectively replacing the old reference with a new one each time.
When you move onto using let, you'll see how immutability works, for instance using things like seqs which can be used over without penalty of something else having also read them (like an iteration over a list would in java for instance), e.g.
(let [myseq (seq [1 2 3 4 5])
f (first myseq)
s (second myseq)
sum (reduce + myseq)]
(println f s sum))
;; 1 2 15
As you can see, it doesn't matter that (first myseq) has "taken" an item from the sequence. because the sequence myseq is immutable, it's still the same, and unaffected by the operations on it. Also, notice that there isn't a single def in the code above, the assignment happened in the let bindings where the values myseq, f, s and sum were created (and are immutable within the rest of the sexp).
Yes, immutability is different from pass-by-value, and you've missed a couple of important details of what's going on in your examples:
value mutation versus variable re-binding. Your code exemplifies re-binding, but doesn't actually mutate values.
shadowing. Your local aStr shadows your global aStr, so you can't see the global one -- although it's still there -- so there's no difference between the effects of (def a ...) and (def aStr ...) here. You can verify that the global is created after running your function.
A final point: Clojure doesn't force you to be purely functional -- it has escape hatches, and it's up to you to use them responsibly. Rebinding variables is one of those escape hatches.
just a note that technically Java, and by extension Clojure (on the JVM) is strictly pass by value. In many cases the thing passed is a reference to a structure that others may be reading, though because it is immutable nobody will be changing out from under you. The important point being that mutability and immutability happen after you pass the reference to something so, and Marcin points out they really are distinct.
I think of much of the immutability in Clojure as residing in (most of) the built-in data structure types and (most of) the functions that allow manipulating ... uh, no, modifying ... no, really, constructing new data structures from them. There are array-like things, but you can't modify them, there are lists, but you can't modify them, there are hash maps, but you can't modify them, etc., and the standard tools for using them actually create new data structures even when they look, to a novice, as if they're performing in-place modifications. And all of that does add up to a big difference.
I have a defrecord called a bag. It behaves like a list of item to count. This is sometimes called a frequency or a census. I want to be able to do the following
(def b (bag/create [:k 1 :k2 3])
(keys bag)
=> (:k :k1)
I tried the following:
(defrecord MapBag [state]
Bag
(put-n [self item n]
(let [new-n (+ n (count self item))]
(MapBag. (assoc state item new-n))))
;... some stuff
java.util.Map
(getKeys [self] (keys state)) ;TODO TEST
Object
(toString [self]
(str ("Bag: " (:state self)))))
When I try to require it in a repl I get:
java.lang.ClassFormatError: Duplicate interface name in class file compile__stub/techne/bag/MapBag (bag.clj:12)
What is going on? How do I get a keys function on my bag? Also am I going about this the correct way by assuming clojure's keys function eventually calls getKeys on the map that is its argument?
Defrecord automatically makes sure that any record it defines participates in the ipersistentmap interface. So you can call keys on it without doing anything.
So you can define a record, and instantiate and call keys like this:
user> (defrecord rec [k1 k2])
user.rec
user> (def a-rec (rec. 1 2))
#'user/a-rec
user> (keys a-rec)
(:k1 :k2)
Your error message indicates that one of your declarations is duplicating an interface that defrecord gives you for free. I think it might actually be both.
Is there some reason why you cant just use a plain vanilla map for your purposes? With clojure, you often want to use plain vanilla data structures when you can.
Edit: if for whatever reason you don't want the ipersistentmap included, look into deftype.
Rob's answer is of course correct; I'm posting this one in response to the OP's comment on it -- perhaps it might be helpful in implementing the required functionality with deftype.
I have once written an implementation of a "default map" for Clojure, which acts just like a regular map except it returns a fixed default value when asked about a key not present inside it. The code is in this Gist.
I'm not sure if it will suit your use case directly, although you can use it to do things like
user> (:earth (assoc (DefaultMap. 0 {}) :earth 8000000000))
8000000000
user> (:mars (assoc (DefaultMap. 0 {}) :earth 8000000000))
0
More importantly, it should give you an idea of what's involved in writing this sort of thing with deftype.
Then again, it's based on clojure.core/emit-defrecord, so you might look at that part of Clojure's sources instead... It's doing a lot of things which you won't have to (because it's a function for preparing macro expansions -- there's lots of syntax-quoting and the like inside it which you have to strip away from it to use the code directly), but it is certainly the highest quality source of information possible. Here's a direct link to that point in the source for the 1.2.0 release of Clojure.
Update:
One more thing I realised might be important. If you rely on a special map-like type for implementing this sort of thing, the client might merge it into a regular map and lose the "defaulting" functionality (and indeed any other special functionality) in the process. As long as the "map-likeness" illusion maintained by your type is complete enough for it to be used as a regular map, passed to Clojure's standard function etc., I think there might not be a way around that.
So, at some level the client will probably have to know that there's some "magic" involved; if they get correct answers to queries like (:mars {...}) (with no :mars in the {...}), they'll have to remember not to merge this into a regular map (merge-ing the other way around would work fine).
In the accepted answer to another question, Setting Clojure "constants" at runtime the clojure function constantly is used.
The definition of constantly looks like so:
(defn constantly
"Returns a function that takes any number of arguments and returns x."
{:added "1.0"}
[x] (fn [& args] x))
The doc string says what it does but not why one would use it.
In the answer given in the previous question constantly is used as follows:
(declare version)
(defn -main
[& args]
(alter-var-root #'version (constantly (-> ...)))
(do-stuff))
So the function returned by constantly is directly evaluated for its result. I am confused as to how this is useful. I am probably not understanding how x would be evaluated with and without being wrapped in `constantly'.
When should I use constantly and why is it necessary?
The constantly function is useful when an API expects a function and you just want a constant. This is the case in the example provided in the question.
Most of the alter-* functions (including alter-var-root) take a function, to allow the caller to modify something based on its old value. Even if you just want the new value to be 7 (disregarding the old value), you still need to provide a function (providing just 7 will result in an attempt to evaluate it, which will fail). So you have to provide a function that just returns 7. (constantly 7) produces just this function, sparing the effort required to define it.
Edit: As to second part of the question, constantly is an ordinary function, so its argument is evaluated before the constant function is constructed. So (constantly #myref) always returns the value referenced by myref at the time constantly was called, even if it is changed later.