I am trying to understand this function from clojuredocs.org:
;; Create a word frequency map out of a large string s.
;; `s` is a long string containing a lot of words :)
(reduce #(assoc %1 %2 (inc (%1 %2 0)))
{}
(re-seq #"\w+" s))
; (This can also be done using the `frequencies` function.)
I dont understand this part: (inc (%1 %2 0))
The first argument (%1 in the anonymous function) to the function passed to reduce is the accumulator, which is initally the empty map {} passed as the second argument to reduce. Maps are functions which lookup the value for the given key, returning the optional default if the key is not found e.g.
({"word" 1} "word") = 1
and
({"word" 1} "other" 0) = 0
so
(%1 %2 0)
looks up the count for the current word (second argument to the reducing function) in the accumulator map, returning 0 if the word has not been added yet. inc increments the current count, so
#(assoc %1 %2 (inc (%1 %2 0))
increments the count of the current word in the intermediate map, or sets it to 1 if this is the first time the word has been encountered.
Here's an easier to read example of the same thing, not using anonymous function syntax:
(reduce
(fn [acc elem]
(assoc acc elem (inc (acc elem 0))))
{}
(re-seq #"\w+" "a dog a cat a dog a banana"))
=> {"a" 4, "dog" 2, "cat" 1, "banana" 1}
Here acc is the map we are building up, and elem is the current word. Let's break down (inc (acc elem 0)):
inc is going to increment the number returned from the inner expression
(acc elem 0) is going to get the current number from the acc map for the word elem, and if there is no number there it'll return 0. This is short for (get acc elem 0) -- maps are functions too and behave like the get function.
You can also achieve the same effect as (assoc acc elem (inc (acc elem 0))) with (update acc elem (fnil inc 0)).
The same logic applies when you replace the reduce function with an anonymous syntax using numbered arguments.
The code you are asking about is essentially that of the standard frequencies function with transients removed:
(defn frequencies [coll]
(reduce
(fn [counts x] (assoc counts x (inc (get counts x 0))))
{}
coll))
This
uses a fn form instead of an anonymous function literal (as does
Taylor Wood above) and
includes a superfluous get, which gives us a clue to the default
0 value.
Related
(defn shuffle-letters
[word]
(let [letters (clojure.string/split word #"")
shuffled-letters (shuffle letters)]
(clojure.string/join "" shuffled-letters)))
But if you put in "test" you can get "test" back sometimes.
How to modify the code to be sure that output will never be equal to input.
I feel embarrassing, I can solve it easily in Python, but Clojure is so different to me...
Thank you.
P.S. I thing we can close the topic now... The loop is in fact all I needed...
You can use loop. When the shuffled letters are the same as the original, recur back up to the start of the loop:
(defn shuffle-letters [word]
(let [letters (clojure.string/split word #"")]
(loop [] ; Start a loop
(let [shuffled-letters (shuffle letters)]
(if (= shuffled-letters letters) ; Check if they're equal
(recur) ; If they're equal, loop and try again
(clojure.string/join "" shuffled-letters)))))) ; Else, return the joined letters
There's many ways this could be written, but this is I think as plain as it gets. You could also get rid of the loop and make shuffle-letters itself recursive. This would lead to unnecessary work though. You could also use let-fn to create a local recursive function, but at that point, loop would likely be cleaner.
Things to note though:
Obviously, if you try to shuffle something like "H" or "HH", it will get stuck and loop forever since no amount of shuffling will cause them to differ. You could do a check ahead of time, or add a parameter to loop that limits how many times it tries.
This will actually make your shuffle less random. If you disallow it from returning the original string, you're reducing the amount of possible outputs.
The call to split is unnecessary. You can just call vec on the string:
(defn shuffle-letters [word]
(let [letters (vec word)]
(loop []
(let [shuffled-letters (shuffle letters)]
(if (= shuffled-letters letters)
(recur)
(clojure.string/join "" shuffled-letters))))))
Here's another solution (using transducers):
(defn shuffle-strict [s]
(let [letters (seq s)
xform (comp (map clojure.string/join)
(filter (fn[v] (not= v s))))]
(when (> (count (into #{} letters)) 1)
(first (eduction xform (iterate shuffle letters))))))
(for [_ (range 20)]
(shuffle-strict "test"))
;; => ("etts" "etts" "stte" "etts" "sett" "tste" "tste" "sett" "ttse" "sett" "ttse" "tset" "stte" "ttes" "ttes" "stte" "stte" "etts" "estt" "stet")
(shuffle-strict "t")
;; => nil
(shuffle-strict "ttttt")
;; => nil
We basically create a lazy list of possible shuffles, and then we take the first of them to be different from the input. We also make sure that there are at least 2 different characters in the input, so as not to hang (we return nil here since you don't want to have the input string as a possible result).
If you want your function to return a sequence:
(defn my-shuffle [input]
(when (-> input set count (> 1))
(->> input
(iterate #(apply str (shuffle (seq %))))
(remove #(= input %)))))
(->> "abc" my-shuffle (take 5))
;; => ("acb" "cba" "bca" "acb" "cab")
(->> "bbb" my-shuffle (take 5))
;; => ()
A reduce call has its f argument first. Visually speaking, this is often the biggest part of the form.
e.g.
(reduce
(fn [[longest current] x]
(let [tail (last current)
next-seq (if (or (not tail) (> x tail))
(conj current x)
[x])
new-longest (if (> (count next-seq) (count longest))
next-seq
longest)]
[new-longest next-seq]))
[[][]]
col))
The problem is, the val argument (in this case [[][]]) and col argument come afterward, below, and it's a long way for your eyes to travel to match those with the parameters of f.
It would look more readable to me if it were in this order instead:
(reduceb val col
(fn [x y]
...))
Should I implement this macro, or am I approaching this entirely wrong in the first place?
You certainly shouldn't write that macro, since it is easily written as a function instead. I'm not super keen on writing it as a function, either, though; if you really want to pair the reduce with its last two args, you could write:
(-> (fn [x y]
...)
(reduce init coll))
Personally when I need a large function like this, I find that a comma actually serves as a good visual anchor, and makes it easier to tell that two forms are on that last line:
(reduce (fn [x y]
...)
init, coll)
Better still is usually to not write such a large reduce in the first place. Here you're combining at least two steps into one rather large and difficult step, by trying to find all at once the longest decreasing subsequence. Instead, try splitting the collection up into decreasing subsequences, and then take the largest one.
(defn decreasing-subsequences [xs]
(lazy-seq
(cond (empty? xs) []
(not (next xs)) (list xs)
:else (let [[x & [y :as more]] xs
remainder (decreasing-subsequences more)]
(if (> y x)
(cons [x] remainder)
(cons (cons x (first remainder)) (rest remainder)))))))
Then you can replace your reduce with:
(apply max-key count (decreasing-subsequences xs))
Now, the lazy function is not particularly shorter than your reduce, but it is doing one single thing, which means it can be understood more easily; also, it has a name (giving you a hint as to what it's supposed to do), and it can be reused in contexts where you're looking for some other property based on decreasing subsequences, not just the longest. You can even reuse it more often than that, if you replace the > in (> y x) with a function parameter, allowing you to split up into subsequences based on any predicate. Plus, as mentioned it is lazy, so you can use it in situations where a reduce of any sort would be impossible.
Speaking of ease of understanding, as you can see I misunderstood what your function is supposed to do when reading it. I'll leave as an exercise for you the task of converting this to strictly-increasing subsequences, where it looked to me like you were computing decreasing subsequences.
You don't have to use reduce or recursion to get the descending (or ascending) sequences. Here we are returning all the descending sequences in order from longest to shortest:
(def in [3 2 1 0 -1 2 7 6 7 6 5 4 3 2])
(defn descending-sequences [xs]
(->> xs
(partition 2 1)
(map (juxt (fn [[x y]] (> x y)) identity))
(partition-by first)
(filter ffirst)
(map #(let [xs' (mapcat second %)]
(take-nth 2 (cons (first xs') xs'))))
(sort-by (comp - count))))
(descending-sequences in)
;;=> ((7 6 5 4 3 2) (3 2 1 0 -1) (7 6))
(partition 2 1) gives every possible comparison and partition-by allows you to mark out the runs of continuous decreases. At this point you can already see the answer and the rest of the code is removing the baggage that is no longer needed.
If you want the ascending sequences instead then you only need to change the < to a >:
;;=> ((-1 2 7) (6 7))
If, as in the question, you only want the longest sequence then put a first as the last function call in the thread last macro. Alternatively replace the sort-by with:
(apply max-key count)
For maximum readability you can name the operations:
(defn greatest-continuous [op xs]
(let [op-pair? (fn [[x y]] (op x y))
take-every-second #(take-nth 2 (cons (first %) %))
make-canonical #(take-every-second (apply concat %))]
(->> xs
(partition 2 1)
(partition-by op-pair?)
(filter (comp op-pair? first))
(map make-canonical)
(apply max-key count))))
I feel your pain...they can be hard to read.
I see 2 possible improvements. The simplest is to write a wrapper similar to the Plumatic Plumbing defnk style:
(fnk-reduce { :fn (fn [state val] ... <new state value>)
:init []
:coll some-collection } )
so the function call has a single map arg, where each of the 3 pieces is labelled & can come in any order in the map literal.
Another possibility is to just extract the reducing fn and give it a name. This can be either internal or external to the code expression containing the reduce:
(let [glommer (fn [state value] (into state value)) ]
(reduce glommer #{} some-coll))
or possibly
(defn glommer [state value] (into state value))
(reduce glommer #{} some-coll))
As always, anything that increases clarity is preferred. If you haven't noticed already, I'm a big fan of Martin Fowler's idea of Introduce Explaining Variable refactoring. :)
I will apologize in advance for posting a longer solution to something where you wanted more brevity/clarity.
We are in the new age of clojure transducers and it appears a bit that your solution was passing the "longest" and "current" forward for record-keeping. Rather than passing that state forward, a stateful transducer would do the trick.
(def longest-decreasing
(fn [rf]
(let [longest (volatile! [])
current (volatile! [])
tail (volatile! nil)]
(fn
([] (rf))
([result] (transduce identity rf result))
([result x] (do (if (or (nil? #tail) (< x #tail))
(if (> (count (vswap! current conj (vreset! tail x)))
(count #longest))
(vreset! longest #current))
(vreset! current [(vreset! tail x)]))
#longest)))))))
Before you dismiss this approach, realize that it just gives you the right answer and you can do some different things with it:
(def coll [2 1 10 9 8 40])
(transduce longest-decreasing conj coll) ;; => [10 9 8]
(transduce longest-decreasing + coll) ;; => 27
(reductions (longest-decreasing conj) [] coll) ;; => ([] [2] [2 1] [2 1] [2 1] [10 9 8] [10 9 8])
Again, I know that this may appear longer but the potential to compose this with other transducers might be worth the effort (not sure if my airity 1 breaks that??)
I believe that iterate can be a more readable substitute for reduce. For example here is the iteratee function that iterate will use to solve this problem:
(defn step-state-hof [op]
(fn [{:keys [unprocessed current answer]}]
(let [[x y & more] unprocessed]
(let [next-current (if (op x y)
(conj current y)
[y])
next-answer (if (> (count next-current) (count answer))
next-current
answer)]
{:unprocessed (cons y more)
:current next-current
:answer next-answer}))))
current is built up until it becomes longer than answer, in which case a new answer is created. Whenever the condition op is not satisfied we start again building up a new current.
iterate itself returns an infinite sequence, so needs to be stopped when the iteratee has been called the right number of times:
(def in [3 2 1 0 -1 2 7 6 7 6 5 4 3 2])
(->> (iterate (step-state-hof >) {:unprocessed (rest in)
:current (vec (take 1 in))})
(drop (- (count in) 2))
first
:answer)
;;=> [7 6 5 4 3 2]
Often you would use a drop-while or take-while to short circuit just when the answer has been obtained. We could so that here however there is no short circuiting required as we know in advance that the inner function of step-state-hof needs to be called (- (count in) 1) times. That is one less than the count because it is processing two elements at a time. Note that first is forcing the final call.
I wanted this order for the form:
reduce
val, col
f
I was able to figure out that this technically satisfies my requirements:
> (apply reduce
(->>
[0 [1 2 3 4]]
(cons
(fn [acc x]
(+ acc x)))))
10
But it's not the easiest thing to read.
This looks much simpler:
> (defn reduce< [val col f]
(reduce f val col))
nil
> (reduce< 0 [1 2 3 4]
(fn [acc x]
(+ acc x)))
10
(< is shorthand for "parameters are rotated left"). Using reduce<, I can see what's being passed to f by the time my eyes get to the f argument, so I can just focus on reading the f implementation (which may get pretty long). Additionally, if f does get long, I no longer have to visually check the indentation of the val and col arguments to determine that they belong to the reduce symbol way farther up. I personally think this is more readable than binding f to a symbol before calling reduce, especially since fn can still accept a name for clarity.
This is a general solution, but the other answers here provide many good alternative ways to solve the specific problem I gave as an example.
I have a list of strings, fx '("abc" "def" "gih") and i would like to be able to search the list for any items containing fx "ef" and get the item or index returned.
How is this done?
Combining filter and re-find can do this nicely.
user> (def fx '("abc" "def" "gih"))
#'user/fx
user> (filter (partial re-find #"ef") fx)
("def")
user> (filter (partial re-find #"a") fx)
("abc")
In this case I like to combine them with partial though defining an anonymous function works fine in that case as well. It is also useful to use re-pattern if you don't know the search string in advance:
user> (filter (partial re-find (re-pattern "a")) fx)
("abc")
If you want to retrieve all the indexes of the matching positions along with the element you can try this:
(filter #(re-find #"ef" (second %)) (map-indexed vector '("abc" "def" "gih")))
=>([1 "def"])
map-indexed vector generates an index/value lazy sequence
user> (map-indexed vector '("abc" "def" "gih"))
([0 "abc"] [1 "def"] [2 "gih"])
Which you can then filter using a regular expression against the second element of each list member.
#(re-find #"ef" (second %))
Just indices:
Lazily:
(keep-indexed #(if (re-find #"ef" %2)
%1) '("abc" "def" "gih"))
=> (1)
Using loop/recur
(loop [[str & strs] '("abc" "def" "gih")
idx 0
acc []]
(if str
(recur strs
(inc idx)
(cond-> acc
(re-find #"ef" str) (conj idx)))
acc))
For just the element, refer to Arthur Ulfeldts answer.
Here is a traditional recursive definition that returns the index. It's easy to modify to return the corresponding string as well.
(defn strs-index [re lis]
(let [f (fn [ls n]
(cond
(empty? ls) nil
(re-find re (first ls)) n
:else (recur (rest ls) (inc n))))]
(f lis 0)))
user=> (strs-index #"de" ["abc" "def" "gih"])
1
user=> (strs-index #"ih" ["abc" "def" "gih"])
2
user=> (strs-index #"xy" ["abc" "def" "gih"])
nil
(Explanation: The helper function f is defined as a binding in let, and then is called at the end. If the sequence of strings passed to it is not empty, it searches for the regular expression in the first element of the sequence and returns the index if it finds the string. This uses the fact that re-find's result counts as true unless it fails, in which case it returns nil. If the previous steps don't succeed, the function starts over with the rest of the sequence and an incremented index. If it gets to the end of the sequence, it returns nil.)
I'm trying to build a set of functions to compare sentences to one another. So I wrote a function called split-to-sentences that takes an input like this:
"This is a sentence. And so is this. And this one too."
and returns:
["This is a sentence" "And so is this" "And this one too."]
What I am struggling with is how to iterate over this vector and get the items that aren't the current value. I tried nosing around with drop and remove but haven't quite figured it out.
I guess one thing I could do is use first and rest in the loop and conj the previous value to the output of rest.
(remove #{current-value} sentences-vector)
Just use filter:
(filter #(not= current-value %) sentences-vector)
I believe you may want something like this function:
(defn without-each [x]
(map (fn [i] (concat (subvec x 0 i) (subvec x (inc i))))
(range (count x))))
Use it like this:
>>> (def x ["foo" "bar" "baz"])
>>> (without-each x)
==> (("bar" "baz") ("foo" "baz") ("foo" "bar"))
The returned elements are lazily concatenated, which is why they are not vectors. This is desirable, since true vector concatenation (e.g. (into a b)) is O(n).
Because subvec uses sharing with the original sequence this should not use an excessive amount of memory.
The trick is to pass your sentences twice into the reduce function...
(def sentences ["abcd" "efg" "hijk" "lmnop" "qrs" "tuv" "wx" "y&z"])
(reduce
(fn [[prev [curr & foll]] _]
(let [aren't-current-value (concat prev foll)]
(println aren't-current-value) ;use it here
[(conj prev curr) foll]))
[[] sentences]
sentences)
...once to see the following ones, and once to iterate.
You might consider using subvec or pop because both operate very quickly on vectors.
How do I get the index of any of the elements on a list of strings as so:
(list "a" "b" "c")
For example, (function "a") would have to return 0, (function "b") 1, (function "c") 2 and so on.
and... will it be better to use any other type of collection if dealing with a very long list of data?
Christian Berg's answer is fine. Also it is possible to just fall back on Java's indexOf method of class String:
(.indexOf (apply str (list "a" "b" "c")) "c")
; => 2
Of course, this will only work with lists (or more general, seqs) of strings (of length 1) or characters.
A more general approach would be:
(defn index-of [e coll] (first (keep-indexed #(if (= e %2) %1) coll)))
More idiomatic would be to lazily return all indexes and only ask for the ones you need:
(defn indexes-of [e coll] (keep-indexed #(if (= e %2) %1) coll))
(first (indexes-of "a" (list "a" "a" "b"))) ;; => 0
I'm not sure I understand your question. Do you want the nth letter of each of the strings in a list? That could be accomplished like this:
(map #(nth % 1) (list "abc" "def" "ghi"))
The result is:
(\b \e \h)
Update
After reading your comment on my initial answer, I assume your question is "How do I find the index (position) of a search string in a list?"
One possibility is to search for the string from the beginning of the list and count all the entries you have to skip:
(defn index-of [item coll]
(count (take-while (partial not= item) coll)))
Example: (index-of "b" (list "a" "b" "c")) returns 1.
If you have to do a lot of look-ups, it might be more efficient to construct a hash-map of all strings and their indices:
(def my-list (list "a" "b" "c"))
(def index-map (zipmap my-list (range)))
(index-map "b") ;; returns 1
Note that with the above definitions, when there are duplicate entries in the list index-of will return the first index, while index-map will return the last.
You can use the Java .indexOf method reliably for strings and vectors, but not for lists. This solution should work for all collections, I think:
(defn index-of
"Clojure doesn't have an index-of function. The Java .indexOf method
works reliably for vectors and strings, but not for lists. This solution
works for all three."
[item coll]
(let [v (if
(or (vector? coll) (string? coll))
coll
(apply vector coll))]
(.indexOf coll item)))
Do you mean, how do you get the nth element of a list?
For example, if you want to get the 2nd element on the list (with zero-based index):
(nth (list "a" "b" "c") 2)
yields
"c"
Cat-skinning is fun. Here's a low-level approach.
(defn index-of
([item coll]
(index-of item coll 0))
([item coll from-idx]
(loop [idx from-idx coll (seq (drop from-idx coll))]
(if coll
(if (= item (first coll))
idx
(recur (inc idx) (next coll)))
-1))))
This is a Lispy answer, I suspect those expert in Clojure could do it better:
(defn position
"Returns the position of elt in this list, or nil if not present"
([list elt n]
(cond
(empty? list) nil
(= (first list) elt) n
true (position (rest list) elt (inc n))))
([list elt]
(position list elt 0)))
You seem to want to use the nth function.
From the docs for that function:
clojure.core/nth
([coll index] [coll index not-found])
Returns the value at the index. get returns nil if index out of
bounds, nth throws an exception unless not-found is supplied. nth
also works for strings, Java arrays, regex Matchers and Lists, and,
in O(n) time, for sequences.
That last clause means that in practice, nth is slower for elements "farther off" in sequences, with no guarantee to work quicker for collections that in principle support faster access (~ O(n)) to indexed elements. For (clojure) sequences, this makes sense; the clojure seq API is based on the linked-list API and in a linked list, you can only access the nth item by traversing every item before it. Keeping that restriction is what makes concrete list implementations interchangeable with lazy sequences.
Clojure collection access functions are generally designed this way; functions that do have significantly better access times on specific collections have separate names and cannot be used "by accident" on slower collections.
As an example of a collection type that supports fast "random" access to items, clojure vectors are callable; (vector-collection index-number) yields the item at index index-number - and note that clojure seqs are not callable.
I know this question has been answered a million time but here is a recursive solution that leverages deconstructing.
(defn index-of-coll
([coll elm]
(index-of-coll coll elm 0))
([[first & rest :as coll] elm idx]
(cond (empty? coll) -1
(= first elm) idx
:else (recur rest elm (inc idx)))))
(defn index-of [item items]
(or (last (first (filter (fn [x] (= (first x) item))
(map list items (range (count items))))))
-1))
seems to work - but I only have like three items in my list