I am trying to rename all files and subdirectories in a given directory.
I create vectors of the old and new names, and map a renaming function over the two vectors.
The vectors of file paths are created ok but the mapping
(map #(re-name (as-file %1) (as-file %2)) flist new-flist))) does not seem to call the renaming function at all (I've put a print call in it to test).
After much tweaking and searching I am still perplexed as to what I'm missing.
Also, I would like to know how I could obviate resorting to forums by debugging it by stepping through the code.
Code:
;;;; Use: Replaces whitespace in names of all directories and files in the given directory with the given separator.
(ns file-renamer.core
(:gen-class)
(:require [clojure.string :as str]))
(use '[clojure.java.io])
(use '[clojure.pprint])
; Constants for testing -> params live
(def direc "/home/john/test/")
(def separator "-")
; ====================================
(defn traverse-dir
"Return a seq of pathnames of directories and files in the given directoryt"
[dir-path]
(map #(.getAbsolutePath %) (file-seq (file dir-path))))
(defn replace-spaces
"Return a copy of string s1 with all sequences of spaces replaced with string s2"
[s1 s2]
(str/replace s1 #"\s+" s2))
(defn re-name
"Rename a file"
[old-file new-file]
(pprint (str "Renaming: " old-file " ==>> " new-file)) ; put here for debugging
(.renameTo old-file new-file))
(defn -main
"Map a fn to rename a file over all files and dirs in the given directory"
[& args]
(let [flist (vec (traverse-dir direc))
new-flist (vec (map #(replace-spaces % separator) flist))]
(pprint flist)
(pprint new-flist)
(map #(re-name (as-file %1) (as-file %2)) flist new-flist)))
Output:
17 Mar 20:53 /file-renamer ⋄ lein run
["/home/john/test"
"/home/john/test/test 108"
"/home/john/test/test 108/ baaaa rrrr"
"/home/john/test/test 108/ baaaa rrrr/Open Document Text .... odt"
"/home/john/test/test 108/ baaaa rrrr/ba z z er ."
"/home/john/test/test 108/ baaaa rrrr/ba z z er ./freed.frm"
"/home/john/test/test 108/ baaaa rrrr/ba z z er ./New Folder"
"/home/john/test/test 108/ baaaa rrrr/ba z z er ./New Folder/Plain Text.txt"
"/home/john/test/test 108/ baaaa rrrr/ba z z er ./fr ed.txt"
"/home/john/test/test 108/s p a c e s------S P A C E S "
"/home/john/test/fox"
"/home/john/test/foo"
"/home/john/test/fog"]
["/home/john/test"
"/home/john/test/test-108"
"/home/john/test/test-108/-baaaa-rrrr"
"/home/john/test/test-108/-baaaa-rrrr/Open-Document-Text-....-odt"
"/home/john/test/test-108/-baaaa-rrrr/ba-z-z-er-."
"/home/john/test/test-108/-baaaa-rrrr/ba-z-z-er-./freed.frm"
"/home/john/test/test-108/-baaaa-rrrr/ba-z-z-er-./New-Folder"
"/home/john/test/test-108/-baaaa-rrrr/ba-z-z-er-./New-Folder/Plain-Text.txt"
"/home/john/test/test-108/-baaaa-rrrr/ba-z-z-er-./fr-ed.txt"
"/home/john/test/test-108/s-p-a-c-e-s------S-P-A-C-E-S-"
"/home/john/test/fox"
"/home/john/test/foo"
"/home/john/test/fog"]
17 Mar 20:53 /file-renamer ⋄
There are a couple of problems here. They are not directly your fault, but it's more the REPL that is playing a trick on you and you not fully understanding (probably) the core concepts of Clojure and lazy operations/sequences.
The map function returns a lazy sequence, that need to be realized at some point in time, which you are not doing. On top of that, the re-name function is not a side-effect free function (pure function).
The REPL is also playing a trick on you: If you call (-main) on the REPL, it will automatically realize those sequences and can cause a lot of confusion for someone new to Clojure.
The most straightforward solution would be to just use the doall function.
(doall (map #(re-name (as-file %1) (as-file %2)) flist new-flist))
But this is the quick and dirty way and I am going to quote Stuart Sierra here:
You might get the advice that you can “force” a lazy sequence to be evaluated with doall or dorun. There are also snippets floating around that purport to “unchunk” a sequence.
In my opinion, the presence of doall, dorun, or even “unchunk” is almost always a sign that something never should have been a lazy sequence in the first place.
A better solution in this case would be to use the doseq function and write something like this:
(defn -main
"Map a fn to rename a file over all files and dirs in the given directory"
[& args]
(doseq [file-string (traverse-dir direc)]
(let [input-file (as-file file-string)
output-file (as-file (replace-spaces file-string separator))]
(re-name input-file output-file))))
This could also be written way shorter.
A good reading that will help a lot in general is the blog post from Stuart Sierra:
Clojure Don’ts: Lazy Effects.
Check the docs on doall.
Here, renaming is a side-effect. The map returns a lazy sequence. You need to force mapping over the whole collections in order for the renaming to happen.
You could force the evaluation of the lazy sequence that map returns using doall as others have suggested (or just use the strict mapv), but map isn't intended to carry out side effects. Adding doall allows it to work, but it's just covering up a smell.
I'd use doseq for this since you're only iterating the sequences to carry out side effects. You don't actually care what the map evaluates to. Unfortunately, this solution requires explicitly zipping the two collections together before they can be given to doseq, which bulk it up a bit:
(let [zipped (map vector flist new-flist)] ; Zip the sequences together
(doseq [[f1 f2] zipped]
(re-name (as-file f1) (as-file f2)))
It's not as tense, but the use of doseq makes it much clearer what the intent is.
Related
I am trying to print the documentation for all functions in a given namespace by invoking the following expression in a REPL:
(doseq
[f (dir-fn 'clojure.repl)]
(doc f))
However the invocation of this expression returns nil without printing the documentation to the REPL. I know this might have to do with doc being a macro, but I'm a Clojure novice and am not entirely sure how to understand the problem.
Why does this expression return nil without printing the documentation?
How can this expression be modified so that it prints the documentation for each function in a given namespace?
Thanks!
Update: Combined both provided answers:
(defn ns-docs [ns']
(doseq [[symbol var] (ns-interns ns')]
(newline)
(println symbol)
(print " ")
(println (:doc (meta var)))))
(ns-docs 'clojure.repl)
I would, instead, start here:
The Clojure CheatSheet
ClojureDocs.org
Clojure-Doc.org (similar name, but different)
The API & Reference sections at Clojure.org
Note that doc is in the namespace clojure.repl, which reflects its intended usage (by a human in a repl). Here is some code that will also iterate on a namespace & print doc strings (using a different technique):
(doseq [[fn-symbol fn-var] (ns-interns 'demo.core)]
(newline)
(println fn-symbol)
(println (:doc (meta fn-var))))
where demo.core is the namespace of interest.
Note that ns-interns gives you both a symbol and var like:
fn-symbol => <#clojure.lang.Symbol -main>
fn-var => <#clojure.lang.Var #'demo.core/-main>
The meta function has lots of other info you may want to use someday:
(meta fn-var) =>
<#clojure.lang.PersistentArrayMap
{ :arglists ([& args]),
:doc "The Main Man!",
:line 9, :column 1,
:file "demo/core.clj",
:name -main,
:ns #object[clojure.lang.Namespace 0x14c35a06 "demo.core"]}>
While this probably won't help you with answering your question, the problem of evaluating macro's comes up a lot when you are learning Clojure.
Macros are responsible for the evaluation of their arguments. In this case clojure.repl/doc will ignore the current lexical context and assume that the symbol f that you're giving it is the name of a function you want to see the documentation for. It does this because it's intended to be used at the REPL, and is assuming you wouldn't want to type quotes all the time.
As f doesn't exist, it prints nothing. Then doseq returns nil, since it exists to do something for side effects only - hence starting in do. In order to pass an argument to a macro that refuses to respect the lexical context like this, you need to write the code for each element in the list.
You can do this by hand, or by constructing the code as data, and passing it to eval to execute. You can do this in an imperative style, using doseq:
(doseq [f (ns-interns 'clojure.repl)]
(eval `(doc ~(symbol "clojure.repl" (str (first f))))))
or in a slightly more Clojurey way (which will allow you to see the code that it would execute by removing eval from the end and running it at the REPL):
(->> (ns-interns 'clojure.repl)
(map #(list 'clojure.repl/doc (symbol "clojure.repl" (str (first %)))))
(cons `do)
eval)
In both of these we use quote and syntax-quote to construct some code from the list of symbols reflected from the namespace, and pass it to eval to actually execute it. This page on Clojure's weird characters should point you in the right direction for understanding what's going on here.
This an example of why you shouldn't write macro's, unless you've got no other options. Macro's do not compose, and are often difficult to work with. For a more in depth discussion, Fogus's talk and Christophe Grand's talk are both good talks.
Why does this expression return nil without printing the documentation?
Because the doc macro is receiving the symbol f from your loop, instead of a function symbol directly.
How can this expression be modified so that it prints the documentation for each function in a given namespace?
(defn ns-docs [ns']
(let [metas (->> (ns-interns ns') (vals) (map meta) (sort-by :name))]
(for [m metas :when (:doc m)] ;; you could filter here if you want fns only
(select-keys m [:name :doc]))))
(ns-docs 'clojure.repl)
=>
({:name apropos,
:doc "Given a regular expression or stringable thing, return a seq of all
public definitions in all currently-loaded namespaces that match the
str-or-pattern."}
...
)
Then you can print those maps/strings if you want.
Question from a total newbie with Clojure. Task is pretty simple, but I'm having hard time finding the best way to do this - I need to set up input, where user could give me a list (user should determine how long) of natural numbers and the program should just return a sum of these numbers.
Maybe this is totally wrong already:
(defn inputlist[naturallist]
(println "Enter list of natural numbers:")
(let[naturallist(read-line)] ))
Here is one way of doing it:
> lein new app demo
> cd demo
Edit the project.clj and src/demo/core.clj so they look as follows:
> cat project.clj
(defproject demo "0.1.0-SNAPSHOT"
:description "FIXME: write description"
:url "http://example.com/FIXME"
:license {:name "Eclipse Public License"
:url "http://www.eclipse.org/legal/epl-v10.html"}
:dependencies [[org.clojure/clojure "1.9.0"]
[org.clojure/tools.reader "1.1.3.1"] ]
:main ^:skip-aot demo.core
:target-path "target/%s"
:profiles {:uberjar {:aot :all}})
> cat src/demo/core.clj
(ns demo.core
(:require
[clojure.tools.reader.edn :as edn]
[clojure.string :as str] ))
(defn -main []
(println "enter numbers")
(let [input-line (read-line)
num-strs (str/split input-line #"\s+")
nums (mapv edn/read-string num-strs)
result (apply + nums) ]
(println "result=" result)))
with result
> lein run
enter numbers
1 2 3 <= you type this, then <enter>
input-line => "1 2 3"
num-strs => ["1" "2" "3"]
nums => [1 2 3]
result => 6
You may wish to start looking at some beginner books as well:
Brave Clojure
Living Clojure
Programming Clojure
The Clojure CheatSheet
The quickest way to do this I can think of is
#(apply + (map #(Integer/parseInt %) (re-seq #"\d+" (read-line))))
This defines an anonymous function which:
(read-line) - reads a line of text, hopefully containing numbers separated by non-numeric characters. So you type in something like "123 456 789"
(re-seq #"\d+" ...) - uses the regular expression \d+ to search for strings of consecutive digits. Each string of consecutive digits is added to a sequence, which is subsequently returned. So, for example, if you type in 123, 456, 789 the re-seq function will return the sequence '("123" "456" "789").
(map #(Integer/parseInt %) ...) - invokes the anonymous function #(Integer/parseInt %) for each element in the list returned by the re-seq invocation, creating another list of the results. So if the input is '("123" "456" "789") the output will be '(123 456 789).
(apply + ...) - applies the + function to the list of numbers, summing them up, and returns the sum.
Et voila! The end result is, you type in a string of numbers, separated by non-numeric characters, you get back the sum of those numbers. If you want to be a little neater, promote code re-use, and in general make this a bit more useful you could break this up into separate functions:
(defn parse-string-list [s]
(re-seq #"\d+" s))
(defn convert-seq-of-int-strings [ss]
(map #(Integer/parseInt %) ss))
(defn sum-numbers-in-seq [ss]
(apply + ss))
Invoking this in a Lisp-y way would look something like
(sum-numbers-in-seq (convert-seq-of-int-strings (parse-string-list (read-line))))
or, in a more Clojure-y way
(-> (read-line)
(parse-string-list)
(convert-seq-of-int-strings)
(sum-numbers-in-seq))
Best of luck.
Welcome to Clojure—and StackOverflow!
Here's how to do it:
(defn input-numbers-and-sum []
(print "Enter list of natural numbers: ")
(flush)
(->> (clojure.string/split (read-line) #"\s+")
(map #(Integer/parseInt %))
(reduce +)))
Here's how it works:
Calling print rather than println avoids printing a newline character at the end of the line. This way, the user's input will appear on the same line as your prompt.
Since there was no newline, you have to call flush to force the output buffer containing the prompt to be printed.
split splits what the user typed into a sequence of strings, divided where a regular expression matches. You have to say clojure.string/split rather than just split because split is not in Clojure's core library. clojure.string/ specifies the library. #"\s+" is a regular expression that matches any number of consecutive whitespace characters. So, if your user types " 6 82 -15 ", split will return ["6" "82" "-15"].
map calls the standard Java library function Integer.parseInt on each of those strings. Integer/parseInt is Clojure's Java interop syntax for calling a static method of a Java class. The #(...) is terse syntax that defines an anonymous function; the % is the argument passed to that function. So, given the sequence of strings above, this call to map will return a sequence of integers: [6 82 -15].
reduce calls the + function repeatedly on each element of the sequence of integers, passing the sum so far as an argument along with the next integer. map and reduce actually take three arguments; the next paragraph tells how the third paragraph gets filled in.
->> is the "thread-last macro". It rewrites the code inside it, to pass the output of each expression but the last as the last argument of the following expression. The result is:(reduce + (map #(Integer/parseInt %) (clojure.string/split (read-line) #"\s+")))Most people find the version with ->> much easier to read.
That might seem like a lot to do something very simple, but it's actually bread and butter once you're used to Clojure. Clojure is designed to make things easy to combine; map, reduce, and ->> are especially useful tools for hooking other functions together.
I've included links to the documentation. Those are worth a look; many contain typical examples of use.
There are other ways to parse numbers, of course, some of which are shown in the answers to this question. What I've written above is an "idiomatic" way to do it. Learn that, and you'll know a lot of the everyday, must-know techniques for programming in Clojure.
Yep, readline is the correct way to do it.
But each element from readlines is essentially an instance of java.lang.Character , and since you want the sum, you'd prefer to convert them to integer before summing the elements of list.
(defn input-list
[]
(print "Enter list of natural numbers")
(let [nums (read-line)]
(reduce + (map #(Integer/parseInt %) (clojure.string/split nums #"\s+")))
This might not be the most idiomatic way to do it, but feel free to tweak it.
Also, please do clean up on your variable/function names.
Edit : (Integer/parseInt %) might cause an error if used directly since the input is an instance of characters, not string. So we can use clojure.string/split to convert user input to a sequence of strings, and use Integer/parseInt % for conversion.
In fact, a more readable version can be written using thread-first macros :
(defn input-list []
(print "Enter list of natural numbers: ")
(->> (clojure.string/split (read-line) #"\s+")
(map #(Integer/parseInt %))
(reduce +)))
This is a more clojurey way to do it.
If you don't care about negative scenarios (wrong input, syntax issues), the quickest solution would be to evaluate the user's input putting it into parens:
(defn sum-ints []
(let [input (read-line)
ints (read-string (str "(" input ")"))]
(reduce + 0 ints)))
Usage:
user=> (sum-ints)
1 2 3 4 5
15
The read-string function evaluates an text expression (1 2 3 4 5) in that case. Since numeral literals turn into numbers, the result will be just a list of numbers.
So I know this isn't the best method of solving this issue, but I'm trying to go through a list of lines from an input file, which end up being expressions. I've got a list of expressions, and each expression has it's own list thanks to the split-the-list function. My next step is to replace characters with id, ints with int, and + or - with addop. I've got the regexes to find whether or not my symbols match any of those, but when I try and replace them, I can only get the last for loop I call to leave any lasting results. I know what it stems down to is the way functional programming works, but I can't wrap my head around the trace of this program, and how to replace each separate type of input and keep the results all in one list.
(def reint #"\d++")
(def reid #"[a-zA-Z]+")
(def readdop #"\+|\-")
(def lines (into () (into () (clojure.string/split-lines (slurp "input.txt")) )))
(defn split-the-line [line] (clojure.string/split line #" " ))
(defn split-the-list [] (for [x (into [] lines)] (split-the-line x)))
(defn tokenize-the-line [line]
(for [x line] (clojure.string/replace x reid "id"))
(for [x line] (clojure.string/replace x reint "int"))
(for [x line] (clojure.string/replace x readdop "addop")))
(defn tokenize-the-list [] (for [x (into [] (split-the-list) )] (tokenize-the-line x)))
And as you can probably tell, I'm pretty new to functional programming, so any advice is welcome!
You're using a do block, which evaluates several expressions (normally for side effects) and then returns the last one. You can't see it because fn (and hence defn) implicitly contain one. As such, the lines
(for [x line] (clojure.string/replace x reid "id"))
(for [x line] (clojure.string/replace x reint "int"))
are evaluated (into two different lazy sequences) and then thrown away.
In order for them to affect the return value, you have to capture their return values and use them in the next round of replacements.
In this case, I think the most natural way to compose your replacements is the threading macro ->:
(for [x line]
(-> x
(clojure.string/replace reid "id")
(clojure.string/replace reint "int")
(clojure.string/replace readdop "addop")))
This creates code which does the reid replace with x as the first argument, then does the reint replace with the result of that as the first argument and so on.
Alternatively you could do this by using comp to compose anonymous functions like (fn [s] (clojure.string/replace s reid "id") (partial application of replace). In the imperative world we get pretty used to running several procedures that "bash the data in place" - in the functional world you more often combine several functions together to do all the operations and then run the result.
I have an incoming lazy stream lines from a file I'm reading with tail-seq (to contrib - now!) and I want to process those lines one after one with several "listener-functions" that takes action depending on re-seq-hits (or other things) in the lines.
I tried the following:
(defn info-listener [logstr]
(if (re-seq #"INFO" logstr) (println "Got an INFO-statement")))
(defn debug-listener [logstr]
(if (re-seq #"DEBUG" logstr) (println "Got a DEBUG-statement")))
(doseq [line (tail-seq "/var/log/any/java.log")]
(do (info-listener logstr)
(debug-listener logstr)))
and it works as expected. However, there is a LOT of code-duplication and other sins in the code, and it's boring to update the code.
One important step seems to be to apply many functions to one argument, ie
(listen-line line '(info-listener debug-listener))
and use that instead of the boring and error prone do-statement.
I've tried the following seemingly clever approach:
(defn listen-line [logstr listener-collection]
(map #(% logstr) listener-collection))
but this only renders
(nil) (nil)
there is lazyiness or first class functions biting me for sure, but where do I put the apply?
I'm also open to a radically different approach to the problem, but this seems to be a quite sane way to start with. Macros/multi methods seems to be overkill/wrong for now.
Making a single function out of a group of functions to be called with the same argument can be done with the core function juxt:
=>(def juxted-fn (juxt identity str (partial / 100)))
=>(juxted-fn 50)
[50 "50" 2]
Combining juxt with partial can be very useful:
(defn listener [re message logstr]
(if (re-seq re logstr) (println message)))
(def juxted-listener
(apply juxt (map (fn [[re message]] (partial listner re message))
[[#"INFO","Got INFO"],
[#"DEBUG", "Got DEBUG"]]))
(doseq [logstr ["INFO statement", "OTHER statement", "DEBUG statement"]]
(juxted-listener logstr))
You need to change
(listen-line line '(info-listener debug-listener))
to
(listen-line line [info-listener debug-listener])
In the first version, listen-line ends up using the symbols info-listener and debug-listener themselves as functions because of the quoting. Symbols implement clojure.lang.IFn (the interface behind Clojure function invocation) like keywords do, i.e. they look themselves up in a map-like argument (actually a clojure.lang.ILookup) and return nil if applied to something which is not a map.
Also note that you need to wrap the body of listen-line in dorun to ensure it actually gets executed (as map returns a lazy sequence). Better yet, switch to doseq:
(defn listen-line [logstr listener-collection]
(doseq [listener listener-collection]
(listener logstr)))
I have the following string
layout: default
title: Envy Labs
What i am trying to do is create map from it
layout->default
title->"envy labs"
Is this possible to do using sequence functions or do i have to loop through each line?
Trying to get a regex to work with and failing using.
(apply hash-map (re-split #": " meta-info))
user> (let [x "layout: default\ntitle: Envy Labs"]
(reduce (fn [h [_ k v]] (assoc h k v))
{}
(re-seq #"([^:]+): (.+)(\n|$)" x)))
{"title" "Envy Labs", "layout" "default"}
The _ is a variable name used to indicate that you don't care about the value of the variable (in this case, the whole matched string).
I'd recommend using clojure-contrib/duck-streams/read-lines to process the lines then split the fields from there. I find this method is usually more robust to errors in the file.