I'm trying to create a sparsely populated multidimensional vector in Clojure, but I'm coming up against the limits of my knowledge.
I have a collection x I'm iterating over, and I want to create a multidimensional vector of size (count x) by (count x). Most of the cells will be empty, but at each point where the x and y axes match (e.g., (1 1), (2 2), (3 3), etc.), I need to run a function to see if a value should be put in that space.
In a procedural language, it would be something like:
for (i = 0; i < length(x); i++) {
for (j = 0; j < length(x); j++) {
if (i == j && testReturnsTrue(x[i])) {
table[i][j] = (list x[i])
}
else {
table[i][j] = ()
}
}
}
But I can't wrap my head around how this would be done in Clojure. I tried using nested for comprehensions and nested loop-recur structures but I can't get either to work.
Alternatively, I could create a mutable table with the right sizes, initialize it all to empty lists, and then set the values as I check each element in x, but I'd like to keep the table immutable if possible.
Use a hashmap? There's no need for a vector, which can't be sparse. Plus, this imperative solution looks like it's not sparse either - it's wasting memory storing zillions of empty cells. Perhaps something like this:
(let [indexed (map-indexed vector xs)]
(reduce (fn [m [i x]]
(if (test? x)
(assoc-in m [i i] x)
m))
{}
indexed))
Nested fors is how I would do it:
(def x [:a :b :c :d])
(vec (for [i (range (count x))]
(vec (for [j (range (count x))]
(if (and (= i j) (identity (x i)))
[(x i)]
[])))))
=> [[[:a] [] [] []] [[] [:b] [] []] [[] [] [:c] []] [[] [] [] [:d]]]
(identity (x i)) is a stand-in for some kind of test.
EDIT: As mentioned in other answers, if this structure remains sparsely populated, a hash-map is a better choice. I was assuming you would be populating the empty portions in later computations.
Related
I'm starting to learn Clojure and have decided that doing some projects on HackerRank is a good way to do that. What I'm finding is that my Clojure solutions are horribly slow. I'm assuming that's because I'm still thinking imperatively or just don't know enough about how Clojure operates. The latest problem I wrote solutions for was Down To Zero II. Here's my Java code
import java.io.BufferedReader;
import java.io.InputStreamReader;
public class Solution {
private static final int MAX_NUMBER = 1000000;
private static final BufferedReader reader = new BufferedReader(new InputStreamReader(System.in));
public static int[] precompute() {
int[] values = new int[MAX_NUMBER];
values[0] = 0;
values[1] = 1;
for (int i = 1; i < MAX_NUMBER; i += 1) {
if ((values[i] == 0) || (values[i] > (values[i - 1] + 1))) {
values[i] = (values[i - 1] + 1);
}
for (int j = 1; j <= i && (i * j) < MAX_NUMBER; j += 1) {
int mult = i * j;
if ((values[mult] == 0) || (values[mult] > (values[i] + 1))) {
values[mult] = values[i] + 1;
}
}
}
return values;
}
public static void main(String[] args) throws Exception {
int numQueries = Integer.parseInt(reader.readLine());
int[] values = Solution.precompute();
for (int loop = 0; loop < numQueries; loop += 1) {
int query = Integer.parseInt(reader.readLine());
System.out.println(values[query]);
}
}
}
My Clojure implementation is
(def MAX-NUMBER 1000000)
(defn set-i [out i]
(cond
(= 0 i) (assoc out i 0)
(= 1 i) (assoc out i 1)
(or (= 0 (out i))
(> (out i) (inc (out (dec i)))))
(assoc out i (inc (out (dec i))))
:else out))
(defn set-j [out i j]
(let [mult (* i j)]
(if (or (= 0 (out mult)) (> (out mult) (inc (out i))))
(assoc out mult (inc (out i)))
out)))
;--------------------------------------------------
; Precompute the values for all possible inputs
;--------------------------------------------------
(defn precompute []
(loop [i 0 out (vec (repeat MAX-NUMBER 0))]
(if (< i MAX-NUMBER)
(recur (inc i) (loop [j 1 new-out (set-i out i)]
(if (and (<= j i) (< (* i j) MAX-NUMBER))
(recur (inc j) (set-j new-out i j))
new-out)))
out)))
;--------------------------------------------------
; Read the number of queries
;--------------------------------------------------
(def num-queries (Integer/parseInt (read-line)))
;--------------------------------------------------
; Precompute the solutions
;--------------------------------------------------
(def values (precompute))
;--------------------------------------------------
; Read and process each query
;--------------------------------------------------
(loop [iter 0]
(if (< iter num-queries)
(do
(println (values (Integer/parseInt (read-line))))
(recur (inc iter)))))
The Java code runs in about 1/10 of a second on my machine, while the Clojure code takes close to 2 seconds. Since it's the same machine, with the same JVM, it means I'm doing something wrong in Clojure.
How do people go about trying to translate this type of code? What are the gotchas that are causing it to be so much slower?
I'm going to do some transformations to your code (which might be slightly outside of what you were originally asking)
and then address your more specific questions.
I know it's almost two years later, but after running across your question and spending way too much time fighting with
HackerRank and its time limits, I thought I would post an answer. Does achieving a solution within HR's environment and
time limits make us better Clojure programmers? I didn't learn the answer to that. But I'll share what I did learn.
I found a slightly slimmer version of your same algorithm. It still has two loops, but the update only happens once in
the inner loop, and many of the conditions are handled in a min function. Here is my adaptation of it:
(defn compute
"Returns a vector of down-to-zero counts for all numbers from 0 to m."
[m]
(loop [i 2 out (vec (range (inc m)))]
(if (<= i m)
(recur (inc i)
(loop [j 1 out out]
(let [ij (* i j)]
(if (and (<= j i) (<= ij m))
(recur (inc j)
(assoc out ij (min (out ij) ;; current value
(inc (out (dec ij))) ;; steps from value just below
(inc (out i))))) ;; steps from a factor
out))))
out)))
Notice we're still using loop/recur (twice), we're still using a vector to hold the output. But some differences:
We initialize out to incrementing integers. This is the worst case number of steps for every value, and once
initialized, we don't have to test that a value equals 0 and we can skip indices 0 and 1 and start the outer loop at
index 2. (We also fix a bug in your original and make sure out contains MAX-NUMBER+1 values.)
All three tests happen inside a min function that encapsulates the original logic: a value will be
updated only if it's a shorter number of steps from the number just below it, or from one of it's factors.
The tests are now simple enough that we don't need to break them out into separate functions.
This code (along with your original) is fast enough to pass some of the test cases in HR, but not all. Here are some
things to speed this up:
Use int-array instead of vec. This means we'll use aset instead of assoc and aget instead of calling out
with an index. It also means that loop/recur isn't the best structure anymore (because we are no longer passing
around new versions of an immutable vector, but actually mutating a java.util.Array); instead we'll use doseq.
Type hints. This alone makes a huge speed difference. When testing your code, include a form at the top (set! *warn-on-reflection* true) and you'll see where Clojure is having to do extra work to figure out what types it is
dealing with.
Use custom I/O functions to read the input. HR's boilerplate I/O code is supposed to let you focus on solving the
challenge and not worry about I/O, but it is basically garbage, and often the culprit behind your program timing out.
Below is a version that incorporates the tips above and runs fast enough to pass all test cases. I've included my custom
I/O approach that I've been using for all my HR challenges. One nice benefit of using doseq is we can include a
:let and a :while clause within the binding form, removing some of the indentation within the body of doseq. Also
notice a few strategically placed type hints that really speed up the program.
(ns down-to-zero-int-array)
(set! *warn-on-reflection* true)
(defn compute
"Returns a vector of down-to-zero counts for all numbers from 0 to m."
^ints [m]
(let [out ^ints (int-array (inc m) (range (inc m)))]
(doseq [i (range 2 (inc m)) j (range 1 (inc i)) :let [ij (* i j)] :while (<= ij m)]
(aset out ij (min (aget out ij)
(inc (aget out (dec ij)))
(inc (aget out i)))))
out))
(let [tokens ^java.io.StreamTokenizer
(doto (java.io.StreamTokenizer. (java.io.BufferedReader. *in*))
(.parseNumbers))]
(defn next-int []
"Read next integer from input. As fast as `read-line` for a single value,
and _much_ faster than `read-line`+`split` for multiple values on same line."
(.nextToken tokens)
(int (.-nval tokens))))
(def MAX 1000000)
(let [q (next-int)
down-to-zero (compute MAX)]
(doseq [n (repeatedly q next-int)]
(println (aget down-to-zero n))))
I am trying to implement a solution for minimum-swaps required to sort an array in clojure.
The code works, but takes about a second to solve for the 7 element vector, which is very poor compared to a similar solution in Java. (edited)
I already tried providing the explicit types, but doesnt seem to make a difference
I tried using transients, but has an open bug for subvec, that I am using in my solution- https://dev.clojure.org/jira/browse/CLJ-787
Any pointers on how I can optimize the solution?
;; Find minimumSwaps required to sort the array. The algorithm, starts by iterating from 0 to n-1. In each iteration, it places the least element in the ith position.
(defn minimumSwaps [input]
(loop [mv input, i (long 0), swap-count (long 0)]
(if (< i (count input))
(let [min-elem (apply min (drop i mv))]
(if (not= min-elem (mv i))
(recur (swap-arr mv i min-elem),
(unchecked-inc i),
(unchecked-inc swap-count))
(recur mv,
(unchecked-inc i),
swap-count)))
swap-count)))
(defn swap-arr [vec x min-elem]
(let [y (long (.indexOf vec min-elem))]
(assoc vec x (vec y) y (vec x))))
(time (println (minimumSwaps [7 6 5 4 3 2 1])))
There are a few things that can be improved in your solution, both algorithmically and efficiency-wise. The main improvement is to remember both the minimal element in the vector and its position when you search for it. This allows you to not search for the minimal element again with .indexOf.
Here's my revised solution that is ~4 times faster:
(defn swap-arr [v x y]
(assoc v x (v y) y (v x)))
(defn find-min-and-position-in-vector [v, ^long start-from]
(let [size (count v)]
(loop [i start-from, min-so-far (long (nth v start-from)), min-pos start-from]
(if (< i size)
(let [x (long (nth v i))]
(if (< x min-so-far)
(recur (inc i) x i)
(recur (inc i) min-so-far min-pos)))
[min-so-far min-pos]))))
(defn minimumSwaps [input]
(loop [mv input, i (long 0), swap-count (long 0)]
(if (< i (count input))
(let [[min-elem min-pos] (find-min-and-position-in-vector mv i)]
(if (not= min-elem (mv i))
(recur (swap-arr mv i min-pos),
(inc i),
(inc swap-count))
(recur mv,
(inc i),
swap-count)))
swap-count)))
To understand where are the performance bottlenecks in your program, it is better to use https://github.com/clojure-goes-fast/clj-async-profiler rather than to guess.
Notice how I dropped unchecked-* stuff from your code. It is not as important here, and it is easy to get it wrong. If you want to use them for performance, make sure to check the resulting bytecode with a decompiler: https://github.com/clojure-goes-fast/clj-java-decompiler
A similar implementation in java, runs almost in half the time.
That's actually fairly good for Clojure, given that you use immutable vectors where in Java you probably use arrays. After rewriting the Clojure solution to arrays, the performance would be almost the same.
I'm playing around with a crafty tutorial here:
http://buildnewgames.com/introduction-to-crafty/
and am wondering how this particular function be implemented in clojurescript/clojure
var max_villages = 5;
for (var x = 0; x < Game.map_grid.width; x++) {
for (var y = 0; y < Game.map_grid.height; y++) {
if (Math.random() < 0.02) {
Crafty.e('Village').at(x, y);
if (Crafty('Village').length >= max_villages) {
return;
}
}
}
}
I know that we can have the (for []) construct but how would you get it to stop when max_villages hits 5?
Here's one approach:
(def max-villages 5)
(->> (for [x (range map-width)
y (range map-height)]
[x y])
(filter (fn [_] (< (rand) 0.02)))
(take max-villages))
Then perhaps add (map make-village-at) or something similar as the next stage of the pipeline; if it's meant to perform side effects, add a dorun or doall as the final stage to force them to happen at once (choosing one or the other depending on whether the return values are interesting).
NB. some extra vectors and random numbers may be generated due to seq chunking, it'll be less than 32 though.
A more imperative approach with a counter for comparison:
(let [counter (atom 0)]
(doseq [x (range map-width)
:while (< #counter max-villages)
y (range map-height)
:while (< #counter max-villages)
:when (< (rand) 0.02)]
(swap! counter inc)
(prn [x y]))) ; call make-village-at here
:while terminates the loop at the current nesting level when its test expression fails; :when moves on to the next iteration immediately. doseq supports chunking too, but :while will prevent it from performing unnecessary work.
Using recursion it would be something like:
(letfn [(op [x y]
(if (= (rand) 0.02)
(do
(village-at x y)
(if (>= (village-length) max-villages) true))))]
(loop [x 0 y 0]
(when (and (< x width) (not (op x y)))
(if (= (inc y) height)
(recur (inc x) 0)
(recur x (inc y))))))
That's a great tutorial!
A variation on Michael's approach (I would have just commented to his answer but I don't have enough stack power yet) would be to use Cartesian products rather than nested for loops:
;; some stub stuff to get the example to run
(ns example.core
(:use clojure.math.combinatorics))
(def max-villages 5)
(def map-width 10)
(def map-height 10)
(defn crafty-e [x y z] (print z))
;; the example, note I use doseq rather than map to empasize the fact that the loop
;; is being performed for its side effects not its return value.
(doseq [coord (take max-villages
(filter
(fn [_] (< (rand) 0.02))
(cartesian-product (range map-width) (range map-height))))]
(crafty-e :village :at coord))
Here is the function I'm trying to run...
(defn mongean [cards times]
(let [_cards (transient cards)]
(loop [i 0 c (get cards i) _count (count cards) _current (/ _count 2)]
(assoc! _cards _current c)
(if ((rem i 2) = 0)
(def _newcur (- _current (inc i)))
(def _newcur (+ _current (inc i))))
(if (<= i _count)
(recur (inc i) (get cards i) _count _newcur )))
(persistent! _cards)))
It's resulting in this Exception...
Exception in thread "main" java.lang.ClassCastException: clojure.lang.PersistentHashSet$TransientHashSet cannot be cast to clojure.lang.ITransientAssociative
Being new to clojure, I'd also appreciate any constructive criticism of my approach above. The goal is to take a List, and return a re-ordered list.
I assume that you are trying to implement the Mongean shuffle. Your approach is very imperative and you should try to use a more functional approach.
This would be a possible implementation, were we calculate the final order of the cards (as per Wikipedia formula) and then we use the built-in replace function to do the mapping:
(defn mongean [cards]
(let [num-cards (count cards)
final-order (concat (reverse (range 1 num-cards 2)) (range 0 num-cards 2))]
(replace cards final-order)))
user> (mongean [1 2 3 4 5 6 7 8])
(8 6 4 2 1 3 5 7)
How do you call that function? It looks like you're passing a set, so that its transient version will also be a set and hence can't be used with any of the assoc functions, as they work on associative data structures and vectors:
user=> (assoc #{} :a 1)
ClassCastException clojure.lang.PersistentHashSet cannot be cast to clojure.lang.Associative clojure.lang.RT.assoc (RT.java:691)
user=> (assoc! (transient #{}) :a 1)
ClassCastException clojure.lang.PersistentHashSet$TransientHashSet cannot be cast to clojure.lang.ITransientAssociative clojure.core/assoc! (core.clj:2959)
; the following works as it uses maps and vectors
user=> (assoc {} :a 1)
{:a 1}
user=> (assoc! (transient {}) :a 1)
#<TransientArrayMap clojure.lang.PersistentArrayMap$TransientArrayMap#65cd1dff>
user=> (assoc [] 0 :a)
[:a]
Now, let's try to discuss the code itself. It's a bit hard to follow your code and try to understand what the goal really is without some more hints on what you want to achieve, but as general comments:
you have a times input parameter you don't use at all
you are supposed to use the result of a transient mutation, not assume that the transient will mutate in place
avoid transients if you can, they're only meant as a performance optimization
the binding _current (/ _count 2) is probably not what you want, as (/ 5 2) really returns 5/2 and it seems that you want to use it as a position in the result
constants like _count don't need to be part of the loop binding, you can use the outer let so that you don't have to pass them at each and every iteration
use let instead of def for naming things inside a function
(if ((rem 1 2) = 0)) is definitely not what you want
Now, leaving aside the shuffling algorithm, if you need to rearrange a sequence you might just produce a sequence of new positions, map them with the original cards to produce pairs of [position card] and finally reduce them by placing the card at the new position, using the original sequence as the seed:
(defn generate [coll] ; counts down from (count coll) to 0, change to
; implement your shuffling algorithm
(range (dec (count coll)) -1 -1))
(defn mongean [cards times]
(let [positions (generate cards) ; get the new positions
assemble (fn [dest [pos card]] ; assoc the card at the wanted position
(assoc dest pos card))]
(reduce assemble cards (map vector positions cards))))
If you simply want to shuffle:
(defn mongean [cards times] (shuffle cards))
I have created a very simple nested loop example and am struggling to write the equivalent Clojure code. I've been trying to do it by list comprehensions but cannot get the same answer. Any help appreciated.
public class Toy {
public static void main(String[] args) {
int maxMod = 0;
for (int i=0;i<1000;i++) {
for (int j=i;j<1000;j++) {
if ((i * j) % 13 == 0 && i % 7 == 0) maxMod = i * j;
}
}
System.out.println(maxMod);
}
}
Here's a list comprehension solution:
(last
(for [i (range 1000)
j (range 1000)
:let [n (* i j)]
:when (and (= (mod n 13) 0)
(= (mod i 7) 0))]
n))
In general, you want to use some sort of sequence operation (like dnolen's answer). However, if you need to do something that is not expressible in some combination of sequence functions, using the loop macro works as well. For this precise problem, dnolen's answer is better than anything using loop, but for illustrative purposes, here is how you would write it with loop.
(loop [i 0
max-mod 0]
(if (>= i 1000)
(println max-mod)
(recur (inc i)
(loop [j 0
max-mod max-mod]
(if (>= j 1000)
max-mod
(recur (inc j)
(if (and (= (mod (* i j) 13) 0)
(= (mod 1 7) 0))
(* i j)
max-mod)))))))
This is pretty much an exact translation of your given code. That said, this is obviously ugly, which is why a solution using for (or other similar functions) is preferred whenever possible.
List comprehensions create lists from other lists, but you want just a single value as result. You can create the input values (i and j) with a list comprehension, and then use reduce to get a single value from the list:
(reduce (fn [max-mod [i j]]
(if (and (zero? (mod (* i j) 13))
(zero? (mod i 7)))
(* i j)
max-mod))
0
(for [i (range 1000) j (range 1000)]
[i j]))