Combining methods in Clojure - clojure

Suppose we have a multimethod foo. It has several realizations. Let's say that one of them is called when argument of foo is a string that contains character \r and another is executed when argument of foo is a string containing character \!. Pseudocode:
(defmulti foo ???) ; can't come up with function..
(defmethod foo \r [_]
(println "one"))
(defmethod foo \! [_]
(println "two"))
So when we call our function like this:
(foo "right!") ;; desired output:
one
two
;; => nil
Important thing here is that list of supported methods should be not rigid, but expandable, so new methods can be added later without touching the original code.
Although I improved my Clojure skill significantly in last few days, I still lack experience. My best idea is to keep a map with pairs 'character - function' and then manually traverse it and execute right functions. In this case I will also need some interface to register new functions, etc. What is idiomatic solution?

I think multimethods don't work the way you expect them to work.
That is: the dispatch in multimethods is called only once for a single multimethod call, so there's no way of getting the result you expect (both 'one' and 'two' printed for "right!" as argument) unless you define one implementation that actually handles the case of having both \r and \! in the input string and prints the output you want.
This will not be easily expandable.
Nicer way to achieve what you want is to make multiple calls explicitly by iterating the input string:
; You want the dispatch function to just return the character passed to it.
(defmulti foo identity)
; The argument list here is mandatory, but we don't use them at all, hence '_'
(defmethod foo \r [_]
(println "one"))
(defmethod foo \! [_]
(println "two"))
; You need the default case for all the other characters
(defmethod foo :default [_]
())
; Iterates the string and executes foo for each character
(defn bar [s]
(doseq [x s]
(foo x)))
so calling
(bar "right!")
will print:
one
two
Edit
If you need to access the whole string inside the multimethod body, then pass it explicitly together with the character:
; You want the dispatch function to just return the character passed to it as the first arg.
(defmulti foo (fn [c _] c))
(defmethod foo \r [c s]
(println "one"))
(defmethod foo \! [c s]
(println "two"))
; The default now takes two arguments which we ignore
(defmethod foo :default [_ _] ())
; Iterates the string and executes foo for each character
(defn bar [s]
(doseq [x s]
(foo x s)))

A plain list of functions would allow arbitrary conditionals. Also Regexs may make your life simpler if you are dealing with strings:
;; start with some functions
(defn on-r [x]
(when (re-find #"r" x)
"one"))
(defn on-! [x]
(when (re-find #"!" x)
"two"))
(def fns (atom [on-r on-!]))
;; call all functions on some value
(keep #(% "right!") #fns)
=> ("one" "two")
(keep #(% "aaaa") #fns)
=> ()
;; later add more functions
(defn on-three [x]
(when (= 3 (count x))
"three"))
(swap! fns conj on-three)
(keep #(% "bar") #fns)
=> ("one" "three")
;; or just use different rules altogether
(def other-fns [#(when (rand-nth [true false])
(str % (rand-int 10)))
#(when (nil? %) "nil")])
(keep #(% nil) other-fns)
=> ("3" "nil")

Related

How to really shuffle sequence in Clojure?

(defn shuffle-letters
[word]
(let [letters (clojure.string/split word #"")
shuffled-letters (shuffle letters)]
(clojure.string/join "" shuffled-letters)))
But if you put in "test" you can get "test" back sometimes.
How to modify the code to be sure that output will never be equal to input.
I feel embarrassing, I can solve it easily in Python, but Clojure is so different to me...
Thank you.
P.S. I thing we can close the topic now... The loop is in fact all I needed...
You can use loop. When the shuffled letters are the same as the original, recur back up to the start of the loop:
(defn shuffle-letters [word]
(let [letters (clojure.string/split word #"")]
(loop [] ; Start a loop
(let [shuffled-letters (shuffle letters)]
(if (= shuffled-letters letters) ; Check if they're equal
(recur) ; If they're equal, loop and try again
(clojure.string/join "" shuffled-letters)))))) ; Else, return the joined letters
There's many ways this could be written, but this is I think as plain as it gets. You could also get rid of the loop and make shuffle-letters itself recursive. This would lead to unnecessary work though. You could also use let-fn to create a local recursive function, but at that point, loop would likely be cleaner.
Things to note though:
Obviously, if you try to shuffle something like "H" or "HH", it will get stuck and loop forever since no amount of shuffling will cause them to differ. You could do a check ahead of time, or add a parameter to loop that limits how many times it tries.
This will actually make your shuffle less random. If you disallow it from returning the original string, you're reducing the amount of possible outputs.
The call to split is unnecessary. You can just call vec on the string:
(defn shuffle-letters [word]
(let [letters (vec word)]
(loop []
(let [shuffled-letters (shuffle letters)]
(if (= shuffled-letters letters)
(recur)
(clojure.string/join "" shuffled-letters))))))
Here's another solution (using transducers):
(defn shuffle-strict [s]
(let [letters (seq s)
xform (comp (map clojure.string/join)
(filter (fn[v] (not= v s))))]
(when (> (count (into #{} letters)) 1)
(first (eduction xform (iterate shuffle letters))))))
(for [_ (range 20)]
(shuffle-strict "test"))
;; => ("etts" "etts" "stte" "etts" "sett" "tste" "tste" "sett" "ttse" "sett" "ttse" "tset" "stte" "ttes" "ttes" "stte" "stte" "etts" "estt" "stet")
(shuffle-strict "t")
;; => nil
(shuffle-strict "ttttt")
;; => nil
We basically create a lazy list of possible shuffles, and then we take the first of them to be different from the input. We also make sure that there are at least 2 different characters in the input, so as not to hang (we return nil here since you don't want to have the input string as a possible result).
If you want your function to return a sequence:
(defn my-shuffle [input]
(when (-> input set count (> 1))
(->> input
(iterate #(apply str (shuffle (seq %))))
(remove #(= input %)))))
(->> "abc" my-shuffle (take 5))
;; => ("acb" "cba" "bca" "acb" "cab")
(->> "bbb" my-shuffle (take 5))
;; => ()

Dispatching function calls on different formats of maps

I'm writing an agar.io clone. I've lately seen a lot of suggestions to limit use of records (like here), so I'm trying to do the whole project only using basic maps.*
I ended up creating constructors for different "types" of bacteria like
(defn new-bacterium [starting-position]
{:mass 0,
:position starting-position})
(defn new-directed-bacterium [starting-position starting-directions]
(-> (new-bacterium starting-position)
(assoc :direction starting-directions)))
The "directed bacterium" has a new entry added to it. The :direction entry will be used to remember what direction it was heading in.
Here's the problem: I want to have one function take-turn that accepts the bacterium and the current state of the world, and returns a vector of [x, y] indicating the offset from the current position to move the bacterium to. I want to have a single function that's called because I can think right now of at least three kinds of bacteria that I'll want to have, and would like to have the ability to add new types later that each define their own take-turn.
A Can-Take-Turn protocol is out the window since I'm just using plain maps.
A take-turn multimethod seemed like it would work at first, but then I realized that I'd have no dispatch values to use in my current setup that would be extensible. I could have :direction be the dispatch function, and then dispatch on nil to use the "directed bacterium"'s take-turn, or default to get the base aimless behavior, but that doesn't give me a way of even having a third "player bacterium" type.
The only solution I can think of it to require that all bacterium have a :type field, and to dispatch on it, like:
(defn new-bacterium [starting-position]
{:type :aimless
:mass 0,
:position starting-position})
(defn new-directed-bacterium [starting-position starting-directions]
(-> (new-bacterium starting-position)
(assoc :type :directed,
:direction starting-directions)))
(defmulti take-turn (fn [b _] (:type b)))
(defmethod take-turn :aimless [this world]
(println "Aimless turn!"))
(defmethod take-turn :directed [this world]
(println "Directed turn!"))
(take-turn (new-bacterium [0 0]) nil)
Aimless turn!
=> nil
(take-turn (new-directed-bacterium [0 0] nil) nil)
Directed turn!
=> nil
But now I'm back to basically dispatching on type, using a slower method than protocols. Is this a legitimate case to use records and protocols, or is there something about mutlimethods that I'm missing? I don't have a lot of practice with them.
* I also decided to try this because I was in the situation where I had a Bacterium record and wanted to create a new "directed" version of the record that had a single field direction added to it (inheritance basically). The original record implemented protocols though, and I didn't want to have to do something like nesting the original record in the new one, and routing all behavior to the nested instance. Every time I created a new type or changed a protocol, I would have to change all the routing, which was a lot of work.
You can use example-based multiple dispatch for this, as explained in this blog post. It is certainly not the most performant way to solve this problem, but arguably more flexible than multi-methods as it does not require you to declare a dispatch-method upfront. So it is open for extension to any data representation, even other things than maps. If you need performance, then multi-methods or protocols as you suggest, is probably the way to go.
First, you need to add a dependency on [bluebell/utils "1.5.0"] and require [bluebell.utils.ebmd :as ebmd]. Then you declare constructors for your data structures (copied from your question) and functions to test those data strucutres:
(defn new-bacterium [starting-position]
{:mass 0
:position starting-position})
(defn new-directed-bacterium [starting-position starting-directions]
(-> (new-bacterium starting-position)
(assoc :direction starting-directions)))
(defn bacterium? [x]
(and (map? x)
(contains? x :position)))
(defn directed-bacterium? [x]
(and (bacterium? x)
(contains? x :direction)))
Now we are going to register those datastructures as so called arg-specs so that we can use them for dispatch:
(ebmd/def-arg-spec ::bacterium {:pred bacterium?
:pos [(new-bacterium [9 8])]
:neg [3 4]})
(ebmd/def-arg-spec ::directed-bacterium {:pred directed-bacterium?
:pos [(new-directed-bacterium [9 8] [3 4])]
:neg [(new-bacterium [3 4])]})
For each arg-spec, we need to declare a few example values under the :pos key, and a few non-examples under the :neg key. Those values are used to resolve the fact that a directed-bacterium is more specific than just a bacterium in order for the dispatch to work properly.
Finally, we are going to define a polymorphic take-turn function. We first declare it, using declare-poly:
(ebmd/declare-poly take-turn)
And then, we can provide different implementations for specific arguments:
(ebmd/def-poly take-turn [::bacterium x
::ebmd/any-arg world]
:aimless)
(ebmd/def-poly take-turn [::directed-bacterium x
::ebmd/any-arg world]
:directed)
Here, the ::ebmd/any-arg is an arg-spec that matches any argument. The above approach is open to extension just like multi-methods, but does not require you to declare a :type field upfront and is thus more flexible. But, as I said, it is also going to be slower than both multimethods and protocols, so ultimately this is a trade-off.
Here is the full solution: https://github.com/jonasseglare/bluebell-utils/blob/archive/2018-11-16-002/test/bluebell/utils/ebmd/bacteria_test.clj
Dispatching a multimethod by a :type field is indeed polymorphic dispatch that could be done with a protocol, but using multimethods allows you to dispatch on different fields. You can add a second multimethod that dispatches on something other than :type, which might be tricky to accomplish with a protocol (or even multiple protocols).
Since a multimethod can dispatch on anything, you could use a set as the dispatch value. Here's an alternative approach. It's not fully extensible, since the keys to select are determined within the dispatch function, but it might give you an idea for a better solution:
(defmulti take-turn (fn [b _] (clojure.set/intersection #{:direction} (set (keys b)))))
(defmethod take-turn #{} [this world]
(println "Aimless turn!"))
(defmethod take-turn #{:direction} [this world]
(println "Directed turn!"))
Fast paths exist for a reason, but Clojure doesn't stop you from doing anything you want to do, per say, including ad hoc predicate dispatch. The world is definitely your oyster. Observe this super quick and dirty example below.
First, we'll start off with an atom to store all of our polymorphic functions:
(def polies (atom {}))
In usage, the internal structure of the polies would look something like this:
{foo ; <- function name
{:dispatch [[pred0 fn0 1 ()] ; <- if (pred0 args) do (fn0 args)
[pred1 fn1 1 ()]
[pred2 fn2 2 '&]]
:prefer {:this-pred #{:that-pred :other-pred}}}
bar
{:dispatch [[pred0 fn0 1 ()]
[pred1 fn1 3 ()]]
:prefer {:some-pred #{:any-pred}}}}
Now, let's make it so that we can prefer predicates (like prefer-method):
(defn- get-parent [pfn x] (->> (parents x) (filter pfn) first))
(defn- in-this-or-parent-prefs? [poly v1 v2 f1 f2]
(if-let [p (-> #polies (get-in [poly :prefer v1]))]
(or (contains? p v2) (get-parent f1 v2) (get-parent f2 v1))))
(defn- default-sort [v1 v2]
(if (= v1 :poly/default)
1
(if (= v2 :poly/default)
-1
0)))
(defn- pref [poly v1 v2]
(if (-> poly (in-this-or-parent-prefs? v1 v2 #(pref poly v1 %) #(pref poly % v2)))
-1
(default-sort v1 v2)))
(defn- sort-disp [poly]
(swap! polies update-in [poly :dispatch] #(->> % (sort-by first (partial pref poly)) vec)))
(defn prefer [poly v1 v2]
(swap! polies update-in [poly :prefer v1] #(-> % (or #{}) (conj v2)))
(sort-disp poly)
nil)
Now, let's create our dispatch lookup system:
(defn- get-disp [poly filter-fn]
(-> #polies (get-in [poly :dispatch]) (->> (filter filter-fn)) first))
(defn- pred->disp [poly pred]
(get-disp poly #(-> % first (= pred))))
(defn- pred->poly-fn [poly pred]
(-> poly (pred->disp pred) second))
(defn- check-args-length [disp args]
((if (= '& (-> disp (nth 3) first)) >= =) (count args) (nth disp 2)))
(defn- args-are? [disp args]
(or (isa? (vec args) (first disp)) (isa? (mapv class args) (first disp))))
(defn- check-dispatch-on-args [disp args]
(if (-> disp first vector?)
(-> disp (args-are? args))
(-> disp first (apply args))))
(defn- disp*args? [disp args]
(and (check-args-length disp args)
(check-dispatch-on-args disp args)))
(defn- args->poly-fn [poly args]
(-> poly (get-disp #(disp*args? % args)) second))
Next, let's prepare our define macro with some initialization and setup functions:
(defn- poly-impl [poly args]
(if-let [poly-fn (-> poly (args->poly-fn args))]
(-> poly-fn (apply args))
(if-let [default-poly-fn (-> poly (pred->poly-fn :poly/default))]
(-> default-poly-fn (apply args))
(throw (ex-info (str "No poly for " poly " with " args) {})))))
(defn- remove-disp [poly pred]
(when-let [disp (pred->disp poly pred)]
(swap! polies update-in [poly :dispatch] #(->> % (remove #{disp}) vec))))
(defn- til& [args]
(count (take-while (partial not= '&) args)))
(defn- add-disp [poly poly-fn pred params]
(swap! polies update-in [poly :dispatch]
#(-> % (or []) (conj [pred poly-fn (til& params) (filter #{'&} params)]))))
(defn- setup-poly [poly poly-fn pred params]
(remove-disp poly pred)
(add-disp poly poly-fn pred params)
(sort-disp poly))
With that, we can finally build our polies by rubbing some macro juice on there:
(defmacro defpoly [poly-name pred params body]
`(do (when-not (-> ~poly-name quote resolve bound?)
(defn ~poly-name [& args#] (poly-impl ~poly-name args#)))
(let [poly-fn# (fn ~(symbol (str poly-name "-poly")) ~params ~body)]
(setup-poly ~poly-name poly-fn# ~pred (quote ~params)))
~poly-name))
Now you can build arbitrary predicate dispatch:
;; use defpoly like defmethod, but without a defmulti declaration
;; unlike defmethods, all params are passed to defpoly's predicate function
(defpoly myinc number? [x] (inc x))
(myinc 1)
;#_=> 2
(myinc "1")
;#_=> Execution error (ExceptionInfo) at user$poly_impl/invokeStatic (REPL:6).
;No poly for user$eval187$myinc__188#5c8eee0f with ("1")
(defpoly myinc :poly/default [x] (inc x))
(myinc "1")
;#_=> Execution error (ClassCastException) at user$eval245$fn__246/invoke (REPL:1).
;java.lang.String cannot be cast to java.lang.Number
(defpoly myinc string? [x] (inc (read-string x)))
(myinc "1")
;#_=> 2
(defpoly myinc
#(and (number? %1) (number? %2) (->> %& (filter (complement number?)) empty?))
[x y & z]
(inc (apply + x y z)))
(myinc 1 2 3)
;#_=> 7
(myinc 1 2 3 "4")
;#_=> Execution error (ArityException) at user$poly_impl/invokeStatic (REPL:5).
;Wrong number of args (4) passed to: user/eval523/fn--524
; ^ took the :poly/default path
And when using your example, we can see:
(defn new-bacterium [starting-position]
{:mass 0,
:position starting-position})
(defn new-directed-bacterium [starting-position starting-directions]
(-> (new-bacterium starting-position)
(assoc :direction starting-directions)))
(defpoly take-turn (fn [b _] (-> b keys set (contains? :direction)))
[this world]
(println "Directed turn!"))
;; or, if you'd rather use spec
(defpoly take-turn (fn [b _] (->> b (s/valid? (s/keys :req-un [::direction])))
[this world]
(println "Directed turn!"))
(take-turn (new-directed-bacterium [0 0] nil) nil)
;#_=> Directed turn!
;nil
(defpoly take-turn :poly/default [this world]
(println "Aimless turn!"))
(take-turn (new-bacterium [0 0]) nil)
;#_=> Aimless turn!
;nil
(defpoly take-turn #(-> %& first :show) [this world]
(println :this this :world world))
(take-turn (assoc (new-bacterium [0 0]) :show true) nil)
;#_=> :this {:mass 0, :position [0 0], :show true} :world nil
;nil
Now, let's try using isa? relationships, a la defmulti:
(derive java.util.Map ::collection)
(derive java.util.Collection ::collection)
;; always wrap classes in a vector to dispatch off of isa? relationships
(defpoly foo [::collection] [c] :a-collection)
(defpoly foo [String] [s] :a-string)
(foo [])
;#_=> :a-collection
(foo "bob")
;#_=> :a-string
And of course we can use prefer to disambiguate relationships:
(derive ::rect ::shape)
(defpoly bar [::rect ::shape] [x y] :rect-shape)
(defpoly bar [::shape ::rect] [x y] :shape-rect)
(bar ::rect ::rect)
;#_=> :rect-shape
(prefer bar [::shape ::rect] [::rect ::shape])
(bar ::rect ::rect)
;#_=> :shape-rect
Again, the world's your oyster! There's nothing stopping you from extending the language in any direction you want.

clojure.lang.LazySeq cannot be cast to java.lang.CharSequence

I am writing a function that, for any given string, replaces any digits within that String with the same number of '.' characters.
Examples:
AT2X -> AT..X
QW3G45 -> QW...G.........
T3Z1 -> T...Z.
I've written the following Clojure function but I am getting an error I don't quite understand:
java.lang.ClassCastException: clojure.lang.LazySeq (in module: Unnamed Module) cannot be case to java.lang.Charsequence
I'm interpreting from the error that I need to force an evaluation of a lazy sequence back into a String (or CharSequence) but I can't figure out where to do so or if this is correct.
(defn dotify
;;Replaces digits with the same number of '.'s for use in traditional board formats
[FEN]
(let [values (doall (filter isDigit (seq FEN)))]
(fn [values]
(let [value (first values)]
(str/replace FEN value (fn dots [number]
(fn [s times]
(if (> times 0)
(recur (str s ".") (dec times)))) "" (Character/digit number 10)) value))
(recur (rest values))) values))
There is a standard clojure.string/replace function that may handle that case. Its last argument might be not just a string or a pattern but also a function that turns a found fragment into what you want.
Let's prepare such a function first:
(defn replacer [sum-str]
(let [num (read-string num-str)]
(apply str (repeat num \.))))
You may try it in this way:
user> (replacer "2")
..
user> (replacer "9")
.........
user> (replacer "22")
......................
user>
Now pass it into replace as follows:
user> (clojure.string/replace "a2b3c11" #"\d+" replacer)
a..b...c...........
Here's a way to do this using reduce:
(defn dotify [s]
(->> s
(reduce (fn [acc elem]
(if (Character/isDigit elem)
(let [dots (Integer/parseInt (str elem))]
(apply conj acc (repeat dots \.)))
(conj acc elem)))
[])
(apply str)))
(dotify "zx4g1z2h")
=> "zx....g.z..h"
And another version using mapcat:
(defn dotify-mapcat [s]
(apply str
(mapcat (fn [c]
(if (Character/isDigit c)
(repeat (Integer/parseInt (str c)) \.)
[c]))
s)))
There are some issues in your example:
Many of the internal forms are themselves functions, but it looks like you just want their bodies or implementations instead of wrapping them in functions.
It's hard to tell by the indentation/whitespace, but the entire function is just recur-ing, the fn above it is not being used or returned.
One of the arguments to str/replace is a function that returns a function.
It helps to break the problem down into smaller pieces. For one, you know you'll need to examine each character in a string and decide whether to just return it or expand it into a sequence of dots. So you can start with a function:
(defn expand-char [^Character c]
(if (Character/isDigit c)
(repeat (Integer/parseInt (str c)) \.)
[c]))
Then use that function that operates on one character at a time in a higher-order function that operates on the entire string:
(apply str (mapcat expand-char s))
=> "zx....g.z..h"
Note this is also ~5x faster than the examples above because of the ^Character type-hint in expand-char function.
You can do this with str/replace too:
(defn expand-char [s]
(if (Character/isDigit ^Character (first s))
(apply str (repeat (Integer/parseInt s) \.))
s))
(str/replace "zx4g1z2h" #"." expand-char)
=> "zx....g.z..h"

Alternative to mutable data structure in clojure [duplicate]

I developed a function in clojure to fill in an empty column from the last non-empty value, I'm assuming this works, given
(:require [flambo.api :as f])
(defn replicate-val
[ rdd input ]
(let [{:keys [ col ]} input
result (reductions (fn [a b]
(if (empty? (nth b col))
(assoc b col (nth a col))
b)) rdd )]
(println "Result type is: "(type result))))
Got this:
;=> "Result type is: clojure.lang.LazySeq"
The question is how do I convert this back to type JavaRDD, using flambo (spark wrapper)
I tried (f/map result #(.toJavaRDD %)) in the let form to attempt to convert to JavaRDD type
I got this error
"No matching method found: map for class clojure.lang.LazySeq"
which is expected because result is of type clojure.lang.LazySeq
Question is how to I make this conversion, or how can I refactor the code to accomodate this.
Here is a sample input rdd:
(type rdd) ;=> "org.apache.spark.api.java.JavaRDD"
But looks like:
[["04" "2" "3"] ["04" "" "5"] ["5" "16" ""] ["07" "" "36"] ["07" "" "34"] ["07" "25" "34"]]
Required output is:
[["04" "2" "3"] ["04" "2" "5"] ["5" "16" ""] ["07" "16" "36"] ["07" "16" "34"] ["07" "25" "34"]]
Thanks.
First of all RDDs are not iterable (don't implement ISeq) so you cannot use reductions. Ignoring that a whole idea of accessing previous record is rather tricky. First of all you cannot directly access values from an another partition. Moreover only transformations which don't require shuffling preserve order.
The simplest approach here would be to use Data Frames and Window functions with explicit order but as far as I know Flambo doesn't implement required methods. It is always possible to use raw SQL or access Java/Scala API but if you want to avoid this you can try following pipeline.
First lets create a broadcast variable with last values per partition:
(require '[flambo.broadcast :as bd])
(import org.apache.spark.TaskContext)
(def last-per-part (f/fn [it]
(let [context (TaskContext/get) xs (iterator-seq it)]
[[(.partitionId context) (last xs)]])))
(def last-vals-bd
(bd/broadcast sc
(into {} (-> rdd (f/map-partitions last-per-part) (f/collect)))))
Next some helper for the actual job:
(defn fill-pair [col]
(fn [x] (let [[a b] x] (if (empty? (nth b col)) (assoc b col (nth a col)) b))))
(def fill-pairs
(f/fn [it] (let [part-id (.partitionId (TaskContext/get)) ;; Get partion ID
xs (iterator-seq it) ;; Convert input to seq
prev (if (zero? part-id) ;; Find previous element
(first xs) ((bd/value last-vals-bd) part-id))
;; Create seq of pairs (prev, current)
pairs (partition 2 1 (cons prev xs))
;; Same as before
{:keys [ col ]} input
;; Prepare mapping function
mapper (fill-pair col)]
(map mapper pairs))))
Finally you can use fill-pairs to map-partitions:
(-> rdd (f/map-partitions fill-pairs) (f/collect))
A hidden assumption here is that order of the partitions follows order of the values. It may or may not be in general case but without explicit ordering it is probably the best you can get.
Alternative approach is to zipWithIndex, swap order of values and perform join with offset.
(require '[flambo.tuple :as tp])
(def rdd-idx (f/map-to-pair (.zipWithIndex rdd) #(.swap %)))
(def rdd-idx-offset
(f/map-to-pair rdd-idx
(fn [t] (let [p (f/untuple t)] (tp/tuple (dec' (first p)) (second p))))))
(f/map (f/values (.rightOuterJoin rdd-idx-offset rdd-idx)) f/untuple)
Next you can map using similar approach as before.
Edit
Quick note on using atoms. What is the problem there is lack of referential transparency and that you're leveraging incidental properties of a given implementation not a contract. There is nothing in the map semantics that requires elements to be processed in a given order. If internal implementation changes it may be no longer valid. Using Clojure
(defn foo [x] (let [aa #a] (swap! a (fn [&args] x)) aa))
(def a (atom 0))
(map foo (range 1 20))
compared to:
(def a (atom 0))
(pmap foo (range 1 20))

In Clojure how can I pass multiple arguments to a defmethod?

I wish to create a multi method which I call like this:
(defmethod some-method "some value"
[ a b ]
b)
: but which selects the function based only on the first paramter 'a'. How can I do this:
(defmulti some-method
WHAT GOES HERE?)
I didn't completely understand your question, but I think you want to
dispatch only on one argument. You can do that like this, I think:
user=> (defmulti even-or-odd (fn [x _] (even? x)))
#'user/even-or-odd
user=> (defmethod even-or-odd true [a _] :even)
#<MultiFn clojure.lang.MultiFn#293bdd36>
user=> (defmethod even-or-odd false [a _] :odd)
#<MultiFn clojure.lang.MultiFn#293bdd36>
user=> (even-or-odd 2 3)
:even
user=> (even-or-odd 3 3)
:odd
user=>
Do you mean select the function based on the value of a?
Then you just need
(defmulti some-method (fn [a b] a))