seq function caveats in clojure - clojure

Question

In the docstring of clojure's seq function, it mentions:
Note that seqs cache values, thus seq
should not be used on any Iterable whose iterator repeatedly
returns the same mutable object.
What does this sentence mean? Why emphasize the same mutable object?

Answer 1

The comment was added later and mentions this ticket:
Some Java libraries return iterators that return the same mutable object on every call:
Hadoop ReduceContextImpl$ValueIterator
Mahout DenseVector$AllIterator/NonDefaultIterator
LensKit FastIterators
While careful usage of seq or iterator-seq over these iterators worked in the past, that is no longer true as of the changes in CLJ-1669 - iterator-seq now produces a chunked sequence. Because next() is called 32 times on the iterator before the first value can be retrieved from the seq, and the same mutable object is returned every time, code on iterators like this now receives different (incorrect) results.
Approach: Sequences cache values and are thus incompatible with holding mutable and mutating Java objects. We will add some clarification about this to seq and iterator-seq docstrings. For those iterators above, it is recommended to either process those iterators in a loop/recur or to wrap them in a lazy-seq that transforms each re-returned mutable object into a proper value prior to caching.

Answer 2

The Clojure's seq function can create sequences from many types of objects like collections and arrays. seq also works on any object that implements the java.util.Iterable interface from the Java Collections framework. Unfortunately, the semantics of Clojure sequences and java.util.Iterator (which is used with the Iterable) are not 100% compatible as pointed out in the answer from #cfrick.
It is, or at some point was, considered ok for each invocation of the next method of the Iterator to return the same (mutable) object. This works only as long as the return value of next is used and discarded prior to the subsequent call to next. However, if return values of next are retained and used later, undefined behavior can result. This is exactly what happens in some implementations of Clojure sequences.
Let me illustrate. The following is a toy implementation of a range of integers in Java. Notice how the implementation of the method next always returns the same object.
package foo.bar;
import java.util.*;
public class MyRange implements Iterable<MyRange.Num> {
public static class Num {
private int n;
public int get() { return n; }
public String toString() { return String.valueOf(n); }
}
private int max;
public MyRange(int max) { this.max = max; }
// Implementation of Iterable
public Iterator<Num> iterator() {
return new Iterator<Num> () {
private int at = 0;
private Num num = new Num();
public boolean hasNext() {
return at < max;
}
public Num next() {
num.n = at++;
return num;
}
};
}
}
This code works fine when consumed in a way intended by the designers of the Java Collections framework. For example:
(loop [i (.iterator (MyRange. 3))]
(when (.hasNext i)
(print (str (.next i) " "))
(recur i)))
;;=> 0 1 2
But once we bring Clojure sequence into the mix, things go wrong:
(map #(.get %) (MyRange. 3))
;;=> (2 2 2)
We got (2 2 2) instead of (0 1 2). This is exactly the type of an issue that the warning in seq is concerned about.
If memory serves, the implementation of the Iterator for EnumhMap in Java 6 used the mutable object implementation in the name of efficiency. Such implementation does not allocate memory on every iteration, so it is faster and does not create garbage. But this "technique" was problematic not only for Clojure but for some Java users as well. So the behavior was changed in Java 7.