Iterate over two lists (of differing length) backwards in Scala - list

What is the most efficient way to iterate over two lists (of differing length) backwards in Scala.
So for two lists
List(a,b,c) and List(1,2)
the pairs would be
(c,2) and (b,1)
Note: I would rather not do a reverse of each list.

A simple way is :
List('a','b','c').reverse zip List(1,2).reverse
Reversing the list is O(n) however, if you're worried about efficiency.
According to List's scaladoc, using reverseIterator might be more efficient. That way you don't creat a new list like with reverse, but traverse it as you keep iterating. That'd be :
val it = list1.reverseIterator zip list2.reverseIterator //returns an Iterator you can force
it.toList // List((c,2), (b,1))

Using parallel collections,
def parRevZip (a: List[String], b: List[Int]) = {
val max = Math.max(a.size, b.size)
val n = Math.abs(a.size - b.size)
if (a.size > b.size)
(max to n+1 by -1).par.map { i => (a(i-1), b(i-n-1)) }
else
(max to n+1 by -1).par.map { i => (a(i-n-1), b(i-1)) }
}
Taking into account different index values for possibly different sized lists, this approach fetches and pairs the same number of elements starting from the end of each list.
Performance needs careful evaluation; for small lists, a plain reverse and zipping may prove much simpler and efficient; for large lists, on the contrary, this parallel approach may be of interest.
Code Refinement
def parRevZip[A,B] (a: List[A], b: List[B]) = {
val aSize = a.size
val bSize = b.size
val max = Math.max(aSize, bSize)
val n = Math.abs(aSize - bSize)
if (aSize > bSize)
(max-1 to n by -1).par.map { i => (a(i), b(i-n)) }
else
(max-1 to n by -1).par.map { i => (a(i-n), b(i)) }
}
Using non recursive collections
Convenient immutable collections here where the computation of size is O(1) (or quasi-constant) (see Recursive collections in Scala such as List) include for instance Array.
Hence,
def parRevZip[A,B] (a: Array[A], b: Array[B])
which does not follow any further the requirement of processing lists.

I think you mean this:
val a = List(1, 2, 3)
val b = List(8, 9)
val result = a.reverse zip b.reverse

Here is my attempt at this problem. The original lists a and b are not duplicated. The operation is O(N) due to List.size:
object test extends App {
val a = List("a", "b", "c") //> a : List[String] = List(a, b, c)
val b = List(1, 2) //> b : List[Int] = List(1, 2)
val aSize = a.size //> aSize : Int = 3
val bSize = b.size //> bSize : Int = 2
// find which is smaller and which is bigger list
val (smaller, bigger) = if (aSize < bSize) (a, b) else (b, a)
//> smaller : List[Any] = List(1, 2)
//| bigger : List[Any] = List(a, b, c)
// skip the extra entries from head of bigger list
val truncated = bigger.drop(Math.abs(aSize - bSize))
//> truncated : List[Any] = List(b, c)
val result = if (a == smaller)
smaller.zip(truncated).reverse
else
truncated.zip(smaller).reverse //> result : List[(Any, Any)] = List((c,2), (b,1))
}

Related

How to subtract two consecutive element in a list in Scala?

I would like to subtract two consecutive element in a list with numbers in Scala.
For example : I have this list :
val sortedList = List(4,5,6)
I would like to have an output list like diffList =(1, 1) where 5-4 = 1 and 6-5 = 1.
I tried the following code:
var sortedList = List[Int]()
var diffList = List[Int]()
for (i <- 0 to (sortedList.length - 1) ;j <- i + 1 to sortedList.length - 1)
{
val diff = (sortedList(j) - sortedList(i))
diffList = diffList :+ diff
}
I have the following result for diffList =(1, 2, 1) but I want diffList = (1,1).
It's because of the for loop. it does not iterate over the two variables (i and j) at once.
You do not mutability nor imperative programming to solve this problem, functional programming got you covered.
def consecutiveDifferences(data: List[Int]): List[Int] =
if (data.isEmpty) List.empty
else data.lazyZip(data.tail).map {
case (x, y) => y - x
}
As I always say, the Scaladoc is your friend.
(Also, as an advice, the best way to learn functional programming is to forbid yourself from mutability)
You can use the sliding method, which according to the docs:
/** Groups elements in fixed size blocks by passing a "sliding window"
* over them (as opposed to partitioning them, as is done in `grouped`.)
*
* An empty collection returns an empty iterator, and a non-empty
* collection containing fewer elements than the window size returns
* an iterator that will produce the original collection as its only
* element.
* #see [[scala.collection.Iterator]], method `sliding`
*
* #param size the number of elements per group
* #return An iterator producing ${coll}s of size `size`, except for a
* non-empty collection with less than `size` elements, which
* returns an iterator that produces the source collection itself
* as its only element.
* #example `List().sliding(2) = empty iterator`
* #example `List(1).sliding(2) = Iterator(List(1))`
* #example `List(1, 2).sliding(2) = Iterator(List(1, 2))`
* #example `List(1, 2, 3).sliding(2) = Iterator(List(1, 2), List(2, 3))`
*/
Then, solving your query is pretty straight forward:
diffList = sortedList.sliding(2).collect {
case Seq(a, b) =>
b - a
}.toList
Which results in List(1,1)
Code run at Scastie.
for(i <- 0 until (sortedList.size - 1)) yield sortedList(i + 1) - sortedList(i)
yield Vector(1,1) which can be converted to list with toList
That's can be also achieved with the following function:
val sortedList = List(4,5,7)
#tailrec
def findDiffs(xs: List[Int])(seed: List[Int]): List[Int] = {
if(xs.isEmpty || xs.size == 1) seed.reverse
else {
val currDiff = xs(1) - xs(0)
findDiffs(xs.tail)(currDiff :: seed)
}
}
val res = findDiffs(sortedList)(Nil)
println(res)
Or just easily with zip:
sortedList.drop(1) zip sortedList map { case (x,y) => x - y }
Sliding (see answer by #Tomer Shetah) over a list delivers an iterator, which may prove convenient for very large collections to avoid/reduce the amount of intermediate structures in the processing. Another approach includes the zipping of the list with itself shifted by one (see answers by #Luis Miguel Mejía Suárez and #Zvi Mints); in this regard another approach to shifting and then zipping is by dropping the first element as in
xs.drop(1) zip xs map {case(a,b) => b-a}
This can be generalised by dropping any number n so that we subtract the first and the n-th elements, then the second and the n+1-th elements, and so forth.

Scala reduce a List based on a condition

I have a List of certain type that I want to reduce based on a condition. I have a type where the Interval is a DateTime interval with a start and an end:
case class MyType(a: Interval, value: Double)
I have got a List[MyType] entries that I want to reduce to a List[MyType] based on MyType that contains same DateTime and value. I do not want to go over the List twice which I already do.
Say I have:
val a = MyType(interval1, 2)
val b = MyType(interval2, 2)
val c = MyType(interval3, 1)
val d = MyType(interval4, 6)
val e = MyType(interval5, 2)
val original = List(a, b, c, d, e)
I have to now reduce the original List based on the following conditions:
1. interval should be continuous, then take the start of the first entry and the end of the second entry
2. the double value should be the same
So assuming that interval1, interval2 are continuous, the result should look like:
val result = Seq(MyType(new Interval(a.interval.start, b.interval.end),2), c, d, e)
Is there a much more elegant solution or an idea?
In the reduce function, check if the condition is true, and if it is, return the current accumulator instead of what would you otherwise compute.
Here's how you would sum only even numbers:
Seq(1,4,6,3).foldLeft(0)( (acc, a) =>
if (a % 2 == 0) acc + a else acc
)
res5: Int = 10
Response to the edited question: It appears you have some conditions that have to hold about the consecuitve elements. Then you can apply the function .sliding.
Seq(a,b,c,d,e).sliding(2).foldLeft(0)(
case (acc, Seq(MyType(ai, a), MyType(bi, b))) =>
if (ai.max == bi.min) acc + a else acc
)
Buuut... You have probably guessed it would not be as performant as you would like. I hope you are not doing any premature optimization, because you know, that's the root of all evil. But if you really need performance, rewrite the code in terms of while loops (fall back to Java).
This should work:
def reduce(xs: List[MyType]) = {
xs match {
case a :: b :: tail =>
if(a.interval.end == b.interval.start && a.value == b.value)
reduce(MyType(new Interval(a.interval.start, b.interval.end) a.value) :: tail)
else
a :: reduce(b :: tail)
case _ => xs
}
}
The if condition might need minor tweaking depending on your exact needs, but the algorithm should work.
Given a list xs
If the first two items a and b can be merged into c, merge them and go back to step 1 with xs = c :: tail
If a and b cannot be merged, try reducing all elements but the first, and append the result to a
Otherwise (list has 1 element or is empty), return xs
Pay attantion that your task could result in multiple distinct solutions, which cannot be further reduced.
So as result you will get a set of solutions: Set[Set[MyType]]
I use Set[MyType] instead of proposed List[MyType] and Seq[MyType] because order is not important and my answer needs possibility to compare different solutions (in order to avoid duplicates).
My answer doesn't make assumptions about order of items, any order is OK.
Besides that in order to simplify the code I have replaced Interval with 2 fields from and to, which can be easily converted.
Here is the code for reduction:
case class MyType(from: Long, to: Long, value: Double)
object MyType {
//Returns all possible varians of reduced source.
//If reduction is not possible, returns empty set.
private def strictReduce(source: Set[MyType]): Set[Set[MyType]] = {
if (source.size <= 1) {Set.empty} else {
val active = source.head //get some item
val otherItems = source.tail //all other items
val reducedWithActive: Set[Set[MyType]] = otherItems.flatMap {
case after if active.to == after.from =>
//we have already found a reduction (active->after),
// so further reductions are not strictly required
reduce(otherItems - after + MyType(active.from, after.to, active.value))
case before if before.to == active.from =>
//we have already found a reduction (before->active),
// so further reductions are not strictly required
reduce(otherItems - before + MyType(before.from, active.to, active.value))
case notContinuos => Set.empty[Set[MyType]]
}
//check if we can reduce items without active
val reducedIgnoringActive = strictReduce(otherItems).
//if so, re-insert active and try to reduce it further, but not strictly anymore
flatMap (reducedOther => reduce(reducedOther + active))
reducedWithActive ++ reducedIgnoringActive
}
}
//Returns all possible varians of reduced source.
//If reduction is not possible, returns source as single result.
private def reduce(source: Set[MyType]): Set[Set[MyType]] = strictReduce(source) match {
case empty if empty.isEmpty => Set(source)
case reduced => reduced
}
//Reduces source, which contains items with different values
def reduceAll(source: Set[MyType]): Set[Set[MyType]] = source.
groupBy(_.value). //divide by values, because they are not merge-able
mapValues(reduce). //reduce for every group
values.reduceLeft((solutionSetForValueA, solutionSetForValueB) =>
//merge solutions for different groups
for(subSolutionForValueA <- solutionSetForValueA;
subSolutionForValueB <- solutionSetForValueB)
yield (subSolutionForValueA ++ subSolutionForValueB) //merge subSolutions
)
}
And here is the sample, which uses it:
object Example extends App {
val source = Set(
MyType(0L, 1L, 1.0),
MyType(1L, 2L, 2.0), //different value
MyType(1L, 3L, 1.0), //competing with next
MyType(1L, 4L, 1.0), //competing with prev
MyType(3L, 5L, 1.0), //joinable with pre-prev
MyType(2L, 4L, 2.0), //joinable with second
MyType(0L, 4L, 3.0) //lonely
)
val solutions: Set[Set[MyType]] = MyType.reduceAll(source)
//here you could choose the best solution (for example by size)
//printing out
solutions.foreach(solution => println(solution.toList.sortBy(_.from).sortBy(_.value).
map(item => s"${item.from}->${item.to}(${item.value})").mkString(", ")))
}
My result is:
0->5(1.0), 1->4(1.0), 1->4(2.0), 0->4(3.0)
0->4(1.0), 1->5(1.0), 1->4(2.0), 0->4(3.0)
Here is what I came up with:
def reduce(accumulator: Seq[MyType], original: Seq[MyType]): Seq[MyType] = original match {
case Nil => accumulator
case head :: xs => {
val found = xs.find(_.timeSpan.getStart().equals(head.timeSpan.getEnd))
if (found.isDefined && found.get.value == head.value) {
reduce(
accumulator :+ (MyType(new Interval(head.timeSpan.getStart, found.get.timeSpan.getEnd), head.value)),
original.diff(Seq(found.get, head))
)
}
else
reduce(
accumulator :+ head,
xs
)
}
}

Scala best way to make uniform crossover operation with permutation list?

I search the best and the most elegant way to make GA crossover operator in Scala functional (No "for" loop, with only immutable type if possible), for example, with this list:
val A = IndexedSeq (5,4,8)
val B = IndexedSeq (3,2,6)
I want to make random bitcoin permutation (with rng.nextBoolean for example) between each element in my IndexedSeq, and finally I get the two lists A' and B' after permutation of their elements.
Example of execution:
rng.nextBoolean <- (true,false,true)
A' = 3,4,6
B' = 5,2,8
Thanks.
def crossover[T](a: Seq[T], b: Seq[T], rs: Seq[Boolean]) =
(a, b, rs).zipped.map((x, y, z) => if (z) Seq(x, y) else Seq(y, x)).transpose
Use with Booleans as third argument:
scala> val Seq(a1, b1) = crossover(A, B, List(true, false, true))
a1: Seq[Int] = Vector(5, 2, 8)
b1: Seq[Int] = Vector(3, 4, 6)
If you want it with a default sequence of Booleans, you could provide a default argument like this:
def crossover[T](a: Seq[T], b: Seq[T], rs: Seq[Boolean] = {
val rng = new util.Random
Stream.continually(rng.nextBoolean) }) =
(a, b, rs).zipped.map((x, y, z) => if (z) Seq(x, y) else Seq(y, x)).transpose
Wow, where's all this code coming from? Here:
val (a1, b1) = A zip B map (t => if (util.Random.nextBoolean) t.swap else t) unzip
There, that's all.
If you already have a list of random booleans, you can do this:
val (a1, b1) = A zip B zip C map { case (t, flag) => if (flag) t.swap else t } unzip
import scala.util.Random
val A = IndexedSeq(5,4,8)
val B = IndexedSeq(3,2,6)
def crossover[T](rng: Random)(a: Seq[T], b: Seq[T]): (Seq[T],Seq[T]) = {
if (a.isEmpty && b.isEmpty) return (Nil,Nil)
val (aTailCrossover,bTailCrossover) = crossover(rng)(a.tail,b.tail)
if (rng.nextBoolean) (b.head +: aTailCrossover, a.head +: bTailCrossover)
else (a.head +: aTailCrossover, b.head +: bTailCrossover)
}
println(crossover(new Random)(A,B))
def rndCombi [T] (a: Seq[T], b: Seq[T]): Seq[T] = {
if (a.size != b.size) sys.error ("sizes don't match: a:" + a.size + " != b: " + b.size)
val rnd = util.Random
val max = (math.pow (2, a.size)).toInt
val r = rnd.nextInt (max)
def pick (a: Seq[T], b: Seq[T], r: Int) : List[T] = {
if (a.size == 0) Nil else
if (r % 2 == 0) a.head :: pick (a.tail , b.tail, r/2) else
b.head :: pick (a.tail , b.tail, r/2)
}
// print all combinations for testing:
// (0 until max).map (i => println (pick (a, b, i).mkString ("-")))
pick (a, b, r).toSeq
}
// I choosed different values for easy testing:
val a = IndexedSeq (7, 8, 9)
val b = IndexedSeq (1, 2, 3)
println (rndCombi (a, b).mkString (" "))
println (rndCombi (a, b.tail).mkString (" "))
Initializing util.Random each time is of course not very clever, if done frequently. So for production code you would rearrange the code.
If you don't restrict the input to 2 sequences, it get's more interesting. Here we go:
def rndCombi [T] (s: Seq[Seq[T]]): Seq[T] = {
val outer = s.size
val inner = s(0).size
val rnd = util.Random
val max = (math.pow (outer, inner)).toInt
val r = rnd.nextInt (max)
def pick (s: Seq[Seq[T]], r: Int, pos: Int = 0) : List[T] =
if (pos == inner) Nil
else s(r % inner)(pos) :: pick (s, r/inner, pos + 1)
// print all combinations for testing:
(0 until max).map (i => println (pick (s, i).mkString ("-")))
println ()
pick (s, r).toSeq
}
val a = IndexedSeq (1, 2, 3)
val b = IndexedSeq (4, 5, 6)
val c = IndexedSeq (7, 8, 9)
println (rndCombi (Seq (a, b, c)).mkString (" "))
The second solution can, of course, be used for 2 sequences as well.

Split list into multiple lists with fixed number of elements

How to split a List of elements into lists with at most N items?
ex: Given a list with 7 elements, create groups of 4, leaving the last group possibly with less elements.
split(List(1,2,3,4,5,6,"seven"),4)
=> List(List(1,2,3,4), List(5,6,"seven"))
I think you're looking for grouped. It returns an iterator, but you can convert the result to a list,
scala> List(1,2,3,4,5,6,"seven").grouped(4).toList
res0: List[List[Any]] = List(List(1, 2, 3, 4), List(5, 6, seven))
There is much easier way to do the task using sliding method.
It works this way:
val numbers = List(1, 2, 3, 4, 5, 6 ,7)
Lets say you want to break the list into smaller lists of size 3.
numbers.sliding(3, 3).toList
will give you
List(List(1, 2, 3), List(4, 5, 6), List(7))
Or if you want to make your own:
def split[A](xs: List[A], n: Int): List[List[A]] = {
if (xs.size <= n) xs :: Nil
else (xs take n) :: split(xs drop n, n)
}
Use:
scala> split(List(1,2,3,4,5,6,"seven"), 4)
res15: List[List[Any]] = List(List(1, 2, 3, 4), List(5, 6, seven))
edit: upon reviewing this 2 years later, I wouldn't recommend this implementation since size is O(n), and hence this method is O(n^2), which would explain why the built-in method becomes faster for large lists, as noted in comments below. You could implement efficiently as follows:
def split[A](xs: List[A], n: Int): List[List[A]] =
if (xs.isEmpty) Nil
else (xs take n) :: split(xs drop n, n)
or even (slightly) more efficiently using splitAt:
def split[A](xs: List[A], n: Int): List[List[A]] =
if (xs.isEmpty) Nil
else {
val (ys, zs) = xs.splitAt(n)
ys :: split(zs, n)
}
I am adding a tail recursive version of the split method since there was some discussion of tail-recursion versus recursion. I have used the tailrec annotation to force the compiler to complain in case the implementation is not indeed tail-recusive. Tail-recursion I believe turns into a loop under the hood and thus will not cause problems even for a large list as the stack will not grow indefinitely.
import scala.annotation.tailrec
object ListSplitter {
def split[A](xs: List[A], n: Int): List[List[A]] = {
#tailrec
def splitInner[A](res: List[List[A]], lst: List[A], n: Int) : List[List[A]] = {
if(lst.isEmpty) res
else {
val headList: List[A] = lst.take(n)
val tailList : List[A]= lst.drop(n)
splitInner(headList :: res, tailList, n)
}
}
splitInner(Nil, xs, n).reverse
}
}
object ListSplitterTest extends App {
val res = ListSplitter.split(List(1,2,3,4,5,6,7), 2)
println(res)
}
I think this is the implementation using splitAt instead of take/drop
def split [X] (n:Int, xs:List[X]) : List[List[X]] =
if (xs.size <= n) xs :: Nil
else (xs.splitAt(n)._1) :: split(n,xs.splitAt(n)._2)

Accessing the next element in list to compare in Scala

I'm new to Scala and i was wondering how you can call the next element of the list because I am trying to compare the current element with the adjacent one.
Given x as the current element, I tried similar to java, x+1 but that didnt work. Any help?
for (x <- list; if (x == (next adj. element))) println("same")
How about sliding?
val list = List(1,2,3,4)
list.sliding(2).foreach(println)
//List(1, 2)
//List(2, 3)
//List(3, 4)
The canonical ways to do this in a for loop would be:
scala> val xs = List(1,2,3,4,3,2)
xs: List[Int] = List(1, 2, 3, 4, 3, 2)
scala> for (List(left,right) <- xs.sliding(2) if (left < right)) println(left + " < " + right)
1 < 2
2 < 3
3 < 4
scala> for ((left,right) <- (xs zip xs.tail) if (left < right)) println(left + " < " + right)
1 < 2
2 < 3
3 < 4
(Incidentally, you're probably better off putting the if statement outside rather than inside the for comprehension in this example.)
If you have indices instead of values, you just dereference them using the same pattern. Personally, I don't find this pattern very clear or useful. It's slow, has weird corner-cases with lists that aren't full, and it's hard to follow what's going on. Instead, I define
class PairedIterable[A](it: Iterable[A]) {
def foreachpair(f: (A,A) => Unit) = {
val i = it.iterator
if (i.hasNext) {
var prev = i.next
while (!ans && i.hasNext) {
val x = i.next
f(prev,x)
prev = x
}
}
}
}
implicit def iterable_has_pairs[A](it: Iterable[A]) = new PairedIterable(it)
which can then be used like so:
scala> xs.foreachpair((left, right) => if (left < right) println(left + " < " + right))
1 < 2
2 < 3
3 < 4
Variants "forallpair", "existspair", and "findpair" are particularly useful.
scala> val xs = 1::3::5::4::Nil
xs: List[Int] = List(1, 3, 5, 4)
scala> (xs, xs.tail).zip.foreach(println)
(1,3)
(3,5)
(5,4)
scala>
As an option you may use match and recursion instead of for:
object Test {
def main(args: Array[String]) {
val list = List(1, 5, 3)
loop(list)
}
def loop(list: List[Int]) {
list match {
case Nil => println("Empty list")
case x :: Nil => println("last " + x)
case x :: tail => {
println(x + " - " + tail.head)
loop(tail)
}
}
}
}
This would be better handled by recursing over the list, instead of iterating through the elements, since elements don't know anything about the list.
For example:
def recurse[T](list: List[T]): Unit = list match {
case List(x, y, _*) if x == y =>
println("same")
recurse(list.tail)
case Nil =>
case _ => recurse(list.tail)
}
As in Scala 2.11.7 the following are valid:
scala> val xs = List(1,2,3,4)
xs: List[Int] = List(1, 2, 3, 4)
1) Zip the tail
scala> xs.zip(xs.tail)
res0: List[(Int, Int)] = List((1,2), (2,3), (3,4))
2) Slide the window
scala> xs.sliding(2)
res1: Iterator[List[Int]] = non-empty iterator
list.tail.head
gives the next element if you want to go through all the elements from the front of the list. This is because the head is the front-most element and tail is the rest of the list.
scala> val li = List (3, 4, 5)
li: List[Int] = List(3, 4, 5)
scala> li.tail.head
res74: Int = 4
If you don't want to compare just a single element, but a sequence of arbitrary length, you can do it in recursive function:
def compAdjectent (l: List [Int]) : Boolean = l match {
case Nil => false
case x :: Nil => false
case x :: y :: xs => if (x.equals (y)) true else compAdjectent (y :: xs)
}