I would like to know how can I create a List of List in the result of a reduce operation.
I've for example this lines
1,2,3,4
0,7,8,9
1,5,6,7
0,6,5,7
And I would like to get something like this
1, [[2,3,4],[5,6,7]]
0, [[7,8,9],[6,5,7]]
Thsi is my code
val parsedData = data.map { line =>
val parts = line.split(",")
val label = Integer.parseInt(parts(0))
(label, List(Integer.parseInt(parts(1)), Integer.parseInt(parts(2)), Integer.parseInt(parts(3)))
}
With this I get
1, [2,3,4]
0, [7,8,9]
1, [5,6,7]
0, [6,5,7]
But if I use a reduceByKey operation with a List.concat(_,_) I get one single List with all items concated.
parsedData.reduceByKey(List.concat(_,_))
I want a List of List, reduced by the Key.
Is there some other operation that i don't know?
Thanks a lot for your help!
Here is a working example:
val data = "1,2,3,4\n0,7,8,9\n1,5,6,7\n0,6,5,7".split("\n")
val parsedData = data.map{ line =>
val parts = line.split(",")
val label = Integer.parseInt(parts(0))
(label, List(Integer.parseInt(parts(1)), Integer.parseInt(parts(2)), Integer.parseInt(parts(3))))
}.toList
//parsedData: List[(Int, List[Int])] = List((1,List(2, 3, 4)), (0,List(7, 8, 9)), (1,List(5, 6, 7)), (0,List(6, 5, 7)))
parsedData.groupBy(_._1).mapValues(_.map(_._2))
// Map(1 -> List(List(2, 3, 4), List(5, 6, 7)), 0 -> List(List(7, 8, 9), List(6, 5, 7)))
I am not sure this is concat you are looking for.
Can you try with that:
parsedData.reduceByKey(_ :: _ :: Nil)
Which should literally create a new list with your elements inside
Related
I have a list that can have certain numbers before an index that I don't need. For example:
val tempList: List[Any] = List(0,0,0,0,0,3,.,5,0,2,5)
How would I be able to remove all of the 0's before 3 without filtering out the 0 after the 3?
You can use dropWhile. Suppose you want to remove everything before 1:
scala> val l = List(0, 0, 0, 0, 1, 2, 0)
l: List[Int] = List(0, 0, 0, 0, 1, 2, 0)
scala> l.dropWhile(_ == 0)
res1: List[Int] = List(1, 2, 0)
What you need to do is a twist on dropWhile, that I would call filterWhile.
One simple solution that leverages the Collection API is the following:
def filterWhile[A](
list: List[A]
)(filterP: A => Boolean, whileP: A => Boolean): List[A] = {
val (toFilter, unfiltered) = list.span(whileP)
toFilter.filter(filterP) ++ unfiltered
}
You can play around with this code here on Scastie, alongside a couple of tests to verify it works as expected, which are the following:
def test[A](
input: List[A],
expected: List[A]
)(filterP: A => Boolean, whileP: A => Boolean): Unit =
assert(
filterWhile(input)(filterP, whileP) == expected,
s"input: $input, expected: $expected, got: ${filterWhile(input)(filterP, whileP)}"
)
test(
input = List(0, 0, 0, 0, 0, 3, '.', 5, 0, 2, 5),
expected = List(3, '.', 5, 0, 2, 5)
)(filterP = _ != 0, whileP = _ != 3)
test(
input = List(0, 0, 0, 1, 0, 3, '.', 5, 0, 2, 5),
expected = List(1, 3, '.', 5, 0, 2, 5)
)(filterP = _ != 0, whileP = _ != 3)
test(
input = List(1, 2, 3, 4),
expected = List(1, 2, 3, 4)
)(filterP = _ < 5, whileP = _ < 5)
In your case in particular all you have to do is invoke the function as follows:
filterWhile(tempList)(_ != 0, _ != 3)
Where the first predicate says how to filter and the second defines the "while" clause. I chose to align the predicate ordering to the name of the function (it first says "filter" and then "while") but feel free to adjust according to your preference. In any case, using named parameters is probably a good thing here.
Combining existing functions on lists, I came up with:
def filterUntil[A](list: List[A], filter: A => Boolean, cond: A => Boolean): List[A] = {
list.takeWhile(cond).filter(filter) ++ list.dropWhile(cond)
}
This produces much the same results as #stefanobaghino's answer.
This might be done with a foldLeft but in this case I would got for a recursive routine:
def remove(list: List[Any]): List[Any] = {
def loop(rem: List[Any], res: List[Any]): List[Any] =
rem match {
case Nil =>
res
case 0 :: tail =>
loop(tail, res)
case 3 :: _ =>
res ++ rem
case x :: tail =>
loop(tail, res :+ x)
}
loop(list, Nil)
}
remove(List(0,0,0,0,0,3,".",5,0,2,5))
// List(3, ., 5, 0, 2, 5)
Note that this won't be efficient for long lists because adding to the tail of a List gets very slow with longs lists.
Also, as noted in the comments, you should avoid Any if possible and use a more specific type.
[ Simplifications thanks to #MikhailIonkin ]
I am a beginner to Scala language. In Scala, List is Immutable as per below code:
scala> var list = List(1,2,3,4,5) // List created named ‘ list ’
list: List[Int] = List(1, 2, 3, 4, 5)
scala> 25 :: list // Prepend with Cons( :: ) , But here new list created.
res2: List[Int] = List(25, 1, 2, 3, 4, 5)
scala> list // print ‘ list ’
res3: List[Int] = List(1, 2, 3, 4, 5)
But,
scala> list
res1: List[Int] = List(1, 2, 3, 4, 5)
scala> list :+= 12 // append list with :+=
scala> list
res2: List[Int] = List(1, 2, 3, 4, 5, 12)
In above example, same "list" is appended. Then how list is immutable? It's confusing me. Any one kindly explain to me?
http://daily-scala.blogspot.co.uk/2010/03/implicit-operator.html
:+= is not just appending, it's appending to a new list and reassigning the variable to point to the new list. It's equivalent to list = list + 12.
25 ++ list is making a new list, but not assigning it anywhere.
val listA = List("one", "two", "three")
val listB = List(1, 3, 4, 9, 2, 6)
Result:
val m: Map[String, Int] = Map(one -> 1, two -> 2, three -> 3)
I want to pair listA and listB to a Map with corresponding key/value. I have tried to use the zip method but it only support sequential merging. How can I achieve the above result?
If f(x) is a function that can return the corresponding numerical value for the given alphabetical value, you can just map each x in listA to a pair (x, f(x)) and then turn the resulting list of pairs into a map using .toMap.
zip with sorted listB:
listA.zip(listB.sorted).toMap
Do you intend:
scala> val listA = List("one", "two", "three", "four")
listA: List[String] = List(one, two, three, four)
scala> val listB = List(1, 3, 4, 9, 2, 6)
listB: List[Int] = List(1, 3, 4, 9, 2, 6)
scala> val m = Map("one"->1, "two"->20, "three"->3)
m: scala.collection.immutable.Map[String,Int] = Map(one -> 1, two -> 20, three -> 3)
scala> listA flatMap { case k if (m contains k) && (listB contains m(k)) => Some(k, m(k)) ; case _ => None }
res4: List[(String, Int)] = List((one,1), (three,3))
I am very new to Scala so this question may be very naive.
I have a list like this List[Int] = List(0, 3, 6, 12, 14, 15, 16, 17). I am trying to create a list like this [(0,3),(3,6),(6,12)..] and so on. So far this is what I have tried:
val l1= List(0, 3, 6, 12, 14, 15, 16, 17)
var l2=scala.collection.mutable.ListBuffer[(Int,Int)]()
l1.zipWithIndex.slice(0,l1.length-1).foreach(x=>{val newval=(x._1,l1(x._2+1)); l2+=newval})
Two questions here:
If I don't use val newval, i.e. try to do l1.zipWithIndex.slice(0,l1.length-1).foreach(x=>l2+=(x._1,l1(x._2+1))), the compiler says:
<console>:10: error: type mismatch;
found : Int
required: (Int, Int)
l1.zipWithIndex.slice(0,l1.length-1).foreach(x=>l2+=(x._1,l1(x._2+1))). Why is that?
What would a way to do it without the mutable listbuffer?
+= is a method on the ListBuffer l2 that accepts repeated parameters. That means when you do something like this:
scala> var l2 = scala.collection.mutable.ListBuffer[(Int, Int)]()
l2: scala.collection.mutable.ListBuffer[(Int, Int)] = ListBuffer()
scala> l2 += (1, 2)
<console>:9: error: type mismatch;
found : Int(1)
required: (Int, Int)
l2 += (1, 2)
.. The compiler thinks you are trying to add multiple Ints to the ListBuffer, when you are trying to add a tuple. You need an extra set of parentheses.
l1.zipWithIndex.slice(0,l1.length-1).foreach(x=> l2 += ((x._1,l1(x._2+1)) ))
You can use sliding, which will create a "sliding window" across the collection to return a list of lists of a specific group size, with a step size of one by default:
scala> List(0, 3, 6, 12, 14, 15, 16, 17).sliding(2)
.map { case List(a, b) => (a, b) }.toList
res10: List[(Int, Int)] = List((0,3), (3,6), (6,12), (12,14), (14,15), (15,16), (16,17))
besides the sliding, you could slide like following:
val l1= List(0, 3, 6, 12, 14, 15, 16, 17)
val l2 = l1.take(l1.size - 1).zip(l1.tail)
updated
l1.zip(l1.tail) works.
Suppose you are given the following list: {1,0,0,3,4,0,8,0,5,6,0}. Is there any way I can assign a particular index to all the 0s in the list in SCALA? This index must then be used as a parameter to another function.
Not exactly sure what you mean, but perhaps this will give you some ideas:
scala> val list = List(3, 4, 0, 0, 3, 0, 2)
list: List[Int] = List(3, 4, 0, 0, 3, 0, 2)
scala> val indexed = list.zipWithIndex
indexed: List[(Int, Int)] = List((3,0), (4,1), (0,2), (0,3), (3,4), (0,5), (2,6))
scala> val zeroIndices = indexed collect { case (value, index) if value == 0 => index }
zeroIndices: List[Int] = List(2, 3, 5)
Bonus:
scala> zeroIndices map list
res1: List[Int] = List(0, 0, 0)