Related
I have some raw test data that I need to split into a map of format:
Map[String, List[(Int, String, Float)]]
I have managed to read in the data as a list and will give an example of one line of data below:
Oor Wullie Route (GCU),1:City Chambers:0.75f,2:Sir Chris Hoy Velodrome:3.8f,3:People's Palace:2.7f,4:Riverside Museum:5.4f,5:Botanic Gardens:2.4f,6:GCU:3.4f
The above represents the following: A Route, a stage number:stage name:total distance of stage
So each set of 3 values (i.e 1:City Chambers:5) should be added to the [Int, String, Float] section of the map, with the route name being the key.
This is my code so far for reading the file and adding it to a list:
var mapBuffer: Map[String, List[(Int, String, Float)]] = Map()
val fitnessData = "C:\\Users\\josep\\Desktop\\Coursework\\Coursework\\src\\cw.txt"
val lines = Source.fromFile("C:\\Users\\josep\\Desktop\\Coursework\\Coursework\\src\\cw.txt").getLines.toList
I would like to write a funciton for splitting the data up and adding it to a map, essentially doing this:
var key ="Oor Wullie Route (GCU)"
var newList = List((1,"City Chambers",0.75f),(2,"Sir Chris Hoy Velodrome",3.8f),(3,"People's Palace",2.7f),(4,"Riverside Museum",5.4f),(5,"Botanic Gardens",2.4f),(6,"GCU",3.4f))
mapBuffer = mapBuffer ++ Map(key -> newList)
How can I add the data to a map in my desired format?
My suggestion would be to use foldLeft. Something like:
val resource = Source.fromFile("src/lines.txt")
val lines = resource.getLines.toList
resource.close()
val map = lines.foldLeft(Map[String, List[(Int, String, Float)]]())((map, line) => {
val keyValuesArray = line.split(",").toList
val key = keyValuesArray.head
val listOfValuesAsString = keyValuesArray.tail
val listOfValues = listOfValuesAsString.map {
case s"$integer:$string:$float" => (integer.toInt, string, float.toFloat)
}
map + (key -> listOfValues)
})
Start with empty map, and add key->values for each line.
Also, try match expressions when you parse data in list (listOfValues part is doing that).
This approach is with pattern matching and tail recursion.
I think it works very well.
First I convert the file into a List[Array[String]].
Second I call loop to go through the list in a recursive way and build the map.
Third inside the loop function I call make List to build the list of tuples in a recursive way.
As an example:
input
Oor Wullie Route (GCU),1:City Chambers:0.75f,2:Sir Chris Hoy Velodrome:3.8f,3:People's Palace:2.7f,4:Riverside Museum:5.4f,5:Botanic Gardens:2.4f,6:GCU:3.4f
Oor Wullie Route2 (GCU),1:City Chambers:0.75f,2:Sir Chris Hoy Velodrome:3.8f,3:People's Palace:2.7f,4:Riverside Museum:5.4f,5:Botanic Gardens:2.4f,6:GCU:3.4f
Oor Wullie Route3 (GCU),1:City Chambers:0.75f,2:Sir Chris Hoy Velodrome:3.8f,3:People's Palace:2.7f,4:Riverside Museum:5.4f,5:Botanic Gardens:2.4f,6:GCU:3.4f
code
import scala.io.Source
object ConverToMap {
#annotation.tailrec
def makeList(lst: List[String], acc: List[(Int, String, Float)]):List[(Int, String, Float)] = {
lst match {
case Nil => acc
case (h :: t) => {
val data = h.split(":")
val tuple = (data(0).toInt, data(1), data(2).substring(0,data(2).length - 1 ).toFloat)
makeList(t, tuple :: acc)
}
}
}
#annotation.tailrec
def loop(lst: List[Array[String]], acc: Map[String, List[(Int, String, Float)]]): Map[String, List[(Int, String, Float)]] = {
lst match {
case Nil => acc
case (h :: t) => {
val key = h(0)
val lTuple = makeList(h.toList.tail, List())
if(acc.contains(key)) loop(t, acc)
else loop(t, Map(key -> lTuple) ++ acc)
}
}
}
def main(args: Array[String]): Unit = {
val fitnessData = "/home/cloudera/files/tests/to_map.csv"
val lines = Source.fromFile(fitnessData)
.getLines
.toList
.map(line => line.split(","))
val mp = loop(lines, Map())
println(mp)
}
}
expected result
Map(Oor Wullie Route3 (GCU) -> List((6,GCU,3.4), (5,Botanic Gardens,2.4), (4,Riverside Museum,5.4), (3,People's Palace,2.7), (2,Sir Chris Hoy Velodrome,3.8), (1,City Chambers,0.7)),
Oor Wullie Route2 (GCU) -> List((6,GCU,3.4), (5,Botanic Gardens,2.4), (4,Riverside Museum,5.4), (3,People's Palace,2.7), (2,Sir Chris Hoy Velodrome,3.8), (1,City Chambers,0.7)),
Oor Wullie Route (GCU) -> List((6,GCU,3.4), (5,Botanic Gardens,2.4), (4,Riverside Museum,5.4), (3,People's Palace,2.7), (2,Sir Chris Hoy Velodrome,3.8), (1,City Chambers,0.7)))
I am trying to create a new list that combines the contents of one single-level list with the hierarchy of another nested list and link them by the IDs of objects in both lists. Note: the ID for the BChain object is stored within the ItemObj inside it.
I have three objects:
ItemObj(id: String, stored: DateTime)
BChain(item: ItemObj, List[BChain])
AChain(aid: String, List[AChain])
I have two lists:
val nestedList: List[AChain]
val singleLevelList: List[BChain]
I want the second list to have the hierarchy of the first, but still contain it's own elements. Therefore, BChain should include the original ItemObj and List[BChain] properties (including all of ItemObj's original property data - ID and DateTime) when it is put into the newly desired list output.
So Instead of (input):
val nestedList = List(
AChain("123", List(AChain("456", [])),
AChain("789", [])
)
val singleLevelList = List(
BChain(ItemObj("123", DateTime), []),
BChain(ItemObj("456", DateTime), []),
BChain(ItemObj("789", DateTime), []))
)
I would like the following output:
val combinedLists = List(
BChain(ItemObj("123", DateTime), List(BChain(ItemObj("456", DateTime), [])),
BChain(ItemObj("789", DateTime), [])
)
as the final list.
Note: There may be more items in the nested list than the single-level list, and if this is the case then the extra items should be ignored. Each of the items in the single-level list should correspond to one of the items in the nested list.
How can I accomplish this?
Any help would be appreciated.
First I had to dummy up a DateTime type and value so that the following would compile.
case class ItemObj(id: String, stored: DateTime)
case class BChain(item: ItemObj, bcl :List[BChain])
case class AChain(aid: String, acl :List[AChain])
Then I changed your singleLevelList into real Scala code and created a Map for fast lookup.
val singleLevelList = List(
BChain(ItemObj("123", DateTime), Nil),
BChain(ItemObj("456", DateTime), Nil),
BChain(ItemObj("789", DateTime), Nil)
)
val sLLMap = singleLevelList.groupBy(_.item.id)
Next a recursive method to change all AChains to BChains.
def a2b(aLst :List[AChain]) :List[BChain] =
aLst.map(a => BChain(sLLMap(a.aid).head.item, a2b(a.acl)))
Now to test it.
val nestedList = List(
AChain("123", List(AChain("456", Nil))),
AChain("789", Nil)
)
a2b(nestedList) //appears to work
Of course this will throw if nestedList has an A without a corresponding B in the singleLevelList.
Maybe something like this is what you want?
final case class ItemObj(id: String, stored: DateTime)
final case class BChain(item: ItemObj, list: List[BChain])
final case class AChain(aid: String, list: List[AChain])
def combine(as: List[AChain], bs: List[BChain]): List[BChain] = {
val asMap = as.iterator.map(a => a.aid -> a.list).toMap
def toBChain(a: AChain): BChain =
BChain(
item = ItemObj(id = a.aid, stored = ???),
list = a.list.map(toBChain)
)
bs.map {
case BChain(item, list) =>
val newElements =
asMap
.getOrElse(key = item.id, default = Nil)
.map(toBChain)
BChain(
item,
list ::: newElements
)
}
}
This code will work even if root element of nested list is absent in single level list
final case class ItemObj(id: String, stored: DateTime = null)
final case class AChain(aid: String, children: List[AChain] = List())
final case class BChain(bid: ItemObj, children: List[BChain] = List())
val nestedList = List(
AChain("123", List(AChain("456"))),
AChain("789")
)
def getCombineList(nestedList: List[AChain], singleList: List[BChain]): List[BChain] = {
val singleListMap = singleList.groupBy(_.bid.id)
def combine(items: List[AChain]): List[BChain] = {
items flatMap {item =>
val children = combine(item.children)
val parent = singleListMap.get(item.aid).map(v =>
List(v.head.copy(children = children))
).getOrElse(
children
)
parent
}
}
combine(nestedList)
}
val s = getCombineList(nestedList, List(BChain(ItemObj("456")), BChain(ItemObj("789"))))
I have a List[T] where the datas inside are like below:
[0]- id : "abc", name: "xyz", others: "value"
[1]- id : "bcd", name: "dsf", others: "value"
Now I want to return the same List[T] but with the id and names i.e the returning List will be:
[0]- id : "abc", name: "xyz"
[1]- id : "bcd", name: "dsf"
I tried with below code:
var temps = templates.map(x => List("id" -> x.id, "name" -> x.name))
But it it produces List inside List i.e:
[0]-
[0] id : "abc"
[1] name: "xyz"
[1]-
[0] id : "bcd"
[1] name: "dsf"
I tried with tuple also but in vain. How can i just map my list such that everything is cleaned out except the id and name value pair??
Unless you want to define a new class with just the id and name fields, I think tuples would be your best bet:
scala> case class obj(id: String, name: String, others: String)
defined class obj
scala> val l = List(new obj("abc", "xyz", "value"), new obj("bcd", "dsf", "value"))
l: List[obj] = List(obj(abc,xyz,value), obj(bcd,dsf,value))
scala> l.map(x => (x.id, x.name))
res0: List[(String, String)] = List((abc,xyz), (bcd,dsf))
Also you are actually using tuples in your example, the -> syntax creates tuples in Scala:
scala> "a" -> "b"
res1: (String, String) = (a,b)
Here is the "define another class" option:
scala> case class obj2(id: String, name: String){
| def this(other: obj) = this(other.id, other.name)
| }
defined class obj2
scala> l.map(new obj2(_))
res2: List[obj2] = List(obj2(abc,xyz), obj2(bcd,dsf))
Given that the List[T] is a List[Map], then you may able to do the following:
//Remove the "others" key.
val temps = templates.map( map => {map - "others"})
I think its pretty clear that your list contains objects with three members: id, name, others. Hence you access them by using
{x => x.name}
What I am not so sure about, is how you imagine your end result. You obviously need some data structure, that holds the members.
You realized yourself, that its not very nice to store each objects members in a list inside the new list, but you seem to not be ok with tuples?
I can imagine, that tuples are just what you want.
val newList = oldList.map{e => e.id -> e.name}
results in alist like this:
List(("id_a", "name_a"), ("id_b", "name_b"), ("id_c","name_c"))
and can be accessed like this(for example):
newList.head._1
for the first tuples ids, and
newList.head._2
for the first tuples name.
Another option could be mapping into a map, since this looks pretty much like, what you want in the first place:
newMap = oldList.map{e => e.id -> e.name}.toMap
This way you can access members like this newMap("key") or safer: newMap.get("key") which returns an option and wont end up in an exception, if the key doesn't exist.
Let's say that I have a Map of strings -> List of Integers. I would like to create a function which takes in as a parameter a List of strings and returns all the integers correlating to all the string in that list. I.e. if the Map X contains the following mappings:
database = [("Bob",[1,2,3]),("John",[1,5,6]),("Trevor",[4,5,7])]
If this function takes in ["Bob","John"] as the list of names, it should return,
[1,2,3,5,6]
Since Bob correlates to 1,2,3 and John correlates to 1,5,6 (same entries for both names aren't duplicated). I also would like to not introduce a mutable variable if I don't have to, thus leading me to believe a for comprehension that yields this list of number values would be the best way to achieve this, but I'm not sure how.
If you want to use a for-comprehension you can so this:
val result = for {
key <- keys
nums <- map.get(key).toSeq
num <- nums
} yield num
result.distinct
Explanation:
for each key in the list try to get an entry and convert it to a Seq (necessary because flatMap expects a Seq in this case) and add every number in the list to the result. If the key is not present in the map the collection will be empty and therefore not yield any results. At the end call distinct to remove the duplicates.
val myMap = Map("Bob" -> List(1,2,3), "John" -> List(1,5,6), "Trevor" -> List(4,5,7))
val names = List("Bob", "John")
You can add default value to Map using method withDefaultValue:
val mapWithDefaul = myMap withDefaultValue Nil
Then you could use Map as function in flatMap:
names.flatMap(mapWithDefaul).distinct
// List(1, 2, 3, 5, 6)
Let
val db = Map("Bob" -> List(1,2,3), "John" -> List(1,5,6), "Trevor" -> List(4,5,7))
val names = List("Bob", "John")
Then a similar approach to #senia's using flatMap,
implicit class mapCorr[A,B](val db: Map[A,List[B]]) extends AnyVal {
def corr(keys: List[A]): List[B] = {
keys.flatMap{ k => db get k }.flatten.distinct
}
}
and
scala> db.corr(keys)
res0: List[Int] = List(1, 2, 3, 5, 6)
Here we allow for key lists of type A and maps from type A to type List[B] .
val myset = Set("Bob","John")
val database = Map(("Bob"->List(1,2,3)),("John"->List(1,5,6)),("Trevor"->List(4,5,7)))
val ids = database.filter(m => myset.contains(m._1)).map(_._2).flatten.toList.distinct
outputs:
ids: List[Int] = List(1, 2, 3, 5, 6)
Something like:
val result = database.filter(elem => list.contains(elem._1)).foldLeft(List())((res,elem) => res ++ elem._2)
where list is the input list of names.
I'm new to scala and I'm trying to remove from a list of tuples elements which their first value is bigger than the second.
For example, From the list:
val list = List[(Int,Int)]((1,3),(3,1),(2,2))
I want to get the list:
val list = List[(Int,Int)]((1,3),(2,2))
So I used the following lines:
var newList = List[(Int, Int)]()
for (element <- list) {
if (element._1 <= element._2) {
newList ::= element;
}
}
But it feels very long for scala.. Is there a shorter way?
Like twillouer's and tzofia's solutions, but with pattern matching:
list filter { case (a, b) => a <= b }
You can simply do:
list.filter(element => element._1 <= element._2)
The filter function filters out elements which do not satisfy the given boolean condition.
you can use filter like this :
scala> val list = List[(Int,Int)]((1,3),(3,1),(2,2))
list: List[(Int, Int)] = List((1,3), (3,1), (2,2))
scala> val newList = list.filter(a => a._1 <= a._2)
newList: List[(Int, Int)] = List((1,3), (2,2))
or filterNot for the example :
scala> val newList = list.filterNot(a => a._1 > a._2)
newList: List[(Int, Int)] = List((1,3), (2,2))