I've struggled to find a way to stay true to functional style in for expressions when I need to collect multiple parameters of an object into a List.
As an example, say I have a Notification object, which has both a fromId (the user id the notification is from) and an objectOwnerId (the id of the user who created the original object). These can differ in facebook style notifications ("X also commented on Y's post").
I can collect the userIds with a for expression like so
val userIds = for { notification <- notifications } yield notification.fromId
however say I want to collect both the fromIds and the objectOwnerIds into a single list, is there any way to do this in a single for expression without the user of vars?
I've done something like this in the past:
var ids = List()
for {
notification <- notifications
ids = ids ++ List(notification.fromId, notification.objectOwnerId)
}
ids = ids.distinct
but it feels like there must be a better way. The use of a var, and the need to call distinct after I complete the collection are both ugly. I could avoid the distinct with some conditionals, but I'm trying to learn the proper functional methods to do things.
Thanks in advance for any help!
For such cases, there is foldLeft:
(notifications foldLeft Set.empty[Id]) { (set, notification) =>
set ++ Seq(notification.fromId, notification.ownerId)
}
or in short form:
(Set.empty[Id] /: notifications) { (set, notification) =>
set ++ Seq(notification.fromId, notification.ownerId)
}
A set doesn't hold duplicates. After the fold you can convert the set to another collection if you want.
val userIds = for {
notification <- notifications
id <- List(notification.fromId, notification.objectOwnerId)
} yield id
Apply distinct afterwards if required. If the id can only be duplicated on a single notification, you can apply distinct on the second generator instead.
Sure, instead of just yielding the fromId, yield a tuple
val idPairs:List[(String, String)] = for(notification <- notifications) yield(notification.fromId, notification.objectOwnerId)
Well, here is my answer to the following:
How to map from [Y(x1, x2), Y(x3, x4)] to [x1,x2,x3,x4]?
Use flatMap (see Collection.Traversable, but note it's actually first defined higher up).
case class Y(a: Int, b: Int)
var in = List(Y(1,2), Y(3,4))
var out = in.flatMap(x => List(x.a, x.b))
> defined class Y
> in: List[Y] = List(Y(1,2), Y(3,4))
> out: List[Int] = List(1, 2, 3, 4)
Also, since for..yield is filter, map and flatMap in one (but also see "sugar for flatMap?" that points out that this isn't as efficient as it could be: there is an extra map):
var out = for { i <- in; x <- Seq(i.a, i.b) } yield x
I would likely pick one of the other answers, however, as this does not directly address the final problem being solved.
Happy coding.
You can also use Stream to transform the pairs into a stream of individual items:
def toStream(xs: Iterable[Y]): Stream[Int] = {
xs match {
case Y(a, b) :: t => a #:: b #:: toStream(t)
case _ => Stream.empty
}
}
But like pst said, this doesn't solve your final problem of getting the distinct values, but once you have the stream it's trivial:
val result = toStream(ys).toList.removeDuplicates
Or a slight modification to the earlier suggestions to use flatten - add a function that turns a Y into a List:
def yToList(y: Y) = List(y.a, y.b)
Then you can do:
val ys = List(Y(1, 2), Y(3, 4))
(ys map yToList flatten).removeDuplicates
I agree with Dave's solution but another approach is to fold over the list, producing your map of id to User object as you go. The function to apply In the fold will query the db for both users and add them to the map being accumulated.
What about simple map? AFAIK for yield gets converted to series of flatMap and map anyway. Your problem could be solved simply as follows:
notifications.map(n => (n.fromId, n.objectOwnerId)).distinct
Related
I am just trying to figure out how immutable things like a List are working, and how I can add things to it?
I am very sorry for asking such dumb questions, but why is here my list always empty when printing it out?
var end = false
val list = List()
while (!end) {
val input = scala.io.StdIn.readLine("input:")
if (input == "stop" ) end = true
else input :: list
}
println(list)
}
Sorry for my inconvenience and this rather stupid question!
I am just trying to figure out how immutable things like a List are working, and how I can add things to it?
You can't. That's what immutable means, after all. If Latin is not your cup of tea, the English translation of immutable is unchangeable. It should be clear now, why you can't change something that is unchangeable.
I am very sorry for asking such dumb questions, but why is here my list always empty when printing it out?
You create an empty list, and you never change it (because it cannot be changed anyway). So, of course it is empty.
What can you can do, however, is create a new list which is almost exactly like the old list, except with a new item prepended to the front. That's what you are doing here:
input :: list
However, you don't assign this new list anywhere, you don't return it, you completely ignore it.
If you want to actually use your list in any way, you need to remember it somehow. The most obvious solution would be to assign it to a variable:
var end = false
var list: List[String] = List() // note: `var` instead of `val`
while (!end) {
val input = scala.io.StdIn.readLine("input:")
if (input == "stop" ) end = true
else list = input :: list // note: assign to `list`
}
println(list)
However, that's not very idiomatic. After all, we have now taken an immutable list and assigned it to a mutable variable … IOW, we have just moved the mutability around.
Instead, we could use a recursive solution:
def buildListFromInput(list: List[String] = List()): List[String] = {
val input = scala.io.StdIn.readLine("input:")
if (input == "stop") list else buildListFromInput(input :: list)
}
println(buildListFromInput())
This solution is not only recursive, the recursive call is also in tail position (IOW, the method is tail-recursive), which means that it will be just as efficient as a while loop (in fact, it will be compiled into a while loop, or more precisely, into a GOTO). The Scala Language Specification guarantees that all implementations of Scala must eliminate direct tail-recursion.
The reason
println(list)
is only printing out an empty list is because the bit
input :: list
isn't actually mutating the list itself. It is simply, in this case, very temporarily, creating a list containing the input at the front.
Try
println(input :: list)
or
val newList = input :: list
println(newList)
and you'll see what I mean.
Try rewriting the code in more functional way. Every operation on Immutable data structures return new instance with change. So :: operator creates new List with input on front. You might want to try rewrite this code as tail recursive function as follows.
#tailrec
def scanInput(continue: Boolean,acc: List[String]): List[String] = {
val input = scala.io.StdIn.readLine("input:")
if(!continue) acc
else scanInput(input != "stop", input :: acc)
}
Above code has no mutating state and it suits more Scala functional style.
In scala List is immutable.
Then how can I add items to the list?
When you add an item to list a new List instance is crated with a item as its head and its tail now contains the previous list.
If you have list of "1,2,3" called intList internally it is represented as
List(3, List(2, List(1, Nil) ) )
If you add an element 4 to this intList
List(4, intList )
Lets call this newList
Note intList still contains List(3, List(2, List(1, Nil) ) ).
If you want the intList to refer the newList You will have to do
intList = intList.add(4)
How can I fix my code
Change list from val to var. Then you can assign resulting List to list variable
list = input :: list
Source: Online course on Scala called Functional Programming Principles in Scala
Thank you for all your help, I appreciate your help so much from all of you!
I should have taken a closer look at recursion since it seems to be really important like in Scala!
But trough your help I am getting an better idea of how it works!
I just tried to figure out how your solutions are working and created my own:
val list = List()
def scanInput(acc: List[String]): List[String] = {
val input = scala.io.StdIn.readLine("input:")
input match {
case "stop" => acc
case _ => scanInput(input :: acc)
}
}
println(scanInput(list))
I want to take this val:
val f = List(Future(1), Future(2), Future(3))
Perform some operation on it (I was thinking flatten)
f.flatten
And get this result
scala> f.flatten = List(1,2,3)
If the flatten method isn't appropriate here, that's fine. As long as I get to the result.
Thanks!
Future.sequence takes a List[Future[T]] and returns a Future[List[T]].
You can do
Future.sequence(f)
and then use map or onComplete on it to access the list of values.
This is a combination of a stylistic question, and my attempts to broaden my Scala understanding.
I've got a list containing Future's, I want to compute the values of the futures, transform into Option's, and flatten the list using a for comprehension:
import scala.util.Try
import scala.concurrent._
import ExecutionContext.Implicits.global
val l= List(Future.successful(1),Future.failed(new IllegalArgumentException))
implicit def try2Traversible[A](xo: Try[A]): Iterable[A] = xo.toOption.toList
val res = for{f <- l; v <- f.value} yield v
scala> res: List[scala.util.Try[Int]] = List(Success(1), Failure(java.lang.IllegalArgumentException))
res.flatten
res16: List[Int] = List(1)
What I want to do is get the flatten stage into the for comprehension, anyone got any suggestions?
Doing this is incorrect:
for{f <- l; v <- f.value} yield v
It appears to work in your case only because the futures are already fulfiled, which is why their value member is defined.
However in the general case they might not yet be fulfilled when you execute the for comprehension, and thus value will return None
(despite the fact that at some point they will eventually be fulfilled).
By example, try this in the REPL:
val f1 = Future{
Thread.sleep(3000) // Just a test to illustrate, never do this!
1
}
val f2 = Future{
Thread.sleep(3000) // Just a test to illustrate, never do this!
throw new IllegalArgumentException
}
val l = List( f1, f2 )
for{f <- l; v <- f.value} yield v
The result is an empty list, because none of the futures in l is fulfilled yet. Then wait a bit (at most 3 seconds) and reexecute the for comprehension (the last line), and you will get a non empty list because the futures have finally been fulfilled.
To fix this, you will have to either block (that is, wait for all the futures to be fulfiled) using scala.concurrent.Await, or stay in the asynchronous world by using something like Future.map or Future.flatMap.
By example, if you want to block, you could do:
Await.result( Future.sequence( l ), duration.Duration.Inf )
Await.result waits for the result of the future, allowing to go from the asynchronous world to the synchronous world. The result of the above is a List[Int]
The problem now is that you lose the failure cases (the result is not List[Try[Int]] as you wanted), and will actually rethrow the first exception.
To fix this, you can use this helper method that I posted in another answer: https://stackoverflow.com/a/15776974/1632462
Using it, you can do:
Await.result( Future.sequence( l map mapValue ), duration.Duration.Inf )
This will wait until all the futures are fulfiled (either with a correct value, or with an error) and return the expected List[Try[Int]]
The idea is to traverse to Try object as if it were an Option (i.e. a 0 or 1 element collection) within the for-comprehension itself.
For this traversal to work there has to be a conversion from the Try type to the Option type.
This should work:
implicit def try2option[A](xo: Try[A]) = xo.toOption
val res = for (f <- l; t <- f.value; x <- t) yield x
You should keep a Future around your final result to retain the asynchronous nature of the computation.
The nice way to do this (and obtain a Future[List[Int]]) would be (probably what you tried):
for {
f <- l // Extract individual future
v <- f // Extract value from future
} yield v
Unfortunately this translates to:
l.flatMap(f => f.map(v => v))
Which does not work, because Future does not inherit GenTraversableOnce (and probably shouldn't), but List needs this trait for its flatMap.
However, we can do this manually:
val res = l.foldRight(Future.successful(List.empty[Int])) {
case (x,xs) => xs.flatMap(vxs => x.map(vx => vx :: vxs))
}
We can use Future.sequence to do that:
Future.sequence(l)
This will return a Future[List[Int]] which only completes when all futures are completed and will contain all values of the futures that completed successfully.
I have defined a type like that:
type s_program =
{ globals : s_var list;
main: s_block; }
and s_var =
{ s_var_name: string;
s_var_type: s_type;
s_var_uniqueId: s_uniqueId }
and s_uniqueId = int
At a point of my program, i have a variable p: s_program, but I need to change s_var_uniqueId of every element of p.globals, for instance, add 1 to every s_var_uniqueId. I have some questions:
1) May I modify directly the related values in p, or do I have to assign new values to a new p':s_program
2) May I write something like that:
let p' =
{ p with
globals = List.map (fun var -> { var with s_var_uniqueId = var.s_var_uniqueId + 1 }) p.globals
Thank you very much.
Edit 1: correct with part as suggested
First you may want to read this section on record in OCaml and this chapter on modifiable data structures. As stated, you will see that records are not mutable. Second, you need to think about what you really want : mutable record, record with mutable fields or mutable record with mutable fields. For example suppose that we have a record for complex numbers (this is the same example in ref 1).
type complex = { re:float; im:float } ;;
If you declare something like
let c = {re=2.;im=3.} ;;
then you cannot change neither c nor re (or im). Indeed c := {re=4.;im=6.} ;; or c.re := 4.;; will both failed with an error. To get a mutable record, you just have to use a reference for c.
let c = ref {re=2.;im=3.} ;;
Then you can change c with c := {re=4.;im=6.} ;;. But I think that you want to have mutable field ! Then you have to precise which fields are mutables. Suppose that you want all fields to be mutable you can write
type complex = { re:float ref; im:float ref }
let make_complex r i = { re = ref r ; im = ref i }
let c = make_complex 3. 4.
;;
and then change fields with
c.re := 6. ; c.im := 7. ;;
It would be easier with a function of type float -> float -> ()
let change_re c r = c.re := r ;;
let change_im c i = c.im := i ;;
let change_complex c r i = change_re c r ; change_im c i ;;
However, I will suggest you to really think about bringing imperative feature like this in your code. This could be a simple way to completly wreck its readibility.
1) That depends whether you want to use Ocaml as an imperative or functional programming language :). If it's the former, you can make both fields of the records mutable (by adding mutable keyword before field name) and then change them in place. However I'd strongly advice not to do that and to:
2) go with your second approach, which looks perfectly fine (except that you seem to be missing {...} for the second record modification.
is there an alternative 'List' syntax in Scala?
Is it possible to define one aditional class/type/operator* called '[' and ']'?
I know 'square brackets' are used to indicate Type, but they are perfect to the repetitive task of declaring lists.
A ';' or '?' would be good also, as a last resource.
Thanks
obs.:
after much search the only alternative I found was to use 'cons':
val list = 1 :: 2 :: 3 :: Nil
but it doesn't reduce any key typing at all.
I am still learning those things in Scala
EDIT:
Just to clarify: Performance is not a priority in my case. And yes, shift is not welcome. :P
Motivation behind the scenes: I like Haskell style, but cannot use it directly with Java.
EDIT 2:
Final solution based on both Rex Kerr solutions
implementing object Types:
package a
object Types {
type \[z] = List[z]
implicit def make_lists[A](z: A) = new ListMaker(z)
class ListMaker[A](a0: A) {
private[this] val buffer = List.newBuilder[A]
buffer += a0
def \(z: A) = {
buffer += z;
this
}
def \\ = buffer.result
}
}
using object Types:
package a
import a.Types._
object Whatever {
def listInListOut (l: \[Int]) = {
1\2\\
}
}
[ and ] are reserved symbols in Scala which are used for type annotations. You can't use them for lists. ; is reserved for end of line. You could use ? in many cases, but it would be awkward.
I recommend that you learn to use the :: notation (and get used to typing the : symbol fast twice in succession) because it really makes the list operations visually clear, plus it is a great syntactic reminder that lists are weird because you put things on the head of the list.
However, if you cannot tolerate this, your best option is probably to define a one-letter list symbol. For example,
List(1,2,3,4)
is a list of the numbers from 1 to 4. What if you could just type L instead of List? It turns out that you can, since this is not a fancy constructor or static method, but a singleton companion object to the class List. So you just
val L = List
L(1,2,3,4)
and you are just one character worse off than your suggestion of brackets.
Define
def l[A](a:A*) = List(a:_*)
Then you can do
l(1,2,3)
which is only one character more than [1,2,3]
I can't help pointing out another way to go here for lists where all the elements are the same type, if you really hate the shift key and don't care if other people can understand your code:
class ListMaker[A](a0: A) {
private[this] val buffer = List.newBuilder[A]
buffer += a0
def \(a: A) = { buffer += a; this }
def \\ = buffer.result
}
implicit def make_lists[A](a: A) = new ListMaker(a)
Now you can list to your heart's content, without ever touching the shift key!
scala> val a = 1\2\3\4\5\\
a: List[Int] = List(1, 2, 3, 4, 5)
scala> val b = 'a'\'b'\\
b: List[Char] = List(a, b)
scala> val c = false\true\false\false\false\false\true\\
c: List[Boolean] = List(false, true, false, false, false, false, true)
This uses exactly as many characters as brackets would. (It doesn't nest well, however.)
Welcome to Scala version 2.10.0.r24777-b20110419020105 (Java HotSpot(TM) Client VM, Java 1.6.0
Type in expressions to have them evaluated.
Type :help for more information.
scala> class LM[A](x: A) {
| def \(y: A) = List(x,y)
| }
defined class LM
scala> implicit def a2l[A](x: A): LM[A] = new LM(x)
a2l: [A](x: A)LM[A]
scala> class LX[A](xs: List[A]) {
| def \(y: A) = xs:::List(y)
| }
defined class LX
scala> implicit def l2lx[A](xs: List[A]): LX[A] = new LX(xs)
l2lx: [A](xs: List[A])LX[A]
scala> 1\2
res0: List[Int] = List(1, 2)
scala> 1\2\3
res1: List[Int] = List(1, 2, 3)
scala>
Not exactly an alternative syntax, but it is by far the most portable solution:
In Intellij IDEA it is possible to create "Live Templates";
press Ctrl+Alt+s; search for "template"; go to "Live Templates" section;
just add one new item named "l" inside Scala entry, add a random description and the following code:
List($END$)
Press Enter, go to the editor, press L followed by Tab.
It is the end of your typing pains.
Do the same for Arrays.