I have a list of Futures and I want to get the first one completed with a certain condition.
Here is an example of a possible code:
val futureList: Future[List[T]] = l map (c => c.functionWithFuture())
val data = for {
c <- futureList
}yield c
data onSuccess {
case x => (x filter (d=> d.condition)).head
}
But it's not efficient, because I'll take only one element of the list, but computed all of them with a lot of latency.
I know firstCompletedOf but it's not what I'm searching.
(Sorry for my bad English.)
Try using a Promise and calling trySuccess on it as soon as a future that satisfies the condition completes. The first to call trySuccess will complete the future, the following ones will have no effect (as opposed to calling success, which can only be called once on a Promise).
Keep in mind that if no future in the list satisfies the condition, you will never have a result, i.e. the promise future will never complete.
import scala.concurrent.{ Await, Future, Promise }
import scala.concurrent.duration._
import scala.concurrent.ExecutionContext.Implicits.global
import scala.util.Random
def condition(x: Int) = x > 50
val futures = (1 to 10) map (x => Future {
// wait a random number of ms between 0 and 1000
Thread.sleep(Random.nextInt(1000))
// return a random number < 100
Random.nextInt(100)
})
val p = Promise[Int]()
// the first one that satisfies the condition completes the promise
futures foreach { _ filter condition foreach p.trySuccess }
val result = p.future
// Watch out: the promise could never be fulfilled if all the futures are <=50
println("The first completed future that satisfies x>50 is: " + Await.result(result, 10.seconds))
Related
My application manages bookings of a user. These bookings are composed by a start_date and end_date, and their current partition in dynamodb is the following:
PK SK DATA
USER#1#BOOKINGS BOOKING#1 {s: '20190601', e: '20190801'}
[GOAL] I would query all reservations which overlap a search time interval as the following:
I tried to find a solution for this issue but I found only a way to query all items inside a search time interval, which solves only this problem:
I decided to make an implementation of it to try to make some change to solve my problem but I didn't found a solution, following you can find my implementation of "query inside interval" (this is not a dynamodb implementation, but I will replace isBetween function with BETWEEN operand):
import { zip } from 'lodash';
const bookings = [
{ s: '20190601', e: '20190801', i: '' },
{ s: '20180702', e: '20190102', i: '' }
];
const search_start = '20190602'.split('');
const search_end = '20190630'.split('');
// s:20190601 e:20190801 -> i:2200119900680011
for (const b of bookings) {
b['i'] = zip(b.s.split(''), b.e.split(''))
.reduce((p, c) => p + c.join(''), '');
}
// (start_search: 20190502, end_search: 20190905) => 22001199005
const start_clause: string[] = [];
for (let i = 0; i < search_start.length; i += 1) {
if (search_start[i] === search_end[i]) {
start_clause.push(search_start[i] + search_end[i]);
} else {
start_clause.push(search_start[i]);
break;
}
}
const s_index = start_clause.join('');
// (end_search: 20190905, start_search: 20190502) => 22001199009
const end_clause: string[] = [];
for (let i = 0; i < search_end.length; i += 1) {
if (search_end[i] === search_start[i]) {
end_clause.push(search_end[i] + search_start[i]);
} else {
end_clause.push(search_end[i]);
break;
}
}
const e_index = (parseInt(end_clause.join('')) + 1).toString();
const isBetween = (s: string, e: string, v: string) => {
const sorted = [s,e,v].sort();
console.info(`sorted: ${sorted}`)
return sorted[1] === v;
}
const filtered_bookings = bookings
.filter(b => isBetween(s_index, e_index, b.i));
console.info(`filtered_bookings: ${JSON.stringify(filtered_bookings)}`)
There’s not going to be a beautiful and simple yet generic answer.
Probably the best approach is to pre-define your time period size (days, hours, minutes, seconds, whatever) and use the value of that as the PK so for each day (or hour or whatever) you have in that item collection a list of the items touching that day with the sort key of the start time (so you can do the inequality there) and you can use a filter on the end time attribute.
If your chosen time period is days and you need to query across a week then you’ll issue seven queries. So pick a time unit that’s around the same size as your selected time periods.
Remember you need to put all items touching that day (or whatever) into the day collection. If an item spans a week it needs to be inserted 7 times.
Disclaimer: This is a very use-case-specific and non-general approach I took when trying to solve the same problem; it picks up on #hunterhacker 's approach.
Observations from my use case:
The data I'm dealing with is financial/stock data, which spans back roughly 50 years in the past up to 150 years into the future.
I have many thousands of items per year, and I would like to avoid pulling in all 200 years of information
The vast majority of the items I want to query spans a time that fits within a year (ie. most items don't go from 30-Dec-2001 to 02-Jan-2002, but rather from 05-Mar-2005 to 10-Mar-2005)
Based on the above, I decided to add an LSI and save the relevant year for every item whose start-to-end time is within a single year. The items that straddle a year (or more) I set that LSI with 0.
The querying looks like:
if query_straddles_year:
# This doesn't happen often in my use case
result = query_all_and_filter_after()
else:
# Most cases end up here (looking for a single day, for instance)
year_constrained_result = query_using_lsi_for_that_year()
result_on_straddling_bins = query_using_lsi_marked_with_0() # <-- this is to get any of the indexes that do straddle a year
filter_and_combine(year_constrained_result, result_on_straddling_bins)
I have programmed a langton's ant and it work nice.
Now I want to run 2 ant simultaneously.I have a run function how make the computation and ant's movement and it's a infinite run loop.
How can I run 2 of this loop at once ?
I have try to look on Thread but i'm not sure that it's the best for my case.
This is some example of my code :
Run function :
let run(f,tab2 : t_fourmi*int array array) =
f.xx := !(f.x);
f.yy := !(f.y);
let d = ref 0
and z = ref 0
and o = ref 0
(* 1 = de bas, 2 = de droite, 3 = de haut, 4 = de gauche *)
in
if tab2.(!(f.x)/5).(!(f.y)/5) = 0
then move_right(f,1,tab2);
if !(f.xx) + 5 = !(f.x)
then d := 4
else if !(f.xx) - 5 = !(f.x)
then d := 2
else if !(f.yy) + 5 = !(f.y)
then d := 1
else if !(f.yy) - 5 = !(f.y)
then d := 3;
while true
do
(*
print_string "step : ";
print_int !o;
print_newline(); *)
o := !o + 1;
f.xx := !(f.x);
f.yy := !(f.y);
z := tab2.(!(f.x)/5).(!(f.y)/5);
if !z = 0
then move_right(f,!d,tab2)
else if !z = 1
then move_left(f,!d,tab2)
else if !z = 2
then move_right(f,!d,tab2)
else if !z = 3
then move_right(f,!d,tab2);
if !(f.xx) + 5 = !(f.x)
then d := 4
else if !(f.xx) - 5 = !(f.x)
then d := 2
else if !(f.yy) + 5 = !(f.y)
then d := 1
else if !(f.yy) - 5 = !(f.y)
then d := 3;
done;
;;
Example of move function :
let move_left(f,d,tab2 : t_fourmi*int*int array array) = (* d = direction d'ou la fourmi viens *)
(* 1 = de bas, 2 = de droite, 3 = de haut, 4 = de gauche *)
f.xx := !(f.x);
f.yy := !(f.y);
if d = 1
then f.x := !(f.x) - 5
else if d = 2
then f.y := !(f.y) - 5
else if d = 3
then f.x := !(f.x) + 5
else if d = 4
then f.y := !(f.y) + 5;
if !(f.x) >= 995
then f.x := 5
else if !(f.x) <= 5
then f.x := 995;
if !(f.y) >= 995
then f.y := 5
else if !(f.y) <= 5
then f.y := 995;
create_fourmi(f);
let n = tab2.(!(f.xx)/5).(!(f.yy)/5) in
drawinv(!(f.xx),!(f.yy),n,tab2);
;;
If you need more function, ask me.
Thanks
If I understand your code correctly (it's unfortunately not very readable), it could be outlined as essentially:
let init params =
...
let step state =
...
let move thing state =
...
let run params thing =
let state = ref (init params) in
while true do
let state := step state in
move thing state
done
where I've factored init, step and move out into separate function, which should be pretty straight-forward to do. And by doing so we can replace the run function with a run_two function that can run two instances virtually at the same time, though not entirely in parallel. They won't run independently, but in synchrony, iteration by iteration:
let run_two params1 thing1 params2 thing2 =
let state1 = ref (init params1) in
let state2 = ref (init params2) in
while true do
state1 := step !state1;
state2 := step !state2;
move !state1;
move !state2;
done
This doesn't use any threads or other complicated concurrency primitives. It's just ordinary code organized in a way that allows composition. It also allows the init and step function to be completely pure, which makes them easy to test and reason about. You could even make the move functions pure, if you factor out the drawing.
You can use threads, though it will open a pandora box for you. You have to synchronize your ants, since probably you want them to move in the same world.
Running two ants in different threads
First of all, you have to represent each ant process as an ever-looping function of type ant -> unit, where ant is the type that describes ant's initial position, moving parameters, and so on (if you don't need this, then just use unit instead. So suppose we have a function val run : ant -> unit. Next, we need to make sure, that we do not write to the same board at the same time from different threads, so we need to create a mutex, using Mutex.create, we then need to update our run function and do Mutex.lock before updating our board, and Mutex.unlock after. Finally, we should ensure that no ants will starve for the board, and that once an ant finishes it moves it yields the control to another ant, we could do this using Thread.delay if we want our simulation to be smooth (i.e., if we want to artificially slow it down to human-observable speed). Otherwise, we can just use Thread.yield. (Note, normally threads are preempted (forced to yield) by the system, but this is done in special yield points. When we do system programming there are usually lots of yield points, but this is not our case, so we have to implement this cooperation explicitly). Finally, our updated run function is ready to be run in a thread, with Thread.createrun ant. This function will return to our main thread of execution, so we can run a second ant and so on. Once all ants start running we have to call Thread.join to wait for all of them. Ideally, this function should never return.
I hope, that the above outline provided enough information for you, and you can enjoy the coding yourself. Feel free to ask questions int the comment section if anything is unclear. Probably, the first question would be how to compile it. It is (there are simpler ways, of course, but the most basic is, assuming that your program is in the ant.ml):
ocamlopt -thread unix.cmxa threads.cmxa ant.ml -o ant
Running two ants in co-routines
While the above will work, it seems too complicated and too system-dependent for such a simple simulation task. Why do we need to use system threads for that? We actually don't, we can be clever and implement co-routines using plain OCaml and continuation-passing style. Don't worry, a continuation is just a function. And a continuation-passing style is when we are calling another function at the end of the other function. In this approach, we will not have the run function that runs infinitely, instead, we will have a step function that advances an ant one step forward at each invocation. For the demonstration purposes, let's simplify our step function so that each ant just greets another, e.g.,
let step other yield =
print_endline ("Hello, " ^ other);
yield ()
That simple. We do our step, and then call the yield function (that is that fancy continuation) to yield the control to the next ant. Now let's tie it together in a simple infinite loop, e.g.,
let rec loop () =
step "Joe" ## fun () ->
step "Doe" ## fun () ->
loop ()
let () = loop ()
Let's make it clear. step "Joe" ## fun () -> <continue> is the same as step "Joe" (fun () -> <continue>), and this (fun () -> <continue>) is the yield function that is passed to the step function that greets Joe. Once Joe is greeted the step function calls us again and we evaluate the <continue>, which is, in our case step "Doe" ## fun () -> loop (), i.e., we pass the fun () -> loop () function as the yield argument to the step function that greets Doe, so once Doe is greeted, we call loop () ... and now we're at the begining of the loop.
In our case, we were passing a value of type unit in our continuations, but we can also pass an arbitrary value, e.g., a value that will represent the state of our simulation (the board with ant positions). This will enable a functional implementation of your simulation if you want to try it.
I have a sorting problem in Scala that I could certainly solve with brute-force, but I'm hopeful there is a more clever/elegant solution available. Suppose I have a list of strings in no particular order:
val keys = List("john", "jill", "ganesh", "wei", "bruce", "123", "Pantera")
Then at random, I receive the values for these keys at random (full-disclosure, I'm experiencing this problem in an akka actor, so events are not in order):
def receive:Receive = {
case Value(key, otherStuff) => // key is an element in keys ...
And I want to store these results in a List where the Value objects appear in the same order as their key fields in the keys list. For instance, I may have this list after receiving the first two Value messages:
List(Value("ganesh", stuff1), Value("bruce", stuff2))
ganesh appears before bruce merely because he appears earlier in the keys list. Once the third message is received, I should insert it into this list in the correct location per the ordering established by keys. For instance, on receiving wei I should insert him into the middle:
List(Value("ganesh", stuff1), Value("wei", stuff3), Value("bruce", stuff2))
At any point during this process, my list may be incomplete but in the expected order. Since the keys are redundant with my Value data, I throw them away once the list of values is complete.
Show me what you've got!
I assume you want no worse than O(n log n) performance. So:
val order = keys.zipWithIndex.toMap
var part = collection.immutable.TreeSet.empty[Value](
math.Ordering.by(v => order(v.key))
)
Then you just add your items.
scala> part = part + Value("ganesh", 0.1)
part: scala.collection.immutable.TreeSet[Value] =
TreeSet(Value(ganesh,0.1))
scala> part = part + Value("bruce", 0.2)
part: scala.collection.immutable.TreeSet[Value] =
TreeSet(Value(ganesh,0.1), Value(bruce,0.2))
scala> part = part + Value("wei", 0.3)
part: scala.collection.immutable.TreeSet[Value] =
TreeSet(Value(ganesh,0.1), Value(wei,0.3), Value(bruce,0.2))
When you're done, you can .toList it. While you're building it, you probably don't want to, since updating a list in random order so that it is in a desired sorted order is an obligatory O(n^2) cost.
Edit: with your example of seven items, my solution takes about 1/3 the time of Jean-Philippe's. For 25 items, it's 1/10th the time. 1/30th for 200 (which is the difference between 6 ms and 0.2 ms on my machine).
If you can use a ListMap instead of a list of tuples to store values while they're gathered, this could work. ListMap preserves insertion order.
class MyActor(keys: List[String]) extends Actor {
def initial(values: ListMap[String, Option[Value]]): Receive = {
case v # Value(key, otherStuff) =>
if(values.forall(_._2.isDefined))
context.become(valuesReceived(values.updated(key, Some(v)).collect { case (_, Some(v)) => v))
else
context.become(initial(keys, values.updated(key, Some(v))))
}
def valuesReceived(values: Seq[Value]): Receive = { } // whatever you need
def receive = initial(keys.map { k => (k -> None) })
}
(warning: not compiled)
I am facing a major issue with my design at this juncture. My method is trying to accomplish the follows:
Insert the passed in object into the database.
Get the autoincremented id from the insert and use it to call webservice1 along with the object.
Get the result from webservice1 and call webservice2 with the original object and some response from webservice1.
Combine the results from webservice1 and 2 and write it into the database.
Get the resulting autoincremented id from the last insert and call webservice3 with the original object that would eventually result into the success or failure of the operation.
I want to design this in a flexible manner since the requirements are in a flux and I do not want to keep on modifying my logic based on any changing. I do realize some amount of change is inevitable but I would like to minimize the damage and respect the open-closed principle.
My initial take was as follows:
def complexOperation(someObject:T) =
dbService.insertIntoDb(someObject) match {
case Left(e:Exception) => Left(e)
case Right(id:Int) => webService.callWebService1(id,someObject) match {
case Left(e:Exception) => Left(e)
case Right(r:SomeResponse1) => webService.callWebservice2(r,someObject) match {
case Left(e:Exception) => webService.rollbackService1();Left(e)
case Right(context:ResponseContext) => dbService.insertContextIntoDb(context) match {
case Left(e:Exception) => Left(e)
case Right(id:Int) => webService.callWebservice3(id,someObject) match {
case Left(e:Exception) => webService.rollbackService3();Left(e)
case Right(r:Response) => Right(r)
}
}
}
}
As you can see, this is a tangled mess. I can neither unit test it, nor extend it nor very easily debug it if things spiral out of control. This code serves its purpose but it will be great to get some ideas on how I should refactor it to make the lives of the people who inherit my code a little more easier.
Thanks
Have a look at scala.util.Try. It's available in Scala 2.10, which may or may not be available to you as an option, but the idea of it is perfect for your scenario.
What you have in your code example is what I like calling the "pyramid" of nesting. The best solution to this is to use flat-mapping wherever you can. But obviously that's an issue when you have stuff like Either[Exception, Result] at every step. That's where Try comes in. Try[T] is essentially a replacement for Either[Exception, T], and it comes with all of the flatMap-ing goodness that you need.
Assuming you can either change the return type of those webService calls, or provide some implicit conversion from Either[Exception, Result] to Try[Result], your code block would become something more like...
for {
id <- dbService.insertIntoDb(someObject)
r <- webService.callWebService1(id,someObject)
context <- webService.callWebservice2(r,someObject)
id2 <- dbService.insertContextIntoDb(context)
response <- webService.callWebservice3(id,someObject).recoverWith {
case e: Exception => webService.rollbackService3(); Failure(e)
}
} yield response
Lift has a similar mechanism in net.liftweb.common.Box. It's like Option, but with a container for Exceptions too.
edit: It looks like you can use the left or right method of an Either, and it will let you use flatMap-ing almost exactly the way I described with Try. The only difference is that the end result is an Either[Exception, Result] instead of a Try[Result]. Check out LeftProjection for details/examples.
You can use for comprehension to reduce the noise in the code.
#Dylan had the right idea above. Let me see if I can help translate what you want to do into idiomatic Scala 2.9.1 code.
This version doesn't attempt any rollbacks:
// 1: No rollbacks, just returns the first exception in Left
def complexOperation1(someObject:T): Either[Exception, Response] = {
for {
id <- dbService.insertIntoDb(someObject).right
r <- webService.callWebService1(id, someObject).right
context <- webService.callWebservice2(idResp, someObject).right
id2 <- dbService.insertContextIntoDb(context).right
response <- webService.callWebservice3(id,someObject).right
} yield response
}
Now, let's try to do the rollbacks exactly as you had them above:
// 2: Rolls back all web services and returns first exception in Left
def complexOperation1(someObject:T): Either[Exception, Response] = {
for {
id <- dbService.insertIntoDb(someObject).right
r <- webService.callWebService1(id, someObject).right
context <- webService.callWebservice2(idResp, someObject).left.map { e =>
webService.rollbackService1()
e
}.right
id2 <- dbService.insertContextIntoDb(context).right
response <- webService.callWebservice3(id,someObject).left.map { e =>
webService.rollbackService3()
e
}.right
} yield response
}
If you define a function which does the effect (the rollback) on the left, it get's a little cleaner and easier to test, for example:
// 3: Factor out the side-effect of doing the follbacks on Left
def rollbackIfLeft[T](f: => Either[Exception, T], r: => Unit): Either[Exception, T] = {
val result = f
result.left.foreach(_ => r) // do the rollback if any exception occured
result
}
def complexOperation1(someObject:T): Either[Exception, Response] = {
for {
id <- dbService.insertIntoDb(someObject).right
r <- webService.callWebService1(id, someObject).right
context <- rollbackIfLeft(webService.callWebservice2(idResp, someObject),
webService.rollbackService1()).right
id2 <- dbService.insertContextIntoDb(context).right
response <- rollbackIfLeft(webService.callWebservice3(id,someObject),
webService.rollbackService3()).right
} yield response
}
You can try out rollbackIfLeft in the scala REPL to get a sense of it:
scala> rollbackIfLeft(Right(42), println("hey"))
res28: Either[Exception,Int] = Right(42)
scala> rollbackIfLeft(Left(new RuntimeException), println("ERROR!"))
ERROR!
res29: Either[Exception,Nothing] = Left(java.lang.RuntimeException)
Hope this helps!
I am seeing a problem with some Scala 2.7.7 code I'm working on, that should not happen if it the equivalent was written in Java. Loosely, the code goes creates a bunch of card players and assigns them to tables.
class Player(val playerNumber : Int)
class Table (val tableNumber : Int) {
var players : List[Player] = List()
def registerPlayer(player : Player) {
println("Registering player " + player.playerNumber + " on table " + tableNumber)
players = player :: players
}
}
object PlayerRegistrar {
def assignPlayersToTables(playSamplesToExecute : Int, playersPerTable:Int) = {
val numTables = playSamplesToExecute / playersPerTable
val tables = (1 to numTables).map(new Table(_))
assert(tables.size == numTables)
(0 until playSamplesToExecute).foreach {playSample =>
val tableNumber : Int = playSample % numTables
tables(tableNumber).registerPlayer(new Player(playSample))
}
tables
}
}
The PlayerRegistrar assigns a number of players between tables. First, it works out how many tables it will need to break up the players between and creates a List of them.
Then in the second part of the code, it works out which table a player should be assigned to, pulls that table from the list and registers a new player on that table.
The list of players on a table is a var, and is overwritten each time registerPlayer() is called. I have checked that this works correctly through a simple TestNG test:
#Test def testRegisterPlayer_multiplePlayers() {
val table = new Table(1)
(1 to 10).foreach { playerNumber =>
val player = new Player(playerNumber)
table.registerPlayer(player)
assert(table.players.contains(player))
assert(table.players.length == playerNumber)
}
}
I then test the table assignment:
#Test def testAssignPlayerToTables_1table() = {
val tables = PlayerRegistrar.assignPlayersToTables(10, 10)
assertEquals(tables.length, 1)
assertEquals(tables(0).players.length, 10)
}
The test fails with "expected:<10> but was:<0>". I've been scratching my head, but can't work out why registerPlayer() isn't mutating the table in the list. Any help would be appreciated.
The reason is that in the assignPlayersToTables method, you are creating a new Table object. You can confirm this by adding some debugging into the loop:
val tableNumber : Int = playSample % numTables
println(tables(tableNumber))
tables(tableNumber).registerPlayer(new Player(playSample))
Yielding something like:
Main$$anon$1$Table#5c73a7ab
Registering player 0 on table 1
Main$$anon$1$Table#21f8c6df
Registering player 1 on table 1
Main$$anon$1$Table#53c86be5
Registering player 2 on table 1
Note how the memory address of the table is different for each call.
The reason for this behaviour is that a Range is non-strict in Scala (until Scala 2.8, anyway). This means that the call to the range is not evaluated until it's needed. So you think you're getting back a list of Table objects, but actually you're getting back a range which is evaluated (instantiating a new Table object) each time you call it. Again, you can confirm this by adding some debugging:
val tables = (1 to numTables).map(new Table(_))
println(tables)
Which gives you:
RangeM(Main$$anon$1$Table#5492bbba)
To do what you want, add a toList to the end:
val tables = (1 to numTables).map(new Table(_)).toList
val tables = (1 to numTables).map(new Table(_))
This line seems to be causing all the trouble - mapping over 1 to n gives you a RandomAccessSeq.Projection, and to be honest, I don't know how exactly they work, but a bit less clever initialising technique does the job.
var tables: Array[Table] = new Array(numTables)
for (i <- 0 to numTables) tables(i) = new Table(i)
Using the first initialisation method I wasn't able to change the objects (just like you), but using a simple array everything seems to be working.