Let's say there's a DBIO composed of a Future and a DBIO, like:
/* Talks to some http service */
def httpFuture: Future[Unit] = ???
def addDbRow: DBIO[Unit] = MyTable += MyRow(1)
val db: DatabaseDef = ???
val dbio: DBIO[Unit] = for {
_ <- DBIO.from(httpFuture())
_ <- addDbRow()
} yield ()
db.run(dbio.transactionally)
Does Slick wait until after the future to begin the database transaction?
It looks like Slick does wait. If you set:
def httpFuture: Future[Unit] = Future(Thread.sleep(10 * 1000)) // sleep for 10seconds
and watch the database logs, you can see that the transaction does not begin until after the future completes.
Related
I have this code in Groovy;
def execution = []
def executor =[:]
for(loopcount=1;loopcount<4;loopcount++){
executor.executor = 'jmeter'
executor.scenario = 'scenario' + loopcount
println executor.scenario
executor.concurrency = 2
execution.add(executor)
}
execution.each{
println executor.scenario
}
It is a list of three maps, all the same apart from the scenario suffix increments. I am expecting;
scenario1
scenario2
scenario3
scenario1
scenario2
scenario3
But I get;
scenario1
scenario2
scenario3
scenario3
scenario3
scenario3
It's definitely adding three different maps in the list because the .each command is returning three values. And they're definitely different values in executor.scenario because the println in the loop is giving the correct '1, 2, 3' count. But why don't they stay as different values in the list?
I've also tried execution.push(executor) but that gives the same results. For context, this yaml is what I'm aiming for eventually;
---
execution:
- executor: "jmeter"
scenario: "scenario1"
concurrency: 2
- executor: "jmeter"
scenario: "scenario2"
concurrency: 2
- executor: "jmeter"
scenario: "scenario3"
concurrency: 2
And apart from the scenario count the rest of it works fine.
problem:
def execution = []
def executor =[:]
for(loopcount=1;loopcount<4;loopcount++){
execution.add(executor) // <<-- this line adds the same variable to the list 4 times
}
to fix this - declare executor inside the for loop
def execution = []
for(loopcount=1;loopcount<4;loopcount++){
def executor =[:] // <<-- creates a new object in a loop
execution.add(executor) // <<-- adds new object to a list
}
probably to make it more clear, let me specify what [] and [:] means:
def execution = new ArrayList()
for(loopcount=1;loopcount<4;loopcount++){
def executor = new LinkedHashMap()
execution.add(executor)
}
however you could declare the variable before the loop but you have to assign a new object into it inside loop
def execution = []
def executor
for(loopcount=1;loopcount<4;loopcount++){
executor = [:]
execution.add(executor)times
}
I have a list of Futures and I want to get the first one completed with a certain condition.
Here is an example of a possible code:
val futureList: Future[List[T]] = l map (c => c.functionWithFuture())
val data = for {
c <- futureList
}yield c
data onSuccess {
case x => (x filter (d=> d.condition)).head
}
But it's not efficient, because I'll take only one element of the list, but computed all of them with a lot of latency.
I know firstCompletedOf but it's not what I'm searching.
(Sorry for my bad English.)
Try using a Promise and calling trySuccess on it as soon as a future that satisfies the condition completes. The first to call trySuccess will complete the future, the following ones will have no effect (as opposed to calling success, which can only be called once on a Promise).
Keep in mind that if no future in the list satisfies the condition, you will never have a result, i.e. the promise future will never complete.
import scala.concurrent.{ Await, Future, Promise }
import scala.concurrent.duration._
import scala.concurrent.ExecutionContext.Implicits.global
import scala.util.Random
def condition(x: Int) = x > 50
val futures = (1 to 10) map (x => Future {
// wait a random number of ms between 0 and 1000
Thread.sleep(Random.nextInt(1000))
// return a random number < 100
Random.nextInt(100)
})
val p = Promise[Int]()
// the first one that satisfies the condition completes the promise
futures foreach { _ filter condition foreach p.trySuccess }
val result = p.future
// Watch out: the promise could never be fulfilled if all the futures are <=50
println("The first completed future that satisfies x>50 is: " + Await.result(result, 10.seconds))
My version of RegEx is being greedy and now working as it suppose to. I need extract each message with timestamp and user who created it. Also if user has two or more consecutive messages it should go inside one match / block / group. How to solve it?
https://regex101.com/r/zD5bR6/1
val pattern = "((a\.b|c\.d)\n(.+\n)+)+?".r
for(m <- pattern.findAllIn(str).matchData; e <- m.subgroups) println(e)
UPDATE
ndn solution throws StackOverflowError when executed:
Exception in thread "main" java.lang.StackOverflowError
at java.util.regex.Pattern$GroupTail.match(Pattern.java:4708)
.......
Code:
val pattern = "(?:.+(?:\\Z|\\n))+?(?=\\Z|\\w\\.\\w)".r
val array = (pattern findAllIn str).toArray.reverse foreach{println _}
for(m <- pattern.findAllIn(str).matchData; e <- m.subgroups) println(e)
I don't think a regular expression is the right tool for this job. My solution below uses a (tail) recursive function to loop over the lines, keep the current username and create a Message for every timestamp / message pair.
import java.time.LocalTime
case class Message(user: String, timestamp: LocalTime, message: String)
val Timestamp = """\[(\d{2})\:(\d{2})\:(\d{2})\]""".r
def parseMessages(lines: List[String], usernames: Set[String]) = {
#scala.annotation.tailrec
def go(
lines: List[String], currentUser: Option[String], messages: List[Message]
): List[Message] = lines match {
// no more lines -> return parsed messages
case Nil => messages.reverse
// found a user -> keep as currentUser
case user :: tail if usernames.contains(user) =>
go(tail, Some(user), messages)
// timestamp and message on next line -> create a Message
case Timestamp(h, m, s) :: msg :: tail if currentUser.isDefined =>
val time = LocalTime.of(h.toInt, m.toInt, s.toInt)
val newMsg = Message(currentUser.get, time, msg)
go(tail, currentUser, newMsg :: messages)
// invalid line -> ignore
case _ =>
go(lines.tail, currentUser, messages)
}
go(lines, None, Nil)
}
Which we can use as :
val input = """
a.b
[10:12:03]
you can also get commands
[10:11:26]
from the console
[10:11:21]
can you check if has been resolved
[10:10:47]
ah, okay
c.d
[10:10:39]
anyways startsLevel is still 4
a.b
[10:09:25]
might be a dead end
[10:08:56]
that need to be started early as well
"""
val lines = input.split('\n').toList
val users = Set("a.b", "c.d")
parseMessages(lines, users).foreach(println)
// Message(a.b,10:12:03,you can also get commands)
// Message(a.b,10:11:26,from the console)
// Message(a.b,10:11:21,can you check if has been resolved)
// Message(a.b,10:10:47,ah, okay)
// Message(c.d,10:10:39,anyways startsLevel is still 4)
// Message(a.b,10:09:25,might be a dead end)
// Message(a.b,10:08:56,that need to be started early as well)
The idea is to take as little characters as possible that will be followed by a username or the end of the string:
(?:.+(?:\Z|\n))+?(?=\Z|\w\.\w)
See it in action
I work with Play! Scala 2.2 and I need to make recursive web services call (until I get all the results).
For the moment I managed to make it with Await.result as following :
def getTitleSetFromJson( jsValue: JsValue ): Set[ String ] = {
val titleReads: Reads[Option[String]] = (__ \\ "title").readNullable[String]
(jsValue \ "response" \ "songs")
.asOpt[Set[Option[String]]](Reads.set(titleReads))
.getOrElse(Set.empty)
.flatten
}
def findEchonestSongs(start: Long, echonestId: String): Future[Set[String]] = {
val titleReads: Reads[Option[String]] = (__ \\ "title").readNullable[String]
val endpoint = s"$baseUrl/artist/songs"
val assembledUrl = s"$endpoint?api_key=$echonestApiKey&id=$artistId&format=json&start=$start&results=20"
WS.url(assembledUrl).get().map { songs =>
if ((songs.json \ "response" \ "total").asOpt[Int].getOrElse(0) > start + 20) {
getTitleSetFromJson(songs.json) ++
Await.result( findEchonestSongs(start + 20, echonestId), 3.seconds )
} else {
getTitleSetFromJson(songs.json)
}
}
}
Now I would like to make the same thing but with the non blocking way i.e. without using Await.result but with everything I tried I got errors : types mismatch because of the nested futures.
Could you please give me the way to do this?
There are many different ways to achieve your requirement. But maybe the simplest way is using flatMap.
Here is an improved version of your function (for making it simpler I took the liberty to make it return a Future[Set[JsValue]]).
import play.api.libs.json.JsValue
import play.api.libs.ws.WS
import scala.concurrent.Future
import play.api.Play.current
import scala.concurrent.ExecutionContext.Implicits.global
object Test {
val apiKey = "//YOUR_ECHONEST_API_KEY//"
val baseUrl = "http://developer.echonest.com/api/v4"
def findSongs(start: Long, artistId: String): Future[Set[JsValue]] = {
def endpoint = s"$baseUrl/artist/songs"
def assembledUrl = s"$endpoint?api_key=$apiKey&id=$artistId&format=json&start=$start&results=20"
def response = WS.url(assembledUrl).get()
def futureJson = response map (_.json)
futureJson flatMap { result =>
def total = (result \ "response" \ "total").asOpt[Int]
def songs = (result \ "response" \ "songs").as[Set[JsValue]]
total exists (_ > start + 20) match {
case false => Future.successful(songs)
case true => findSongs(start + 20, artistId) map (songs ++ _)
}
}
}
}
Working on the result
If you really want a Set[String] you can easily use map on the result.
val sample = findSongs(0, "someone")
sample.map(_.map(_ \ "title").map(_.as[String]))
Improvements
There is a lot of room for improvement here. For instance there is no need to pass the artistId every time you cal the function recursively. So we can define an inner function which is recursive.
It is not that difficult to make it tail recursive by using an accumulator.
Finally it might make sense to use a Seq[Future] and use fold instead of recursion, which is again a common pattern in functional programming. But all these stuff are already provided nicely by Stream api. You can refer to the Stream API for more info.
You can map the results from other ( from nested call ) future to be added to current Set.
Also... One very important thing. Readability Counts... Specially on StackOverFLow.
def findEchonestSongs(start: Long, echonestId: String): Future[Set[String]] = {
val titleReads: Reads[Option[String]] = (__ \\ "title").readNullable[String]
val requestUrl = "http://developer.echonest.com/api/v4/artist/songs?api_key=" + echonestApiKey + "&id=" + echonestId +
"&format=json&start=" + start + "&results=20"
WS.url( requestUrl ).get().flatMap { songs =>
var nTotal = (songs.json \ "response" \ "total").asOpt[Int].getOrElse(0)
if( nTotal > start + 20 ) {
val titleSet = getTitleSetFromJson( songs.json )
val moreTitleSetFuture = findEchonestSongs( start + 20, echonestId )
moreTitleSetFuture.map( moreTitleSet => titleSet ++ moreTitleSet )
}
else {
Future.successful( getTitleSetFromJson( songs.json ) )
}
}
}
def getTitleSetFromJson( jsValue: JsValue ): Set[ String ] = {
(songs.json \ "response" \ "songs")
.asOpt[Set[Option[String]]](Reads.set(titleReads))
.getOrElse(Set.empty)
.flatten
}
I get this as a response to an API hit.
1735 Queries
Taking 1.001303 to 31.856310 seconds to complete
SET timestamp=XXX;
SELECT * FROM ABC_EM WHERE last_modified >= 'XXX' AND last_modified < 'XXX';
38 Queries
Taking 1.007646 to 5.284330 seconds to complete
SET timestamp=XXX;
show slave status;
6 Queries
Taking 1.021271 to 1.959838 seconds to complete
SET timestamp=XXX;
SHOW SLAVE STATUS;
2 Queries
Taking 4.825584, 18.947725 seconds to complete
use marketing;
SET timestamp=XXX;
SELECT * FROM ABC WHERE last_modified >= 'XXX' AND last_modified < 'XXX';
I have extracted this out of the response html and have it as a string now.I need to retrieve values as concisely as possible such that I get a map of values of this format Map(Query -> T1 to T2 seconds) Basically what this is the status of all the slow queries running on MySQL slave server. I am building an alert system over it . So from this entire paragraph in the form of String I need to separate out the queries and save the corresponding time range with them.
1.001303 to 31.856310 is a time range . And against the time range the corresponding query is :
SET timestamp=XXX; SELECT * FROM ABC_EM WHERE last_modified >= 'XXX' AND last_modified < 'XXX';
This information I was hoping to save in a Map in scala. A Map of the form (query:String->timeRange:String)
Another example:
("use marketing; SET timestamp=XXX; SELECT * FROM ABC WHERE last_modified >= 'XXX' AND last_modified xyz ;"->"4.825584 to 18.947725 seconds")
"""###(.)###(.)\n\n(.*)###""".r.findAllIn(reqSlowQueryData).matchData foreach {m => println("group0"+m.group(1)+"next group"+m.group(2)+m.group(3)}
I am using the above statement to extract the the repeating cells to do my manipulations on it later. But it doesnt seem to be working;
THANKS IN ADvance! I know there are several ways to do this but all the ones striking me are inefficient and tedious. I need Scala to do the same! Maybe I can extract recursively using the subString method ?
If you want use scala try this:
val regex = """(\d+).(\d+).*(\d+).(\d+) seconds""".r // extract range
val txt = """
|1735 Queries
|
|Taking 1.001303 to 31.856310 seconds to complete
|
|SET timestamp=XXX; SELECT * FROM ABC_EM WHERE last_modified >= 'XXX' AND last_modified < 'XXX';
|
|38 Queries
|
|Taking 1.007646 to 5.284330 seconds to complete
|
|SET timestamp=XXX; show slave status;
|
|6 Queries
|
|Taking 1.021271 to 1.959838 seconds to complete
|
|SET timestamp=XXX; SHOW SLAVE STATUS;
|
|2 Queries
|
|Taking 4.825584, 18.947725 seconds to complete
|
|use marketing; SET timestamp=XXX; SELECT * FROM ABC WHERE last_modified >= 'XXX' AND last_modified < 'XXX';
""".stripMargin
def logToMap(txt:String) = {
val (_,map) = txt.lines.foldLeft[(Option[String],Map[String,String])]((None,Map.empty)){
(acc,el) =>
val (taking,map) = acc // taking contains range
taking match {
case Some(range) if el.trim.nonEmpty => //Some contains range
(None,map + ( el -> range)) // add to map
case None =>
regex.findFirstIn(el) match { //extract range
case Some(range) => (Some(range),map)
case _ => (None,map)
}
case _ => (taking,map) // probably empty line
}
}
map
}
Modified ajozwik's answer to work for SQL commands over multiple lines :
val regex = """(\d+).(\d+).*(\d+).(\d+) seconds""".r // extract range
def logToMap(txt:String) = {
val (_,map) = txt.lines.foldLeft[(Option[String],Map[String,String])]((None,Map.empty)){
(accumulator,element) =>
val (taking,map) = accumulator
taking match {
case Some(range) if element.trim.nonEmpty=> {
if (element.contains("Queries"))
(None, map)
else
(Some(range),map+(range->(map.getOrElse(range,"")+element)))
}
case None =>
regex.findFirstIn(element) match {
case Some(range) => (Some(range),map)
case _ => (None,map)
}
case _ => (taking,map)
}
}
println(map)
map
}