How to create Akka Sink from a function that consumes Source - akka

I have a function something like
def send[T, R](src: Source[T, _]): Future[R]
Conceptually, it is a sink that drains the source and streams it out to another system. It should be simple to make an actual Akka Sink out of it, but I cannot find a way. Can you please help me write a Sink-generating function along these lines:
def fromFunction[T, Mat](fn: Source[T, _] => Mat): Sink[T, Mat] = ???
I tried to wire up a graph, but it gets stuck on function trying to materialise the value before the graph is connected. I guess I need to defer the function call, but do not know how.
class FuncSinkShape[T, R](fn: Source[T, _] ⇒ Future[R])
extends GraphStageWithMaterializedValue[SinkShape[T], Future[R]] {
val in = Inlet[T]("SourceFlowShape.in")
override val shape = SinkShape(in)
override def createLogicAndMaterializedValue(inheritedAttributes: Attributes): (GraphStageLogic, Future[R]) = {
val sourceStage = new GraphStage[SourceShape[T]]() {
override val shape = SourceShape(Outlet[T]("FuncSinkShape.out"))
override def createLogic(inheritedAttributes: Attributes): GraphStageLogic = {
new GraphStageLogic(shape) {
setHandler(shape.out, new OutHandler {
override def onPull(): Unit = {
//pull(in)
}
})
}
}
}
val logic = new GraphStageLogic(shape) {
setHandler(in, new InHandler {
override def onPush(): Unit = {
val elm=grab(in)
err.println(s"-- onPush $elm")
emit(sourceStage.shape.out, elm)
}
})
}
(logic, fn(Source.fromGraph(sourceStage)))
}
}

Related

performance implications of holding state in large data classes and updating it using copy()

In order to store state in jetpack compose I have so far used the the following pattern:
private val _largeDataClass:MutableState<LargeDataClass> = mutableStateOf(LargeDataClass())
val largeDataClass :State<LargeDataClass> = _largeDataClass
then I display some or all of the properties of this class in my composition. When the user changes a property of this data class I need to update the state and I do it in the following way:
fun onUserEvent(somePropertyChange:String){
_largeDataClass.value=largeDataClass.value.copy(someProperty = somePropertyChange)
}
I got this approach from the following post. It works and has the benefit of keeping my codebase relatively small (as there might be 20+ different properties in LargeDataClass that I dont need to declare individually as mutable state) BUT, if I am not mistaken, following this approach will trigger the recomposition of my entire screen even if the user just types a single letter in one of my many TextFields. As all my composables display some property of largeDataClass and they have just been notified that its value has changed.
My first question is wether I am right in this last assumption. Will my current way of holding state negatively affect my apps performance because I am forcing the screen to redraw itself completely constantly? Or are there some optimizations, that I'm unaware of, in compose that prevent this from happening and render my appoach safe?
my second question: I would really love it if there was a way of turning a data class, say:
data class Student(
val key: String = "s0",
val firstName: String = "",
val lastName: String = "")
into an equivalent state holder class (something similar to the following)
class StudentState(s:Student){
val key= mutableStateOf(s:Key),
val firstName= mutableStateOf(s.firstName),
val lastName= mutableStateOf(s.lastName)}
(Ideally without having to explicitly write such a class myself every time) Does this exist already? is there a way of using reflection or the like to achieve this for a generic data class?
I am still learning to deal with state in jetpack compose and I want to get it right. It seems to me that tracking the properties of my data classes individually either in the ViewModel or in a State Holder class is the right thing to do, but on the other hand this makes my code a lot longer and it just feels like I am doing a lot of stuff twice and my code becomes less readable and maintainable. Any insights are much appreciated
You could use reflection to create mutableStates for the members of any class like so:
fun createStateMap(baseObject: Any) = with(baseObject::class.memberProperties) {
map { it.name }.zip(map { mutableStateOf(it.getter.call(baseObject)) }).toMap()
}
And then use it like this:
class MyViewModel : ViewModel() {
private val student = Student(firstName = "John", lastName = "Doe")
val studentStateMap = createStateMap(student)
}
#Composable
fun MyComposable(viewModel: MyViewModel) {
val student = viewModel.studentStateMap
Button(
onClick = { student["firstName"]?.value = "Jack" }
) {
Text(text = student["firstName"]?.value.toString())
}
}
I wouldn't use it, it's not typesafe, it's messy and ugly, but it's possible.
An Annotation to store data class #AsState
Well I am still not sure about wether it is fine to simply .copy(changedValue = "...") a large data class or if this is inefficient because it triggers unecessary recompositions.
In any case just to be safe (and cause I think its cleaner) I wrote an AnnotationProcessor that takes my data classes and creates both a mutable and immutable version of the class holding all properties as state. It supports both lists and maps but is shallow (meaning that it wont repeat the same process for nested classes). Here an example of the class with the annotation and the result.
please let me know if you consider this to be usefull in order to track state more cleanly when a data class is displayed/edited in a composable (and also if you dont)
The original class
#AsState
data class Test(val name:String, val age:Int, val map:HashMap<String,Int>, val list:ArrayList<String>)
The mutable verion of the class with a custonm constructor and rootClass getter
public class TestMutableState {
public val name: MutableState<String>
public val age: MutableState<Int>
public val map: SnapshotStateMap<String, Int>
public val list: SnapshotStateList<String>
public constructor(rootObject: Test) {
this.name=mutableStateOf(rootObject.name)
this.age=mutableStateOf(rootObject.age)
this.map=rootObject.map.map{Pair(it.key,it.value)}.toMutableStateMap()
this.list=rootObject.list.toMutableStateList()
}
public fun getTest(): Test = Test(name = this.name.value,
age = this.age.value,
map = HashMap(this.map),
list = ArrayList(this.list),
)
}
The immutable version that can be public in the ViewModel
public class TestState {
public val name: State<String>
public val age: State<Int>
public val map: SnapshotStateMap<String, Int>
public val list: SnapshotStateList<String>
public constructor(mutableObject: TestMutableState) {
this.name=mutableObject.name
this.age=mutableObject.age
this.map=mutableObject.map
this.list=mutableObject.list
}
}
TL;DR
Next I am pasting the source code for my annotation processor so you can implement it. I basically followed this article and implemented some of my own changes based on arduous googling. I might make this a module in the future so that other people can more easily implement this in their projects i there is any interest:
Annotation class
#Target(AnnotationTarget.CLASS)
#Retention(AnnotationRetention.SOURCE)
public annotation class AsState
Annotation processor
#AutoService(Processor::class)
class AnnotationProcessor : AbstractProcessor() {
companion object {
const val KAPT_KOTLIN_GENERATED_OPTION_NAME = "kapt.kotlin.generated"
}
override fun getSupportedAnnotationTypes(): MutableSet<String> {
return mutableSetOf(AsState::class.java.name)
}
override fun getSupportedSourceVersion(): SourceVersion = SourceVersion.latest()
override fun process(annotations: MutableSet<out TypeElement>?, roundEnv: RoundEnvironment): Boolean {
roundEnv.getElementsAnnotatedWith(AsState::class.java)
.forEach {
if (it.kind != ElementKind.CLASS) {
processingEnv.messager.printMessage(Diagnostic.Kind.ERROR, "Only classes can be annotated")
return true
}
processAnnotation(it)
}
return false
}
#OptIn(KotlinPoetMetadataPreview::class, com.squareup.kotlinpoet.DelicateKotlinPoetApi::class)
private fun processAnnotation(element: Element) {
val className = element.simpleName.toString()
val pack = processingEnv.elementUtils.getPackageOf(element).toString()
val kmClass = (element as TypeElement).toImmutableKmClass()
//create vessel for mutable state class
val mutableFileName = "${className}MutableState"
val mutableFileBuilder= FileSpec.builder(pack, mutableFileName)
val mutableClassBuilder = TypeSpec.classBuilder(mutableFileName)
val mutableConstructorBuilder= FunSpec.constructorBuilder()
.addParameter("rootObject",element.asType().asTypeName())
var helper="return ${element.simpleName}("
//create vessel for immutable state class
val stateFileName = "${className}State"
val stateFileBuilder= FileSpec.builder(pack, stateFileName)
val stateClassBuilder = TypeSpec.classBuilder(stateFileName)
val stateConstructorBuilder= FunSpec.constructorBuilder()
.addParameter("mutableObject",ClassName(pack,mutableFileName))
//import state related libraries
val mutableStateClass= ClassName("androidx.compose.runtime","MutableState")
val stateClass=ClassName("androidx.compose.runtime","State")
val snapshotStateMap= ClassName("androidx.compose.runtime.snapshots","SnapshotStateMap")
val snapshotStateList=ClassName("androidx.compose.runtime.snapshots","SnapshotStateList")
fun processMapParameter(property: ImmutableKmValueParameter) {
val clName =
((property.type?.abbreviatedType?.classifier) as KmClassifier.TypeAlias).name
val arguments = property.type?.abbreviatedType?.arguments?.map {
ClassInspectorUtil.createClassName(
((it.type?.classifier) as KmClassifier.Class).name
)
}
val paramClass = ClassInspectorUtil.createClassName(clName)
val elementPackage = clName.replace("/", ".")
val paramName = property.name
arguments?.let {
mutableClassBuilder.addProperty(
PropertySpec.builder(
paramName,
snapshotStateMap.parameterizedBy(it), KModifier.PUBLIC
)
.build()
)
}
arguments?.let {
stateClassBuilder.addProperty(
PropertySpec.builder(
paramName,
snapshotStateMap.parameterizedBy(it), KModifier.PUBLIC
)
.build()
)
}
helper = helper.plus("${paramName} = ${paramClass.simpleName}(this.${paramName}),\n")
mutableConstructorBuilder
.addStatement("this.${paramName}=rootObject.${paramName}.map{Pair(it.key,it.value)}.toMutableStateMap()")
stateConstructorBuilder
.addStatement("this.${paramName}=mutableObject.${paramName}")
}
fun processListParameter(property: ImmutableKmValueParameter) {
val clName =
((property.type?.abbreviatedType?.classifier) as KmClassifier.TypeAlias).name
val arguments = property.type?.abbreviatedType?.arguments?.map {
ClassInspectorUtil.createClassName(
((it.type?.classifier) as KmClassifier.Class).name
)
}
val paramClass = ClassInspectorUtil.createClassName(clName)
val elementPackage = clName.replace("/", ".")
val paramName = property.name
arguments?.let {
mutableClassBuilder.addProperty(
PropertySpec.builder(
paramName,
snapshotStateList.parameterizedBy(it), KModifier.PUBLIC
)
.build()
)
}
arguments?.let {
stateClassBuilder.addProperty(
PropertySpec.builder(
paramName,
snapshotStateList.parameterizedBy(it), KModifier.PUBLIC
)
.build()
)
}
helper = helper.plus("${paramName} = ${paramClass.simpleName}(this.${paramName}),\n")
mutableConstructorBuilder
.addStatement("this.${paramName}=rootObject.${paramName}.toMutableStateList()")
stateConstructorBuilder
.addStatement("this.${paramName}=mutableObject.${paramName}")
}
fun processDefaultParameter(property: ImmutableKmValueParameter) {
val clName = ((property.type?.classifier) as KmClassifier.Class).name
val paramClass = ClassInspectorUtil.createClassName(clName)
val elementPackage = clName.replace("/", ".")
val paramName = property.name
mutableClassBuilder.addProperty(
PropertySpec.builder(
paramName,
mutableStateClass.parameterizedBy(paramClass), KModifier.PUBLIC
).build()
)
stateClassBuilder.addProperty(
PropertySpec.builder(
paramName,
stateClass.parameterizedBy(paramClass),
KModifier.PUBLIC
).build()
)
helper = helper.plus("${paramName} = this.${paramName}.value,\n")
mutableConstructorBuilder
.addStatement(
"this.${paramName}=mutableStateOf(rootObject.${paramName}) "
)
stateConstructorBuilder
.addStatement("this.${paramName}=mutableObject.${paramName}")
}
for (property in kmClass.constructors[0].valueParameters) {
val javaPackage = (property.type!!.classifier as KmClassifier.Class).name.replace("/", ".")
val javaClass=try {
Class.forName(javaPackage)
}catch (e:Exception){
String::class.java
}
when{
Map::class.java.isAssignableFrom(javaClass) ->{ //if property is of type map
processMapParameter(property)
}
List::class.java.isAssignableFrom(javaClass) ->{ //if property is of type list
processListParameter(property)
}
else ->{ //all others
processDefaultParameter(property)
}
}
}
helper=helper.plus(")") //close off method
val getRootBuilder= FunSpec.builder("get$className")
.returns(element.asClassName())
getRootBuilder.addStatement(helper.toString())
mutableClassBuilder.addFunction(mutableConstructorBuilder.build()).addFunction(getRootBuilder.build())
stateClassBuilder.addFunction(stateConstructorBuilder.build())
val kaptKotlinGeneratedDir = processingEnv.options[KAPT_KOTLIN_GENERATED_OPTION_NAME]
val mutableFile = mutableFileBuilder
.addImport("androidx.compose.runtime", "mutableStateOf")
.addImport("androidx.compose.runtime","toMutableStateMap")
.addImport("androidx.compose.runtime","toMutableStateList")
.addType(mutableClassBuilder.build())
.build()
mutableFile.writeTo(File(kaptKotlinGeneratedDir))
val stateFile = stateFileBuilder
.addType(stateClassBuilder.build())
.build()
stateFile.writeTo(File(kaptKotlinGeneratedDir))
}
}
gradle annotation
plugins {
id 'java-library'
id 'org.jetbrains.kotlin.jvm'
}
dependencies {
implementation fileTree(dir: 'libs', include: ['*.jar'])
implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk8:$kotlin_version"
}
java {
sourceCompatibility = JavaVersion.VERSION_1_8
targetCompatibility = JavaVersion.VERSION_1_8
}
gradle processor
plugins {
id 'kotlin'
id 'kotlin-kapt'
}
dependencies {
implementation fileTree(dir: 'libs', include: ['*.jar'])
implementation project(':annotations')
implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.6.10"
// https://mvnrepository.com/artifact/com.squareup/kotlinpoet
implementation 'com.squareup:kotlinpoet:1.10.2'
implementation "com.squareup:kotlinpoet-metadata:1.7.1"
implementation "com.squareup:kotlinpoet-metadata-specs:1.7.1"
implementation "com.google.auto.service:auto-service:1.0.1"
// https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-metadata-jvm
implementation "org.jetbrains.kotlinx:kotlinx-metadata-jvm:0.4.2"
implementation 'org.json:json:20211205'
kapt "com.google.auto.service:auto-service:1.0.1"
}
java {
sourceCompatibility = JavaVersion.VERSION_1_8
targetCompatibility = JavaVersion.VERSION_1_8
}

Akka: testing actor got message

I have the next code:
//TestActor got some message
class TestActor extends Actor {
def receive = {
case string: String => //....
}
}
//TestReg when create get ActorRef, when i call `pass` method, then should pass text to ActorRef
class TestReg(val actorRef: ActorRef) {
def pass(text: String) {
actorRef ! text
}
}
When i wrote test:
class TestActorReg extends TestKit(ActorSystem("system")) with ImplicitSender
with FlatSpecLike with MustMatchers with BeforeAndAfterAll {
override def afterAll() {
system.shutdown()
}
"actorReg" should "pass text to actorRef" in {
val probe = TestProbe()
val testActor = system.actorOf(Props[TestActor])
probe watch testActor
val testReg = new TestReg(testActor)
testReg.pass("test")
probe.expectMsg("test")
}
}
I got error:
java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsg while waiting for test
How to check what the actor got a text?
probe.expectMsg() is calling the assertion on the probe. But you passed the testActor into your TestReg class
change it to the following line and it will work
val testReg = new TestReg(probe.ref)
have to call .ref to make the probe into an ActorRef
And you want to do it here not at the instantiation of the variable to avoid
certain bugs that are outside of the scope of this response
the error in the logic as I see it is you are thinking that the watch method makes probe see what test actor does. but its death watch not message watch. which is different.
Create application.conf file with this:
akka {
test {
timefactor = 1.0
filter-leeway = 999s
single-expect-default = 999s
default-timeout = 999s
calling-thread-dispatcher {
type = akka.testkit.CallingThreadDispatcherConfigurator
}
}
actor {
serializers {
test-message-serializer = "akka.testkit.TestMessageSerializer"
}
serialization-identifiers {
"akka.testkit.TestMessageSerializer" = 23
}
serialization-bindings {
"akka.testkit.JavaSerializable" = java
}
}
}

How to use mocks with the Cake Pattern

I have the following class:
class LinkUserService() {
//** cake pattern **
oauthProvider: OAuthProvider =>
//******************
def isUserLinked(userId: String, service: String) = {
val cred = oauthProvider.loadCredential(userId)
cred != null
}
def linkUserAccount(userId: String, service: String): (String, Option[String]) = {
if (isUserLinked(userId, service)) {
("SERVICE_LINKED", None)
} else {
val authUrl = oauthProvider.newAuthorizationUrl
("SERVICE_NOT_LINKED", Some(authUrl))
}
}
def setLinkAuthToken(userId: String, service:String, token:String):String = {
oauthProvider.createAndStoreCredential(userId, token)
}
}
Typically I'd use this class in production like so:
val linkService = LinkUserService with GoogleOAuthProvider
When it comes to testing, I want to replace the oauthProvider with a mock such that's been trained by my unit test to respond like so: oauthProvider.loadCredential("nobody") returns null. Is this possible? If so, how would I set up my unit test to do so?
You have this problem because you are not using cake pattern to full extent. If you write something like
trait LinkUserServiceComponent {
this: OAuthProviderComponent =>
val linkUserService = new LinkUserService
class LinkUserService {
// use oauthProvider explicitly
...
}
}
trait GoogleOAuthProviderComponent {
val oauthProvider = new GoogleOAuthProvider
class GoogleOAuthProvider {
...
}
}
And then you use a mock like this:
val combinedComponent = new LinkUserServiceComponent with OAuthProviderComponent {
override val oauthProvider = mock(...)
}
Then your problem disappears. If you also make generic interface traits like this (and make other components depend on interface, not on implementation):
trait OAuthProviderComponent {
def oauthProvider: OAuthProvider
trait OAuthProvider {
// Interface declaration
}
}
then you also would have generic reusable and testable code.
This is very similar to your suggestion and it really is the essence of cake pattern.
The only solution I've been able to come up wiht is a sort of delegate mock trait as demonstrated:
trait MockOAuthProvider extends OAuthProvider {
val mockProvider = mock[OAuthProvider]
def loadCredential(userId: String) = mockProvider.loadCredential(userId)
def newAuthorizationUrl() = mockProvider.newAuthorizationUrl
def createAndStoreCredential(userId: String, authToken: String) = mockProvider.createAndStoreCredential(userId, authToken)
}
I place that at the top of my Spec, then when I declare my LinkUserService I mix in this Mock trait like so:
val linkUserService = new LinkUserService() with MockOAuthProvider
val mockOAuth = linkUserService.mockProvider
Finally in my unit tests I do things like:
mockOAuth.loadCredential("nobody") returns null
It works, but I could see that being a PITA if my trait was larger

How to mock spray-client response

I have a simple spray client :
val pipeline = sendReceive ~> unmarshal[GoogleApiResult[Elevation]]
val responseFuture = pipeline {Get("http://maps.googleapis.com/maps/api/elevation/jsonlocations=27.988056,86.925278&sensor=false") }
responseFuture onComplete {
case Success(GoogleApiResult(_, Elevation(_, elevation) :: _)) =>
log.info("The elevation of Mt. Everest is: {} m", elevation)
shutdown()
case Failure(error) =>
log.error(error, "Couldn't get elevation")
shutdown()
}
Full code can be found here.
I want to mock the response of the server to test the logic in the Success and Failure cases. The only relevant information i found was here but I haven't been able to use the cake pattern to mock the sendReceive method.
Any suggestion or example would be greatly appreciated.
Here's an example of one way to mock it using specs2 for the test spec and mockito for the mocking. First, the Main object refactored into a class setup for mocking:
class ElevationClient{
// we need an ActorSystem to host our application in
implicit val system = ActorSystem("simple-spray-client")
import system.dispatcher // execution context for futures below
val log = Logging(system, getClass)
log.info("Requesting the elevation of Mt. Everest from Googles Elevation API...")
import ElevationJsonProtocol._
import SprayJsonSupport._
def sendAndReceive = sendReceive
def elavation = {
val pipeline = sendAndReceive ~> unmarshal[GoogleApiResult[Elevation]]
pipeline {
Get("http://maps.googleapis.com/maps/api/elevation/json?locations=27.988056,86.925278&sensor=false")
}
}
def shutdown(): Unit = {
IO(Http).ask(Http.CloseAll)(1.second).await
system.shutdown()
}
}
Then, the test spec:
class ElevationClientSpec extends Specification with Mockito{
val mockResponse = mock[HttpResponse]
val mockStatus = mock[StatusCode]
mockResponse.status returns mockStatus
mockStatus.isSuccess returns true
val json = """
{
"results" : [
{
"elevation" : 8815.71582031250,
"location" : {
"lat" : 27.9880560,
"lng" : 86.92527800000001
},
"resolution" : 152.7032318115234
}
],
"status" : "OK"
}
"""
val body = HttpEntity(ContentType.`application/json`, json.getBytes())
mockResponse.entity returns body
val client = new ElevationClient{
override def sendAndReceive = {
(req:HttpRequest) => Promise.successful(mockResponse).future
}
}
"A request to get an elevation" should{
"return an elevation result" in {
val fut = client.elavation
val el = Await.result(fut, Duration(2, TimeUnit.SECONDS))
val expected = GoogleApiResult("OK",List(Elevation(Location(27.988056,86.925278),8815.7158203125)))
el mustEqual expected
}
}
}
So my approach here was to first define an overridable function in the ElevationClient called sendAndReceive that just delegates to the spray sendReceive function. Then, in the test spec, I override that sendAndReceive function to return a function that returns a completed Future wrapping a mock HttpResponse. This is one approach for doing what you want to do. I hope this helps.
There's no need to introduce mocking in this case, as you can simply build a HttpResponse much more easily using the existing API:
val mockResponse = HttpResponse(StatusCodes.OK, HttpEntity(ContentTypes.`application/json`, json.getBytes))
(Sorry for posting this as another answer, but don't have enough karma to comment)

NullPointerException on a listener pattern in scala

I'm trying to create a listener design Pattern like that:
abstract class Listener(g: Engine) {
g.addListener(this)
}
class Listener1(g: Engine) extends Listener(g)
class Listener2(g: Engine) extends Listener(g)
class Engine {
val listener1 = new Listener1(this)
val listener2 = new Listener2(this)
var listeners: List[Listener] = Nil
def addListener(g: Listener) = {
listeners = g::listeners
}
}
But if fails with a NullPointerException, because listeners is initially equal to null when listener1 and listener2 are created.
How do I overcome this problem?
EDIT: I tried the following:
def addListener(g: Listener) = {
if(listeners == null) {
listeners = List(g)
} else {
listeners = g::listeners
}
}
But the problem is that after the class is initialized, listeners = Nil.
I need a better design pattern to achieve my goal.
SOLUTION: Without using lazy val or mutable collections:
var listeners: List[Listener] = _
def addListener(g: Listener) = {
if(listeners == null)
listeners = List(g)
else
listeners = g::listeners
}
Using _ for initialization instead of 'null' or 'Nil' makes the compiler not to overwrite listeners after listener1 and listener2 have been created.
With a help of lazy value and mutable collection:
class Engine {
val listener1 = new Listener1(this)
val listener2 = new Listener2(this)
lazy val listeners = collection.mutable.ListBuffer.empty[Listener]
def addListener(g: Listener) {
listeners += g
}
}
If you badly need immutable List somewhere down the road, just turn buffer into it with .toList method in the callplace.