How to Efficiently Parse SQS Event Message using Kotlin - amazon-web-services

I have SQS Queue that send Batch Event to Lambda
Now I want to get body from each message from Record Event
Following code I implemented in Kotlin but can't optimize it furthur , also there are warnings.
Event Data is similar to - Example Amazon SQS message event (standard queue)
https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html
Any pointers will be appreciated :
class ApplicationHandler (
private val mapper: ObjectMapper = ApplicationModule.mapper
override fun handleRequest(input: Any, context: Context) {
val eventList: List<Event<*>> = parseEvent(input) // Passing whole input
for(event in eventList) {
when (event.type) {
*** do some code ***
}
}
}
}
private fun parseEvent(input: Any): List<Event<*>> {
val eventString: List<String> = when (input) {
// Iterating each message in Record as KeyValue Pair
is Map<*, *> -> {
val records = input["Records"] as List<Map<String, Any>>
records.map { it["body"] as String }
}
else -> throw IllegalArgumentException("Unexpected event type received: ${input.javaClass} [$input]")
}
return eventString.map { mapper.readValue(it,Event::class.java) }
}
}
Event.class:
data class Event<T>(
val id: String,
val time: Instant? = null,
val type: String
}

Related

Why sending different message to AWS SQS return same message ID

I want to send messages to a FIFO sqs queue. Given a array of list different user ids, for each id, I want to call sendMessage command to send the id as message body. I'm expecting every time it will return a different message id, but actually they all return same messageId. Sample code below:
const sendMessage = async (params:ISqsRequestParam) => {
try {
const sqsResponse = await sqsClient.send(new SendMessageCommand(params));
console.log(`send message response: ${JSON.stringify(sqsResponse)}`);
return sqsResponse.MessageId; // For unit tests.
} catch (err) {
console.error('SQS sending', err);
}
};
export const handler = async function (event: IEventBridgeAddtionalParams, context: Context): Promise<string[]> {
console.info(`${context.functionName} triggered at ${event.time} under ${process.env.NODE_ENV} mode`);
console.info(`customor parameter value is ${event.custom_parameter}`);
try {
const sqsUrl: string = event.custom_parameter === 'Creator' ? process.env.BATCH_CREATOR_QUEUE_URL : process.env.BATCH_PROCESSOR_QUEUE_URL;
console.info(`SQS Url is: ${sqsUrl}`);
const tenantData: ITenantResponse = await fetchAllTenantIds();
console.info(`ResponseDate from tenant service: ${JSON.stringify(tenantData.value)}`);
// change to fix tenantId for development environment for better debugging and test
const data : ITenantDetails[] = process.env.NODE_ENV !== 'production' ? fixedTenantDataForNonProd() : tenantData.value;
const promise = data.map(async tenantDetails => {
if (!tenantDetails.tenantFailed) {
console.info(`Tenant Id in message body: ${tenantDetails.id}`);
const params: ISqsRequestParam = {
MessageBody: tenantDetails.id,
MessageDeduplicationId: `FP_Tenant_populator_${event.custom_parameter}`, // Required for FIFO queues
MessageGroupId: `FP_Tenant_populator_Group_${event.custom_parameter}`, // Required for FIFO queues
QueueUrl: sqsUrl //SQS_QUEUE_URL; e.g., 'https://sqs.REGION.amazonaws.com/ACCOUNT-ID/QUEUE-NAME'
};
const messageId:string = await sendMessage(params);
return messageId; //for unit testing
} else {
return null; //for unit testing
}
});
const messageIds:string[] = await Promise.all(promise); //for unit testing
const activeMessageIds:string[] = messageIds.filter(id=> id!= null); //for unit testing
console.info(`Success, ${activeMessageIds.length} messages sent. MessageID:${activeMessageIds[0]}`);
return activeMessageIds; //for unit testing
} catch (error) {
console.error(`Fetch tenant details error: ${error}`);
}
};

BillingClient.querySkuDetails(SkuDetailsParams.builder()) not working?

Getting an empty list when fetching with querySkuDetails()?
So in case you've been having this issue lately, where you want to fetch your Google Play Console list of SkuDetail, maybe to show the price of one of the SkuDetail and show it to the user has it was in my case or to display some other information about a SkuDetail from your Google Play Console merchant account. Anyways, here's what's worked for me:
First you need to add this to your build.gradle app file:
implementation "com.android.billingclient:billing-ktx:4.0.0"
Then Inside of my Fragment's ViewModel, I did the following:
class MainViewModel(application: Application) : AndroidViewModel(application) {
private val billingClient by lazy {
BillingClient.newBuilder(application.applicationContext)
.setListener(purchasesUpdatedListener)
.enablePendingPurchases()
.build()
}
/**
#param result Returns true if connection was successful, false if otherwise
*/
private inline fun billingStartConnection(crossinline result: (Boolean) -> Unit) {
billingClient.startConnection(object : BillingClientStateListener {
override fun onBillingSetupFinished(billingResult: BillingResult) {
if (billingResult.responseCode == BillingClient.BillingResponseCode.OK) {
// The BillingClient is ready. You can query purchases here.
result(true)
}
}
override fun onBillingServiceDisconnected() {
// Try to restart the connection on the next request to
// Google Play by calling the startConnection() method.
result(false)
}
})
}
sealed class BillingClientObserver {
object Loading : BillingClientObserver()
object ClientDisconnected : BillingClientObserver()
object HasNoPurchases : BillingClientObserver()
object HasNoAdsPrivilege : BillingClientObserver()
object UserCancelledPurchase : BillingClientObserver()
data class UnexpectedError(val debugMessage: String = "") : BillingClientObserver()
}
private val _billingClientObserver: MutableStateFlow<BillingClientObserver> =
MutableStateFlow(BillingClientObserver.Loading)
val billingClientObserver: StateFlow<BillingClientObserver> = _billingClientObserver
suspend fun checkSkuDetailById(productId: String) =
billingStartConnection { billingClientReady ->
if (billingClientReady) {
val skuList = ArrayList<String>()
skuList.add(productId)
val params = SkuDetailsParams.newBuilder()
params.setSkusList(skuList).setType(BillingClient.SkuType.INAPP)
viewModelScope.launch(Dispatchers.Main) {
val skuDetailList = withContext(Dispatchers.IO) {
billingClient.querySkuDetails(params.build())
}
skuDetailList.skuDetailsList?.let {
Timber.d("Timber> List<SkuDetails>: $it")
if (it.isNotEmpty()) {
val skuDetails: SkuDetails = it[0]
_goAdsFreePricing.value = skuDetails.price
} else {
_billingClientObserver.value =
BillingClientObserver.UnexpectedError(context.getString(R.string.unable_to_get_price_msg))
}
} ?: run {
_billingClientObserver.value =
BillingClientObserver.UnexpectedError(context.getString(R.string.unable_to_get_price_msg))
}
}
} else {
_billingClientObserver.value =
BillingClientObserver.UnexpectedError(context.getString(R.string.unable_connect_to_play_store))
}
}
}
The most important thing to do for the
BillingClient.querySkuDetails(SkuDetailsParams.builder())
to be successful, is to first establish a successful connection through the
BillingClient.startConnection( listener: BillingClientStateListener)
then do the query on the background thread, very important that the
querySkuDetails()
happens in the background thread as it is made to fail if done on the
#MainThread
then listen for its result on the
#MainThread
like in the above example.

How can I collect values from a mutableSharedFlow in a unit test?

The following test doesn't pass. The string "before adding XXX" is never printed.
#Test
fun testFoo() = runBlocking {
val testCoroutineScope = TestCoroutineScope().apply {
pauseDispatcher() // This needs to be here because the actual test handles time.
}
val sharedFlow = MutableSharedFlow<Int>()
val values = mutableListOf<Int>()
println("before launch")
val job = testCoroutineScope.launch {
println("before collect")
sharedFlow.collect {
println("before adding $it")
values.add(it)
}
}
println("before emits")
sharedFlow.emit(1)
sharedFlow.emit(2)
testCoroutineScope.runCurrent()
assertEquals(mutableListOf(1, 2), values)
job.cancel()
}
I would like to have a way to emit values once the collect part is handled. I can't set a replay value, nor can I use an onSubscription because that would change the business logic.
Adding delay or yield don't seem to make any difference so it might not be a racing condition.
The test passes if pauseDispatcher() is commented.
My understanding was that emit would suspend until the collect lambda were called.
The way I solved it is via extraBufferCapacity and one more testCoroutineScope.runCurrent()
#Test
fun testFoo() = runBlocking {
val testCoroutineScope = TestCoroutineScope().apply {
pauseDispatcher()
}
val sharedFlow = MutableSharedFlow<Int>(
extraBufferCapacity = 2 // Without it, sharedFlow.emit won't have a space to save data. It will be collected
// next time there's a testCoroutineScope.runCurrent()
)
val values = mutableListOf<Int>()
println("before launch")
val job = testCoroutineScope.launch {
println("before collect")
sharedFlow.collect {
println("before adding $it")
values.add(it)
}
}
testCoroutineScope.runCurrent() // Allows the previous launch to start collecting
println("before emits")
sharedFlow.emit(1)
sharedFlow.emit(2)
testCoroutineScope.runCurrent()
assertEquals(mutableListOf(1, 2), values)
job.cancel()
}
UPDATE: Turns out my initial attempt wasn't bulletproof either, a race condition still existed, here's my revised solution, so far so good:
val msf = MutableSharedFlow<Int>()
runBlocking {
val job1 = GlobalScope.launch {
msf
.onStart {
println("${Thread.currentThread().name} - on start ")
}
.onEach { println("got $it") }
.collect()
}
val job2 = launch {
msf.subscriptionCount
.filter { it > 0 }
.onEach {
msf.emit(42)
cancel()
}
.collect()
}
delay(100)
job1.cancel()
println(job2)
}
println("${Thread.currentThread().name} - escaped!")
Old solution (incorrect):
runBlocking {
val msf = MutableSharedFlow<Int>()
val d = CompletableDeferred<Boolean>()
launch {
msf
.onStart {
d.complete(true)
}
.onEach { println("got $it") }
.collect()
}
d.join()
msf.emit(42)
}
using the invokeOnCompletion() handler cannot use suspended functions, using it with tryEmit() did not work for me (returns true but does nothing).

mock retrofit suspend function infinite response

I would like to test case when server does not return response, and we trigger the next network call ( like for example search query).
So we basically have a method inside ViewModel and Retrofit method
interface RetrofitApi {
#GET("Some Url")
suspend fun getVeryImportantStuff(): String
}
class TestViewModel(private val api: RetrofitApi) : ViewModel() {
private var askJob: Job? = null
fun load(query: String) {
askJob?.cancel()
askJob = viewModelScope.launch {
val response = api.getVeryImportantStuff()
//DO SOMETHING WITH RESPONSE
}
}
}
And I would like to test case when new query is asked, and the old one didn't returns.
for case when response returns test is easy
#Test
fun testReturnResponse() {
runBlockingTest {
//given
val mockApi:RetrofitApi = mock()
val viewModel = TestViewModel(mockApi)
val response = "response from api"
val query = "fancy query"
whenever(mockApi.getVeryImportantStuff()).thenReturn(response)
//when
viewModel.load(query)
//then
//verify what happens
}
}
But I don't know how to mock suspend function that did't come back, and test case when new request is triggered like this
#Test
fun test2Loads() {
runBlockingTest {
//given
val mockApi:RetrofitApi = mock()
val viewModel = TestViewModel(mockApi)
val response = "response from api"
val secondResponse = "response from api2"
val query = "fancy query"
whenever(mockApi.getVeryImportantStuff())
.thenReturn(/* Here return some fancy stuff that is suspend* or something like onBlocking{} stub but not blocking but dalayed forever/)
.thenReturn(secondResponse)
//when
viewModel.load(query)
viewModel.load(query)
//then
//verify that first response did not happens , and only second one triggered all the stuff
}
}
Any ideas ?
EDIT: I'm not really attached to mockito, any mock library will be good :)
regards
Wojtek
I came up with kind of solution to the problem, but slightly different than I was thinking at the beginning
interface CoroutineUtils {
val io: CoroutineContext
}
interface RetrofitApi {
#GET("Some Url")
suspend fun getVeryImportantStuff(query: String): String
}
class TestViewModel(private val api: RetrofitApi,
private val utils: CoroutineUtils) : ViewModel() {
private val text = MutableLiveData<String>()
val testStream: LiveData<String> = text
private var askJob: Job? = null
fun load(query: String) {
askJob?.cancel()
askJob = viewModelScope.launch {
val response = withContext(utils.io) { api.getVeryImportantStuff(query) }
text.postValue(response)
}
}
}
And the test scenario would look like this
class TestViewModelTest {
#get:Rule
val coroutineScope = MainCoroutineScopeRule()
#get:Rule
val instantTaskExecutorRule = InstantTaskExecutorRule()
lateinit var retrofit: RetrofitApi
lateinit var utils: CoroutineUtils
val tottalyDifferentDispatcher = TestCoroutineDispatcher()
lateinit var viewModel: TestViewModel
#Before
fun setup() {
retrofit = mock()
utils = mock()
viewModel = TestViewModel(retrofit, utils)
}
#UseExperimental(ExperimentalCoroutinesApi::class)
#Test
fun test2Loads() {
runBlockingTest {
//given
val response = "response from api"
val response2 = "response from api2"
val query = "fancy query"
val query2 = "fancy query2"
whenever(utils.io)
.thenReturn(tottalyDifferentDispatcher)
val mutableListOfStrings = mutableListOf<String>()
whenever(retrofit.getVeryImportantStuff(query)).thenReturn(response)
whenever(retrofit.getVeryImportantStuff(query2)).thenReturn(response2)
//when
viewModel.testStream.observeForever {
mutableListOfStrings.add(it)
}
tottalyDifferentDispatcher.pauseDispatcher()
viewModel.load(query)
viewModel.load(query2)
tottalyDifferentDispatcher.resumeDispatcher()
//then
mutableListOfStrings shouldHaveSize 1
mutableListOfStrings[0] shouldBe response2
verify(retrofit, times(1)).getVeryImportantStuff(query2)
}
}
}
It is not exactly what I wanted, because retrofit call is not triggered when load method is called for the first time, but it is the closest solution.
What would be a perfect test for me will be assertion that retrofit was called twice , but only the second one returned to ViewModel. Solution for that will be to wrap Retrofit around method that returns suspend function like this
interface RetrofitWrapper {
suspend fun getVeryImportantStuff(): suspend (String)->String
}
class TestViewModel(private val api: RetrofitWrapper,
private val utils: CoroutineUtils) : ViewModel() {
private val text = MutableLiveData<String>()
val testStream: LiveData<String> = text
private var askJob: Job? = null
fun load(query: String) {
askJob?.cancel()
askJob = viewModelScope.launch {
val veryImportantStuff = api.getVeryImportantStuff()
val response = withContext(utils.io) {
veryImportantStuff(query)
}
text.postValue(response)
}
}
}
and test for it
#Test
fun test2Loads() {
runBlockingTest {
//given
val response = "response from api"
val response2 = "response from api2"
val query = "fancy query"
val query2 = "fancy query2"
whenever(utils.io)
.thenReturn(tottalyDifferentDispatcher)
val mutableListOfStrings = mutableListOf<String>()
whenever(retrofit.getVeryImportantStuff())
.thenReturn(suspendCoroutine {
it.resume { response }
})
whenever(retrofit.getVeryImportantStuff()).thenReturn(suspendCoroutine {
it.resume { response2 }
})
//when
viewModel.testStream.observeForever {
mutableListOfStrings.add(it)
}
tottalyDifferentDispatcher.pauseDispatcher()
viewModel.load(query)
viewModel.load(query2)
tottalyDifferentDispatcher.resumeDispatcher()
//then
mutableListOfStrings shouldHaveSize 1
mutableListOfStrings[0] shouldBe response2
verify(retrofit, times(2)).getVeryImportantStuff()
}
}
But in my opinion it is a little bit too much in interference in code only to be testable. But maybe I'm wrong :P
Looks like you want to test scenario when you have unreachable server, timeout or something similar.
In this case while doing your mock you can say that on first try it returns object and then on second executions throws exception like java.net.ConnectException: Connection timed out.
whenever(mockApi.getVeryImportantStuff())
.thenReturn(someObjet)
.thenThrow(ConnectException("timed out"))
And this this should work but you will have to do try/catch block in ViewModel witch is not ideal. I would suggest you to add additional abstraction.
You could you Repository or UseCase or whatever pattern/name you like to move the network call there. Then introduce sealed class Result to encapsulate behaviour and make your ViewModel more readable.
class TestViewModel(val repo: Repo): ViewModel() {
private var askJob: Job? = null
fun load(query: String) {
askJob?.cancel()
askJob = viewModelScope.launch {
when (repo.getStuff()) {
is Result.Success -> TODO()
is Result.Failure -> TODO()
}
}
}
}
class Repo(private val api: Api) {
suspend fun getStuff() : Result {
return try {
Result.Success(api.getVeryImportantStuff())
} catch (e: java.lang.Exception) {
Result.Failure(e)
}
}
}
sealed class Result {
data class Success<out T: Any>(val data: T) : Result()
data class Failure(val error: Throwable) : Result()
}
interface Api {
suspend fun getVeryImportantStuff() : String
}
With that level of abstraction your ViewModelTest only checks what happens in two cases.
Hope that's helpful!

Akka: testing actor got message

I have the next code:
//TestActor got some message
class TestActor extends Actor {
def receive = {
case string: String => //....
}
}
//TestReg when create get ActorRef, when i call `pass` method, then should pass text to ActorRef
class TestReg(val actorRef: ActorRef) {
def pass(text: String) {
actorRef ! text
}
}
When i wrote test:
class TestActorReg extends TestKit(ActorSystem("system")) with ImplicitSender
with FlatSpecLike with MustMatchers with BeforeAndAfterAll {
override def afterAll() {
system.shutdown()
}
"actorReg" should "pass text to actorRef" in {
val probe = TestProbe()
val testActor = system.actorOf(Props[TestActor])
probe watch testActor
val testReg = new TestReg(testActor)
testReg.pass("test")
probe.expectMsg("test")
}
}
I got error:
java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsg while waiting for test
How to check what the actor got a text?
probe.expectMsg() is calling the assertion on the probe. But you passed the testActor into your TestReg class
change it to the following line and it will work
val testReg = new TestReg(probe.ref)
have to call .ref to make the probe into an ActorRef
And you want to do it here not at the instantiation of the variable to avoid
certain bugs that are outside of the scope of this response
the error in the logic as I see it is you are thinking that the watch method makes probe see what test actor does. but its death watch not message watch. which is different.
Create application.conf file with this:
akka {
test {
timefactor = 1.0
filter-leeway = 999s
single-expect-default = 999s
default-timeout = 999s
calling-thread-dispatcher {
type = akka.testkit.CallingThreadDispatcherConfigurator
}
}
actor {
serializers {
test-message-serializer = "akka.testkit.TestMessageSerializer"
}
serialization-identifiers {
"akka.testkit.TestMessageSerializer" = 23
}
serialization-bindings {
"akka.testkit.JavaSerializable" = java
}
}
}