Kestrel gives bad request after FIN and end of request - amazon-web-services

I have a API made in Dotnet 6, running in a docker container and hosted on AWS.
The client is built in Wordpress and uses php curl to make requests to the api.
The Api is built with using minimal APIs and as much as I can see it doesn't do anything too fancy.
After going live with the API I did start notice strange logs where requests end with a BadHttpRequestException.
I have not been able to trace what is causing it but if we look at the server logs that traces the entire request we can see that the request is fulfilled and it sends back the response with a status of 200.
After that at row 18 it starts to close the connection, but at row 19 it starts to read the body again which after a while ends with a Bad request error at row 22 before the connection is stopped.
This error is not caught in my exception middleware so I'm lost to what to do or how to fix in.
Any suggestions would be highly appreciated.
Both Logs below are for the same endpoint on the api
Log with error in it
1 "Connection id \"{ConnectionId}\" accepted.",
2 "Connection id \"{ConnectionId}\" started.",
3 "{HostingRequestStartingLog:l}",
4 "{CandidateCount} candidate(s) found for the request path '{Path}'",
5 "Request matched endpoint '{EndpointName}'",
6 "The request has an origin header: '{origin}'.",
7 "CORS policy execution successful.",
8 "Successfully validated the token.",
9 "AuthenticationScheme: {AuthenticationScheme} was successfully authenticated.",
10"AuthenticationScheme: {AuthenticationScheme} was successfully authenticated.",
11"Authorization was successful.",
12"Executing endpoint '{EndpointName}'",
13"Writing value of type '{Type}' with status code '200'.",
14"Executed endpoint '{EndpointName}'",
15"HTTP {RequestMethod} {RequestPath} responded 200 in {Elapsed:0.0000} ms",
16"Connection id \"{ConnectionId}\" completed keep alive response.",
17"{HostingRequestFinishedLog:l}",
18"Connection id \"{ConnectionId}\" received FIN.",
19"Connection id \"{ConnectionId}\", Request id \"{TraceIdentifier}\": started reading request body.",
20"Connection id \"{ConnectionId}\" sending FIN because: \"{Reason}\"",
21"Connection id \"{ConnectionId}\" disconnecting.",
22"Connection id \"{ConnectionId}\" bad request data: \"{message}\"",
23"Exception": "Microsoft.AspNetCore.Server.Kestrel.Core.BadHttpRequestException: Unexpected end of request content.\n at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1ContentLengthMessageBody.TryReadInternal(ReadResult& readResult)\n at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1MessageBody.OnConsumeAsync()"
24"Connection id \"{ConnectionId}\", Request id \"{TraceIdentifier}\": done reading request body.",
25"Connection id \"{ConnectionId}\" stopped.",
Log that is fine
1 "Connection id \"{ConnectionId}\" accepted.",
2 "Connection id \"{ConnectionId}\" started.",
3 "{HostingRequestStartingLog:l}",
4 "{CandidateCount} candidate(s) found for the request path '{Path}'",
5 "Request matched endpoint '{EndpointName}'",
6 "The request has an origin header: '{origin}'.",
7 "CORS policy execution successful.",
8 "Successfully validated the token.",
9 "AuthenticationScheme: {AuthenticationScheme} was successfully authenticated.",
10"AuthenticationScheme: {AuthenticationScheme} was successfully authenticated.",
11"Authorization was successful.",
12"Executing endpoint '{EndpointName}'",
13"Writing value of type '{Type}' with status code '200'.",
14"Executed endpoint '{EndpointName}'",
15"HTTP {RequestMethod} {RequestPath} responded 200 in {Elapsed:0.0000} ms",
16"Connection id \"{ConnectionId}\" completed keep alive response.",
17"{HostingRequestFinishedLog:l}",
18"Connection id \"{ConnectionId}\", Request id \"{TraceIdentifier}\": started reading request body.",
19"Connection id \"{ConnectionId}\", Request id \"{TraceIdentifier}\": done reading request body.",
20"Connection id \"{ConnectionId}\" received FIN.",
21"Connection id \"{ConnectionId}\" disconnecting.",
22"Connection id \"{ConnectionId}\" sending FIN because: \"{Reason}\"",
23"Connection id \"{ConnectionId}\" stopped.",

Related

WSO2 API-M v4.1.0: 404 with log message STATUS = Message dispatched to the main sequence. Invalid URL., RESOURCE = , HEALTH CHECK URL

I was able to create and publish APIs on my dev WSO2 installation based on H2 DB. Unfortunately started from the fifth APIs (the first four still working very well) I have this error.
If I'm calling the API from Postman I receive
{
"code": 404,
"type": "Status report",
"message": "Not Found",
"description": "The requested resource is not available."
}
Into the logs I see
INFO {org.apache.synapse.mediators.builtin.LogMediator} - STATUS = Message dispatched to the main sequence. Invalid URL., RESOURCE = XXXXXXXXXXXXXXX, HEALTH CHECK URL = XXXXXXXXXXXXXXX
Any help will appriciate.
API-M version 4.1.0
Regards,

To change the status code from 200 to 404 when stepfunction not found

I'm trying to Execute the AWS step function from API Gateway, It's working as expected.
Whenever I'm passing the input, statemachinearn(stepfunction name to execute) It's triggering the step function.
But It's still returning the status code 200, whenever it's not able to find the stepfunction, I want to return the status code 404 if the apigateway not found that stepfunction.
Could you please help me on that
Response:
Status: 200ok
Expected:
Status: 404
Thanks,
Harika.
As per the documentation StartExecution API call do return 400 Bad Request for non existent statemachine which is correct as RESTful API standard.
StateMachineDoesNotExist
The specified state machine does not exist.
HTTP Status Code: 400
From the RESTful API point of view, endpoint /execution/(which I created in API Gateway for the integration setup) is a resource, no matter it accepts GET or POST or something else. 404 is only appropriate when the resource /execution/ itself does not exist. If /execution/ endpoint exists, but its invocation failed (no matter what the reasons), the response status code must be something other than 404.
So in the case of the returned response(200) for POST call with non-existent statemachine it is correct. But when API Gateway tried to make the call to non-existent statemachine it got 404 from StartExecution api call which it eventually wrapped into a proper message instead of returning 404 http response.
curl -s -X POST -d '{"input": "{}","name": "MyExecution17","stateMachineArn": "arn:aws:states:eu-central-1:1234567890:stateMachine:mystatemachine"}' https://123456asdasdas.execute-api.eu-central-1.amazonaws.com/v1/execution|jq .
{
"__type": "com.amazonaws.swf.service.v2.model#StateMachineDoesNotExist",
"message": "State Machine Does Not Exist: 'arn:aws:states:eu-central-1:1234567890:stateMachine:mystatemachine1'"
}
Let's say you create another MethodResponse where you can provide an exact HTTP Status Code in your case 404 which you want to return and you do an Integration Response where you have to choose the Method Response by providing either Exact HTTP Responce Code(400 -> Upstream response from the **StartExecution** API Call) OR a Regex -> (4\{d}2) matching all the 4xx errors.
In that case you will be giving 404 for all the responses where the upstream error 4xx StartExecution Errors
ExecutionAlreadyExists -> 400
ExecutionLimitExceeded -> 400
InvalidArn -> 400
InvalidExecutionInput -> 400
InvalidName -> 400
StateMachineDeleting -> 400
StateMachineDoesNotExist -> 400
Non Existent State Machine:
curl -s -X POST -d '{"input": "{}","name": "MyExecution17","stateMachineArn": "arn:aws:states:eu-central-1:1234567890:stateMachine:mystatemachine1"}' https://123456asdasdas.execute-api.eu-central-1.amazonaws.com/v1/execution|jq .
< HTTP/2 404
< date: Sat, 30 Jan 2021 14:12:16 GMT
< content-type: application/json
...
{
"__type": "com.amazonaws.swf.service.v2.model#StateMachineDoesNotExist",
"message": "State Machine Does Not Exist: 'arn:aws:states:eu-central-1:1234567890:stateMachine:mystatemachine1'"
}
Execution Already Exists
curl -s -X POST -d '{"input": "{}","name": "MyExecution17","stateMachineArn": "arn:aws:states:eu-central-1:1234567890:stateMachine:mystatemachine"}' https://123456asdasdas.execute-api.eu-central-1.amazonaws.com/v1/execution|jq .
* We are completely uploaded and fine
< HTTP/2 404
< date: Sat, 30 Jan 2021 14:28:27 GMT
< content-type: application/json
{
"__type": "com.amazonaws.swf.service.v2.model#ExecutionAlreadyExists",
"message": "Execution Already Exists: 'arn:aws:states:eu-central-1:1234567890:execution:mystatemachine:MyExecution17'"
}
Which I think will be misleading.

Jenkins send email with part of log line

I have SOAPUI project with 10 test cases in Jenkins. I set up Jenkins to send me email with information from console output (log). I have set up email notification content as HTML (text/html).
I have this log in console output in Jenkins:
07:25:05,957 INFO [SoapUITestCaseRunner] Running SoapUI testcase [Login with username and password]
07:25:05,957 INFO [SoapUITestCaseRunner] running step [Clear access token]
07:25:05,957 INFO [log] Environment URL: url.test.environment
07:25:05,958 INFO [SoapUITestCaseRunner] running step [Retrieve accessToken]
07:25:05,959 DEBUG [HttpClientSupport$SoapUIHttpClient] Stale connection check
07:25:05,960 DEBUG [HttpClientSupport$SoapUIHttpClient] Attempt 1 to execute request
07:25:05,960 DEBUG [SoapUIMultiThreadedHttpConnectionManager$SoapUIDefaultClientConnection] Sending request: POST /api/v2/path HTTP/1.1
07:25:06,010 DEBUG [SoapUIMultiThreadedHttpConnectionManager$SoapUIDefaultClientConnection] Receiving response: HTTP/1.1 200
07:25:06,011 DEBUG [HttpClientSupport$SoapUIHttpClient] Connection can be kept alive indefinitely
07:25:06,017 INFO [SoapUITestCaseRunner] Assertion [JsonPath Existence Match] has status VALID
07:25:06,017 INFO [SoapUITestCaseRunner] Assertion [Valid HTTP Status Codes] has status VALID
07:25:06,017 INFO [SoapUITestCaseRunner] Assertion [JsonPath Existence Match 1] has status VALID
07:25:06,017 INFO [SoapUITestCaseRunner] Assertion [JsonPath Existence Match 2] has status VALID
07:25:06,017 INFO [SoapUITestCaseRunner] running step [Pass accessToken]
07:25:06,019 INFO [SoapUITestCaseRunner] Finished running SoapUI testcase [Login with username and password], time taken: 51ms, status: FINISHED
I have set up Jenkins to send email just with this line from log:
07:25:06,019 INFO [SoapUITestCaseRunner] Finished running SoapUI testcase [Login with username and password], time taken: 51ms, status: FINISHED
To that I use this regex to find just that line:
<pre>${BUILD_LOG_REGEX, regex="Finished running SoapUI testcase \\[Login with username and password\\]", showTruncatedLines=false}</pre>
But I want to have in email notification just parts of that line, something like this:
"Login with username and password: FINISHED"
or
"Login with username and password: FAILED"
Is there any way to send email with just part of that line?
Thanks to Aaron I found a solution.
Now I use this regex:
<b>Login with username and password: </b> <font color="green">${BUILD_LOG_REGEX, regex=".*Finished running SoapUI testcase \\[Login with username and password\\].*status: FINISHED", showTruncatedLines=false, substText="SUCCESS"}</font>
<font color="red">${BUILD_LOG_REGEX, regex=".*Finished running SoapUI testcase \\[Login with username and password\\].*status: FAILED", showTruncatedLines=false, substText="FAILED"}</font>
E-mail notification now looks like this:
Login with username and password: SUCCESS
or this:
Login with username and password: FAILED
"SUCCESS" text has green color and "FAILED" text has red color
It seems like you can use a substText argument to define a replacement pattern :
<pre>${BUILD_LOG_REGEX, regex="Finished running SoapUI testcase \\[Login with username and password\\].*status: ([A-Z]+)", showTruncatedLines=false, substText="Login with username and password : \\1"}</pre>
This will match only the lines such as the last one from your sample, select the success/failure status in the first capturing group which will be referred in the replacement pattern.

Amazon Lex: can't make mobile app work

I have a bot that works perfectly in the AWS console.
I've tried to make it work in a mobile app using AWS MobileHub but it doesn't work well and I don't know why I'm totally stuck.
Description: I make a request with a project name to get the list of tasks on this project. But first I have to connect. The "connection" request gets a token from my backend and put it in sessionAttributes. The other requests must have this token in sessionAttributes or the bot will ask to authenticate.
The workflow works with the AWS console bot:
Request "tasks for project 123" -> Answer "you must authenticate"
Request "connection" -> Answer "you are now connected"
Request "tasks for project 123" -> Answer "tasks for project 123
are 4, 8, 11. Choose a task to get details."
Now what happens with the app, in text mode:
Request "tasks for project 123" -> Answer "you must authenticate"
Request "connection" -> No answer
Request "tasks for project 123" -> The app crashes
Logs in Android Studio at step 3 (crash):
06-28 11:05:58.942 1719-1719/com.amazon.mysampleapp E/AndroidRuntime: FATAL EXCEPTION: main
Process: com.amazon.mysampleapp, PID: 1719
Theme: themes:{default=overlay:com.cyngn.hexo, iconPack:com.cyngn.hexo, fontPkg:com.cyngn.hexo, com.android.systemui=overlay:com.cyngn.hexo, com.android.systemui.navbar=overlay:com.cyngn.hexo}
java.lang.NullPointerException: Attempt to invoke virtual method 'void com.amazonaws.mobileconnectors.lex.interactionkit.continuations.LexServiceContinuation.continueWithTextInForTextOut(java.lang.String)' on a null object reference
at com.mysampleapp.demo.bots.ConversationalBotTextFragment.textEntered(ConversationalBotTextFragment.java:135)
at com.mysampleapp.demo.bots.ConversationalBotTextFragment.access$000(ConversationalBotTextFragment.java:42)
at com.mysampleapp.demo.bots.ConversationalBotTextFragment$1.onKey(ConversationalBotTextFragment.java:86)
at android.view.View.dispatchKeyEvent(View.java:9236)
at android.view.ViewGroup.dispatchKeyEvent(ViewGroup.java:1640)
at android.view.ViewGroup.dispatchKeyEvent(ViewGroup.java:1640)
at android.view.ViewGroup.dispatchKeyEvent(ViewGroup.java:1640)
at android.view.ViewGroup.dispatchKeyEvent(ViewGroup.java:1640)
at android.view.ViewGroup.dispatchKeyEvent(ViewGroup.java:1640)
at android.view.ViewGroup.dispatchKeyEvent(ViewGroup.java:1640)
at android.view.ViewGroup.dispatchKeyEvent(ViewGroup.java:1640)
at android.view.ViewGroup.dispatchKeyEvent(ViewGroup.java:1640)
at android.view.ViewGroup.dispatchKeyEvent(ViewGroup.java:1640)
at com.android.internal.policy.PhoneWindow$DecorView.superDispatchKeyEvent(PhoneWindow.java:2651)
at com.android.internal.policy.PhoneWindow.superDispatchKeyEvent(PhoneWindow.java:1777)
at android.app.Activity.dispatchKeyEvent(Activity.java:2731)
at android.support.v7.app.AppCompatActivity.dispatchKeyEvent(AppCompatActivity.java:543)
at android.support.v7.view.WindowCallbackWrapper.dispatchKeyEvent(WindowCallbackWrapper.java:53)
at android.support.v7.app.AppCompatDelegateImplBase$AppCompatWindowCallbackBase.dispatchKeyEvent(AppCompatDelegateImplBase.java:312)
at android.support.v7.view.WindowCallbackWrapper.dispatchKeyEvent(WindowCallbackWrapper.java:53)
at com.android.internal.policy.PhoneWindow$DecorView.dispatchKeyEvent(PhoneWindow.java:2363)
at android.view.ViewRootImpl$ViewPostImeInputStage.processKeyEvent(ViewRootImpl.java:4135)
at android.view.ViewRootImpl$ViewPostImeInputStage.onProcess(ViewRootImpl.java:4097)
at android.view.ViewRootImpl$InputStage.deliver(ViewRootImpl.java:3650)
at android.view.ViewRootImpl$InputStage.onDeliverToNext(ViewRootImpl.java:3703)
at android.view.ViewRootImpl$InputStage.forward(ViewRootImpl.java:3669)
at android.view.ViewRootImpl$AsyncInputStage.forward(ViewRootImpl.java:3795)
at android.view.ViewRootImpl$InputStage.apply(ViewRootImpl.java:3677)
at android.view.ViewRootImpl$AsyncInputStage.apply(ViewRootImpl.java:3852)
at android.view.ViewRootImpl$InputStage.deliver(ViewRootImpl.java:3650)
at android.view.ViewRootImpl$InputStage.onDeliverToNext(ViewRootImpl.java:3703)
at android.view.ViewRootImpl$InputStage.forward(ViewRootImpl.java:3669)
at android.view.ViewRootImpl$InputStage.apply(ViewRootImpl.java:3677)
at android.view.ViewRootImpl$InputStage.deliver(ViewRootImpl.java:3650)
at android.view.ViewRootImpl.deliverInputEvent(ViewRootImpl.java:5932)
at android.view.ViewRootImpl.doProcessInputEvents(ViewRootImpl.java:5906)
at android.view.ViewRootImpl.enqueueInputEvent(ViewRootImpl.java:5867)
at android.view.ViewRootImpl$ViewRootHandler.handleMessage(ViewRootImpl.java:3442)
at android.os.Handler.dispatchMessage(Handler.java:102)
at android.os.Looper.loop(Looper.java:148)
at android.app.ActivityThread.main(ActivityThread.java:5461)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:726)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616)
06-28 11:05:58.944 789-3750/? W/ActivityManager: Force finishing activity com.amazon.mysampleapp/com.mysampleapp.MainActivity
And here with the voice on mobile app:
Request "tasks for project 123" -> Answer "you must authenticate"
Request "connection" -> Answer "you are now connected"
Request "tasks for project 123" -> Answer "you must authenticate"
Logs in Android Studio at steps 1 and 3:
06-28 11:24:15.227 4265-4265/com.amazon.mysampleapp E/Lex: InteractiveVoiceViewAdapter: Interaction error
com.amazonaws.mobileconnectors.lex.interactionkit.exceptions.DialogFailedException: Failed to fulfill current request.
at com.amazonaws.mobileconnectors.lex.interactionkit.InteractionClient$8.run(InteractionClient.java:593)
at android.os.Handler.handleCallback(Handler.java:739)
at android.os.Handler.dispatchMessage(Handler.java:95)
at android.os.Looper.loop(Looper.java:148)
at android.app.ActivityThread.main(ActivityThread.java:5461)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:726)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616)
06-28 11:24:15.227 4265-4265/com.amazon.mysampleapp E/ConversationalBotVoice: Error: You must authenticate.
com.amazonaws.mobileconnectors.lex.interactionkit.exceptions.DialogFailedException: Failed to fulfill current request.
at com.amazonaws.mobileconnectors.lex.interactionkit.InteractionClient$8.run(InteractionClient.java:593)
at android.os.Handler.handleCallback(Handler.java:739)
at android.os.Handler.dispatchMessage(Handler.java:95)
at android.os.Looper.loop(Looper.java:148)
at android.app.ActivityThread.main(ActivityThread.java:5461)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:726)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616)
So I tried to change the lambda so I don't need the connection part (but still it uses the token for the requests, I've just written the token in the lambda instead of getting it from the backend).
In AWS console, I get the tasks list
On mobile with text, I get the tasks list
On mobile with voice, it doesn't work
And the logs for that last case:
06-28 11:38:40.619 7655-7655/com.amazon.mysampleapp E/Lex: InteractiveVoiceViewAdapter: Interaction error
com.amazonaws.services.lexrts.model.DependencyFailedException: Received error response from Lambda: Handled (Service: AmazonLexRuntime; Status Code: 424; Error Code: DependencyFailedException; Request ID: 8f30ae21-5be5-11e7-bc7b-8d61d03f20da)
at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:729)
at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:405)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:212)
at com.amazonaws.services.lexrts.AmazonLexRuntimeClient.invoke(AmazonLexRuntimeClient.java:651)
at com.amazonaws.services.lexrts.AmazonLexRuntimeClient.postContent(AmazonLexRuntimeClient.java:450)
at com.amazonaws.mobileconnectors.lex.interactionkit.InteractionClient$2.run(InteractionClient.java:375)
at java.lang.Thread.run(Thread.java:818)
06-28 11:38:40.620 7655-7655/com.amazon.mysampleapp E/ConversationalBotVoice: Error: Error from Bot
com.amazonaws.services.lexrts.model.DependencyFailedException: Received error response from Lambda: Handled (Service: AmazonLexRuntime; Status Code: 424; Error Code: DependencyFailedException; Request ID: 8f30ae21-5be5-11e7-bc7b-8d61d03f20da)
at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:729)
at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:405)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:212)
at com.amazonaws.services.lexrts.AmazonLexRuntimeClient.invoke(AmazonLexRuntimeClient.java:651)
at com.amazonaws.services.lexrts.AmazonLexRuntimeClient.postContent(AmazonLexRuntimeClient.java:450)
at com.amazonaws.mobileconnectors.lex.interactionkit.InteractionClient$2.run(InteractionClient.java:375)
at java.lang.Thread.run(Thread.java:818)
I don't know what to do, I don't even know how to log things, how to check what is sent to Lex from the app and what Lex sends back. Thanks for your help.

WSO2 ESB HTTPS Endpoint, call mediator - But a callback is not registered ( anymore) to process this response error

While using Call mediator and Send Mediator (calling HTTPS endpoint with username token by using policy at the endpoint)
Getting the following error :
[2015-01-03 03:38:23,058] DEBUG - SynapseCallbackReceiver Callback removed for request message id :
urn:uuid:160c12bd-1286-4ef0-873e-93777380a4a2. Pending callbacks count : 1
[2015-01-03 03:38:23,061] DEBUG - TargetHandler http-outgoing-19: Closed
[2015-01-03 03:38:23,065] DEBUG - TargetHandler http-outgoing-19: Keep-Alive Connection closed
[2015-01-03 03:38:23,064] WARN - SynapseCallbackReceiver Synapse received a response for the reques
t with message Id : urn:uuid:160c12bd-1286-4ef0-873e-93777380a4a2 But a callback is not registered (
anymore) to process this response
[2015-01-03 03:38:23,067] DEBUG - LoggingNHttpClientConnection http-outgoing-19: Shutdown connection
[2015-01-03 03:38:51,472] DEBUG - access - 68.232.203.67 - - [03/Jan/2015:03:38:51 +0530] "POST /Ser
vice.asmx HTTP/1.1" - - "-" "Synapse-PT-HttpComponents-NIO"
[2015-01-03 03:38:51,476] DEBUG - access - 68.232.203.67 - [03/Jan/2015:03:38:51 +0530] "- - " 200
I can see the expected response on the console WIRE>>
but getting this error and the response is not being correlated
please let me know if anyone has faced this issue earlier and how to resolve this.
Thanks
Reason might be, your service endpoint takes much time to respond. Increase the endpoint timeout value in the endpoint configuration and synapse.global_timeout_interval value in synapse properties file.