WSO2 SP, Is there any limitation in JSON event length? - wso2

Question

WSO2 SP 4.3.0 seems not to work with JSON events bigger than 1024 characters. Is there any limitation?
I'm using WSO2 Stream Processor version 4.3.0. I have a Shiddi application deployed using an HTTP source with JSON mapping. Everything works ok as long as the JSON event is less than 1024 characters. If the JSON's lenght is more than 1024 characters, then it looks like SP is splitting the message into several different events, and it fails when trying to parse the JSON content.
It is easy to reproduce using the included samples, "ReceiveHTTPInJsonFormatWithDefaultMapping".
#Source(type = 'http', receiver.url='http://localhost:8006/productionStream', basic.auth.enabled='false',
#map(type='json'))
define stream SweetProductionStream (name string, amount double);
#sink(type='log')
define stream LowProductionAlertStream (name string, amount double);
-- passthrough data in the SweetProductionStream into LowProducitonAlertStream
#info(name='query1')
from SweetProductionStream
select *
insert into LowProductionAlertStream;
Then try to send a small event:
curl -X POST -d "{\"event\": {\"name\":\"sugar\",\"amount\": 20.5}}" http://localhost
:8006/productionStream --header "Content-Type:application/json"
This will work. Then send a long event (more than 1024 characters)
curl -X POST -d "{\"event\": {\"name\":\"sugarlongggggggggggggggggg....gggggg\",\"amount\": 20.5}}" http://localhost
:8006/productionStream --header "Content-Type:application/json"
In the logs you can see:
ERROR {org.wso2.extension.siddhi.map.json.sourcemapper.JsonSourceMapper} - Invalid Json String :{"event": {"name":"sugarlonggggggggggggggggggggggggggggggggggggggggggggggggggggggggg
26/3/2019 20:38:15[2019-03-26 19:38:15,789] ERROR {org.wso2.extension.siddhi.map.json.sourcemapper.JsonSourceMapper} - Invalid Json String :gggggggggggggggggggggggggggggggggggggggggggggggggggggggg","amount": 20.5}}
26/3/2019 20:38:15[2019-03-26 19:38:15,791] ERROR {org.wso2.transport.http.netty.common.Util} - Remote client closed the connection before completing outbound response io.netty.handler.codec.EncoderException: java.lang.IllegalStateException: cannot send more responses than requests
26/3/2019 20:38:15 at io.netty.handler.codec.MessageToMessageEncoder.write(MessageToMessageEncoder.java:106)
26/3/2019 20:38:15 at io.netty.handler.codec.MessageToMessageCodec.write(MessageToMessageCodec.java:116)
26/3/2019 20:38:15 at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:738)
26/3/2019 20:38:15 at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:730)
26/3/2019 20:38:15 at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:816)
26/3/2019 20:38:15 at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:723)
26/3/2019 20:38:15 at io.netty.handler.stream.ChunkedWriteHandler.doFlush(ChunkedWriteHandler.java:304)
26/3/2019 20:38:15 at io.netty.handler.stream.ChunkedWriteHandler.flush(ChunkedWriteHandler.java:137)
26/3/2019 20:38:15 at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:776)
26/3/2019 20:38:15 at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:768)
26/3/2019 20:38:15 at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:749)
26/3/2019 20:38:15 at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
26/3/2019 20:38:15 at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:776)
26/3/2019 20:38:15 at io.netty.channel.AbstractChannelHandlerContext.invokeWriteAndFlush(AbstractChannelHandlerContext.java:802)
26/3/2019 20:38:15 at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:814)
26/3/2019 20:38:15 at io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:794)
26/3/2019 20:38:15 at io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:831)
26/3/2019 20:38:15 at org.wso2.transport.http.netty.contractimpl.HttpOutboundRespListener.writeOutboundResponseHeaderAndBody(HttpOutboundRespListener.java:186)
26/3/2019 20:38:15 at org.wso2.transport.http.netty.contractimpl.HttpOutboundRespListener.writeOutboundResponse(HttpOutboundRespListener.java:138)
26/3/2019 20:38:15 at org.wso2.transport.http.netty.contractimpl.HttpOutboundRespListener.lambda$null$35(HttpOutboundRespListener.java:94)
26/3/2019 20:38:15 at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
26/3/2019 20:38:15 at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
26/3/2019 20:38:15 at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
26/3/2019 20:38:15 at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
26/3/2019 20:38:15 at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
26/3/2019 20:38:15 at java.lang.Thread.run(Thread.java:748)
26/3/2019 20:38:15Caused by: java.lang.IllegalStateException: cannot send more responses than requests
26/3/2019 20:38:15 at io.netty.handler.codec.http.HttpContentEncoder.encode(HttpContentEncoder.java:110)
26/3/2019 20:38:15 at io.netty.handler.codec.http.HttpContentEncoder.encode(HttpContentEncoder.java:51)
26/3/2019 20:38:15 at io.netty.handler.codec.MessageToMessageCodec$1.encode(MessageToMessageCodec.java:67)
26/3/2019 20:38:15 at io.netty.handler.codec.MessageToMessageEncoder.write(MessageToMessageEncoder.java:88)
26/3/2019 20:38:15 ... 25 more

Answer 1

It is a known issue and is fixed from the siddhi-io-http-1.0.40.
In order to apply the latest version on your Stream Processor pack, please follow the below steps,
Download siddhi-io-http-1.0.40.jar from the following link.
https://store.wso2.com/store/assets/analyticsextension/details/63a5db75-586e-48d0-8244-4c16774e8846
Replace the existing siddhi-io-http-1.0.39.jar which can be found in the /lib directory with the latest siddhi-io-http-1.0.40.jar.

Answer 2

This issue has been already identified[1] and fixed in the latest version.
In order to apply the latest version to SP, please follow the steps given below.
Download the new version of siddhi-io-http extension from WSO2 Extension Store[2].
Replace the existing siddhi-io-http-x.x.x.jar which is in SP-HOME/lib directory with the above downloaded one.
Restart the server.
[1] https://github.com/wso2-extensions/siddhi-io-http/issues/84
[2] https://store.wso2.com/store/assets/analyticsextension/details/63a5db75-586e-48d0-8244-4c16774e8846