WSO2 error when converting Text message to o Siddhi Event - wso2
I am using WSO2SP version 4.3.0 with regex extension : siddhi-execution-regex-4.1.2.jar. My code was working fine but whenever i restart my wso2sp instance i start getting error "Exception occurred when converting Text message to Siddhi Event in the stream transactionstream1 of siddhi text input mapper. java.lang.StackOverflowError "
I have to restart wso2sp multiple times to get this working,which is not a viable solution. I am also sending text message via http using extension siddhi-io-http-1.0.40.jar. Text map extension : siddhi-map-text-1.1.2.jar and siddhi-map-text-1.0.22.jar
Test source :
#source(type = 'http', receiver.url = 'http://localhost:5005/text',
#map(type='text',fail.on.missing.attribute='false', regex.A=""""(after)":("(\\"|[^"])*"|\[("(\\"|[^"])*"(,"(\\"|[^"])*")*)?\])""",
#attributes(payload = 'A[2]')))
define stream transactionstream1(payload string);
Test Text:
curl -X POST \
http://localhost:5005/text \
-H 'content-type: application/json' \
-d '{"schema":{"type":"struct","fields":[{"type":"string","optional":false,"field":"id"}],"optional":false,"name":"db.db.Key"},"payload":{"id":"{ \"$oid\" : \"dafeafwe3\"}"}} {"schema":{"type":"struct","fields":[{"type":"string","optional":true,"name":"io.debezium.data.Json","version":1,"field":"after"},{"type":"string","optional":true,"name":"io.debezium.data.Json","version":1,"field":"patch"},{"type":"struct","fields":[{"type":"string","optional":false,"field":"version"},{"type":"string","optional":false,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"ts_ms"},{"type":"string","optional":true,"name":"io.debezium.data.Enum","version":1,"parameters":{"allowed":"true,last,false"},"default":"false","field":"snapshot"},{"type":"string","optional":false,"field":"db"},{"type":"string","optional":false,"field":"rs"},{"type":"string","optional":false,"field":"collection"},{"type":"int32","optional":false,"field":"ord"},{"type":"int64","optional":true,"field":"h"}],"optional":false,"name":"io.debezium.connector.mongo.Source","field":"source"},{"type":"string","optional":true,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"}],"optional":false,"name":"db.db.collection.Envelope"},"payload":{"after":"{\"_id\": {\"$oid\": \"54444342fdsfsdfs\"},\"code\": \"42432432423\",\"name\": \"name1\",\"desc\": \"description\",\"transRefId\": \"cgvvjbjhvjy6979yjbv\",\"origAmount\": 1000,\"amount\": 1000,\"currency\": \"USD\",\"redeemedCashcode\": \"\",\"sender\": {\"id\": \"342edesfsfes\",\"name\": \"rose\",\"phone\": \"123456789\"},\"receiver\": {\"id\": \"fr3wfwrw342\",\"name\": \"daass\",\"phone\": \"453452423\",\"client\": \"CLIENT2\"}}}'
Part of Stack trace:
ERROR {org.wso2.extension.siddhi.map.text.sourcemapper.TextSourceMapper}Exception occurred when converting Text message {"type":"string","optional":false,"field":"id"}],"optional":false,"name":"db.db.Key"},"payload":{"id":"{ \"$oid\" : \"dafeafwe3\"}"}} {"schema":{"type":"struct","fields":[{"type":"string","optional":true,"name":"io.debezium.data.Json","version":1,"field":"after"},{"type":"string","optional":true,"name":"io.debezium.data.Json","version":1,"field":"patch"},{"type":"struct","fields":[{"type":"string","optional":false,"field":"version"},{"type":"string","optional":false,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"ts_ms"},{"type":"string","optional":true,"name":"io.debezium.data.Enum","version":1,"parameters":{"allowed":"true,last,false"},"default":"false","field":"snapshot"},{"type":"string","optional":false,"field":"db"},{"type":"string","optional":false,"field":"rs"},{"type":"string","optional":false,"field":"collection"},{"type":"int32","optional":false,"field":"ord"},{"type":"int64","optional":true,"field":"h"}],"optional":false,"name":"io.debezium.connector.mongo.Source","field":"source"},{"type":"string","optional":true,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"}],"optional":false,"name":"db.db.collection.Envelope"},"payload":{"after":"{\"_id\": {\"$oid\": \"54444342fdsfsdfs\"},\"code\": \"42432432423\",\"name\": \"name1\",\"desc\": \"description\",\"transRefId\": \"cgvvjbjhvjy6979yjbv\",\"origAmount\": 1000,\"amount\": 1000,\"currency\": \"USD\",\"redeemedCashcode\": \"\",\"sender\": {\"id\": \"342edesfsfes\",\"name\": \"rose\",\"phone\": \"123456789\"},\"receiver\": {\"id\": \"fr3wfwrw342\",\"name\": \"daass\",\"phone\": \"453452423\",\"client\": \"CLIENT2\"}}},"op":"r","ts_ms":1575862492251}} to Siddhi Event in the stream transactionstream1 of siddhi text input mapper. java.lang.StackOverflowError
at java.util.regex.Pattern$Branch.match(Pattern.java:4606)
at java.util.regex.Pattern$GroupHead.match(Pattern.java:4660)
at java.util.regex.Pattern$Loop.match(Pattern.java:4787)
at java.util.regex.Pattern$GroupTail.match(Pattern.java:4719)
at java.util.regex.Pattern$BranchConn.match(Pattern.java:4570)
at java.util.regex.Pattern$CharProperty.match(Pattern.java:3779)
at java.util.regex.Pattern$Branch.match(Pattern.java:4606)
at java.util.regex.Pattern$GroupHead.match(Pattern.java:4660)
at java.util.regex.Pattern$Loop.match(Pattern.java:4787)
at java.util.regex.Pattern$GroupTail.match(Pattern.java:4719)
at java.util.regex.Pattern$BranchConn.match(Pattern.java:4570)
at java.util.regex.Pattern$CharProperty.match(Pattern.java:3779)
at java.util.regex.Pattern$Branch.match(Pattern.java:4606)
at java.util.regex.Pattern$GroupHead.match(Pattern.java:4660)
at java.util.regex.Pattern$Loop.match(Pattern.java:4787)
at java.util.regex.Pattern$GroupTail.match(Pattern.java:4719)
at java.util.regex.Pattern$BranchConn.match(Pattern.java:4570)
at java.util.regex.Pattern$CharProperty.match(Pattern.java:3779)
at java.util.regex.Pattern$Branch.match(Pattern.java:4606)
at java.util.regex.Pattern$GroupHead.match(Pattern.java:4660)
at java.util.regex.Pattern$Loop.match(Pattern.java:4787)
at java.util.regex.Pattern$GroupTail.match(Pattern.java:4719)
at java.util.regex.Pattern$BranchConn.match(Pattern.java:4570)
at java.util.regex.Pattern$CharProperty.match(Pattern.java:3779)
at java.util.regex.Pattern$Branch.match(Pattern.java:4606)
at java.util.regex.Pattern$GroupHead.match(Pattern.java:4660)
at java.util.regex.Pattern$Loop.match(Pattern.java:4787)
at java.util.regex.Pattern$GroupTail.match(Pattern.java:4719)
at java.util.regex.Pattern$BranchConn.match(Pattern.java:4570)
at java.util.regex.Pattern$CharProperty.match(Pattern.java:3779)
at java.util.regex.Pattern$Branch.match(Pattern.java:4606)
at java.util.regex.Pattern$GroupHead.match(Pattern.java:4660)
at java.util.regex.Pattern$Loop.match(Pattern.java:4787)
at java.util.regex.Pattern$GroupTail.match(Pattern.java:4719)
at java.util.regex.Pattern$BranchConn.match(Pattern.java:4570)
at java.util.regex.Pattern$Slice.match(Pattern.java:3974)
at java.util.regex.Pattern$Branch.match(Pattern.java:4606)
at java.util.regex.Pattern$GroupHead.match(Pattern.java:4660)
at java.util.regex.Pattern$Loop.match(Pattern.java:4787)
at java.util.regex.Pattern$GroupTail.match(Pattern.java:4719)
at java.util.regex.Pattern$BranchConn.match(Pattern.java:4570)
at java.util.regex.Pattern$CharProperty.match(Pattern.java:3779)
at java.util.regex.Pattern$Branch.match(Pattern.java:4606)
at java.util.regex.Pattern$GroupHead.match(Pattern.java:4660)
at java.util.regex.Pattern$Loop.match(Pattern.java:4787)
at java.util.regex.Pattern$GroupTail.match(Pattern.java:4719)
Is there a version compatible issue ? How do i resolve this permanently ?
It seems like this behavior is related to Java notoriously doesn't liking alternations in certain circumstances where there are potential backtrack problems.
So, this part ("(\"|[^"])"|[("(\"|[^"])"(,"(\"|[^"])"))?])" creates an undo burden on the backtrack mechanism.
Please try updating the regex as below to reduce the impact of the implementation flaw which causes StackOverflowError.
(after)":("(\\"|[|]|[^"])*+")
Please refer Sporadic Stack Overflow error in java Matcher for more details.
Related
Request param is logged in access log with embedded jetty server of spring boot application
I have got an issue with my application, it logs request along with its query param which may contain sensitive data in access log. application is configured with logback.xml & embedded jetty. jetty server is customized with below accessLogCustomer public JettyServerCustomizer accessLogCustomizer() { return server -> { Slf4jRequestLog requestLog = new Slf4jRequestLog(); requestLog.setExtended(true); requestLog.setLogLatency(true); requestLog.setPreferProxiedForAddress(true); requestLog.setLogTimeZone(userTimezone == null ? ZoneId.systemDefault().getId() : userTimezone); requestLog.setLogDateFormat("Y-MM-dd HH:mm:ss, SSS Z"); RequestLogHandler requestLogHandler = new RequestLogHandler(); requestLogHandler.setRequestLog(requestLog); requestLogHandler.setHandler(server.getHandler()); server.setHandler(requestLogHandler); }; } logback.xml <appender name="access" class="ch.qos.logback.core.rolling.RollingFileAppender"> <File>${logs.dir}/abc-access.log</File> <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder"> <layout class="ch.qos.logback.classic.PatternLayout"> <Pattern>%m %n</Pattern> </layout> <charset>UTF-8</charset> </encoder> <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <FileNamePattern>${logs.dir}/abc-access.%d.log.gz</FileNamePattern> </rollingPolicy> </appender> <logger name="org.eclipse.jetty.server.RequestLog" additivity="false"> <appender-ref ref="access"/> </logger> request logged in access log 192.168.0.100 - - [2021-05-20 15:48:15,093 +0530] "POST /myAPI/v2/customer/message?myID=123&messageText=hello HTTP/1.0" 200 0 "-" "PostmanRuntime/7.26.8" 475 I am trying to avoid messageText from access log, but not getting any solution.
Use the CustomRequestLog and Slf4jRequestLogWriter instead. You'll want the special format option %U which emits the URL path, without the query string (which is available as %q btw) Your resulting configuration would look like this ... Slf4jRequestLogWriter slfjRequestLogWriter = new Slf4jRequestLogWriter(); String format = "%{client}a - %u %t %m \"%U\" %s %O \"%{Referer}i\" \"%{User-Agent}i\""; CustomRequestLog customRequestLog = new CustomRequestLog(slfjRequestLogWriter, format); server.setRequestLog(customRequestLog); Play with the format line, read the Javadoc on CustomRequestLog to know what you can do. Some notes: The example format is not strictly following the Extended NCSA format (as it's missing the HTTP version portion, and the HTTP method is outside of the quoted section, but that is usually not a problem for many users) Slf4jRequestLogWriter is only concerned with taking the formatted log line and sending it to the slf4j-api, it does nothing else. RequestLogHandler is deprecated and not a recommended usage anymore (as it does not log bad requests and context-less requests), use the Server.setRequestLog(RequestLog) instead. Jetty will use the CustomRequestLog's Pattern to produce a String, this String is forwarded to the Slf4jRequestLogWriter as a slf4j logging event message, which is then logged per your existing slf4j + logback configuration.
WSO2 IS Unable to compile class for JSP
I'm using WSO2 IS 5.10 with docker and after making a change to the image, which has nothing to do with JSPs, opening the dashboard on the service provider list I see a white screen. In wso2 log I found errors like this: Servlet.service() for servlet [bridgeservlet] threw exception org.apache.jasper.JasperException: Unable to compile class for JSP: An error occurred at line: [17] in the generated java file: [/home/wso2carbon/wso2is-5.10.0/lib/tomcat/work/Catalina/localhost/ROOT/proxytemp/hc_1893914628/org/apache/jsp/application/list_002dservice_002dproviders_jsp.java] Only a type can be imported. org.wso2.carbon.identity.application.common.model.xsd.ApplicationBasicInfo resolves to a package An error occurred at line: [118] in the jsp file: [/application/list-service-providers.jsp] ApplicationBasicInfo cannot be resolved to a type 115: <% 116: String BUNDLE = "org.wso2.carbon.identity.application.mgt.ui.i18n.Resources"; 117: ResourceBundle resourceBundle = ResourceBundle.getBundle(BUNDLE, request.getLocale()); 118: ApplicationBasicInfo[] applications = null; 119: 120: String filterString = request.getParameter(ApplicationMgtUIConstants.SP_NAME_FILTER); 121: filterString = ApplicationMgtUIUtil.resolveFilterString(filterString); this disappears when restarting the image. I’d like to know what it’s due to
XmlStreamException - Got character[0] expected a valid XML character
I am seeing a strange behavior in invoking a webservice. I get the response successfully after this line is executed- OMElement result = client.sendReceive(payload); Now if I hover the mouse and inspect the result which is an OMElement, then everything goes good and the following code executes successfully - System.out.println(result.toString); However if don't inspect the OMElement result variable, I get the following exception - Caused by: javax.xml.stream.XMLStreamException: Error at line:2 col:4000 Got character[0] expected a valid XML character at weblogic.xml.stax.XMLStreamReaderBase.advance(XMLStreamReaderBase.java:206) at weblogic.xml.stax.XMLStreamReaderBase.next(XMLStreamReaderBase.java:237) at org.apache.axiom.util.stax.wrapper.XMLStreamReaderWrapper.next(XMLStreamReaderWrapper.java:225) at org.apache.axiom.util.stax.dialect.DisallowDoctypeDeclStreamReaderWrapper.next(DisallowDoctypeDeclStreamReaderWrapper.java:34) at org.apache.axiom.util.stax.wrapper.XMLStreamReaderWrapper.next(XMLStreamReaderWrapper.java:225) at org.apache.axiom.util.stax.dialect.BEAStreamReaderWrapper.next(BEAStreamReaderWrapper.java:80) at org.apache.axiom.om.impl.builder.StAXOMBuilder.parserNext(StAXOMBuilder.java:668) at org.apache.axiom.om.impl.builder.StAXOMBuilder.next(StAXOMBuilder.java:214) at org.apache.axiom.om.impl.llom.OMElementImpl.buildNext(OMElementImpl.java:709) at org.apache.axiom.om.impl.llom.OMNodeImpl.getNextOMSibling(OMNodeImpl.java:121) at org.apache.axiom.om.impl.traverse.OMChildrenIterator.getNextNode(OMChildrenIterator.java:36) at org.apache.axiom.om.impl.traverse.OMAbstractIterator.hasNext(OMAbstractIterator.java:69) at org.apache.axiom.om.impl.util.OMSerializerUtil.serializeChildren(OMSerializerUtil.java:555) at org.apache.axiom.om.impl.llom.OMElementImpl.internalSerialize(OMElementImpl.java:846) at org.apache.axiom.om.impl.util.OMSerializerUtil.serializeChildren(OMSerializerUtil.java:556) at org.apache.axiom.om.impl.llom.OMElementImpl.internalSerialize(OMElementImpl.java:846) at org.apache.axiom.om.impl.util.OMSerializerUtil.serializeChildren(OMSerializerUtil.java:556) at org.apache.axiom.om.impl.llom.OMElementImpl.internalSerialize(OMElementImpl.java:846) at org.apache.axiom.om.impl.util.OMSerializerUtil.serializeChildren(OMSerializerUtil.java:556) at org.apache.axiom.om.impl.llom.OMElementImpl.internalSerialize(OMElementImpl.java:846) at org.apache.axiom.om.impl.llom.OMSerializableImpl.serialize(OMSerializableImpl.java:120) at org.apache.axiom.om.impl.llom.OMSerializableImpl.serialize(OMSerializableImpl.java:108) at org.apache.axiom.om.impl.llom.OMElementImpl.toString(OMElementImpl.java:957) at com.thehartford.pi.opc.serviceproxy.policyinquiry.impl.PolicyInquiryServiceProxyImpl.parseResponse(PolicyInquiryServiceProxyImpl.java:134) ... 76 more Caused by: Error at line:2 col:4000 Got character[0] expected a valid XML character at weblogic.xml.babel.scanner.ScannerState.checkedRead(ScannerState.java:628) at weblogic.xml.babel.scanner.CharData.read(CharData.java:65) at weblogic.xml.babel.scanner.Scanner.startState(Scanner.java:296) at weblogic.xml.babel.scanner.Scanner.scan(Scanner.java:178) at weblogic.xml.babel.baseparser.BaseParser.accept(BaseParser.java:533) at weblogic.xml.babel.baseparser.BaseParser.accept(BaseParser.java:510) at weblogic.xml.babel.baseparser.EndElement.parse(EndElement.java:34) at weblogic.xml.babel.baseparser.BaseParser.parseElement(BaseParser.java:457) at weblogic.xml.babel.baseparser.BaseParser.parseSome(BaseParser.java:326) at weblogic.xml.stax.XMLStreamReaderBase.advance(XMLStreamReaderBase.java:195) Could some one give me some lead on where the problem could be?
C++-CLI Pipeline only returns the first row of the error message
I'm executing exchange powershell commands through a pipeline from my code. The problem is that when the command is executed on the powershell directly the error returned is four rows long, but when I run the command through the pipeline it only returns the first row of the error message. This is the code I have so far: bool ps_calls::check_invoke(Pipeline^ pipeline) { if (pipeline->Error->Count <= 0) return true; Collection<System::Object^> errorCollection = pipeline->Error->ReadToEnd(); for (int i=0; i< errorCollection.Count; i++) { String^ msg = String::Format("Error {0}",errorCollection[i]->ToString()); m_errors->Add(msg); m_logger->info_msg(String::Format("ERROR: {0}", msg)); } return false; } This is an example of what my pipeline returns now: ERROR: Error The operation couldn't be performed because object 'Testing' couldn't be found on 'server'. This is an example of what the error message is if the same command is executed on the powershell console manually: The operation couldn't be performed because object 'test' couldn't be found on 'server'. + CategoryInfo : NotSpecified: (:) [Get-MailboxDatabase], ManagementObjectNotFoundException + FullyQualifiedErrorId : 2F753561,Microsoft.Exchange.Management.SystemConfigurationTasks.GetMailboxDatabase + PSComputerName : server I would like to be able to get the rest of this error message as well so that I can see the ErrorID when an error occurs.
PowerShell doesn't call ToString when it does formatting, that's why you don't see the same output. Some host applications typically pipe output to Out-Default - if you did that, you'd see the expected format for errors. Other host applications will pipe output to Out-String - this will format objects in the same way that the would be formatted if piped to Out-Default, but you'll get a string object that you can log however you need to.
'Cannot parse input stream' error when updating defects in Rally via pyral
I am using the Python Toolkit for Rally REST API to update defects on our Rally server. I have confirmed that I am able to make contact with the server and authenticate fine by getting a list of current defects. I am running into issues with updating them. I am using Python 2.7.3 with pyral 0.9.1 and requests 0.13.3. Also, I am passing 'verify=False' to the Rally() call and have made appropriate chages to the restapi module to compensate for this. Here is my test code: import sys from pyral import Rally, rallySettings server = "rallydev.server1.com" user = "user#mycompany.com" password = "trial" workspace = "trialWorkspace" project = "Testing Project" defectID = "DE192" rally = Rally(server, user, password, workspace=workspace, project=project, verify=False) defect_data = { "FormattedID" : defectID, "State" : "Closed" } try: defect = rally.update('Defect', defect_data) except Exception, details: sys.stderr.write('ERROR: %s \n' % details) sys.exit(1) print "Defect %s updated" % defect.FormattedID When I run the script: [temp]$ ./updefect.py ERROR: Unable to update the Defect If I change the code in the RallyRESTResponse function to print out the value of self.errors when found (line 164 of rallyresp.py), I get this output: [temp]$ ./updefect.py [u"Cannot parse input stream due to I/O error as JSON document: Parse error: expected '{' but saw '\uffff' [ chars read = >>>\uffff<<< ]"] ERROR: Unable to update the Defect I did find another question that sounds like it might possibly be related to mine here: App SDK: Erorr parsing input stream when running query Can you provide any assistance?
Pairing Michael's observation regarding the GZIP encoding with that of another astute Rally customer working a Support case on the issue - it appears that some versions of the requests module will default to GZIP compression if the content-type is not specifically defined. The fix is to set content-type to application/json in the REST Headers section of pyral's config.py: RALLY_REST_HEADERS = \ { 'X-RallyIntegrationName' : 'Python toolkit for Rally REST API', 'X-RallyIntegrationVendor' : 'Rally Software Development', 'X-RallyIntegrationVersion' : '%s.%s.%s' % __version__, 'X-RallyIntegrationLibrary' : 'pyral-%s.%s.%s' % __version__, 'X-RallyIntegrationPlatform' : 'Python %s' % platform.python_version(), 'X-RallyIntegrationOS' : platform.platform(), 'User-Agent' : 'Pyral Rally WebServices Agent', 'Content-Type' : 'application/json', }
What you are seeing is probably not related to the Python 2.7.3 / requests 0.13.3 versions being used. The error message you saw has also been reported using the Javascript based App SDK and .NET Toolkit for Rally (2 separate reports here on SO) and at least one other person using Python 2.6.6 and requests 0.9.2. It appears that the error verbiage is being generated on the Rally WSAPI back-end. Current assessment by fellow Rally'ers is that it is an encoding related issue. The question is where the encoding issue originates. I have yet to be able to repro this issue, having tried with several versions of Python (2.6.x and 2.7.x), several versions of requests and on Linux, MacOS and Win7. As you seem to be pretty comfortable with diving in to the code and running in debug mode, one avenue to try is to capture the defective POST URL and POST data and attempting the update via a browser based REST client like 'Simple REST Client' or Poster and observing if you get the same error message in the WSAPI response.
I'm seeing similar behavior with pyral while trying to add an attachment to a defect. With debugging and logging on I see this request on stdout: 2012-07-20T15:11:24.855212 PUT https://rally1.rallydev.com/slm/webservice/1.30/attachmentcontent/create.js?workspace=workspace/123456789 Then the json in the logfile: 2012-07-20 15:11:24.854 PUT attachmentcontent/create.js?workspace=workspace/123456789 {"AttachmentContent": {"Content": "iVBORw0KGgoAAAANSUhEUgAABBQAAAJrCAIAAADf2VflAAAXOWlDQ... Then this in the logfile (after a bit of fighting with restapi.py to get around the unicode error): 2012-07-20 15:11:25.260 404 Cannot parse input stream due to I/O error as JSON document: Parse error: expected '{' but saw '?' [ chars read = >>>?<<< ] The notable thing there is the 404 error code. Also, the "Cannot parse input stream..." error message is not coming from pyral, it's coming from Rally's server. So pyral is sending Rally something Rally can't understand. I also logged the response headers, which may be a clue: {'rallyrequestid': 'qs-app-03ml3akfhdpjk7c430otjv50ak.qs-app-0387404259', 'content-encoding': 'gzip', 'transfer-encoding': 'chunked', 'expires': 'Fri, 20 Jul 2012 19:18:35 GMT', 'vary': 'Accept-Encoding', 'cache-control': 'no-cache,no-store,max-age=0,must-revalidate', 'date': 'Fri, 20 Jul 2012 19:18:36 GMT', 'p3p': 'CP="NON DSP COR CURa PSAa PSDa OUR NOR BUS PUR COM NAV STA"', 'content-type': 'text/javascript; charset=utf-8'} Note there the 'content-encoding': 'gzip'. I suspect the requests module (I'm using 0.13.3 in Macos Python 2.6) is gzip encoding its PUT request but the Rally API server is not properly decoding that.