Wowza Transcoder Add-on throws ArrayIndexOutOfBoundsException - wowza

I'm running Wowza 3.6.2 on Windows 8.1 (64 bit) and have enabled the Transcoder add-on. I'm using the transcoder to take JPEG-snapshots from the live stream. I've built a custom HTTPProvider, similar to what is described here. This works fine and I can get JPG-snapshots from the stream through my HTTPProvider.
The problem is that since I enabled the transcoder I get irritating error messages in my server log on onPublish and onUnPublish of any stream.
As a transcoder template I used the default transrate.xml that comes with the Wowza installation, without any modifications made to it.
When i publish to a stream asdf I get errors similar to this:
ERROR server comment - TranscoderSessionDestination.init[livereceiver/_definst_/asdf]: [asdf_160p]:java.lang.ArrayIndexOutOfBoundsException: 1
java.lang.ArrayIndexOutOfBoundsException: 1
at com.foo.wms.module.IncomingStreamEventHandler.getQueryStringMap(IncomingStreamEventHandler.java:191)
at com.foo.wms.module.IncomingStreamEventHandler.onPublish(IncomingStreamEventHandler.java:83)
at com.wowza.wms.stream.MediaStream.notifyActionPublish(Unknown Source)
at com.wowza.wms.stream.publish.Publisher.publish(Unknown Source)
at com.wowza.wms.stream.publish.Publisher.publish(Unknown Source)
at com.wowza.wms.transcoder.model.TranscoderSessionDestination.init(Unknown Source)
at com.wowza.wms.transcoder.model.TranscoderSession.a(Unknown Source)
at com.wowza.wms.transcoder.model.TranscoderSession.handleOnMetadata(Unknown Source)
at com.wowza.wms.transcoder.model.LiveStreamTranscoder.handleOnMetadata(Unknown Source)
at com.wowza.wms.stream.live.LiveStreamTranscoderRunner.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
And when I unpublish the stream I get this:
ERROR server comment - TranscoderSessionDestination.shutdown: [asdf_160p]:java.lang.NullPointerException
java.lang.NullPointerException
at com.foo.wms.module.IncomingStreamEventHandler.onUnPublish(IncomingStreamEventHandler.java:166)
at com.wowza.wms.stream.MediaStream.notifyActionUnPublish(Unknown Source)
at com.wowza.wms.stream.publish.Publisher.publish(Unknown Source)
at com.wowza.wms.stream.publish.Publisher.unpublish(Unknown Source)
at com.wowza.wms.transcoder.model.TranscoderSessionDestination.shutdown(Unknown Source)
at com.wowza.wms.transcoder.model.TranscoderSession.c(Unknown Source)
at com.wowza.wms.transcoder.model.TranscoderSession.shutdown(Unknown Source)
at com.wowza.wms.transcoder.model.LiveStreamTranscoder.shutdown(Unknown Source)
at com.wowza.wms.stream.live.LiveStreamTranscoderRunner.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
I get three of each exception when I publish/unpublish (one for each Encode block that is enabled in the transrate.xml file).
Does anyone have an idea on what might be causing this?

Here is a better way to determine if the stream is a transcoded stream and not a source stream
if(stream.isTranscodeResult()) return;

After posting my question I had another look at the stacktrace and realized what the problem was - I had been looking in the wrong direction the whole time. Since the problem appeared when I enabled the transcoder, I concluded that's where the problem must be. What I didn't realize was that onPublish and onUnPublish fires multiple times when you use the transcoder - one time for the incoming stream, and one time for every transcoded stream.
Within the onPublish and onUnPublish methods of my module I do stuff, like read querystring-paramters, which are not present on the transcoded streams. That is why the exceptions are thrown when the onPublish and onUnPublish methods are called for the transcoded streams.
To remedy this, I added two lines at the beginning of onPublish and onUnPublish methods.
if (streamName.contains("_"))
return;
A somewhat ugly solution, but I am in control of all stream names and do not allow underscore in them anyway, so in my case this works fine.
Update:
#flux has provided a much nicer solution for how to check if the stream is the result of a transcode operation. See his answer for more info.

Related

How to find the cause of a broken wso2mi-dashboard?

Good day everyone.
We have wso2 implemented on three Docker containers. Everything worked fine until after adding the next flow, the entrance to the Micro Integration Dashboard stopped working. Moreover, the system itself continues to work, only you can not enter the panel.
Error image
login failed internal server error
No changes were made to the users or to the container itself.
I've searched and read everything I can, but I can't even grasp where the problem is.
Please push me in the right direction.
Here is the error from the logs:
[2023-01-12 07:40:08,602] ERROR {LoginDelegate} - Error logging into dashboard server java.lang.IndexOutOfBoundsException: Index 0 out of bounds for length 0
at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64)
at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70)
at java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:248)
at java.base/java.util.Objects.checkIndex(Objects.java:372)
at java.base/java.util.ArrayList.get(ArrayList.java:459)
at org.wso2.ei.dashboard.core.rest.delegates.auth.LoginDelegate.getTokenFromMI(LoginDelegate.java:80)
at org.wso2.ei.dashboard.core.rest.delegates.auth.LoginDelegate.authenticateUser(LoginDelegate.java:51)
at org.wso2.ei.dashboard.core.rest.api.LoginApi.receiveLogin(LoginApi.java:56)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191)
at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104)
at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268)
at org.glassfish.jersey.internal.Errors.process(Errors.java:316)
at org.glassfish.jersey.internal.Errors.process(Errors.java:298)
at org.glassfish.jersey.internal.Errors.process(Errors.java:268)
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289)
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256)
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703)
at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416)
at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229)
at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1450)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:550)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410)
at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
at java.base/java.lang.Thread.run(Thread.java:829)

How to solve stability problems in Google Dataflow

I have a Dataflow job that has been running stable for several months.
The last 3 days or so, I've problems with the job, it's getting stuck after a certain amount of time and the only thing I can do is stop the job and start a new one. This happened after 2, 6 and 24 hours of processing. Here is the latest exception:
java.lang.ExceptionInInitializerError
at org.apache.beam.runners.dataflow.worker.options.StreamingDataflowWorkerOptions$WindmillServerStubFactory.create (StreamingDataflowWorkerOptions.java:183)
at org.apache.beam.runners.dataflow.worker.options.StreamingDataflowWorkerOptions$WindmillServerStubFactory.create (StreamingDataflowWorkerOptions.java:169)
at org.apache.beam.sdk.options.ProxyInvocationHandler.returnDefaultHelper (ProxyInvocationHandler.java:592)
at org.apache.beam.sdk.options.ProxyInvocationHandler.getDefault (ProxyInvocationHandler.java:533)
at org.apache.beam.sdk.options.ProxyInvocationHandler.invoke (ProxyInvocationHandler.java:158)
at com.sun.proxy.$Proxy54.getWindmillServerStub (Unknown Source)
at org.apache.beam.runners.dataflow.worker.StreamingDataflowWorker.<init> (StreamingDataflowWorker.java:677)
at org.apache.beam.runners.dataflow.worker.StreamingDataflowWorker.fromDataflowWorkerHarnessOptions (StreamingDataflowWorker.java:562)
at org.apache.beam.runners.dataflow.worker.StreamingDataflowWorker.main (StreamingDataflowWorker.java:274)
Caused by: java.lang.RuntimeException: Loading windmill_service failed:
at org.apache.beam.runners.dataflow.worker.windmill.WindmillServer.<clinit> (WindmillServer.java:42)
Caused by: java.io.IOException: No space left on device
at sun.nio.ch.FileDispatcherImpl.write0 (Native Method)
at sun.nio.ch.FileDispatcherImpl.write (FileDispatcherImpl.java:60)
at sun.nio.ch.IOUtil.writeFromNativeBuffer (IOUtil.java:93)
at sun.nio.ch.IOUtil.write (IOUtil.java:65)
at sun.nio.ch.FileChannelImpl.write (FileChannelImpl.java:211)
at java.nio.channels.Channels.writeFullyImpl (Channels.java:78)
at java.nio.channels.Channels.writeFully (Channels.java:101)
at java.nio.channels.Channels.access$000 (Channels.java:61)
at java.nio.channels.Channels$1.write (Channels.java:174)
at java.nio.file.Files.copy (Files.java:2909)
at java.nio.file.Files.copy (Files.java:3027)
at org.apache.beam.runners.dataflow.worker.windmill.WindmillServer.<clinit> (WindmillServer.java:39)
Seems like there is no space left on a device, but shouldn't this be managed by Google? Or is this an error in my job somehow?
UPDATE:
The workflow is as follows:
Reading mass data from PubSub (up to 1500/s)
Filter some messages
Keeping session window on key and grouping by it
Sort the data and do calculations
Output the data to another PubSub
You can increase the storage capacity in the parameter of your pipelise. Look at this one diskSizeGb in this page
In addition, more you keep data in memory, more you need memory. It's the case for the windows, if you never close them, or if you allow late data for too long time, you need a lot of memory to keep all these data up.
Tune either your pipeline, or your machine type. Or both!

Corda - problem while executing flow with multiple output states

I'm trying to execute a corda flow with 3000 output states (Java) but I got the error:
[Thread-8 (ActiveMQ-IO-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$4#6a8da5c5)] impl.JournalImpl.run - appendAddRecord::java.lang.IllegalArgumentException: Record is too large to store 18603342 {}
java.lang.IllegalArgumentException: Record is too large to store 18603342
at org.apache.activemq.artemis.core.journal.impl.JournalImpl.switchFileIfNecessary(JournalImpl.java:2915) ~[artemis-journal-2.2.0.jar:2.2.0]
at org.apache.activemq.artemis.core.journal.impl.JournalImpl.appendRecord(JournalImpl.java:2640) ~[artemis-journal-2.2.0.jar:2.2.0]
at org.apache.activemq.artemis.core.journal.impl.JournalImpl.access$200(JournalImpl.java:88) ~[artemis-journal-2.2.0.jar:2.2.0]
at org.apache.activemq.artemis.core.journal.impl.JournalImpl$1.run(JournalImpl.java:778) [artemis-journal-2.2.0.jar:2.2.0]
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42) [artemis-commons-2.2.0.jar:2.2.0]
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) [artemis-commons-2.2.0.jar:2.2.0]
at org.apache.activemq.artemis.utils.actors.ProcessorBase$ExecutorTask.run(ProcessorBase.java:53) [artemis-commons-2.2.0.jar:2.2.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:1.8.0_181]
at java.lang.Thread.run(Unknown Source) [?:1.8.0_181]
To avoid this problem I divided the execution of the flow into more steps and call it n times (in this case 6) processing 500 output states in every execution.
This solution works, but there is a better/efficient solution to solve this problem?
Thank you in advance.
This error indicates that a message you are trying to send exceeds the network's max message size.
As of Corda 3.x, this max message size is hardcoded to 10MB (10,485,760 bytes).
In a future version of Corda, the network operator will be able to configure the max message size for the network as part of the network parameters.
The purpose of setting a max message size is to prevent large nodes from bullying smaller nodes by forcing them to process excessively large messages.

com.ctc.wstx.exc.WstxParsingException: Text size limit

I am sending a big attachment to a CXF webservice and I get the following exception:
Caused by: javax.xml.bind.UnmarshalException
- with linked exception:
[com.ctc.wstx.exc.WstxParsingException: Text size limit (134217728) exceeded
at [row,col {unknown-source}]: [1,134855131]]
at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.handleStreamException(UnmarshallerImpl.java:426)
at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:362)
at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:339)
at org.apache.cxf.jaxb.JAXBEncoderDecoder.doUnmarshal(JAXBEncoderDecoder.java:769)
at org.apache.cxf.jaxb.JAXBEncoderDecoder.access$100(JAXBEncoderDecoder.java:94)
at org.apache.cxf.jaxb.JAXBEncoderDecoder$1.run(JAXBEncoderDecoder.java:797)
at java.security.AccessController.doPrivileged(Native Method)
at org.apache.cxf.jaxb.JAXBEncoderDecoder.unmarshall(JAXBEncoderDecoder.java:795)
... 25 more
The issue seems to come from the Woodstox library that says
Text size limit (134217728) exceeded
Does someone know if it is possible to increase this limit? if yes, how to do?
If it's coming from woodstox like that, then you aren't sending it as an MTOM attachment. My first suggestion would be to flip it to MTOM so it can be handled outside the XML parsing. Much more efficient as we can keep it as an inputstream or similar and not have it in memory.
If you want to keep it in the XML, you can set the property: "org.apache.cxf.stax.maxTextLength" to some larger value. Keep in mind, stuff coming in from the stax parser like this are held in memory as either a String or byte[] and will thus consume memory.

Running NiTE and OpenNI on Java - Handviewer Java Sample Crashes

I am trying to get the Handviewer java sample project running. When running the application crashes after (1-2 sec) and firing the following exceptions!
Please note that I have managed to run the UserViewers.java sample without problems. and I am running on OSX 10.8.3, NiTE-MacOSX-x64-2.2, OpenNI-MacOSX-x64-2.2, and Asus XtionPro Live.
Exception in thread "Thread-17" java.util.NoSuchElementException: Unknown pixel format: 0
at org.openni.PixelFormat.fromNative(PixelFormat.java:30)
at org.openni.VideoMode.<init>(VideoMode.java:38)
at com.primesense.nite.NativeMethods.niteReadHandTrackerFrame(Native Method)
at com.primesense.nite.HandTracker.readFrame(HandTracker.java:139)
at com.primesense.nite.Samples.HandViewer.HandViewer.onNewFrame(HandViewer.java:69)
at com.primesense.nite.HandTracker.onFrameReady(HandTracker.java:360)
Exception in thread "AWT-EventQueue-0" java.lang.NullPointerException
at com.primesense.nite.Samples.HandViewer.HandViewer.paint(HandViewer.java:55)
at javax.swing.JComponent.paintChildren(JComponent.java:884)
at javax.swing.JComponent.paint(JComponent.java:1046)
at javax.swing.JComponent._paintImmediately(JComponent.java:5106)
at javax.swing.JComponent.paintImmediately(JComponent.java:4890)
at javax.swing.RepaintManager$3.run(RepaintManager.java:814)
at javax.swing.RepaintManager$3.run(RepaintManager.java:802)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.AccessControlContext$1.doIntersectionPrivilege(AccessControlContext.java:87)
at javax.swing.RepaintManager.paintDirtyRegions(RepaintManager.java:802)
at javax.swing.RepaintManager.paintDirtyRegions(RepaintManager.java:745)
at javax.swing.RepaintManager.prePaintDirtyRegions(RepaintManager.java:725)
at javax.swing.RepaintManager.access$1000(RepaintManager.java:46)
at javax.swing.RepaintManager$ProcessingRunnable.run(RepaintManager.java:1684)
at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:209)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:708)
at java.awt.EventQueue.access$400(EventQueue.java:82)
at java.awt.EventQueue$2.run(EventQueue.java:669)
at java.awt.EventQueue$2.run(EventQueue.java:667)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.AccessControlContext$1.doIntersectionPrivilege(AccessControlContext.java:87)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:678)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:296)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:211)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:201)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:196)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:188)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:122)
This probably won't help you much because it's already been a while... but I have had a similar problem with my Kinect. I could resolve everything by reinstalling everything and switching to OpenNI 1.5.4. Everything worked after that. Good luck.