I'm looking for a solution to play HD videos on a multimonitor OSX environment for a projector/desktop application. It could be one huge video, or a video split in parts.
So far I've been using Flash StageVideo successfully to play 1080p and 720p on single monitors. This works great with flash projectors. The problem with flash projectors is you can't span multiple monitors, or multiple windows. I still haven't tried opening multiple projectors, because I wouldn't know how to position each projector in a different monitor consistently.
In Adobe AIR you can have multiple windows and control their position, but AFAIK you can't use StageVideo to decode videos with the GPU... and using the classic Video class is really out of the question.
With C++ there are multiple frameworks (cinder/openFrameworks) but AFAIK opening multiple windows, or spaning multiple monitors is not such a good idea because of bad performance. I stil haven't figured out if it's possible or even a good idea to open one app per monitor and control it's position.
Has anyone succeeded in this problem using AS3 or any other language/framework like C++ with openFrameworks?
With AIR you can have a single window span multiple windows.
I have a 1920x1080 video scaled by 2 on stage of 3840x2160. I haven't yet tried using StageVideo to up the resolution, but I am hopeful it will work.
Related
Problem I need help with
Assist with optimizing latency in its "Short-Video Feed" and solving intermittent performance bugs. A central feature of my app requires seamless plays 15 to 60 second clips when users use "swipe up" hand gestures similarly to TikTok and Instagram reels. Right now, I have performance bugs (intermittent) such as black screens, delayed loading screens, sometimes long loading, etc.
The bugs may be caused because Flutter is slower than Native iOS. However, our "Short-Video Feed" has lots of bugs whether I use a M3U8 (Mux), or whether I use a MP4 based approach with AWS S3.
If I use the Mux based approach with M3U8, "Short-Video Feed" there is a noticeable few milliseconds black screen for each short-video playback.
If I use the Amazon based approach with MP4, "Short-Video Feed" intermittently loads for a few seconds (sometimes minutes) when there is low bandwidth, and some videos get stuck even when a user returns to a location with faster bandwidth.
Open issue on Flutter
https://github.com/flutter/flutter/issues/25558
Approaches I have tried with no success:
Native Player. I tried to use a native video player for Android/iOS, with MP4 and M3U8, but the UI was still very laggy (because of data transfer between Android/iOS and flutter latenices).
Flutter Player. I tried to use a Flutter video player for Android/iOS, with MP4 and M3U8, but the UI shows a black screen with M3U8 & heavy loading for poor internet connections with MP4.
Approaches I need help to try:
Optimize M3U8 player to minimize the black screen issue. or...
Create MP4 chunks to optimize for poor reception areas (this is what I think TikTok Instagram Reels, and similar applications do based on what I can see).
Has anyone solved this issue?
How about isolating whether these lags are due to network buffering or due to Flutter (or even a device hardware limitation such as memory or GPU)?
Perhaps use a few local MP4 files with identical frame rates and encoding parameters (both video and audio) and see whether the UI lag is reproducible upon swipe-up scrolling?
I have a project that uses libmpv with the opengl widget (as per the examples that come with libmpv) along with a QtWebEngine widget that displays information, graphics and animations (a scrolling ticker for example).
I found that of the video playback options in Qt, mpv was the smoothest and most reliable. It will play back perfectly smoothly any video up to 1080p.
However while video is playing, any animations in QtWebEngine are unsmooth and jittery. Video is also slightly less smooth when something is moving in the webpage.
The system I'm testing with is not struggling for resources (cpu use is around 45%). There is also not any video decoding, as it's playing back raw video (but while it plays back encoded video, the effect is the same, regardless if hardware acceleration is enabled or not).
I figure that the mpv widget is interrupting the MainWindow thread while it processes frames and causing it to freeze every few milliseconds.
As far as I know there is no way to separate the mpv thread from the MainWindow thread though.
I don't know if it'll be possible to make mpv and webengine work together smoothly. I feel like there must be some way to run two widgets at once in one window and not have them mess with each other.
I'm testing with Ubuntu 18.04, QT 5.11 and the latest mpv from git.
Does anyone have any advice or pointers for what to try first? I realise this is somewhat of a broad question but my knowledge of graphics is limited. If anyone has advice on a conceptual level (I don't need someone to code me a fix) I can investigate myself.
Thank you.
So I am trying to figure out how get a video feed (or screenshot feed if I must) of the Desktop using OpenGL in Windows and display that in a 3D environment. I plan to integrate this with ARToolkit to make essentially a virtual screen. The only issue is that I have tried manually getting the pixels in OpenGl, but I have been unable to properly display them in a 3D environment?
I apologize in advance that I do not have minimum runnable code, but due to all the dependencies and whatnot trying to get an ARToolkit code running would be far from minimal. How would I capture the desktop on Windows and display it in ARToolkit?
BONUS: If you can grab each desktop from the 'virtual' desktops in Windows 10, that would be an excellent bonus!
Alternative: If you know another AR library that renders differently, or allows me to achieve the same effect, I would be grateful.
There are 2 different problems here:
a) Make an augmentation that plays video
b) Stream the desktop to somewhere else
For playing video on an augmentation you basically need to have a texture that gets updated on each frame. I recall that ARToolkit for Unity has an example that plays video.However.
Streaming the desktop to the other device is a problem of its own. There are tools that do screen recording, but you probably don't want that.
It sounds to me that what you want to do it to make a VLC viewer and put that into an augmentation. If I am correct, I suggest you to start by looking at existing open source VLC viewers.
I'm using ofVideoPlayer and for some reason there is some flicker/tear the moment of the transitions between the videos. I've tried to change the video format, i added ofSetVerticalSync(true);and it stills gets that error.
How can i fix this?
I had a lot of struggle with Openframeworks and Tearing.
Basically, it depends a lot on your Hardware.
You may install drivers to force the graphic cards vsync.
It is more true if you run OF with Linux OS.
If you are on windows certainly look into the graphics cards settings panel and experiment with different settings for vertical sync.
If you are on on OSX there are some other video players you might want to test.
http://forum.openframeworks.cc/t/ofxavfvideoplayer/12770
Recently I have had the best results with the HAP codec and the hap player. It loads super quick and supports alpha channels. I would definitely try that.
https://github.com/bangnoise/ofxHapPlayer
What I need to do is create a program that overlays the whole screen and every 30 seconds the screen needs to flash black once.
the program just needs to be on top of everything, doesn't have to work over the top of games, but wouldn't say no if it did!
But i've got no idea where to start. Ideally the solution would be cross-platform for both windows and osx.
Does anybody have any ideas about where I should start or could whip up a quick demo?
OpenGL (you tagged it as such) will not help you with this.
Create a program, that overlays the whole screen,
The canonical way to do this is by creating a decorationless, borderless top level window with some stay-on-top property being set.
and every 30 seconds the screen needs to flash black once.
How do you define "flash back once"? You mean you want the display become visible for one single vertical retrace period or a given amount of time? Being the electronics tinkerer I am, honestly, I'd do this using a handfull of transistors, resistors and capacitors, blanking the analog VGA signal.
Anyway, if you want to do this using software, this is going to be hard work. If you'd do this using the aforementioned stay-on-top window, when you "flash" it away, all the programs with visible output would receive redraw events, which to process would take some time. In the best case scenario the system uses a compositing window manager which can practically immediately show the desktop. Without a compositor its going to be impossible to "flash" the screen.
Ideally the solution would be cross-platform for both windows and osx
A task like this can not be solved cross plattform. There's too much OS dependent work to do for this.
I presume this is for some kind of nerological or psychological experiment. I think doing this using some VGA intercepting circurity would be actually the easier, quicker to implement solution. I can help you with that. But I think there's another StackExchange better suited for this. Unfortunately digital display interfaces (DVI, HDMI and Display Port) use a complex line code scheme, which can not be blanked as easily as VGA, so you must have a computer capable of analog (=VGA) output and a display with a VGA input.