I have a pet project with webrtc audio-video calls. Currently calls from browser are working exactly as expected.
And I'm trying C++/Qt client based on Native C++ WebRTC (it is just a modified peer_connection_client example with modified signalling).
It works, but I've found an issue: when I make a call from one PC to another, the quality of the video is extremely reducing until bandwidth became around 250-300kbps (or 500kbps total, not sure).
As I told, there is a web version working on same signalling and I tested it out on the same PCs. The result was really surprised for me: no quality reducing and around 2000kbps network load.
Moreover, if I connect PC with cable to my router, the quality is fine and 2000kbps load, as expected.
I suppose that the problem is somewhere around wifi bandwidth estimator or so, but can't realize how can I control it's behavior.
Have someone any ideas how can I improve the quality and make webrtc use 2000kbps not 500kbps?
Thanks in advance,
Br,
Sergey
While going forward, I've found out that I have to call rtc::Thread::ProcessMessages() eventually to prevent being stuck on signalling_threads events.
But after that a new problem occured. It is "UDP send of XXX bytes failed with error 10035" this is described at https://groups.google.com/forum/#!topic/discuss-webrtc/wmYo7AU3evI.
Related
For my work, I am currently working on a existing system (C++) that communicates with a beckhoff PLC. This PLC reads the current speed of the process and controls a light.
This program needs an update to work with the new thickness measurement. The code for this new measurements works, we tested it on another computer.
The problem is that the communication between the beckhoff and the computer is not working anymore. This problem starts when we rebuild (a part of) the program. To check what went wrong, we used wireshark to check the communication.
In the figure below, you can see the correct communication with the Beckhoff PLC (with Ethercat).
In this figure, you can see the communication after we build the new program.
Of course, we tested a lot of option to elimate this problem but at the moment we cannot think of any other method to get the communication back.
Things we tried:
Compile the old code to see if the communication still works. This doesn't work. Only the originale .exe file works.
Go a few version back to test the communication. This also doesn't work.
We check the hardware configuration and the mac adress of the Beckhoff PLC. This is all correct.
It would be great if someone knows a solution for this problem. Please, don't hesitate to ask question for more information about the problem.
I use C++ (Visual Studio 2015) and OpenCV (ver 3.2.0) to process data sent from Kinect v1. My C++ program has no problem when it starts debugging for the first time. After it stops debugging and re-start debugging, however, it gets very slow.
I am suspecting that the program closes without releasing some memory (i.e., memory leak). I am aware of that I would need to use the delete function to release the memory if I use the new function. But I didn't use the new function in the C++ program (I neither used the malloc() function, which is equivalent to the new function in C programs).
For OpenCV, I use the destroyAllWindows function at the end of the program. For Kinect v1, I also use the NuiShutdown(), Release(), and CloseHandle() functions at the end of the program.
Is there anything else I need do to release the memory (e.g., releasing memory associated with Mat in OpenCV)? Or is something else causing the decrease in processing speed?
I'd appreciate your help. Thanks.
After first run disconnect Kinect then reconnect and try second run.
If all goes well now then the problem is most likely stuck thread. The device access is usually handled by separate threads and especially with USB they can get stuck (in case of error or sync problem between accessing form host and expecting on device side) until you disconnect device (not sure which Kinect driver are you using but JUNGO version which NuiShutdown() infers have this problem). You can also check task manager before disconnection if there are not some stuck processes left after first run.
To remedy this you need to find out what are you doing wrong during access. It could be:
wrong USB port
use the back side not front slots.
invalid USB transfer request
device is always waiting for specific set of commands or stream and waits until it does not receive it so it blocks all other things. So using unsupported commands or reading in wrong times or sizes of packets can cause this.
USB communication is out of sync
PC host can timeout in case you do not have enough CPU power while critical operation is processed (or have opened too many apps on background).
This can be caused also by wrong gfx driver as I suspect you are using rendering ... Intel HD graphics can generate such problems with ease especially on notebooks. Try to disable any rendering in your app or at least limit rendering to OpenGL 1.0 to see if speed is the same in between runs. If this is the case the whole desktop usually flickers or is not repainting parts of apps ... and animations are sometimes sluggish.
Another problem might be a debugger. If without it all is well then debugger is the problem and you can not solve it. Debugging while accessing IO can cause sync and timeout problems especially with USB.
To check for memory leaks you can simply see how much free memory you got before 1st run and compare it to values after 1st,2nd,3th .. runs if the value lowers you got something stuck somewhere. After app close all the memory belonging to app is freed by OS so even if you forget some delete that does not matter unless some thread is still running ...
Some USB drivers based on libUSB I encountered got also problem with Handle leaks. But that behaves differently ... all runs fine until there are no free handles. After that OS is non functional you can not open any window,app, anything ... until any app is closed.
[Edit1] Front USB slots
Front slots are usually connected to motherboard with relatively long cable (usually flat and not very well shielded) so it is more susceptible to noise. Also as it is located usually around HDD and above high frequency parts of the motherboard it also induce it into the USB feed. All this degrades the quality of USB signal causing much much bigger rejection rate hence lowering sync capability and also the overall usable bandwidth.
If you compare that with backside USB ports they have no cables but are connected directly in PCB with short and well shielded paths so the connection quality is much much better.
So if you use device demanding high bandwith or synchronism then front ports are a bad choice.
I'm trying to understand the userland part of the Raspberry Pi graphics driver code from https://github.com/raspberrypi/userland
My understanding so far is:
- a firmware blob runs in the GPU and offers an OpenGL-like interface which, on lower levels, is based on message (byte-array) passing on top of one of multiple 28-bit-word FIFOs called VCHIQ (the other VCHIQ queues are irrelevant for graphics)
- on the CPU part, OpenGL calls are turned into messages to the GPU. Access to the low-level facility (either the message queue or VCHIQ -- I haven't found that part yet in the code) requires a Linux kernel module, but no high-level logic happens in there.
- the GPU part is closed, but that's okay for my purposes. The (ARM) CPU part is, AFAIK, open
My ultimate goal is to get communication with the GPU working on bare metal (without Linux), but with the closed firmware blob intact. As a first goal, I want to understand how an OpenGL call is actually passed to the GPU. Anything beyond that is not part of this question.
However, I'm stuck at finding the actual code for this. The OpenGL calls use RPC_CALL* and in turn RPC_DO, which calls khronos_server_lock_func_table(). However, that function seems to be missing from the code, and to my surprise, I couldn't find anything useful about it on Google.
My questions:
- am I still on the ARM CPU side, or did I move to GPU land without noticing? If the latter is the case, where did I cross that line?
- Assuming I'm still on the CPU side -- where is the code for that function? Is it open at all, or do we actually have closed parts left around on the CPU side here? All sources on the web seem to indicate that the code for the CPU is 100% open.
- at which point does the implementation of the C OpenGL functions actually send a message to the GPU? I'm somewhat expecting a call to the kernel functionality that represents VCHIQ to be happening at some point, probably implemented as a device file.
I don't fully understand how do you intend to access the GPU without using Linux, and I am not that familiar with the technicalities, but some time ago I've been digging into the GPU for my private project so I'll tell you what I know.
The GPU is VideoCore IV and its documentation is available on Broadcom's website.
Also, on the Raspberry Pi Wiki you can see on the picture on the left that VCHIQ is in the kernel driver, so you might look for the implementation details in the kernel's source code.
Maybe this might be of some help too: VideoCore IV Programmer's Manual. About the document:
This is a independent documentation project based on a combination of static analysis and trial and error on real hardware. This work is 100% independent from and not sanctioned by or connected with Broadcom or its agents. No Broadcom documents or materials were used beyond those publicly available.
As for the software itself, The Khronos Group provides OpenGL ES and OpenVG implementation, but it's not open source. You can get the documentation from their website, but I doubt you'll find anything on such low level.
Hope it helps.
I want to try some C and C++ programming with audio processing, such as synthesizers, chorus, delay etc, but I only know working with a console as output. I wish to have, instead of a console application, a window that would be capable of sending an audio signal to the speakers, running the code at the background and working with it in a similar way as it goes with printf: every time I would call the "output function", it would send to the speakers (or sound card) a sample value, indicating current oscilator position. This output operation could be executed every time it is requested or in the end of a built in loop. Doing all this with a high sample rate would be just great.
I think I could do all this using AudioWorker on Web Audio API, plus a flexible GUI on HTML5 canvas, but I'm new at this API and I'm not sure whether its resulting sound quality is good enough.
Thanks in advance.
Edit: I use Windows 8.1, but any answer for other platform is welcome.
Edit2: Any programming languages other than C, C++ or JavaScript suggestions are also welcome.
I do lots of sound synthesis with the Web Audio API and I think it sounds great. Javascript is really all you need. Well, if you want to use audio files, you need a web server to serve those audio files, but the audio synthesis all happens in javascript.
It doesn't matter so much what OS you use, but different browsers have different levels of support for the web audio API. Chrome tends to have the best support, and Internet Explorer definitely has the worst support.
In my application we will present the video stream from a traffic camera to a client viewer. (And eventually several client viewers.) The client should have the ability to watch the live video or rewind the video and watch earlier footage including video that occurred prior to connecting with the video stream. We intend to use wxWidgets to view the video and within that we will probably use the wxMediaCtrl.
Now, from the above statements some of you might be thinking "Hey, he doesn't know what he's talking about." And you would be correct! I'm new to these concepts and I confused by the surplus of information. Are the statements above reasonable? Can anyone recommend a basic server/client architecture for this? We will definitely be using C++ wxWidgets for the GUI, but perhaps wxMediaCtrl is not what I want... should I be directly using something like the ffmpeg libraries?
Our current method seems less than optimal. The server extracts a bitmap from each video frame and then waits for the single client to send a "next frame" message, at which point the server sends the bitmap. Effectively we've recreated our own awkward, non-standard, inefficient, and low-functionality video streaming protocol and viewer. There has to be something better!
You should check out this C++ RTMP Server: http://www.rtmpd.com/. I quickly downloaded, compiled and successfully tested it without any real problems (on Ubuntu Maverick). The documentation is pretty good if a little all over the place. I suspect that once you have a streaming media server capable of supporting the typical protocols (which rtmpd seems to do), then writing a client should fall into place naturally, especially if you're using wxWidgets as the interface api. Of course, it's easy to write that here, from the comfort of my living room, it'll be a different story when you're knee deep in code :)
you can modify your software such that:
The server connect, server grabs an image, passes it to ffmpeg establishing stream, then copy the encoded data from ffmpeg stream and send to client via network, if the connection drops, close the ffmpeg stream.
Maybe you can use the following to your own advantage:
http://www.kirsle.net/blog.html?u=kirsle&id=63
There is a player called VLC. It has a library for c++ and you can use it to embed the player in your GUI application. It supports a very wide range of protocols. So you should leave connecting, retrieving and playing jobs to VLC and take care of the starting and stopping jobs only. It would be easy and probably a better solution than doing it yourself.
For media playing facility, both music and audio, you can a look on GStream. And talking about the server, I think Twisted (A network library in Python) should be good option. The famous live video social website justin.tv is based on Twisted. Here you can read the story from here. Also, I built a group of server for streaming audio on Twisted, too. They can serve thousands of listener on line in same time.