I've done some research on this and have already tried using IP WebCam and gstreamer to pipe the video stream from the IP based source to /dev/videoX and reading from that.
It works fine, but it isn't exactly what I am looking for. I have several old useless phones that I would like to use on a robot as video sources from multiple angles for computer vision based tasks.
I've also read that DroidCam does this, but I have read terrible reviews about the app and especially the USB based part.
Most likely this robot will not be near any WiFi (or rather should not require it), and the phones will not have a network connection (no SIM card), so no USB tethering.
It seems like there should be a very simple way to do this. Is there a way to uselibusb and maybe adb to capture the video stream from an Android phone connected to an Ubuntu box with USB?
Related
In my Android app, I use Android NDK to play music by doing the following:
extract audio samples from an OGG file using the Vorbis library
process the audio samples
redirect the processed samples to the audio output using the Oboe library
In order to avoid underruns, I do the first 2 steps in a separate thread, to extract and process the sound a little bit in advance (it adds a bit of latency, but this is not a problem for my app). That solution works great on every device I've tested so far.
But for some reason, when I pair my device with a bluetooth speaker and when I play music, I have what seems to be underruns on some devices like Samsung S7 or Nokia 1 (but not on every device).
This bug looks so random to me that I don't know where to start. It acts like the bluetooth connection is using quite a lot of CPU so my app doesn't have enough resource to run properly.
Does anyone have experienced something similar? Should I do anything in my code to handle the bluetooth connection so it doesn't use CPU (for example to avoid audio resampling)?
Thanks for your help.
Android + Bluetooth audio is a world of pain. The major thing to appreciate about Bluetooth is the audio sink runs at a rate independent of other audio devices, which is why the native mediaplayer will do things like display video in accordance with whatever rate the attached audio device consumes samples, essentially slaving itself to the clock of the BT audio device. If you want to drive the speed from Android (i.e. SystemClock timebase) you'll need to use a timestretching AudioTrack. (This can be done, but driver support is erratic and overall system stability tanks).
Firstly, you want to eliminate the devices themselves being problems. Can you play the ogg files in a media player to a Bluetooth speaker from the S7 or Nokia 1 without problems? If so, it's your code!
It sounds to me like the speaker is consuming samples faster than the device is producing them, for whatever reason. Basically check your callbacks to make sure whenever the audio subsystem requests more data you are actually providing it. Be sure to drive your decoding pipeline according to the callbacks being made and not the system clock or any other assumptions about timing.
Finally, Bluetooth audio, at least A2DP, as opposed to directly streaming MP3, is going to require some processing to recompress the audio as it is sent out, but those devices should have plenty of headroom for this, maybe even special DSPs. I've done it with 1080P video playback at the same time before, and it starts to fall apart with two videos at once!
I am doing a speech processing project with a Raspberry Pi 3 (running Raspbian) using a USB Microphone. I can see the Microphone show up as a selectable audio device for the Pi and it produces/captures sound perfectly.
I cannot figure out how to use this in my code; I have done a ton of research regarding this and have found some tutorials but nothing that making sense. I come from more of a hardware background and have done something like this with controllers where I hook up an actual Mic and process the Analog Signal into Digital on IO Pins; I am so frustrated with this that I am about to pump data over from an Arduino using a Mic and A2D conversion.
-------------------------------------------------------My questions----------------------------------------------------
1) I want to know how to access a USB data stream or USB device in C or C++. My Linux abilities are not the best. Do I open a Serial Connection or open a filestream in "/dev/USB/...."? Can you provide a code example?
2) Regardless of the fidelity of the USB Mic Input, I want to know how to access its Input in C/C++. I have been looking at ALSA but cannot really understand a lot of its complexity. Is there something that gives me access to a raw input signal on the USB Port that I can process ( where I extrapolate out frequency, amplitude, etc.)?
I have already gone through a lot of the similar posts on here. I am really stuck on this one. I 'm really looking to understand what is going on from the OS perspective; I'll use a library given but I want to understand how it works.
Thanks!
So an update:
So I did all of my code in C with some .sh scripts. I went ahead and figured out how to use the Alsa asoundlib (asound.h specifically). As of now, I am able to generate and record sound via USB Mic/Headset with my Pi 3. Doing so is rather arduous but here is a useful link (1).
For my project, I also found a CMU tutorial/repos for their PocketSphinx Audio Recognition Device at link (2) and video link (3). This project utilizes the Alsa asoundlib as well and was a great help for me. It takes a while to download and you need to crawl through its .sh scripts to figure out its gcc linking. But I am able to now give audio cues which are interpreted by my Pi 3 and pushed to speaker output and GPIO pins.
LINKS(1)http://www.alsa-project.org/alsa-doc/alsa-lib/_2test_2pcm_8c-example.html
(2)https://wolfpaulus.com/embedded/raspberrypi2-sr/
(3)https://www.youtube.com/watch?v=5kp5qpwVh_8
I'm looking into coding a very simple LAN based home monitoring system using python 2.7 on Windows. I have a number of computers around the house each with a usb webcam attached. The cameras have built in microphones. I'm looking for the best way to capture and stream the audio and video over the network, then recieve and view/listen to it. I'm guessing I'd have to use PyAudio to get the audio from the microphone and CV2 to get the video, past that, I'm not sure how I'd stream that data to another computer, recieve it and then view/listen to it.
Not realy a question here, but still I guess I have a solution for you
(but there a millions of ways to solve this.)
My way is ROS (Robot Operating System), which is basically a TCP/IP Service Wrapper. Anyway you can simple broadcast and receive streams via your network.
ROS can be implemented in C++ or Python.
ROS usually applies OpenCV as CV-library, so it should suite your wishes.
Straight forward examples for video are (i.e.) here:
mjpeg_server
web_video_server
Audio streaming is introduced here:
audio_common (there might be solutions via PyAudio. Not sure if other solutions might come more easy and still suitable.)
I have a Windows native desktop app (C++/Delphi), and I'm successfully using Directshow to display live video in it from a 'local' video capture device.
The next thing I want to do is display video from a 'remote' capture device, streamed over the LAN.
To stream the video, I guess I can use something like Expression Encoder or VLC, but I'm not sure what's the easiest way to receive/decode the streamed video. Inserting an ActiveX VLC or Flash player might be one option (although the licensing may be an issue then), but I was wondering if there's any way to achieve this with Directshow...
Application needs to run on XP, and the video decoding should ideally be royalty free.
Suggestions, please!
Using Directshow for receiving and displaying your video can work, but the simplicity, "openness" and performances will depend on the video format and streaming method you'll be using.
A lot of open/free source filters exist for RTSP (e.g. based on live555), but you may also find that creating your own source filter is a better fit.
The best solution won't be the same for H264 diffusion through RTP/RTSP and for MJPEG diffusion through simple UDP.
In our project, we would like to access the webcam image programmatically.
The main problem we have is that the webcam automatically adjusts the sensitivity depending on the brightness of the captured image.
Is there any (platform-independent) way to change this kind of parameters for the webcam (preferably any model)?
We're currently using Ubuntu 10.04, Microsoft Windows XP & 7. Programming language is C/C++.
Any idea is appreciated.
Thanks and regards
Tobias
There most likely won't be any platform independent way to do what you need. If there is, it's probably by using some high level language, which likely won't suit.
I don't know about the linux platform, but I'm a C++/windows/COM/DirectShow developer who works on internet based video applications.
On the Windows platform, capture devices are communicated with via COM and DirectShow.
For a general overview of video capture on windows, see the Video Capture section of MSDN.
Have a look at Selecting a Capture Device for information on how to enumerate the capture devices on your system. You'll need to enumerate the devices in the CLSID_VideoInputDeviceCategory, in order to discover (programmatically) the webcam as a video input device - there may be many devices in this category.
Video capture devices have a "FriendlyName" to help identify your webcam that you can store and retrieve the device for later use.
Once you've got the device, your query said you'd wanted to configure the device. Check out the Configuring a Video Capture Device for this.
DirectShow is one of Microsoft's most comprehensive (and difficult) APIs to learn. The MSDN developer forum on DirectShow is very active and beginner friendly and I highly recommend you check it out.
Finally, capture graphs aren't the easiest thing to build in DirectShow, I'd start off with a simple playback graph - e.g. playback a media file from disk and progress from there to capture graphs.
The VLC project is open source and cross-platform and it uses DirectShow for playback on the windows platform.
Good luck!