I am doing a speech processing project with a Raspberry Pi 3 (running Raspbian) using a USB Microphone. I can see the Microphone show up as a selectable audio device for the Pi and it produces/captures sound perfectly.
I cannot figure out how to use this in my code; I have done a ton of research regarding this and have found some tutorials but nothing that making sense. I come from more of a hardware background and have done something like this with controllers where I hook up an actual Mic and process the Analog Signal into Digital on IO Pins; I am so frustrated with this that I am about to pump data over from an Arduino using a Mic and A2D conversion.
-------------------------------------------------------My questions----------------------------------------------------
1) I want to know how to access a USB data stream or USB device in C or C++. My Linux abilities are not the best. Do I open a Serial Connection or open a filestream in "/dev/USB/...."? Can you provide a code example?
2) Regardless of the fidelity of the USB Mic Input, I want to know how to access its Input in C/C++. I have been looking at ALSA but cannot really understand a lot of its complexity. Is there something that gives me access to a raw input signal on the USB Port that I can process ( where I extrapolate out frequency, amplitude, etc.)?
I have already gone through a lot of the similar posts on here. I am really stuck on this one. I 'm really looking to understand what is going on from the OS perspective; I'll use a library given but I want to understand how it works.
Thanks!
So an update:
So I did all of my code in C with some .sh scripts. I went ahead and figured out how to use the Alsa asoundlib (asound.h specifically). As of now, I am able to generate and record sound via USB Mic/Headset with my Pi 3. Doing so is rather arduous but here is a useful link (1).
For my project, I also found a CMU tutorial/repos for their PocketSphinx Audio Recognition Device at link (2) and video link (3). This project utilizes the Alsa asoundlib as well and was a great help for me. It takes a while to download and you need to crawl through its .sh scripts to figure out its gcc linking. But I am able to now give audio cues which are interpreted by my Pi 3 and pushed to speaker output and GPIO pins.
LINKS(1)http://www.alsa-project.org/alsa-doc/alsa-lib/_2test_2pcm_8c-example.html
(2)https://wolfpaulus.com/embedded/raspberrypi2-sr/
(3)https://www.youtube.com/watch?v=5kp5qpwVh_8
Related
I'm a beginner when it comes to programming and I wanted to do a personal project in C++ to develop my skills. The project I had in mind involves playing audio on my laptop (running Windows 10), analyzing it, and sending data to an arduino that will change the color and brightness of LED lights in sync with the audio that's playing. I would like it so that I can simply, for example, just play a song on Spotify or a music video on Youtube etc. and the program will get data from that audio stream as an input. Elsewhere I've seen programs use audio from recorded WAV files or streams from a microphone as input, but not what I have in mind. I want to use this program for parties, so using a microphone as a workaround wouldn't be ideal.
Is this even possible? And if so how should I approach this problem? Are there certain APIs I should look to or what? If the program gets audio as the input, would I still be able to play music on something like a bluetooth speaker as well? Or can it only send data to one place at a time?
My roommate who is much better at programming than me accomplished this on Mac using Swift, and while I don't have a Mac, would using Linux instead make this easier?
Modern windows has “Stereo Mix” recording device, just for that. Here’s how to enable: https://technicalustad.com/enable-stereo-mix-in-windows-10/
After that setup, in your C++ program use any recording API you want.
Here’s a sample that does what you ask for, opens a recording device, starts recording, and sends audio samples to the class provided in the argument: https://learn.microsoft.com/en-us/windows/win32/coreaudio/capturing-a-stream You probably want to trade CPU time for latency for your application, i.e. don’t Sleep for hnsActualDuration/REFTIMES_PER_MILLISEC/2, change into Sleep( 0 ) or Sleep( 1 )
In my Android app, I use Android NDK to play music by doing the following:
extract audio samples from an OGG file using the Vorbis library
process the audio samples
redirect the processed samples to the audio output using the Oboe library
In order to avoid underruns, I do the first 2 steps in a separate thread, to extract and process the sound a little bit in advance (it adds a bit of latency, but this is not a problem for my app). That solution works great on every device I've tested so far.
But for some reason, when I pair my device with a bluetooth speaker and when I play music, I have what seems to be underruns on some devices like Samsung S7 or Nokia 1 (but not on every device).
This bug looks so random to me that I don't know where to start. It acts like the bluetooth connection is using quite a lot of CPU so my app doesn't have enough resource to run properly.
Does anyone have experienced something similar? Should I do anything in my code to handle the bluetooth connection so it doesn't use CPU (for example to avoid audio resampling)?
Thanks for your help.
Android + Bluetooth audio is a world of pain. The major thing to appreciate about Bluetooth is the audio sink runs at a rate independent of other audio devices, which is why the native mediaplayer will do things like display video in accordance with whatever rate the attached audio device consumes samples, essentially slaving itself to the clock of the BT audio device. If you want to drive the speed from Android (i.e. SystemClock timebase) you'll need to use a timestretching AudioTrack. (This can be done, but driver support is erratic and overall system stability tanks).
Firstly, you want to eliminate the devices themselves being problems. Can you play the ogg files in a media player to a Bluetooth speaker from the S7 or Nokia 1 without problems? If so, it's your code!
It sounds to me like the speaker is consuming samples faster than the device is producing them, for whatever reason. Basically check your callbacks to make sure whenever the audio subsystem requests more data you are actually providing it. Be sure to drive your decoding pipeline according to the callbacks being made and not the system clock or any other assumptions about timing.
Finally, Bluetooth audio, at least A2DP, as opposed to directly streaming MP3, is going to require some processing to recompress the audio as it is sent out, but those devices should have plenty of headroom for this, maybe even special DSPs. I've done it with 1080P video playback at the same time before, and it starts to fall apart with two videos at once!
I'm looking into coding a very simple LAN based home monitoring system using python 2.7 on Windows. I have a number of computers around the house each with a usb webcam attached. The cameras have built in microphones. I'm looking for the best way to capture and stream the audio and video over the network, then recieve and view/listen to it. I'm guessing I'd have to use PyAudio to get the audio from the microphone and CV2 to get the video, past that, I'm not sure how I'd stream that data to another computer, recieve it and then view/listen to it.
Not realy a question here, but still I guess I have a solution for you
(but there a millions of ways to solve this.)
My way is ROS (Robot Operating System), which is basically a TCP/IP Service Wrapper. Anyway you can simple broadcast and receive streams via your network.
ROS can be implemented in C++ or Python.
ROS usually applies OpenCV as CV-library, so it should suite your wishes.
Straight forward examples for video are (i.e.) here:
mjpeg_server
web_video_server
Audio streaming is introduced here:
audio_common (there might be solutions via PyAudio. Not sure if other solutions might come more easy and still suitable.)
I'm trying to write a C++ app which plays midi sounds, but I'm having some difficulty. I got the examples that come with RtMidi to compile out of the box, but running the midiout example, I don't hear anything. Running the midiprobe example I get:
There are 1 MIDI input sources available.
Input Port #1: AudioFire4 (2357) Plug 1
There are 1 MIDI output ports available.
Output Port #1: AudioFire4 (2357) Plug 1
But that's just an audio interface that I use for using multiple speakers. I don't use any of the midi functionality on it.
Does anyone know how to get RtMidi to use the built in system general midi sound bank so that I can actually hear something?
Thanks,
Gabe
If anyone else is curious about this, I ended up installing FluidSynth and downloading the FluidR3 soundfont. That seems to work great. Just run it, and it presents itself as a midi device.
I want to develop code to transmit data from system to PIC through USB.
Can anybody give good link regarding data transmission through USB.
because i am new to this.
NOTE: Very simple is code is enough to me.
Thanks in Advance
The PIC16F877 does not have a USB peripheral built in. I assume that the product that you are building is a "USB Device" and that the "system" that you are referring to will provide the USB host functionality. If this is the case then you will have to add a USB interface chip to your hardware to provide async serial to USB connectivity. Suitable devices are made by TI (TUSB series) or FTDI. You then connect one of these to the PIC internal USART and pretend that you are transferring the data over a simple serial port. If you are using a pc then the connected device will appear as a standard COMn: port.
I totally agree with the FTDI route, however if you're not looking for a UART tunnel over USB then you have to get a bit more in-depth.
You'll have to write the USB routines yourself or find libraries/projects for your processor. What you will definitely have to have is a fast clock - 12MHz is necessary More is better because on small uC's like these you'll spend most of your time just handling the basics of USB - signaling and so forth. You'll also need a fair bit of memory because the USB code takes up around 1.5K I think. Then you need room afterwards for your own code.
I've seen the V-USB (http://www.obdev.at/products/vusb/index.html). It's for AVR not PIC, but it says it provides all of the USB functionality you'll need and even provides vendor and product IDs for you to use (non-commercial I believe). There's also a PIC project for USB that doesn't run on the same hardware as yours here: http://www.alanmacek.com/usb/
To make the driver you'll have to use libusb - here's a link for the win32 version: http://libusb-win32.sourceforge.net/
It's surprisingly more simple than I had expected, but I just looked at example code and not actually made anything. Good luck!
As your task will involve Windows Device Driver development, I do recommend to downlaod and install the WDK and look through the contained sample USB drivers.
This might get you some hints about the complexity of the topic. Device driver development is not the easiest thing to start with.
Maybe you can start with UART communication (RS232).
check http://www.beyondlogic.org/usbnutshell/pic16f876.gif will surely help u
also
ttp://www.beyondlogic.org/usbnutshell/usb7.htm , type h in front of 2nd link.