I want to try some C and C++ programming with audio processing, such as synthesizers, chorus, delay etc, but I only know working with a console as output. I wish to have, instead of a console application, a window that would be capable of sending an audio signal to the speakers, running the code at the background and working with it in a similar way as it goes with printf: every time I would call the "output function", it would send to the speakers (or sound card) a sample value, indicating current oscilator position. This output operation could be executed every time it is requested or in the end of a built in loop. Doing all this with a high sample rate would be just great.
I think I could do all this using AudioWorker on Web Audio API, plus a flexible GUI on HTML5 canvas, but I'm new at this API and I'm not sure whether its resulting sound quality is good enough.
Thanks in advance.
Edit: I use Windows 8.1, but any answer for other platform is welcome.
Edit2: Any programming languages other than C, C++ or JavaScript suggestions are also welcome.
I do lots of sound synthesis with the Web Audio API and I think it sounds great. Javascript is really all you need. Well, if you want to use audio files, you need a web server to serve those audio files, but the audio synthesis all happens in javascript.
It doesn't matter so much what OS you use, but different browsers have different levels of support for the web audio API. Chrome tends to have the best support, and Internet Explorer definitely has the worst support.
Related
I am trying to write a pro music/audio processing application, and I would like to be able to interact with the audio inputs/outputs at a very low level - ideally something allowing me to apply effects to the audio inputs and output this in real-time, similar to programs like Logic, Ableton etc.
I have written a pretty basic program that detects audio endpoint devices and can change their volumes using the MMDevice interface, but this is nowhere near the functionality I would like.
I have learned from the Microsoft docs that the four core-audio APIs are:
MMDevice
WASAPI
DeviceTopology
EndpointVolume
but it doesn't seem like any of these have the capabilities that I need. I'm thinking that I will need to be able to interact with the speakers at the level of setting the position of the membrane at a given time.
Is this even possible? If so, what can I use to do this?
The Windows Audio Session API (WASAPI) is the best bet for this purpose. It allows interaction with audio endpoints and setting up audio streams (which are streams of data that you can send or receive in real time). A good example is here.
My company is currently working on what could be called an audio analysis program which needs to process to multiple audio inputs (8 or so) in real time. This means that we need a framework that can handle multichannel audio interface devices that have up to 8 input channels. On top of this, the framework should be as portable as possible. We actually started our development using Java but it ran into issues with the sound API.
When looking for alternate ways to do what we need, I started thinking about using C++ and Qt. I have some experience with both, but I've never done anything remotely similar (in any language for that matter)
Now, the question is, can Qt/Phonon handle audio interfaces/sound cards with over 2 input channels (assuming that the OS can see the devices just fine)? Would it dependent on the backend being used?
Phonon as no input function. it's for playback only if i'm right. but if you want to process input audio you can use QAudioInput. I've used it with just one audio input but I think this constructor with the right QAudioDeviceInfo could do what you want.
I want to capture all audio that is played to the user (all sounds together). Currently I'm working on Windows but it would be nice if the code was cross-platform (but not necessarily). Is it possible to do it with OpenAL? How? Code examples would be great.
Language: C++
The only way to do this I believe is to create a replacement audio device driver that receives all audio requests, and then forwards them to the original device driver. There are a number of existing applications that work in this way including Freecorder, MP3myMP3 Recorder, SoundTap and Wondershare to name but a few (Google "Streaming Audio Recorder").
As for cross-platform, I would say not a chance since it is OS driver model dependent.
Depending on what you have in your system, some (not all) sound cards offer a "Stereo Mix" feature, which can be used like any other recording device. This is basically exactly what you want, as it is literally a mix of all stereo sounds being played.
I have some basic effect algorithms (i.e chrous, LP filtering..) which I would like to build a GUI application to be able to use these algorithms.
For example I want to be able to open an audio file, process the audio file in some way with my algorithms and playback the processed file.
Later on I would like to, if possible be able to see the waveforms of the original file and the processed file in the GUI application. This is my objective now.
In the future I want to be able to create a user interface through which users can be able to use my own audio processing algorithms on files of their own.
Is it possible to design such a GUI with the Qt programming framework? If so, could someone point me in the right direction to get started? Right now I have the Qt SDK 1.1 beta running on Windows 7 OS and also using Qt creator. I would really appreciate some guidance.
Qt is a very powerful application framework, but do not expect any extra help with DSP tasks from it. It contains API for some basic and common tasks, like playing an audio/video file, working with audio devices, creating audio effects (search QAudio and Phonon in the Qt's help) etc. You can use some ready-to-use widgets and create your own multimedia player in a few moments.
But in the DSP you are -mostly, on your own. There is, for instance, only a limited audio file format support, so if you want to work with more formats than .wav and .aiff, use some specialized library. I recommend libsndfile (http://www.mega-nerd.com/libsndfile/) which is most powerful free audio file library available. And if you plan your effects to be more universal, use rather VST technology by Steinberg - today's audio plug-in standard, but it is relatively complicated, not suitable for beginners.
There is no built-in widget which can show a waveform, you have to create it yourself, but it is not much complicated. Qt has a really cool drawing functions, brushes, texts, gradients, transformations, antialiasing, even OpenGL wrapper - everything ready and very simple to use.
So the answer is definitely yes. I use Qt in my multimedia applications for three years and now I can't see how I could live without it (using VST GUI and Windows APIs before).
Sure its possible, QT is a framework for writing applications, you can write any application you want using it, you'll probably end up needing to write some custom controls. As an example, here's an Open source QT based application that does pretty much everything you are talking about and much more:
http://qtractor.sourceforge.net/qtractor-index.html
In my application we will present the video stream from a traffic camera to a client viewer. (And eventually several client viewers.) The client should have the ability to watch the live video or rewind the video and watch earlier footage including video that occurred prior to connecting with the video stream. We intend to use wxWidgets to view the video and within that we will probably use the wxMediaCtrl.
Now, from the above statements some of you might be thinking "Hey, he doesn't know what he's talking about." And you would be correct! I'm new to these concepts and I confused by the surplus of information. Are the statements above reasonable? Can anyone recommend a basic server/client architecture for this? We will definitely be using C++ wxWidgets for the GUI, but perhaps wxMediaCtrl is not what I want... should I be directly using something like the ffmpeg libraries?
Our current method seems less than optimal. The server extracts a bitmap from each video frame and then waits for the single client to send a "next frame" message, at which point the server sends the bitmap. Effectively we've recreated our own awkward, non-standard, inefficient, and low-functionality video streaming protocol and viewer. There has to be something better!
You should check out this C++ RTMP Server: http://www.rtmpd.com/. I quickly downloaded, compiled and successfully tested it without any real problems (on Ubuntu Maverick). The documentation is pretty good if a little all over the place. I suspect that once you have a streaming media server capable of supporting the typical protocols (which rtmpd seems to do), then writing a client should fall into place naturally, especially if you're using wxWidgets as the interface api. Of course, it's easy to write that here, from the comfort of my living room, it'll be a different story when you're knee deep in code :)
you can modify your software such that:
The server connect, server grabs an image, passes it to ffmpeg establishing stream, then copy the encoded data from ffmpeg stream and send to client via network, if the connection drops, close the ffmpeg stream.
Maybe you can use the following to your own advantage:
http://www.kirsle.net/blog.html?u=kirsle&id=63
There is a player called VLC. It has a library for c++ and you can use it to embed the player in your GUI application. It supports a very wide range of protocols. So you should leave connecting, retrieving and playing jobs to VLC and take care of the starting and stopping jobs only. It would be easy and probably a better solution than doing it yourself.
For media playing facility, both music and audio, you can a look on GStream. And talking about the server, I think Twisted (A network library in Python) should be good option. The famous live video social website justin.tv is based on Twisted. Here you can read the story from here. Also, I built a group of server for streaming audio on Twisted, too. They can serve thousands of listener on line in same time.