Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I am writing a program that displays a random natural note and waits for the user to play that note on the guitar. The audio input is processed to see if the correct pitch was played, and if it was, the next note is shown and the score of the user is updated. The idea is to teach basic guitar notes.
I intend to use SFML for audio processing and QT4 for the gui. I will have a widget derived from the relevant QObject and SFML classes.
Question: How do I detect the pitch of microphone input using SFML? Is it possible to simply store a part of the input in an sf::sound object and call it's getPitch() method?
Is it possible to simply store a part of the input in an sf::sound object and call it's getPitch() method?
GetPitch() from sf::SoundSource will return the value you used on SetPitch(pitch) or teh default 1.0f. It is to edit the sound, not to get information about it. I think the only way to do it is to get the array of sound samples and process it with some kind of algorithm. You can get this array with it:
sf::SoundBufferRecorder recorder;
recorder.Start();
// ...
recorder.Stop();
const sf::SoundBuffer& buffer = recorder.GetBuffer();
size_t sample_count = buffer.GetSamplesCount();
const sf::Int16* samples = buffer.GetSamples();
unsigned int samples_per_second = buffer.GetSampleRate();
As it turns out, SFML does not have any algorithms for detecting pitch built in. Thanks to LBg for getting my mind working in the right direction. SFML only provides the tools needed to record the sounds and store them in a buffer.
I found out that I can use a Fast Fourier transform to evaluate the buffer for a frequency. This frequency can then be compared to a list of known pitch frequencies, together with a pitch threshold.
While SFML doesn't have an fft algorithm built in, it does have the tools needed to get a sound buffer. I will have to check and see if this is the most cross-platform way of doing things.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 1 year ago.
Improve this question
I'm trying to create a custom GUI in OpenGL from scratch in C++, but I was wondering is possible or not?
I'm getting started on some code right now, but I'm gonna stop until I get an answer.
YES.
If you play a video game, in general, every UIs should be implemented by APIs like OpenGL, DXD, Metal or Vulkan. Since a rendering surface has higher frame rate than OS UI APIs, using them together slows down the game.
Starting with making a view class as a base class, implement actual UI classes like button, table and so on inherited from the base class.
Making UIs using a GFX API is similar to making a game in terms of using same graphics techniques such as Texture Compression, Mipmap, MSAA and some special effects and so on. However, handling a font is a sort of huge part, for this reason, many game developers use a game engine/UI libraries.
https://www.twitch.tv/heroseh
Works on a Pure C + OpenGL User Interface Library daily at about 9AM(EST).
Here is their github repo for the project:
https://github.com/heroseh/vui
I myself am in the middle of stubbing in a half-assed user interface that
is just a list of clickable buttons. ( www.twitch.com/kanjicoder )
The basic idea I ran with is that both the GPU and CPU need to know about your
data. So I store all the required variables for my UI in a texture and then
sync that texture with the GPU every time it changes.
On the CPU side its a uint8 array of bytes.
On the GPU side it's unsigned 32 bit texture.
I have getters and setters both on the GPU (GLSL code) and CPU (C99) code that
manage the packing and unpacking of variables in and out of the pixels of the texture.
It's a bit crazy. But I wanted the "lowest-common-denominator" method of creating
a UI so I can easily port this to any graphics library of my choice in the future.
For example... Eventually I might want to switch from OpenGL to Vulkan. So if I keep
most of my logic as just manipulations of a big 512x512 array of pixels, I shoudn't
have too much refactoring work ahead of me.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
In my current app I need to share screens ala skype or discord, I'd prefer to not use external libs, but will if I have to.
So far I have been sending the screenshots in down-scaled bitmap form over TCP sockets and repainting the window every few milliseconds, this is of course an effort I knew was doomed from the start, is there any api that could save me?
Any help appreciated.
While I haven't implemented it myself, I believe that what's usually done is the screen is broken into 16x16 pixel blocks. You can keep the previous screenshot, take a new one, compare which blocks have changed and send only the 16x16 blocks that have changes in them.
You can further improve performance by having a change threshold. If fewer than x pixels have changed in a block, don't send yet. Or if the cumulative sum of the changes in a block (the difference between corresponding pixels) is below some threshold, don't send that block.
The blocks are also often compressed using a lossy compression scheme that really shrinks down the required size you need to send per block. The image blocks are often also sent in 4:2:2 mode, meaning you store the red and blue channels at half the resolution of the green channel. This is based on how the visual system works, but it explains why things that are pure red or pure blue sometimes get blockiness or fringing around them when screen sharing.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
A video can be edit by deleting some frames from it.that can be done by deleting consecutive frames or random frames in that video.
we need to detect the tampered videos, we hope to do this by detecting frames which has been altered , simply we need to have an algorithm for way to identify deleted frames.Because we are creating a tool which could use for tamper detection as a video surveillance tool.Which could use in law enforcements and courts
Is there any method to identify that frames of a video has been deleted by malicious attacks, there are already available methods such as watermarking, digital signature, but we need an algorithm to detect that frames by using proper algorithm.
In general, whatever image sequence I am handed, some or all of it could have come from a real camera, from Photoshop or from a 3D renderer, and there is no general purpose image processing technique that will be able to tell the difference based on analysis of the image content alone, either well enough to stand up in court, or, indeed, at all.
You'll need to embed some easy to recognise yet difficult to fake, relocate or tamper with signal into the video at recording time. You've tagged the question "opencv", but the fields you want to investigate are cryptography and watermarking, not computer vision.
Have the video surveillance equipment use public key crypto to visibly watermark each frame with an identifier unique to the piece of equipment, a timestamp, a frame sequence number and a hash or other suitable function of the frame image, using a scheme robust to compression.
Tampering with the video sequence will then require either knowledge of the device's private key, or removal of the watermark. This isn't great (keeping the private key secret will be a logistical headache) but is probably the best solution you can get.
this can't be done in general. However some approaches may be possible.
the used video format may support frame wise meta data that stores the index or time index and that is not touched during editing
the image sensor itself may be configured to write some meta data to some specific region of the image
you may have some external reference that was imaged by the camera and may help identify missing frames
precise clock
fast blinking indicator
some uniform motion
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 1 year ago.
Improve this question
I need to convert text (string+font) into mesh (vertices, indices, triangles etc), but I don't need to draw anything. I'll just get a string from one API and push it as vertices & indices to another. What's the simplest/easiest/best way of doing this? Font metrics and text placing are of course available and no other transforms are needed.
I'm currently working with VC++. However, any kind of OpenSource (C/C++, C#, VB,...) and "non-open but free" COM/.NET -libraries would be great.
I've heard of FreeType. Does it answer my prayers or is there something even better?
EDIT: As Nico Schertler commented, there seems to be Mesh.TextFromFont -function in DirectX -libs that probably does the trick. Thank you Nico! I'll update when I have time to test this in practise.
Mesh.TextFromFont sounded good but it didn't save the day since I couldn't figure out how to get the actual point/triangle data from the mesh -object.
But then I found this. In this project, GraphicsPath is used to create a point-path from a glyph. Then the points are coverted into Polygons and the polygons are then tesselated into triangles using Poly2Tri.
A quick browse through the source code and with some small modifications and code stripping I ended up with a nice .NET -dll with one simple static function that does everything I need.
To convert a text into a mesh you can use the ttf2mesh library. This library consists of just one C-file and allows to open truetype font (.ttf) and convert it glyphs to a mesh objects in 2d or 3d space. There is an examples in the repository.
An interesting feature is the lack of dependency on any third party library (like libfreetype). Also in the examples there is a ttf2obj program that allows you to convert a font file to an OBJ file.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I'm interested in learning to use OpenGL and I had the idea of writing a music visualizer. Can anyone give me some pointers of what elements I'll need and how I should go about learning to do this?
If you use C++/CLI, here's an example that uses WPF four (fourier that is;) display.
He references this site (archived) that has considerable information about what your asking, here's anoutline from the specific page;
How do we split sound into
frequencies? Our ears do it by
mechanical means, mathematicians do it
using Fourier transforms, and
computers do it using FFT.
The Physics of Sound
Harmonic Oscillator
Sampling Sounds
Fourier Analysis
Complex Numbers
Digital Fourier Transform
FFT
Ahhh, I found this (archived) a few minutes later, it's a native C++ analyzer. Code included, that should get you off and running.
My approach for creating BeatHarness (http://www.beatharness.com) :
record audio in real time
have a thread that runs an FFT on the audio to get the frequency intensities
calculate audio-volume for left and right channel
filter the frequencies in bands (bass, midtones, treble)
now you have some nice variables to use in your graphics display.
For example, show a picture where the size is multiplied by the bass - this will give you a picture that'll zoom in on the beat.
From there on it's your own imagination ! :)
Are you trying to write your own audio/music player? Perhaps you should try writing a plugin for an existing player so you can focus on graphics rather than the minutia of codecs, dsp, and audio output devices.
I know WinAMP and Foobar have APIs for visualization plugins. I'm sure Windows Media Player and iTunes also have them. Just pick a media player and start reading. Some of them may even have existing OpenGL plugins from which you can start so you can focus on pure OpenGL.
If you're just after some basic 3D or accelerated 2D then I'd recommend purchasing a copy of Dave Astle's "Beginning OpenGL Game Programming" which covers the basics of OpenGL in C++.
For the music analysis part, you should study the basis of Fourier series, then pick a free implementation of a DFFT (digital fast fourier transform) algorithm.
You can find implementation of FFT algorithms and other useful informations in Numerical Recipes in C book. The book is free AFAIK. There is also Numerical Recipes in C++ book.
You may want to consider using libvisual's FFT/DCT functions over FFTW; they're much simpler to work with and provide data that's similarly easy to work with for the sake of generating visuals. Several media players and visualization plugins use libvisual to some extent for their visuals. Examples: Totem (player), GOOM (plugin for Totem and other players), PsyMP3 2.x (player)
From my point of view...check this site:
http://nehe.gamedev.net/
really good Information and Tutorials for using OpenGL
edit:
http://www.opengl.org/code/