Large number of Sound Effects with SimpleAudioEngine - cocos2d-iphone

Background:
My background music is being played as a sound effect because I want to change the pitch between each repetition of the tune.
The Issue:
While the background music is playing there's a lot of other short sound effects happening. After a certain number of sound effects play, my background music (also a sound effect) cuts out. The queue seems to cycle so that after say 50 sound effects have played, when the 51st plays, the 1st will be released whether it has completed or not.
Request for direction:
There are two directions I can see possibly going with this issue.
1. Not play the background music as an effect and figure out how to change the pitch as background music instead of an effect
2. Some how make sure that the effect will be retained until it has completed.
Thanks

The SimpleAudioEngine doesn't allow you to control channels, so it'll just play your sound effects in one of the channels it has allocated internally. Eventually all channels are playing audio. In that case, when a new audio file is to be played and all channels are already playing, SimpleAudioEngine will cancel one of the existing audio files. On occasion this will be your background music. Dang.
What you normally do to fix this is to assign (allocate) one audio channel specifically for the background music. No other audio will ever attempt to play audio on that channel. You will have to use the "regular" CocosDenshion API. It looks like you either need to use CDAudioManager or CDSoundEngine, but I have no experience with using channels with CocosDenshion.
Personally I never used CocosDenshion, whenever I needed more control I found the ObjectAL API to be easier to understand and it has excellent documentation. I made the following helper function for ObjectAL that plays a sound on a particular channel. I simply instantiated a ALChannelSource for my background music, and only played background music through that channel. The returned ALSoundSource even allows you to change pitch, pan, etc. while the audio is playing.
+(id<ALSoundSource>) playEffect:(NSString*)effect channel:(ALChannelSource*)channel loop:(bool)loop
{
id<ALSoundSource> soundSource = nil;
if (channel)
{
ALBuffer* buffer = [[OALAudioSupport sharedInstance] bufferFromFile:effect];
soundSource = [channel play:buffer loop:loop];
}
return soundSource;
}
All other audio files I'm playing through the OALSimpleAudio class.

Related

Flutter video_player initialization error when initializing multiple videos

Problem I need help with
Assist with optimizing latency in its "Short-Video Feed" and solving intermittent performance bugs. A central feature of my app requires seamless plays 15 to 60 second clips when users use "swipe up" hand gestures similarly to TikTok and Instagram reels. Right now, I have performance bugs (intermittent) such as black screens, delayed loading screens, sometimes long loading, etc.
The bugs may be caused because Flutter is slower than Native iOS. However, our "Short-Video Feed" has lots of bugs whether I use a M3U8 (Mux), or whether I use a MP4 based approach with AWS S3.
If I use the Mux based approach with M3U8, "Short-Video Feed" there is a noticeable few milliseconds black screen for each short-video playback.
If I use the Amazon based approach with MP4, "Short-Video Feed" intermittently loads for a few seconds (sometimes minutes) when there is low bandwidth, and some videos get stuck even when a user returns to a location with faster bandwidth.
Open issue on Flutter
https://github.com/flutter/flutter/issues/25558
Approaches I have tried with no success:
Native Player. I tried to use a native video player for Android/iOS, with MP4 and M3U8, but the UI was still very laggy (because of data transfer between Android/iOS and flutter latenices).
Flutter Player. I tried to use a Flutter video player for Android/iOS, with MP4 and M3U8, but the UI shows a black screen with M3U8 & heavy loading for poor internet connections with MP4.
Approaches I need help to try:
Optimize M3U8 player to minimize the black screen issue. or...
Create MP4 chunks to optimize for poor reception areas (this is what I think TikTok Instagram Reels, and similar applications do based on what I can see).
Has anyone solved this issue?
How about isolating whether these lags are due to network buffering or due to Flutter (or even a device hardware limitation such as memory or GPU)?
Perhaps use a few local MP4 files with identical frame rates and encoding parameters (both video and audio) and see whether the UI lag is reproducible upon swipe-up scrolling?

Do not show Desktop between 2 videos

I´m working with OMXPlayer on Raspberry.
Right now I have a loop(with Python2.7) to show videos and it works correctly.
But I have two problems:
1. When one video is finished, the Desktop will be shown for one second. And I don't want it. How can I change quickly to another video without showing the Desktop?
2. Another problem is that I wanna show some pictures too.. I know that OMXPlayer does not show images... Can I use another program in my code? But the user should not notice the change.
Thanks.
I was trying to figure this out too but I'm afraid it's not possible. It seams to me that omxplayer is only able to play a single video at a time and to play another video you have to run a new instance of the program. It takes a while to initialize, hence the gap between the videos.
That said, I figured a nasty way to get around it. When playing a video, you can extract it's last frame into a PNG file with ffmpeg. Then, you can play the video with omxplayer and use whatever graphical tools you have to display the picture fullscreen in the background. When the video ends, it disappears but the picture stays there and since it's the last frame of the video, it appears to just freeze at the end for a second, before the next video starts. Then, you just repeat the process. Hope it helps.

How to display smooth video in FireMonkey (FMX, FM3)?

Has anyone figured out how to display smooth video (i.e. a series of bitmaps) in a FireMonkey application, HD or 3D? In VCL you could write to a canvas from a thread and this would work perfectly, but this does not work in FMX. To make things worse, the apparently only reliable way is to use TImage, and that seems to be updated from the main thread (open a menu and video freezes temporarily). All EMB examples I could find all either write to TImage from the main thread, or use Synchronize(). These limitations make FMX unusable for decent video display so I am looking for a hack or possibly bypass of FMX. I use XE5/C++ but welcome any suggestions. Target OS is both Windows 7+ & OS X. Thanks!
How about putting a TPaintbox on your form to hold the video. In the OnPaint method you simply draw the next frame to the paintbox canvas. Now put a TTimer on the form, set the interval to the frame rate required. In the OnTimer event for the timer just write paintbox1.repaint
This should give you regular frames no matter what else the program is doing.
For extra safety, you could increment a frame number in the OnTimer event. Now in the paintbox paint method you know which frame to paint. This means you won't jump frames if something else calls the paint method as well as the timer - you will just end up repainting the same frame for the extra call to OnPaint.
I use this for marching ants selections although I go one step further and use an overlaid canvas so I can draw independently to the selection and the underlying paintbox canvas to remove the need to repaint the main canvas when the selection changes. That requires calls to API but I guess you won't need it unless you are doing videos with a transparent colour.
Further research, including some talks with the Itinerant developer, has unfortunately made it clear that, due to concurrency restrictions, FM has been designed so that all GPU access goes through the main thread and therefore painting will always be limited. As a result I have decided FM is not suitable for my needs and I am re-evaluating my options.

XAudio2 delayed sound - when playing multiple voices

I am using XAudio2 to play sounds in my game. I have about 16 source voices out of which 3 are looping effects which need to be played just like a background music. I have another background music which starts playing at regular intervals(once completed it waits for 1 minute and restarts). Whenever another effect has to be played like button click, gun shot etc, I search for an available source and attach the buffer to it and starts playing.
My problem is once the background music has started playing, just the looping effects will get played and other effects which need to be played on game actions(such as button click) do not get played. And all of these delayed effects get played once the background music is stopped. Also if I use one looping effect(instead of 3) all these get played in the right order.
I am using a submix voice for effects and background music is submitted directly to the mastering voice.
Can someone please help me as I am stuck here
Whenever one intends to play different sounds in the same time, he or she should make sure that these sounds are not in a queue, displayed in a sequential order, but they are displayed using different threads.
In Milsim's case the problem was exactly this, apparently, he used tasks to display the sounds, but they were not issued in different threads. He solved his problem using IAsyncAction.

cocos2d-iphone games stop playing background music sometimes

cocos2d-iphone 1.0.1.
I have noticed this with other cocos2d-iphone games installed in my device, like Kingdom Rush.
Basically, most of the times, the audio is fine (almost always). But suddenly, at an unexpected moment, the background music stops playing and only the sound effects work. Sometimes, killing the application will not be enough to fix it.
With my cocos2d-iphone game this happens as well, with no hint in the console. I use SimpleAudioEngine to play background music and sound effects.
Killing my application, restarting Xcode will not fix it. I usually just ignore the problem and, in the near future, it is suddenly gone. I suspect that rebooting the device tends to fix this, but that's beyond the point: I should know why is it happening.
I also tried preloading my background music. Doesn't change a thing.
I believe I have experienced this problem with both .mp3 and .wav formats.
Why might this be happening?
No idea to the exact reason, but I can think of a few:
memory warning causes audio stream to be interrupted
audio interruptions (calendar notification, incoming SMS/call) not handled properly by CocosDenshion
other streaming music is played (ie perhaps videoplayer, iPod music player)
music isn't streamed but buffered, which means music is fighting over audio buffers with all other effects - eventually so many audio effects are played that older buffers have to cancel playback (which might be the music) in order to allow the new effect to play
defective device (since it happens in other apps …)
bug in CocosDenshion (check the cocos2d issue tracker and forum for any unresolved audio bugs)
I think you can exclude the latter if you're using the playBackgroundMusic API to stream music instead of buffering it.