I have a list of .mp3 files over the web and I would like to get the highest quality file.
Quality in multimedia files equals the bit rate of them.
The bit rate itself should be found in the file's headers. If not, length of the audio track could be used too. (Filesize / Track Length = Bit Rate)
These things would be easy if I would have these files locally, but I would like to fetch this information over HTTP and determine which file has the highest quality.
Can I get an audio track's length out of HTTP headers? If not, is it possible to fetch only the bits that describes the length/bit rate instead of downloading the whole file?
I'm writing the code in python, but the question is quite general so I'm not tagging it as a python question.
Assuming that the remote server is behaving nicely, you could issue a HEAD request to the file and check the contents of the Content-Length header field. It doesn't give you track length or bit rate but you can get the size of the file.
EDIT: MP3s consist of multiple frames, each of which can be of a different bit rate (VBR). Track length is calculated from the bit rate of each of these frames, rather than the length itself being stored. If you want the bit rate reliably, you'd need two get the whole file and get the bit rate of each of the frames. It may be possible to grab the first few KB of the file and read the bit rate from the first frame, but this is not always at the same point in the file (e.g. due to position of ID3 tag etc.).
Related
I'm working on an mp3 Steganography project and i want to encode text inside the mp3 file by manipulating Least Significant Bits(LSB) at regular intervals. I want to encode that text without making any significant changes in the audio. And according to this link http://www.datavoyage.com/mpgscript/mpeghdr.htm there are mp3 headers which carry information of the leading mp3 chunk. So i want a guidance on how can i make this possible?
Mp3 file is made of sequences of "Frames" (It's about 11000 frames for a mp3 file with 4 minutes playing). At front and end of each MP3 file there are two fields of information (Id3 Tag v1, v2) contains information about Mp3 file - these two fields are optional and can exist or not without any impact on the quality of Mp3 file. You should not hide staga-message here because they can easily be found. Frame consists of frame header (32 bits) and frame body (contains compressed sound). According to your question, steganography will affect on the frame header (32 bits), so I'll focus on frame header!
In 32 bits of frame header still exists some "unimportant bit" due to their functions (read more detail on their function). In short you can use bit in index of: 24, 27, 28, 29, 30, 31, 32 (with bit 27 and 28 will have a small impact on the sound quality) with index in this picture in this link: https://en.wikipedia.org/wiki/MP3#/media/File:Mp3filestructure.svg.
So it depends on whether you want just 5 bits per frame of 7 bits per frame. 7 bits is the max number of bits that you can use on each frame due to my working (both theory and test by source code) but someone else can find a larger bit!
In order to access byte array of each frame, you can write your own class but there are many free available classes on the Internet - NAudio.dll by Mark Heath - (I cannot post link due to forum laws, you can search Google) - is a useful one.
Having accessed the byte array of each frame, you can embed/extract data in/from Mp3 file. Note that: 32 first bits of byte array of each frame is the Frame Header, so you can easily identify the precise index of unimportant bits!
I've recently completed my final year thesis on this topic (steganography on images -LSB, Parity Coding and MP3 - Unused Bits Header). The following source codes from my thesis (written in C#) is a runnable steganography program. I hope that it can help: http://www.mediafire.com/download/aggg33i5ydvgrpg/ThesisSteganography%2850900483%29.rar
Ps: I'm a Vietnamese, so it can exist some errors in my sentences!
I'm trying to find a solution to store a binary file in it's smallest size on disk. I'm reading vehicles VIN and plate number from a database that is 30 Bytes and when I put it in a txt file and save it, its size is 30B, but its size on disk is 4KB, which means if I save 100000 files or more, it would kill storage space.
So my question is that how can I write this 30B to an individual binary file to its smallest size on disk, and what is the smallest possible size of 30B on disk including other info such as file name and permissions?
Note: I do not want to save those text in database, just I want to make separate binary files.
the smallest size of a file is always the cluster size of your disk, which is typically 4k. for data like this, having many records in a single file is really the only reasonable solution.
although another possibility would be to store those files in an archive, a zip file for example. under windows you can even access the zip contents pretty similar to ordinary files in explorer.
another creative possibility: store all the data in the filename only. a zero byte file takes only 1024 bytes in the MFT. (assuming NTFS)
edit: reading up on resident files, i found that on the newer 4k sector drives, the MFT entry is actually 4k, too. so it doesn't get smaller than this, whether the data size is 0 or not.
another edit: huge directories, with tens or hundreds of thousands of entries, will become quite unwieldy. don't try to open one in explorer, or be prepared to go drink a coffee while it loads.
Most file systems allocate disk space to files in chunks. It is not possible to take less than one chunk, except for possibly a zero-length file.
Google 'Cluster size'
You should consider using some indexed file library like gdbm: it is associating to arbitrary key some arbitrary data. You won't spend a file for each association (only a single file for all of them).
You should reconsider your opposition to "databases". Sqlite is a library giving you SQL and database abilities. And there are noSQL databases like mongodb
Of course, all this is horribly operating system and file system specific (but gdbm and sqlite should work on many systems).
AFAIU, you can configure and use both gdbm and sqlite to be able to store millions of entries of a few dozen bytes each quite efficienty.
on filesystems you have the same problem. the smallest allocate size is one data-node and also a i-node. For example in IBM JFS2 is the smallest blocksize 4k and you have a inode to allocate. The second problem is you will write many file in short time. It makes a performance problems, to write in short time many inodes.
Every write operation must jornaled and commit. Or you us a old not jornaled filesystem.
A Idear is, grep many of your data recorders put a separator between them and write 200-1000 in one file.
for example:
0102030400506070809101112131415;;0102030400506070809101112131415;;...
you can index dem with the file name. Sequence numbers or so ....
I've been trying to convert a huge amount of data (about 900 MB) to an audible file format for a few days now. I've been given a .dat file containing 900 millions floating-point samples (one per line) representing 90 seconds of music at 10 MHz.
I downsampled to 40 KHz but now I don't know how I could possibly listen to the audio hidden in those bytes. I'm writing a C++ program in a Linux environment but if any one knows how to accomplish this task using Matlab, Octave, Python, Audacity, MPlayer or any other tool, please come forth and speak :) Contributions in any amount are greatly appreciated.
head -n 5 ~/input.dat
-2.4167
-7.5322e-016
-0.2283
0.13581
-0.51926
Target your sample rate to 44100 hz (or 48000, 22050, 11025, or 8000 hz)
Convert your audio samples to 16-bit signed integers (-32768 to +32767).
Follow the instructions on WAV file synthesis here:
WAV File Synthesis From Scratch - C
The wav file format is a rather simple one.
You just need to write the 44 byte header block defined in that link, followed by your data converted to integers.
If you have a sequence of bytes and want to convert it to audio, all you need to do is write a header to it. Since you mentioned that you can use MatLAB, I would recommend wavwrite command. It is simple, tried and tested and excellent for prototyping. Here is the link to the documentation:
http://www.mathworks.in/help/matlab/ref/wavwrite.html
Here are some steps you may need to take in case you are using wavwrite.
- Since your input data is floating point, scale the data in your file to within a range of [-1, 1].
- Once data is scaled plug and chug into the function call.
- Play the wav file using wavplay command.
WAV allows for floating-point sample data and a wide range of sample-rates (1Hz to 4.2GHz in 1Hz increments, if memory serves).
You don't need to bother with converting to integer values. Just set up the WAV file's header appropriately and write 32-bit floats as binary data in the data section.
From a storage perspective, the 10MHz sample-rate is no problem for a WAV file. Playback, however, will require conversion to something the hardware can handle. The upper limit these days is typically 96 or 192 kHz.
First of all, sorry for my poor English. I am writting video streaming server in C++. I have multiple mpeg2-ts files (movies and advertisements) which I need to stream via HTTP as one single TS-FILE. The problem is that every mpeg-ts file has its own timestamps (PCR, PTS, DTS). And, as I understand, to make a continuous streaming flow, every new PCR (PTS, DTS) value should continue from the last PCR (PTS, DTS) value.
Here is a picture for better understanding of what I am saying about: http://i.stack.imgur.com/vL1m6.png (I can't include my picture directly in the message. Sorry)
I need to replace pcr`1, pcr`2, pcr`3 timestamps with the new ones. For example, I sent ts-packet containing the pcr3 timestamp and after a few more ts packets (not containing any value of PCR) I want to insert my advertisement. And my question is: how do I calculate the new values for pcr`1, pcr`2, pcr`3 and so on?
Is it correct to calculate the bitrate of the current video and then divide the amount of bits that the program have sent since the last PCR timestamp (in our case, it's pcr3) by this bitrate? I mean the following: (new timestamp) = (previous timestamp) + (the amount of bits) / (bitrate). Or is there a more efficient way to do it?
As for PTS and DTS timestamps, I read here that these timestamps can be non-linear. Is it will be correct to calculate it relative to the last original PCR that I received? I mean:
pts_new = (original_pts - last_original_pcr) + pcr_new.
dts_new = (original_dts - last_original_pcr) + pcr_new.
(original_pts - last_original_pcr) is the difference between pts and pcr values
pcr_new is the last modified pcr value
My program can read and edit these timestamps in mpeg-ts stream. Fortunately, there's a lot of literature on how to do it. But how do I calculate the new values for these timestamps?
I have just started to learn the specification of mpeg2-ts, and please correct my mistakes if I am wrong in something.
Thanks in advance. Hope you understood me.
Mpeg2 "Splicing" is an art-form, and is much more complicated than concatenating two streams. It requires manipulations which many companies have patented (http://www.google.com/patents/US6380991, http://www.google.com/patents/US6806909, http://www.google.com/patents/US6993081)
to answer some of your questions:
your calculation of the next pcr looks ok, although you need take into account many compliancy issues (etr290 for example)
for DTS/PTS you have much more work to do. the most basic splice will just restamp the ad's pts/dts in such a way that they continue from the last timestamp of the first TS.
ad first timestamp = last timestamp + frame interval
the trick lies within making sure that you have no "holes" in either the presentation time stamps or the decoding timestamps. This is the difficult part and requires deep understanding in MPEG2 buffers (tstd, eb, mb).
Good luck.
I'm working on a Qt GUI for visualizing 'live' data which is received via a TCP/IP connection. The issue is that the data is arriving rather quickly (a few dozen MB per second) - it's coming in faster than I'm able to visualize it even though I don't do any fancy visualization - I just show the data in a QTableView object.
As if that's not enough, the GUI also allows pressing a 'Freeze' button which will suspend updating the GUI (but it will keep receiving data in the background). As soon as the Freeze option was disabled, the data which has been accumulated in the background should be visualized.
What I'm wondering is: since the data is coming in so quickly, I can't possibly hold all of it in the memory. The customer might even keep the GUI running over night, so gigabytes of data will accumulate. What's a good data storage system for writing this data to disk? It should have the following properties:
It shouldn't be too much work to use it on a desktop system
It should be fast at appending new data at the end. I never need to touch previously written data anymore, so writing into anywhere but the end is not needed.
It should be possible to randomly access records in the data. This is because scrolling around in my GUI will make it necessary to quickly display the N to N+20 (or whatever the height of my table is) entries in the data stream.
The data which is coming in can be separated into records, but unfortunately the records don't have a fixed size. I'd rather not impose a maximum size on them (at least not if it's possible to get good performance without doing so).
Maybe some SQL database, or something like CouchDB? It would be great if somebody could share his experience with such scenarios.
I think that sqlite might do the trick. It seems to be fast. Unfortunately, I have no data flow like yours, but it works well as a backend for a log recorder. I have a GUI where you can view the n, n+k logs.
You can also try SOCI as a C++ database access API, it seems to work fine with sqlite (I have not used it for now but plan to).
my2c
I would recommend a simple file based solution.
If you can use fixed size records: If the you get the data continuously with constant sample rate, random access to data is easy and very fast when you know the time stamp of first data point and the sample rate. If the sample rate varies, then write time stamp with each data point. Now random access requires binary search, but it is still fast enough.
If you have variable size records: Write the variable size data to one file and to other file write indexes (which are fixed size) to the data file. And if the sample rate varies, write time stamps too. Now you can do the random access fast using the index file.
If you are using Qt to implement this kind of solution, you need two sets of QFile and QDataStream instances, one for writing and one for reading.
And a note about performance: don't flush the file after every data point write. But remember to flush the file before doing any random access to it.