I have a boost dynamic_bitset that I am trying to extract the set bits from:
boost::dynamic_bitset<unsigned long> myBitset(1000);
My first thought was to do a simple 'dump' loop through each index and ask if it was set:
for(size_t index = 0 ; index < 1000 ; ++index)
{
if(myBitset.test(index))
{
/* do something */
}
}
But then I saw two interesting methods, find_first() and find_next() that I thought for sure were meant for this purpose:
size_t index = myBitset.find_first();
while(index != boost::dynamic_bitset::npos)
{
/* do something */
index = myBitset.find_next(index);
}
I ran some tests and it seems like the second method is more efficient, but this concerns me that there might be another 'more correct' way to perform this iteration. I wasn't able to find any examples or notes in the documentation indicating the correct way to iterate over the set bits.
So, is using find_first() and find_next() the best way to iterate over a dynamic_bitset, or is there another way?
find_first and find_next are the fastest way. The reason is that these can skip over an entire block (of dynamic_bitset::bits_per_block bits, probably 32 or 64) if none of them are set.
Note that dynamic_bitset does not have iterators, so it will behave a bit un-C++'ish no matter what.
Depends on your definition of more correct. A correct method probably must yield correct results on all valid inputs and be fast enough.
find_first and find_next are there so that they can be optimized to scan entire blocks of bits in one comparison. If a block is, say, an unsigned long of 64 bits, one block comparison analyses 64 bits at once, where a straightforward loop like you posted would do 64 iterations for that.
Related
I am looking for a C++ bitset implementation that can answer if a bit is set in a range. std::bitset, vector, and boost::dynamic_bitset all give access to individual bits that I can loop over, but that isn't the most efficient way to query a range of bits to ask if any bit is set- I don't even need to know which one.
bitset b;
if(b.any(33, 199))
{
// ...
}
Is there a library that provides this? I would like to run some benchmarks against other implementations (including one I may have to write), but I can't find any that appear to implement this functionality.
Unfortunately in C++11 bitset it is not possible to set a range of bits to the given value by just specifying the boundaries of the range. Iterating over individual bits seems the most we can do. It is also not possible to check if all bits inside the range are set to the same value (1,0).
There is an open source project in Git that provides an alternative implementation of the BitSet (RangedBitset) that supports these operations. It uses an array of uint_64t_ words of any size internally, but can also handle ranges specified with the accuracy of the single bit. There you can do the things like
a.set(4, 8, true); // set the range [ 4 .. 8 [ to true
bool is_all_range = a.check(2, 6, true); // check if all range is set to 1.
To check if some bit is set in the range [x,y] in bitset, you can use bs._Find_next(x-1). It returns the next set bit after the position x-1.
Then you can check if the returned value is <=y or not.
bool Find_if_bitset_has_any_set_bit_in_range(bitset<M> &bs, int x, int y){
if(bs._Find_next(x-1)<=y) return 1; //TRUE
return 0; //FALSE
}
C++11's bitset provides the any() method that you are after, but if that isn't an option then just use b.to_ulong() and check for non zero.
I think this is not answered on this site yet.
I made a code which goes through many combinations of 4 numbers. The number values are from 0 to 51, so they can be stored in 6 bits, so in 1 byte, am I right? I use these 4 numbers in nested for cycles and then use them in the lowest level for cycle. So what c++ type from those which can store at least 52 values is the fastest for iterating through 4 nested for cycles?
The code looks like:
for(type first = 0; first != 49; ++first)
for(type second = first+1; second != 50; ++second)
for(type third = second+1; third != 51; ++third)
for(type fourth = third+1; fourth != 52; ++fourth) {
//using those values for about 1 bilion bit operations made in another for cycles
}
That code is very simplified and maybe there is also a better way for this kind of iterating, you can help me also with that.
Use the typedef std::uint_fast8_t from the header <cstdint>. It is supposed to be the "fastest" unsigned integer type with at least 8 bits.
The fastest is whatever the underlying processor ALU can natively work with. Now registers may be addressable in multiple formats. In that case all those formats are equally fast.
So this becomes very processor architecture specific rather than C++ specific.
If you are working on a modern day PC processor then an int is as fast as anything else for your for loops.
On an embedded system there are more things to consider. Eg. Whether the variable is stored in an aligned location or not?
On most machines, int is the fastest integer type. On all of the computers I work with, int is faster than unsigned, significantly faster than signed char.
Another issue, perhaps a bigger one, is what you are doing with those numbers. You didn't show the code, so there's no way of telling. Use int if you expect first*second to produce the expected integral value.
Yet another issue is how widely portable you expect this code to be. There's a huge distinction between code that will be ported to a number of different architectures, different compilers versus code that will be used in a limited and controlled setting. If it's the latter, write some benchmarks, and use the type under which the benchmarks perform best. The problem is a bit tougher if you are writing something for wide consumption.
Is there some reasonably fast code out there which can help me quickly search a large bitmap (a few megabytes) for runs of contiguous zero or one bits?
By "reasonably fast" I mean something that can take advantage of the machine word size and compare entire words at once, instead of doing bit-by-bit analysis which is horrifically slow (such as one does with vector<bool>).
It's very useful for e.g. searching the bitmap of a volume for free space (for defragmentation, etc.).
Windows has an RTL_BITMAP data structure one can use along with its APIs.
But I needed the code for this sometime ago, and so I wrote it here (warning, it's a little ugly):
https://gist.github.com/3206128
I have only partially tested it, so it might still have bugs (especially on reverse). But a recent version (only slightly different from this one) seemed to be usable for me, so it's worth a try.
The fundamental operation for the entire thing is being able to -- quickly -- find the length of a run of bits:
long long GetRunLength(
const void *const pBitmap, unsigned long long nBitmapBits,
long long startInclusive, long long endExclusive,
const bool reverse, /*out*/ bool *pBit);
Everything else should be easy to build upon this, given its versatility.
I tried to include some SSE code, but it didn't noticeably improve the performance. However, in general, the code is many times faster than doing bit-by-bit analysis, so I think it might be useful.
It should be easy to test if you can get a hold of vector<bool>'s buffer somehow -- and if you're on Visual C++, then there's a function I included which does that for you. If you find bugs, feel free to let me know.
I can't figure how to do well directly on memory words, so I've made up a quick solution which is working on bytes; for convenience, let's sketch the algorithm for counting contiguous ones:
Construct two tables of size 256 where you will write for each number between 0 and 255, the number of trailing 1's at the beginning and at the end of the byte. For example, for the number 167 (10100111 in binary), put 1 in the first table and 3 in the second table. Let's call the first table BBeg and the second table BEnd. Then, for each byte b, two cases: if it is 255, add 8 to your current sum of your current contiguous set of ones, and you are in a region of ones. Else, you end a region with BBeg[b] bits and begin a new one with BEnd[b] bits.
Depending on what information you want, you can adapt this algorithm (this is a reason why I don't put here any code, I don't know what output you want).
A flaw is that it does not count (small) contiguous set of ones inside one byte ...
Beside this algorithm, a friend tells me that if it is for disk compression, just look for bytes different from 0 (empty disk area) and 255 (full disk area). It is a quick heuristic to build a map of what blocks you have to compress. Maybe it is beyond the scope of this topic ...
Sounds like this might be useful:
http://www.aggregate.org/MAGIC/#Population%20Count%20%28Ones%20Count%29
and
http://www.aggregate.org/MAGIC/#Leading%20Zero%20Count
You don't say if you wanted to do some sort of RLE or to simply count in-bytes zeros and one bits (like 0b1001 should return 1x1 2x0 1x1).
A look up table plus SWAR algorithm for fast check might gives you that information easily.
A bit like this:
byte lut[0x10000] = { /* see below */ };
for (uint * word = words; word < words + bitmapSize; word++) {
if (word == 0 || word == (uint)-1) // Fast bailout
{
// Do what you want if all 0 or all 1
}
byte hiVal = lut[*word >> 16], loVal = lut[*word & 0xFFFF];
// Do what you want with hiVal and loVal
The LUT will have to be constructed depending on your intended algorithm. If you want to count the number of contiguous 0 and 1 in the word, you'll built it like this:
for (int i = 0; i < sizeof(lut); i++)
lut[i] = countContiguousZero(i); // Or countContiguousOne(i)
// The implementation of countContiguousZero can be slow, you don't care
// The result of the function should return the largest number of contiguous zero (0 to 15, using the 4 low bits of the byte, and might return the position of the run in the 4 high bits of the byte
// Since you've already dismissed word = 0, you don't need the 16 contiguous zero case.
I'm trying to create a simple DBMS and although I've read a lot about it and have already designed the system, I have some issues about the implementation.
I need to know what's the best method in C++ to use a series of bits whose length will be dynamic. This series of bits will be saved in order to figure out which pages in the files are free and not free. For a single file the number of pages used will be fixed, so I can probably use a bitset for that. However the number of records per page AND file will not be fixed. So I don't think bitset would be the best way to do this.
I thought maybe to just use a sequence of characters, since each character is 1 byte = 8 bits maybe if I use an array of them I would be able to create the bit map that I want.
I never had to manipulate bits at such a low level, so I don't really know if there is some other better method to do this, or even if this method would work at all.
thanks in advance
If you are just wanting the basics on the bit twiddling, the following is one way of doing it using an array of characters.
Assume you have an array for the bits (the length needs to be (totalitems / 8 )):
unsigned char *bits; // this of course needs to be allocated somewhere
You can compute the index into the array and the specific bit within that position as follows:
// compute array position
int pos = item / 8; // 8 bits per byte
// compute the bit within the byte. Could use "item & 7" for the same
// result, however modern compilers will typically already make
// that optimization.
int bit = item % 8;
And then you can check if a bit is set with the following (assumes zero-based indexing):
if ( bits[pos] & ( 1 << bit ))
return 1; // it is set
else
return 0; // it is not set
The following will set a specific bit:
bits[pos] |= ( 1 << bit );
And the following can be used to clear a specific bit:
bits[pos] &= ~( 1 << bit );
I would implement a wrapper class and simply store your bitmap in a linked list of chunks where each chunk would hold a fixed size array (I would use a stdint type like uint32_t to ensure a given number of bits) then you simply add links to your list to expand. I'll leave contracting as an exercise to the reader.
I need to keep track of n samples. The information I am keeping track of is of boolean type, i.e. something is true or false. As soon as I am on sample n+1, i basically want to ignore the oldest sample and record information about the newest one.
So say I keep track of samples, I may have something like
OLDEST 0 0 1 1 0 NEWEST
If the next sample is 1, this will become
OLDEST 0 1 1 0 1 NEWEST
if the next one is 0, this will become...
OLDEST 1 1 0 1 0 NEWEST
So what is the best way to implement this in terms of simplicity and memory?
Some ideas I had:
Vector of bool (this would require shifting elements so seems expensive)
Storing it as bits...and using bit shifting (memorywise --cheap? but is there a limit on the number of samples?)
Linked lists? (might be an overkill for the task)
Thanks for the ideas and suggestions :)
You want a set of bits. Maybe you can look into a std::bitset
http://www.sgi.com/tech/stl/bitset.html
Very straightfoward to use, optimal memory consumption and probably the best performance
The only limitation is that you need to know at compile-time the value of n. If you want to set it on runtime, have a look at boost http://www.boost.org/doc/libs/1_36_0/libs/dynamic_bitset/dynamic_bitset.html
Sounds like a perfect use of a ring buffer. Unfortunately there isn't one in the standard library, but you could use boost.
Alternately roll your own using a fixed-length std::list and splice the head node to the tail when you need to overwrite an old element.
It really depends on how many samples you want to keep.
vector<bool> could be a valid option; I would expect an
erase() on the first element to be reasonably efficient.
Otherwise, there's deque<bool>. If you know how many elements
you want to keep at compile time, bitset<N> is probably better
than either.
In any case, you'll have to wrap the standard container in some
additional logic; none have the actual logic you need (that of
a ring buffer).
If you only need 8 bits... then use a char and do logical shifts "<<, >>" and do a mask to look at the one you need.
16 Bits - short
32 Bits - int
64 Bits - long
etc...
Example:
Oldest 00110010 Newest -> Oldest 1001100101 Newest
Done by:
char c = 0x32; // 50 decimal or 00110010 in binary
c<<1; // Logical shift left once.
c++; // Add one, sense LSB is the newest.
//Now look at the 3rd newest bit
print("The 3rd newest bit is: %d\n", (c & 0x4));
Simple and EXTREMELY cheap on resources. Will be VERY VERY high performance.
From your question, it's not clear what you intend to do with the samples. If all you care about is storing the N most recent samples, you could try the following. I'll do it for "chars" and let you figure out how to optimize for "bool" should you need that.
char buffer[N];
int samples = 0;
void record_sample( char value )
{
buffer[samples%N] = value;
samples = samples + 1;
}
Once you've stored N samples (once you've called record_sample N times) you can read the oldest and newest samples like so:
char oldest_sample()
{
return buffer[samples%N];
}
char newest_sample()
{
return buffer[(samples+N-1)%N];
}
Things get a little trickier if you intend to read the oldest sample before you've already stored N samples - but not that much trickier. For that, you want a "ring buffer" which you can find in boost and on wikipedia.