Shannon Entropy - c++

Question

The following C++ code (as is) is from http://rosettacode.org/wiki/Entropy. There are mistakes - can anyone correct them?
#include <string>
#include <map>
#include <iostream>
#include <algorithm>
#include <cmath>
double log2( double number ) {
return log( number ) / log( 2 ) ;
}
int main( int argc , char *argv[ ] ) {
std::string teststring( argv[ 1 ] ) ;
std::map<char , int> frequencies ;
for ( char c : teststring )
frequencies[ c ] ++ ;
int numlen = teststring.length( ) ;
double infocontent = 0 ;
for ( std::pair<char , int> p : frequencies ) {
double freq = static_cast<double>( p.second ) / numlen ;
infocontent += freq * log2( freq ) ;
}
infocontent *= -1 ;
std::cout << "The information content of " << teststring
<< " is " << infocontent << " !\n" ;
return 0 ;
}
The first error seems to be fixed with:
double log2( double n )
{
// log(n)/log(2) is log2.
return log( n ) / log( 2. );
}
I'm unsure what they are trying to say with:
for ( char c : teststring )

Answer 1

This one work nice
template <typename T> static float ShannonEntropy(T data[],int elements){
float entropy=0;
std::map<T,long> counts;
typename std::map<T,long>::iterator it;
//
for (int dataIndex = 0; dataIndex < elements; ++dataIndex) {
counts[data[dataIndex]]++;
}
//
it = counts.begin();
while(it != counts.end()){
float p_x = (float)it->second/elements;
if (p_x>0) entropy-=p_x*log(p_x)/log(2);
it++;
}
return entropy;
}

Answer 2

The loop is a foreach loop. It means: For each character in teststring, put it into the variable c and do the loop body.
The same can be expressed with a regular for-loop and an index variable, but this way is much shorter and easier to read. Other languages like C# and Java had this for a long time, but C++ only had some template feature in the STL that were not as readable.
This is a new feature of C++ 11 and your compiler will complain if it's not compatible. If you get errors, try to get a better compiler.

Answer 3

Here is my java code for entropy of image
public static double getShannonEntropy_Image(BufferedImage actualImage){
List<String> values= new ArrayList<String>();
int n = 0;
Map<Integer, Integer> occ = new HashMap<>();
for(int i=0;i<actualImage.getHeight();i++){
for(int j=0;j<actualImage.getWidth();j++){
int pixel = actualImage.getRGB(j, i);
int alpha = (pixel >> 24) & 0xff;
int red = (pixel >> 16) & 0xff;
int green = (pixel >> 8) & 0xff;
int blue = (pixel) & 0xff;
//0.2989 * R + 0.5870 * G + 0.1140 * B greyscale conversion
//System.out.println("i="+i+" j="+j+" argb: " + alpha + ", " + red + ", " + green + ", " + blue);
int d= (int)Math.round(0.2989 * red + 0.5870 * green + 0.1140 * blue);
if(!values.contains(String.valueOf(d)))
values.add(String.valueOf(d));
if (occ.containsKey(d)) {
occ.put(d, occ.get(d) + 1);
} else {
occ.put(d, 1);
}
++n;
}
}
double e = 0.0;
for (Map.Entry<Integer, Integer> entry : occ.entrySet()) {
int cx = entry.getKey();
double p = (double) entry.getValue() / n;
e += p * log2(p);
}
return -e;
}

Answer 4

The first error is because of the fiasco regarding names from the C library. It's not specified which overloads of log are dumped into the global namespace; presumably, the author's implementation only had one, so that log(2) is unambiguous, but yours has all of them, giving an ambiguity since there isn't one that takes type int. For portability, it should be std::log(2.). Better still, use std::log2 rather than reinventing it.
The second is a "range-based for statement", introduced to the language in 2011, which iterates over each element of an array, container, or other sequence. You'll need a reasonably modern compiler, and you may need to specifically enable C++11 support. For example, with GCC, you need the command-line argument -std=c++11 (or c++0x with older versions).

Answer 5

The above generic answer is very good, but for the particular case of strings, this one is simpler:
#include <cmath>
#include <string>
float shannon_entropy(const std::string & s)
{
int counts[256] = {};
for (unsigned char c: s)
{
counts[c]++;
}
float entropy = 0;
float length = (float)s.size();
for (int count: counts)
{
if (count == 0)
continue;
float p = (float)count / length;
entropy -= p * std::log2f(p);
}
return entropy;
}