Write multimap fo file Fast way c++ - c++

Question

I am using the following code to cound the frequency of words :
// Program for Owen written by Briana Morrison
//#pragma warning (disable : 4786)
#include <stdio.h>
#include <iostream>
#include <fstream>
#include <string>
#include <map>
#include <algorithm>
#include <vector>
using namespace std;
// program assumes that the filename is the only thing passed into program
// if you are using standard argc and argv, then arguments to main should change, and uncomment
// first line.
int main(int argc, char * argv[])
{
string filename(argv[1]);
// string filename;
//cout << "Enter filename" << endl;
//cin >> filename;
ifstream infile(filename.c_str());
//ifstream infile("poe.txt");
string word;
bool debug = false; // for debugging purposes
int count = 0; // count of words for debugging
// create a map of words to frequencies
map<string, int, less<string> > words;
// create a multimap of frequencies to words
multimap<int, string, greater<int> > freq;
// loop while there is input in the file
infile >> word; //priming read
while (infile)
{
count++;
// convert word to lowercase
for (int i = 0; i < word.length(); i++)
if ('A' <= word[i] && word[i] <= 'Z')
word[i] = tolower(word[i]);
if (debug) cout << word << endl;
// if word not found, add to map, otherwise increment count
if (words.find(word) != words.end())
{
words[word]++;
if (debug) cout << word << " found and count incremented to " << words[word] << endl;
}
else
{
words[word] = 1;
if (debug) cout << word << " not found and count incremented to " << words[word] << endl;
}
infile >> word;
}
if (debug) cout << "count is " << count << " and map has " << words.size() << endl;
// now go through map and add everything to multimap...words still in alphabetical order
map<string, int, less<string> >::iterator it = words.begin();
for (it = words.begin(); it != words.end(); it++)
{
pair<int, string> p(it->second, it->first);
freq.insert(p);
}
if (debug) cout << "map has " << words.size() << " and multimap has " << freq.size() << endl;
ofstream outfile("myout.txt");
multimap<int, string, greater<int> >::iterator myit=freq.begin();
for (myit = freq.begin(); myit != freq.end(); myit++)
{
outfile << myit->first << "\t" << myit->second << endl;
}
outfile.close();
return 0;
}
The problem is not in here I think
When i am writing the words to a file, it gets slower by every iteration why?
ofstream outfile("myout.txt");
multimap<int, string, greater<int> >::iterator myit=freq.begin();
for (myit = freq.begin(); myit != freq.end(); myit++)
{
outfil<< myit->first << "\t" << myit->second << endl;
}
outfile.close();
How can I write the multimap in a fast way to a file?

Answer 1

You can use '\n' instead of std::endl to avoid flushing it for every line.
outfil << myit->first << '\t' << myit->second << '\n';

Answer 2

for (myit = freq.begin(); myit != freq.end(); ++myit)
{
outfil<< myit->first << "\t" << myit->second << "\n";
}
this should be faster.
Or you can buffer the data and write them all at once, not line by line.

Answer 3

I don't see why your loop should get slower with every single iteration, but note that you're using formatted output (which is what operator<< does), which is notoriously slow. In case your strings don't contain null bytes you could make your code more efficient by writing the std::string via ostream::write i.e.
outfil << myit->first;
outfil.write( "\t", 1 );
outfil.write( myit->second.c_str(), myit->second.size() );
outfil.write( "\n", 1 );