putting a list (.txt) file into a 2D array - c++

I'm trying to separate a text file (which has a list of 200 strings) and store each other string (even number and odd number in the list) into a 2D Array.
The text file is ordered in this way (without the numbers):
Alabama
Brighton
Arkansas
Bermuda
Averton
Burmingham
I would like to store it in a 2 dimensional array called strLine[101][2] iterating throughout so the first string in the list is in location [0][0] and the second string of the list is in location [0][1], etc until the file finishes reading and the list becomes organized like this (without the numbers):
Alabama | Brighton
Arkansas | Bermuda
Avertinon | Burmingham
My code outputs the original unsorted list at the moment, i would like to know how to implement the 2d array (with correct syntax) and how to implement an i, j for-loop in the getline() function so it can iterate through each element of the 2D array.
Any help would be greatly appreciated.
My code:
bool LoadListBox()
{
// Declarations
ifstream fInput; // file handle
string strLine[201]; // array of string to hold file data
int index = 0; // index of StrLine array
TCHAR szOutput[50]; // output to listbox,
50 char TCHAR
// File Open Process
fInput.open("data.txt"); // opens the file for read only
if (fInput.is_open())
{
getline( // read a line from the file
fInput, // handle of file to read
strLine[index]); // storage destination and index iterator
while (fInput.good()) // while loop for open file
{
getline( // read line from data file
fInput, // file handle to read
strLine[index++]); // storage destination
}
fInput.close(); // close the file
index = 0; // resets back to start of string
while (strLine[index] != "") // while loop for string not void
{
size_t pReturnValue; // return code for mbstowcs_s
mbstowcs_s( // converts string to TCHAR
&pReturnValue, // return value
szOutput, // destination of the TCHAR
50, // size of the destination TCHAR
strLine[index].c_str(), // source of string as char
50); // max # of chars to copy
SendMessage( // message to a control
hWnd_ListBox, // handle to listbox
LB_ADDSTRING, // append string to listbox
NULL, // window parameter not used
LPARAM(szOutput)); // TCHAR to add
index++; // next element of string array
}
return true; // file loaded okay
}
return false; // file did not load okay
}

Step 1
Transform string strLine[201]; to string place[100][2];. Also consider making a
struct place
{
std::string state;
std::string city;
};
because it is a bit more explicit what exactly is being stored. More expressive code is easier to read, generally prevents mistakes (harder to accidentally use strLine[x][2] or something like that), and requires less commenting. Code that comments itself should be a personal goal. The compiler doesn't care, of course, but few people are compilers.
Step 2
Use two separate index variables. Name the first something like num_entries because what it's really doing is counting the number of items in the array.
Step 3
Read two lines into the inner array and test the result of the reads. If they read successfully, increment the index.
while (getline(fInput, place[num_entries][0]) && getline(fInput, place[num_entries][1]))
{
num_entries++;
}
Step 4 (optional clean-up)
Step 2 turns while (strLine[index] != "") into while (index < num_entries)
Replace all of the 50s with a constant. That way you can't change the value and miss a few 50s AND it's easier to infer meaning from a good, descriptive identifier than a raw number.

Related

C++ storing 0 and 1 more efficiently, like in a binary file?

I want to store multiple arrays which all entries consist of either 0 or 1.
This file would be quite large if i do it the way i do it.
I made a minimalist version of what i currently do.
#include <iostream>
#include <fstream>
using namespace std;
int main(){
ofstream File;
File.open("test.csv");
int array[4]={1,0,0,1};
for(int i = 0; i < 4; ++i){
File << array[i] << endl;
}
File.close();
return 0;
}
So basically is there a way of storing this in a binary file or something, since my data is 0 or 1 in the first place anyways?
If yes, how to do this? Can i also still have line-breaks and maybe even commas in that file? If either of the latter does not work, that's also fine. Just more importantly, how to store this as a binary file which has only 0 and 1 so my file is smaller.
Thank you very much!
So basically is there a way of storing this in a binary file or something, since my data is 0 or 1 in the first place anyways? If yes, how to do this? Can i also still have line-breaks and maybe even commas in that file? If either of the latter does not work, that's also fine. Just more importantly, how to store this as a binary file which has only 0 and 1 so my file is smaller.
The obvious solution is to take 64 characters, say A-Z, a-z, 0-9, and + and /, and have each character code for six entries in your table. There is, in fact, a standard for this called Base64. In Base64, A encodes 0,0,0,0,0,0 while / encodes 1,1,1,1,1,1. Each combination of six zeroes or ones has a corresponding character.
This still leaves commas, spaces, and newlines free for your use as separators.
If you want to store the data as compactly as possible, I'd recommend storing it as binary data, where each bit in the binary file represents one boolean value. This will allow you to store 8 boolean values for each byte of disk space you use up.
If you want to store arrays whose lengths are not multiples of 8, it gets a little bit more complicated since you can't store a partial byte, but you can solve that problem by storing an extra byte of meta-data at the end of the file that specifies how many bits of the final data-byte are valid and how many are just padding.
Something like this:
#include <iostream>
#include <fstream>
#include <cstdint>
#include <vector>
using namespace std;
// Given an array of ints that are either 1 or 0, returns a packed-array
// of uint8_t's containing those bits as compactly as possible.
vector<uint8_t> packBits(const int * array, size_t arraySize)
{
const size_t vectorSize = ((arraySize+7)/8)+1; // round up, then +1 for the metadata byte
vector<uint8_t> packedBits;
packedBits.resize(vectorSize, 0);
// Store 8 boolean-bits into each byte of (packedBits)
for (size_t i=0; i<arraySize; i++)
{
if (array[i] != 0) packedBits[i/8] |= (1<<(i%8));
}
// The last byte in the array is special; it holds the number of
// valid bits that we stored to the byte just before it.
// That way if the number of bits we saved isn't an even multiple of 8,
// we can use this value later on to calculate exactly how many bits we should restore
packedBits[vectorSize-1] = arraySize%8;
return packedBits;
}
// Given a packed-bits vector (i.e. as previously returned by packBits()),
// returns the vector-of-integers that was passed to the packBits() call.
vector<int> unpackBits(const vector<uint8_t> & packedBits)
{
vector<int> ret;
if (packedBits.size() < 2) return ret;
const size_t validBitsInLastByte = packedBits[packedBits.size()-1]%8;
const size_t numValidBits = 8*(packedBits.size()-((validBitsInLastByte>0)?2:1)) + validBitsInLastByte;
ret.resize(numValidBits);
for (size_t i=0; i<numValidBits; i++)
{
ret[i] = (packedBits[i/8] & (1<<(i%8))) ? 1 : 0;
}
return ret;
}
// Returns the size of the specified file in bytes, or -1 on failure
static ssize_t getFileSize(ifstream & inFile)
{
if (inFile.is_open() == false) return -1;
const streampos origPos = inFile.tellg(); // record current seek-position
inFile.seekg(0, ios::end); // seek to the end of the file
const ssize_t fileSize = inFile.tellg(); // record current seek-position
inFile.seekg(origPos); // so we won't change the file's read-position as a side effect
return fileSize;
}
int main(){
// Example of packing an array-of-ints into packed-bits form and saving it
// to a binary file
{
const int array[]={0,0,1,1,1,1,1,0,1,0};
// Pack the int-array into packed-bits format
const vector<uint8_t> packedBits = packBits(array, sizeof(array)/sizeof(array[0]));
// Write the packed-bits to a binary file
ofstream outFile;
outFile.open("test.bin", ios::binary);
outFile.write(reinterpret_cast<const char *>(&packedBits[0]), packedBits.size());
outFile.close();
}
// Now we'll read the binary file back in, unpack the bits to a vector<int>,
// and print out the contents of the vector.
{
// open the file for reading
ifstream inFile;
inFile.open("test.bin", ios::binary);
const ssize_t fileSizeBytes = getFileSize(inFile);
if (fileSizeBytes < 0)
{
cerr << "Couldn't read test.bin, aborting" << endl;
return 10;
}
// Read in the packed-binary data
vector<uint8_t> packedBits;
packedBits.resize(fileSizeBytes);
inFile.read(reinterpret_cast<char *>(&packedBits[0]), fileSizeBytes);
// Expand the packed-binary data back out to one-int-per-boolean
vector<int> unpackedInts = unpackBits(packedBits);
// Print out the int-array's contents
cout << "Loaded-from-disk unpackedInts vector is " << unpackedInts.size() << " items long:" << endl;
for (size_t i=0; i<unpackedInts.size(); i++) cout << unpackedInts[i] << " ";
cout << endl;
}
return 0;
}
(You could probably make the file even more compact than that by running zip or gzip on the file after you write it out :) )
You can indeed write and read binary data. However having line breaks and commas would be difficult. Imagine you save your data as boolean data, so only ones and zeros. Then having a comma would mean you need an special character, but you have only ones and zeros!. The next best thing would be to make an object of two booleans, one meaning the usual data you need (c++ would then read the data in pairs of bits), and the other meaning whether you have a comma or not, but I doubt this is what you need. If you want to do something like a csv, then it would be easy to just fix the size of each column (int would be 4 bytes, a string of no more than 32 char for example), and then just read and write accordingly. Suppose you have your binary
To initially save your array of the an object say pets, then you would use
FILE *apFile;
apFile = fopen(FILENAME,"w+");
fwrite(ARRAY_OF_PETS, sizeof(Pet),SIZE_OF_ARRAY, apFile);
fclose(apFile);
To access your idx pet, you would use
Pet m;
ifstream input_file (FILENAME, ios::in|ios::binary|ios::ate);
input_file.seekg (sizeof(Pet) * idx, ios::beg);
input_file.read((char*) &m,sizeof(Pet));
input_file.close();
You can also add data add the end, change data in the middle and so on.

Arduino function can't edit array

Background
I've written an Arduino function which receives strings in the format "a,b,c,d,e" over a serial connection, where a,b,c,d,e are integers, and I'm trying to update an array with these integers every time a new string is received. The data is received and parsed into individual integers fine, but the array won't update properly.
Attempt
Below is the code, I've left the getData() function out as all it does is receive the string from the serial connection and store it in an array of characters input (that part is working fine).
void setup() {
Serial.begin(9600);
}
void loop() {
getData();
if(parsed == false){
parseData(readings);
}
}
void parseData(int readings[]) {
x = 0;
char * split;
split = strtok(input,",");
while (split != NULL)
{
readings[x] = split;
split = strtok (NULL, ",");
x++;
}
parsed = true;
}
Problem
If I send a string like "6,7,8,9,0", the array readings[] is updated to [289,291,293,295,297] no matter what values I send, I have checked what values split takes inside the function and they are correct, however the line readings[x] = split; fails to update the array elements to anything other than those 5 numbers in that order. This is the case when the value of readings[n] is checked inside or outside the parseData function.
Also, if I send fewer than 5 integers in the string, e.g. a,b,c, only the first array elements will change and the others will remain at 0, e.g. [289,291,293,0,0]
Before I found out about passing array pointers to functions, the exact same thing was happening with slightly different code - when I called the function in the loop, I just used parseData();, and when I defined the function I just used void parseData(){
Question
Why isn't the array updating properly and how can I fix it?
Your readings is an array of integers.
And split is a pointer-to-char.
The statement
readings[x] = split
stores split, which is the address of a character, as an integer value.
If I write
const char *pointer = "42";
int address = pointer;
I am not storing the integer value 42 in address - I'm storing the number identifying the memory location of the first character in my string.
If you want to convert a string into an integer, you need to parse it with a function like strtol.

What is the best solution for writing numbers into file and than read them?

I have 640*480 numbers. I need to write them into a file. I will need to read them later. What is the best solution? Numbers are between 0 - 255.
For me the best solution is to write them binary(8 bits). I wrote the numbers into txt file and now it looks like 1011111010111110 ..... So there are no questions where the number starts and ends.
How am I supposed to read them from the file?
Using c++
It's not good idea to write bit values like 1 and 0 to text file. The file size will bigger in 8 times. 1 byte = 8 bits. You have to store bytes, 0-255 - is byte. So your file will have size 640*480 bytes instead of 640*480*8. Every symbol in text file has size of 1 byte minimum. If you want to get bits, use binary operators of programming language that you use. To read bytes much easier. Use binary file for saving your data.
Presumably you have some sort of data structure representing your image, which somewhere inside holds the actual data:
class pixmap
{
public:
// stuff...
private:
std::unique_ptr<std::uint8_t[]> data;
};
So you can add a new constructor which takes a filename and reads bytes from that file:
pixmap(const std::string& filename)
{
constexpr int SIZE = 640 * 480;
// Open an input file stream and set it to throw exceptions:
std::ifstream file;
file.exceptions(std::ios_base::badbit | std::ios_base::failbit);
file.open(filename.c_str());
// Create a unique ptr to hold the data: this will be cleaned up
// automatically if file reading throws
std::unique_ptr<std::uint8_t[]> temp(new std::uint8_t[SIZE]);
// Read SIZE bytes from the file
file.read(reinterpret_cast<char*>(temp.get()), SIZE);
// If we get to here, the read worked, so we move the temp data we've just read
// into where we'd like it
data = std::move(temp); // or std::swap(data, temp) if you prefer
}
I realise I've assumed some implementation details here (you might not be using a std::unique_ptr to store the underlying image data, though you probably should be) but hopefully this is enough to get you started.
You can print the number between 0-255 as the char value in the file.
See the below code. in this example I am printing integer 70 as char.
So this result in print as 'F' on the console.
Similarly you can read it as char and then convert this char to integer.
#include <stdio.h>
int main()
{
int i = 70;
char dig = (char)i;
printf("%c", dig);
return 0;
}
This way you can restrict the file size.

Best way to store string of known maximum length in file for fast load into vector<string> in C++

I've got big amount of text data which I need to save to file for next reprocessing. These data are stored in table like vector< vector< string > > - every record (vector) has same number of attributes(vector). So, going through the vector I can find the max length of every attribute in table and count of records. Now I have to write these data to file (can be binary) in that way that I will be able to load them back into vector< vector< string > > very fast. It doesn't matter how much time will writing take but I need reading to vector implement in the fastest way.
Due to fact that data will be processed "record by record" the whole file may not will be load to memory. But for fast reading I want to use buffer 256 MB or 512 MB.
So for now I implemented this in this way:
Data are stored in two files - description file and data file. Description file contains the count of records, count of attributes and maximum length of every attribute. Data file is binary file of chars. There are no values or records separators, just values. Every value in concrete attribute has same length so if some value has smaller length than maximum length, the remaining chars are null characters '\0'.
Then I read chunk of file to char array buffer (256 MB or 512 MB) with std::fread. When application calls function vector getNext(), I read the chunk of chars from buffer (because I know length of every attribute) and append every char to concrete string to create vector.
But, this way seems not so fast for my purpose when I need parse big count of records in loop from buffer to vector. Is another better way to do whole this problem?
This part of code is parsing chars from buffer to values:
string value;
vector<string> record;
int pos = bfrIndex(); // returns current position in buffer. position of values of next record
for(unsigned int i = 0; i < d.colSize.size(); i++) { // d.colSize is vector of every attribute
value.clear();
value.reserve(d.colSize[i]);
for(unsigned int j = pos; j < pos + d.colSize[i]; j++) {
if (buffer[j] == '\0') break;
value += buffer[j];
}
record.push_back(value);
pos += d.colSize[i]; // set position in buffer to next value
}
return record;
I'd consider a binary approach that used the method employed in Doom's .wad files. I.e a directory with length & file offsets of each resource, followed by the resources themselves. With a small amount of overhead for the directory, you get instant knowledge of both where to find each string and how long they each are.
vector<vector<string> > is a 3d character "cube" where every dimension vary in size along the others. Unless you are able to predict each "size", you risk to read one-by one and reallocate every time.
Fast reading happens when you can "load up" the data all in once, and than define how to split. The data structure will probably be a single string, and a vector<vector<range> > where range is a std::pair<std::string::const_iterator>.
The problem -here- is that you cannot manipulate the strings being them tightened together.
A second chance is maintain the dynamic nature of vector<vector<string> >,but store the dataso that each "size" can be read before the data tehnselves, so that you can resize the vectos and then read the content into its componets.
In pseudocode:
template<class Stream, class Container>
void save(const Container& c, const stream& s)
{ s.write(c.size()); for(auto& e: c) save(e,s) }
template<class Stream, class Container>
void load(Container& c, const stream& s)
{
int sz=0; s.read(c.size()); c.resize(sz);
for(auto& i:c) load(i,s);
}
Of course, specialized for string-s so that saving/loading a string actually writes/reads its own chars.

Newline character in Text Document?

I wrote a pretty simple function that reads in possible player names and stores them in a map for later use. Basically in the file, each line is a new possible player name, but for some reason it seems like all but the last name has some invisible new line character after it. My print out is showing it like this...
nameLine = Georgio
Name: Georgio
0
nameLine = TestPlayer
Name: TestPlayer 0
Here is the actual code. I assume I need to be stripping something out but I am not sure what I need to be checking for.
bool PlayerManager::ParsePlayerNames()
{
FileHandle_t file;
file = filesystem->Open("names.txt", "r", "MOD");
if(file)
{
int size = filesystem->Size(file);
char *line = new char[size + 1];
while(!filesystem->EndOfFile(file))
{
char *nameLine = filesystem->ReadLine(line, size, file);
if(strcmp(nameLine, "") != 0)
{
Msg("nameLine = %s\n", nameLine);
g_PlayerNames.insert(std::pair<char*, int>(nameLine, 0));
}
for(std::map<char*,int>::iterator it = g_PlayerNames.begin(); it != g_PlayerNames.end(); ++it)
{
Msg("Name: %s %d\n", it->first, it->second);
}
}
return true;
}
Msg("[PlayerManager] Failed to find the Player Names File (names.txt)\n");
filesystem->Close(file);
return false;
}
You really need to consider using iostreams and std::string. The above code is SO much more simpler if you used the C++ constructs available to you.
Problems with your code:
why do you allocate a buffer for a single line which is the size of the file?
You don't clean up this buffer!
How does ReadLine fill the line buffer?
presumably nameLine points to the begining of the line buffer, if so, given in the std::map, the key is a pointer (char*) rather than a string as you were expecting, and the pointer is the same! If different (i.e. somehow you read a line and then move the pointer along for each name, then std::map will contain an entry per player, however you'll not be able to find an entry by player name as the comparison will be a pointer comparison rather than a string comparison as you are expecting!
I suggest that you look at implementing this using iostreams, here is some example code (without any testing)
ifstream fin("names.txt");
std::string line;
while (fin.good())
{
std::getline(fin, line); // automatically drops the new line character!
if (!line.empty())
{
g_PlayerNames.insert(std::pair<std::string, int>(line, 0));
}
}
// now do what you need to
}
No need to do any manual memory management, and std::map is typed with std::string!
ReadLine clearly includes the newline in the data it returns. Simply check for and remove it:
char *nameLine = filesystem->ReadLine(line, size, file);
// remove any newline...
if (const char* p_nl = strchr(nameLine, '\n'))
*p_nl = '\0';
(What this does is overwrite the newline character with a new NUL terminator, which effectively truncates the ASCIIZ string at that point.
Most likely the ReadLinefunction also reads the newline character. I suppose your file does not have a newline at the very last line, thus you do not get a newline for that name.
But until I know what filesystem, FileHandle_t, and Msg is, it is very hard to determine where the issue could be.