Effective method for reading large CSV files? [closed] - c++

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking for code must demonstrate a minimal understanding of the problem being solved. Include attempted solutions, why they didn't work, and the expected results. See also: Stack Overflow question checklist
Closed 9 years ago.
Improve this question
I currently have 5 CSV files with about 45,000 records per each file. Whats the best method to go about this? Ive done I/O before, but never on this scale. Parse into a vector string?

Yes, read them into a vector would be reasonable.
The choice of storage does depend a little on what you are planning to do, and what the data is. If you know what the format of the data is, you may want to create a struct, and read the data into a more organised form. E.g. if the file is like this:
name, score, err
Mats, 89, 2.1%
Steve, 79, 8%
then you could have a structure like this:
struct Row
{
string name;
int score;
float err;
}
As comments say, 45K lines is not very much, and it shouldn't cause any major problems unless you are running it on something with the computing capacity of a wrist-watch.

Just keep doing what you are doing: read all lines into a Vector of strings, a Vector of a Vector of strings, or a Vector of objects. We are talking 200 to 500 MB RAM, and nowadays most computers have much more than that available. From a processing time point of view, that's going to be 5 to 10 minutes in an average computer (depending on the amount of processing, of course).
If you run into any problem, ask a new question with more information.

Related

How do I sort int without stl liberaries [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 10 months ago.
Improve this question
I have a problem where I need to sort buses arriving at a bus station on the basis of time of arrival without using STL (standard template library) in ascending order
You may first want to read about sorting algorithms in general. A good staring point is here.
There you see many of them.
The recommendation for newbies is to start with bubble sort.
Please see here for an example including source code.
Then, you need to store your bus data in a struct. Along with the timing information. All those struct shoulb be stored in an array, best a std::vector.
Then you need to write a compare function for times. The complexity of this depends, if you have one varaible that stores the complete time, like in a unix timestamp, or in a struct, for example tm. Then you need to compare hours, minutes and seconds and some boolean relation.
But first, you need to read a lot, then think even longer on how to implement, and then write the code.

Is my company doing this right, sharing data between exes? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 6 years ago.
Improve this question
First off, my company is into power grid, not IT, so software is kinda a secondary role here.
I work on a power system simulation software, very old code base in C++ with MFC, like 15 years old. What we do is take large amounts of data, ~100,000 floating point values then format and write to a text file (most of the code actually uses the C FILE structure to do this). Then, it's read by a separate engine exe which computes the electrical algorithm (Electrical algorithms are mostly numeric solutions of system of diffn equations) and then writes back huge amount of data to another text file, which we read and update the UI.
My question is, is this how it should be done? It there a way to skip writing into the text file and directly pass the data to the exes?
exes are called using CreateProcess() MFC function.
EDIT::
Sorry, site won't let me comment.
#Vlad Feinstein Well, yes, it's like a Ladder. A thing called load flow solves power flow through the lines, which in turn will be used to find stability of the systems, which in turn for overvoltage ect. It's huge, the UI is million+ lines of code, engine exes another million maybe.
Doesn't MFC already implement IPC using Dynamic Data Exchange? I can pass strings to another process's PreTranslateMessage() func. A scaled up version of that?
There is no such a thing as "should be done as ..." there are multiple methods to do IPC and while the method you describe might not be the fastest, it is a viable solution nevertheless. If the performance doesn't bother you in this particular case you should not bother with modifying it. It is exactly the case where the phrase "if it ain't broke, don't fix it" applies.
Probably, you would not want to make any new IPC in the application that way, though.

How is it possible to create an information fetcher for a game like League of Legends? [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking for code must demonstrate a minimal understanding of the problem being solved. Include attempted solutions, why they didn't work, and the expected results. See also: Stack Overflow question checklist
Closed 9 years ago.
Improve this question
Basically what I'm wondering is, how could you get like a list of all mobs, champions their hp, mana etc with programming? I know this is possible because it has been done before but I just can't see how you would be able to do this. Is looking in the assembler code necessary or can you do it in some other way? I'm mostly wondering about the theory behind it. (Using C++ if that helps anything at all)
Such things are usually done using crawling (e.g. retrieving the data from the web pages provided by Riot Games; might be partially outdated) or using reverse engineering to get this data from the game client's files (might not contain everything). In either way you'd get datasets which you'll have to read or interpret (look for values or replicate the way the game client reads and interprets the data).
I'm not sure whether there are some tools or APIs released somewhere, at least I haven't heard of anything officially supported or endorsed; most of this is essentially in a gray zone usage wise.

Work easily on a sound file [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking for code must demonstrate a minimal understanding of the problem being solved. Include attempted solutions, why they didn't work, and the expected results. See also: Stack Overflow question checklist
Closed 9 years ago.
Improve this question
I'd like to do some tests in C++ with some sound file (16 bit, mono, 44khz)
for (int i = 0; i++, i < lengthofthesoundfile)
{
mysound[i] ...
}
How to convert a WAV file to RAW data only (get rid of the metadatas, etc.) so that I can easily open it in C++ ?
Would someone have a minimal working example of how to load such a file, and then doing a loop on the soundfile's raw data like above?
Thanks in advance!
A .wav file is simply a RIFF file of type WAVE, with (at least) the fmt and data chunks. The fmt chunk contains information about the format of the data chunk, which contains the raw data.
So if you want, you can extract just the data chunk for your usage. The RIFF format is really simple (read the Wikipedia page I linked). You should not have any trouble extracting the data chunk.

Printing numbers column by column [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking for code must demonstrate a minimal understanding of the problem being solved. Include attempted solutions, why they didn't work, and the expected results. See also: Stack Overflow question checklist
Closed 9 years ago.
Improve this question
I am trying to print numbers column by column.
I am trying to run 50 simulations that will produce a list of numbers. For every simulation, the length of such list is unknown.
For every simulation I want to print the numbers down a column.
When the next simulation starts I want the program to go back up to the top of the file and start printing the list down without entering into the previous list.
All I can find are ways to print the numbers by row, but I will not know the numbers down the row unless I do all the simulations.
Any advice would be much appreciated!
You have a few choices:
Write each sim result to a separate file and collate later;
Hold all results in memory until finished;
After each simulation, read the existing file, and output it again with the new results added in;
Knowing the number of simulations you intend to produce, write results in binary and leave padding for those that are not yet done.
I would probably go with 1.
By "goes back to the top of the file", do you actually mean rewind the file pointer to access memory starting at the beginning of the file? I will assume that's what you meant. Since you want the first input after the last output, then you can run your simulations and print them in reverse order, or simply call the appropriate library functions as you need them.