Here is the problem, I dont know how many attributes or which type are the attributes in table and I need simple select statment like: SELECT * FROM TABLE1; to write down to file.
And this need to be done with otlv4 wrapper.
Please help.
otl_stream i(50, // buffer size
"select * from test_tab where f1>=:f<int> and f1<=:f*2", // SELECT statement
db // connect object
);
int ac=0;
char bc[64];
memset(bc, 0, 64);
while(!i.eof())
{
i >> ac >> bc;
cout << "Boooo " << ac << " " << bc << endl;
}
This is example where I know how many attributes are there and which type are there. But what if I dont know that??
A file stream along with OTL's check_end_of_row() and set_all_column_types() functions should do what you're asking. While looping on an OTL stream's eof check, for each row you could loop on the end of row check, and send each attribute's value from the OTL stream to the file stream. After each end of row check, send your line break code(s) to the file stream. Setting all column types to str should allow you to use just one variable to process all attribute values in a row.
This example from the OTL docs demonstrates how to use the set_all_column_types function. You must create a stream first, set the option, then open the stream. If you create and open the stream at the same time, it won't work.
Related
I know the title is a bit confusing, so i will try my best to explain it here.
When outputting to an excel file you can move the output location down by outputting a "\n" or by using std::endl;
is there something similar to move up?
to make it easier to understand i have some code:
std::ofstream outFile;
outFile.open( "outfile.csv" );
outFile << "test \n";
outFile << "test2" << "," << "test3";
in this case the "test" text is found ( in the excel file ) at A1, the test2 is found at A2, after that i can move it right, test3 is located at B2.
the question is: is there a way to move the next output up and left ( in this case, how can i move the next output to e.g. A1 )
a way to reset the position would work too.
Moving the position in a file is possible, but it won't help you much, because your entries are all of different length. Text files are just a bunch of characters, there is no A1 or B2 in the text file.
Instead of working on the text file directly, if feasible, you should load all the contents into a data structure that allows you to modify specifc entries given an index.
You could use a 2d array or a std::map<char,std::vector<std::string>> data, so you can access an element at A1 via data['A'][1].
A fragment of my code is :
for (int iter = 0; iter < flags.total_iterations_; ++iter) {
if (iter%20==0) {
std::ofstream mf(flags.model_file_.c_str());
accum_model.AppendAsString(word_index_map, mf); }
else {
std::cout << "Model not created for "; }
std::cout << "Iteration " << iter << " ...\n";
So, I am trying to generate outputs from method accum_model at every 20th iteration. But, the problem is I have to write the output in new file everytime the 20th iteration is reached. Now, my output is being overwritten.
I execute this code with the help of a executible, which is as:
./lda --num_topics 15 --alpha 0.1 --beta 0.01 --training_data_file testdata/test_data.txt --model_file MF/lda_model.txt --burn_in_iterations 120 --total_iterations 150
The MF/lda_model.txt is the output file given. I am not understanding how to link the file that contains the code and this executible command as I would need 5 different new files (for 100 iterations - as data is written into a new file every 20th iteration).
I am new to coding and so far, I was coding in python. I tried till this loop, I am confused about how to create new files and get corresponding outputs. Please help! Thanks in advance.
Use std::stringstream, and build a new file name to open each time.
std::string uniquePathFileNamePostFix (int i) {
std::stringstream ss;
ss << '-' << i ;
return (ss.str());
}
The idea is to use the stringstream to create (or append or prepend) a unique modifier based on i. (or anything else convenient - I have used time stamps).
If I understand your question correctly, you are overwriting the ofstream instead of appending to it.
You'll want to specify the 'app' flag in the mode of the ofstream constructor:
std::ofstream mf(flags.model_file_.c_str(), std::ios_base::app);
If you need to start the output with an new, "empty" file, just leave out the mode (ofstream defaults to std::ios_base:out, whether you specify it or not): std::ofstream::ofstream
...and if you need a new output file (according to your OP), you need to change the file name in flags.model_file_.
I'm not sure that I understand your question correctly, but I think you want to output every 20th iteration in a new file. To do so, you just need to append the value of theiter to the name of the file or otherwise add a "dynamic" element to it.
The way to do it using only standard C++ is using stringstream:
std::stringstream file_name;
file_name << flags.model_file_ << iter;
result = file_name.str();
I have a program which reads information about 3D meshes in from a text file, stores the data as a mesh,performs some post-processing on the mesh, then collects information like volume, center of mass, etc.
The program has two threads:
Main thread starts the second thread, reads the file, does the processing, then waits for the second thread. As a part of the processing, it puts information about the meshes its just read onto a queue.
Second thread connects to SQL Server using Poco ODBC, puts some initial information about the file its reading into a database table, then gets information off the queue and assembles a potentially lengthy insert command. When it is done doing so, it submits the command, performs a final update command regarding the results of operations performed, then lets the main thread know it's done, and terminates the 2nd thread.
Everything works, right up until the moment it submits the large insert command. It throws an exception, and i can't figure out why.
Here i will give a simplistic outline of the code that is executing. Assume variables exist and are initialized.
The poco commands i run are:
using namespace Poco;
using namespace Poco::Data::Keywords;
using namespace Poco::Data;
ODBC::Connector::registerConnector();
Session session(SessionFactory::instance().create("ODBC", "Driver={SQL Server};Server=<hostname>;Database=<DB name>;Uid=<username>;Pwd=<password>;"));
session << "INSERT INTO TableName1 (SourceFileName,UserName) VALUES (?,?)",use(_filename),use(username),now;
session << "SELECT SCOPE_IDENTITY()", into(runID),now; //This always runs and returns 0.
string queryString = "Insert into TableName2 (Field1, field2, field3, field4, ...) "+
"VALUES (val1, val2, val3, val4, ...)"+
",(valA, valB, valC, valD, ...),..."
session << queryString,now;
Statement update(session);
update << "UPDATE TableName1 SET Field2 = ?, Field3 = ?, Field4 = ? WHERE Field1 = ?", use(data2), use(data3), use(data3), use(identity);
update.execute();
ODBC::Connector::unregisterConnector();
<send signal to main thread indicating being done.>
I'm trying to figure out a few key things.
How can I tell what state the Session is in?
Is there a way to ask Poco what went wrong and have it print an error message?
Are there any special things I need to set up to be able to specify a big insert statement all together in text like I am? I have tried it using ? placeholders, or executing individual statements, but it always gives me an exception on that part.
Is there a way to have statements execute under the same connection for sure? Normally I would do my INSERT INTO TableName1(...)VALUES(...) SELECT SCOPE_IDENTITY() all as a single operation. I've tried my commands in SQL Server Management Studio and it works properly. Right now, it is always returning a 0 aka NULL, like the statements run in separate connections.
More information:
String query = "INSERT INTO TableName1 (SourceFileName,UserName) VALUES ('"+ __file + "','" + __username + "')";
Statement insertStmt = (session << query);
try{insertStmt.execute(true);}
catch(Poco::Exception exc)
{
cout << exc.displayText() << endl;
}
try{session << "SELECT SCOPE_IDENTITY() as SCOPE_IDENTITY", into(runID), now;
cout << "Run ID: " << runID << endl;}
catch(Poco::Exception exc)
{
cout << exc.displayText() << endl;
}
I greatly appreciate your help or any suggestions on how I can improve this question.
1.:
There are various query members in the Session class - isConnected(), canTransact(), isTransaction() ... (if that is what you are looking for; if not, see the next answer)
1. and 2.:
Wrap your statement into try/catch block:
#include "Poco/Data/ODBC/ODBCException.h"
//...
try
{
session << "INSERT INTO TableName1 (SourceFileName, UserName) VALUES (?, ?) ",use(_filename),use(username),now;
}
catch(Poco::Data::ODBC::ConnectionException& ce){ std::cout << ce.toString() << std::endl; }
catch(Poco::Data::ODBC::StatementException& se){ std::cout << se.toString() << std::endl; }
3.:
I don't think the problem is too large statement. There is a configurable internal setting limiting the string size to 1K, but this applies to value strings, not the whole SQL statement.
If you still think that is the problem, you can increase the max field size, e.g.:
std::size_t fieldSize = 4096; //4K
session.impl()->setProperty("maxFieldSize", Poco::Any(fieldSize));
4.:
Poco::Data::ODBC does not parse or analyze the SQL statement in any way; so from that standpoint, whatever works with your ODBC driver will be fine.
The problem with my code is that in the while loop, the program is not properly reading the data in the file. If I were to output each individual member of the structure out of the while loop it would output stuff like zero, blank and even random numbers to different members of the structure. This also means that nothing is getting added to the vector because the vector size is zero.
Note 1-I'm using codeblocks as my IDE.
Note 2-The file that I'm using is an excel file. This means that I'm assuming that you've worked with excel files before and that you know that they're lined up in columns and rows. Also, I only want a certain amount of data going to each member in the structure.
Here's a very small sample from my input file.
EVENT_ID CZ_NAME_STR BEGIN_DATE BEGIN_TIME
9991511 MIAMI-DADE CO. 10/18/1955 800
9991516 MIAMI-DADE CO. 4/10/1956 1730
9991517 MIAMI-DADE CO. 4/10/1956 1730
Here's my code
#include <iostream>
#include <fstream>
#include <cstdlib>
#include <vector>
#include <string>
using namespace std;
// Structure.
struct Weather_Event
{
int eventID; // is the unique int id associated with the weather event.
string CZ_NAME; // is the location associated with the weather event.
char beginDate[11]; // is the date the event started.
char beginTime[5]; // is the time the event started.
string eventType; // is the type of weather event.
int deaths; // are the number of people killed by the weather.
int injuries; // are the number of people injured by the event.
int propertyDamage; /* is the $ worth of property damage caused
by the event. */
float beginLatitude; // is the starting latitude for the event.
float beginLongitude; // is the starting longitude for the event.
};
int main()
{
// Create an empty vector that will contain the structure.
vector<Weather_Event>weatherInformation(0);
Weather_Event data; /* Create an object to access each member in
the structure. */
// Declare an object and open the file.
ifstream weather("weatherdata.csv");
if(!weather) // Check to see if the file opens.
{
cerr << "Error, opening file. ";
exit(EXIT_FAILURE); // terminate the program early.
}
/* While you're not at the end of the file, keep reading each new
piece of data. */
while(weather >> data.eventID >> data.CZ_NAME >> data.beginDate
>> data.beginTime >> data.eventType >> data.deaths
>> data.injuries >> data.propertyDamage >> data.beginLatitude
>> data.beginLongitude)
{
/* Add all the data that was added to each member of the
structure, into the vector. */
weatherInformation.push_back(data);
}
weather.close();
// Next display the result
for(size_t i=0; i<weatherInformation.size(); i++)
{
cout << "EventID: " << weatherInformation[i].eventID << endl;
}
return 0;
}
For starters, the format of the input doesn't seem to contain all the fields being read. That is, the input contains columns for event ID, name, begin data, and begin time but the code also reads event type, deaths, injuries, property damage, latitude, and longitude. Once this problem is address, you'll run into different other problems, though:
The first line doesn't match the format you are trying to read at all. You'll need to ignore it, e.g., using weather.ignore(std::numeric_limits<std::streamsize>::max(), '\n');
The formatted input operator for string reads a single word. However, your name consists of multiple words, i.e., only the first of these words is read. I don't know if your file contains a suitable separators (e.g. tab characters '\t') you could use to delimit the name or if the name as a fixed length or something. You'll need something to tell the stream where to stop reading the name.
When reading into a char array using the formatted input operators, you should set up a limit for the characters to be read to avoid an overflow: if you don't set up a limit, as many characters as matching the format will be read! That is, you want to read the beginDate and the beginTime using something like this:
weather >> std::setw(sizeof(data.beginDate)) >> data.beginDate
>> std::setw(sizeof(data.beginTime)) >> data.beginTime;
Setting the width limits the amount of data being read. If there are more characters the input will fail rather than overflowing the buffer (if I recall correctly).
These are the problems I spotted based on the code and data you posted.
AccountNumber Type Amount
15 checking 52.42
23 savings 51.51
11 checking 12.21
is my tab delmited file
i would like to be able to search for rows by the account number. say if i put in 23, i want to get that specific row. how would id do that?
also more advance, if i wanted to change a specific value, say amount 51.51 in account 23. how do i fetch that value and replace it with a new value?
so far im just reading in row by row
string line;
ifstream is("account.txt");
if (is.is_open())
{
while (std::getline(is, line)) // read one line at a time
{
string value;
string parseline;
std::istringstream iss(line);
getline(line, parseline);
cout << parseline << endl; // do something with the value
while (iss >> value) // read one value at at time from the line
{
//cout << line << " "; // do something with the value
}
}
is.close();
}
else
cout << "File cant be opened" << endl;
return 0;
Given that each line is of variable length there is no way to index to particular row without first parsing the entire file.
But I suspect your program will want to manipulate random rows and columns. So I'd start by parsing out the entire file. Put each row into its own data structure in an array, then index that row in the array.
You can use "strtok" to split the input up into rows, and then strtok again to split each row into fields.
If I were to do this, I would first write a few functions that parse the entire file and store the data in an appropriate data structure (such as an array or std::map). Then I would use the data structure for the required operations (such as searching or editing). Finally, I would write the data structure back to a file if there are any modifications.