I have been writing a wrapper for the Cassandra cppdriver for CQL3.0 and I have come across some odd behavior, and I am not sure if it is typical or a bug.
For reference, I am working with with the cppdriver code release on 4 September (from the repository), libuv0.10 and off of the songs / playlist example posted on the datastax website (http://www.datastax.com/documentation/cql/3.1/cql/ddl/ddl_music_service_c.html)
The problem that I am having is with executing query strings. There seems to be some threshold of characters after which the query string being sent to Cassandra becomes garbage. The code that I am using to construct and send the string to the cppdriver library (and parse the results) is provided below. I added a function (cass_session_print_query) to the cassandra.h and session.cpp files to print out generated statement.
map<string, vector<string> > retresults;
int i = 0, ccount;
stringstream ss;
vector<string> keys = get.GetList();
vector<string>::iterator kit = keys.begin();
map<int, pair<string, string> > primkeys = get.GetMap();
map<int, pair<string, string> >::iterator mit = primkeys.begin();
if (!keys.empty())
{
ss << "SELECT " << (*kit);
++kit;
for ( ; kit != keys.end(); ++kit)
ss << "," << (*kit);
ss << " FROM " << tablename;
if (!primkeys.empty())
{
ss << " WHERE ";
ss << mit->second.first << " = ?";
++mit;
for ( ; mit != primkeys.end(); ++mit)
ss << " and " << mit->second.first << " = ?";
mit = primkeys.begin();
}
ss << ";";
cass_bool_t has_more_pages = cass_false;
const CassResult* result = NULL;
CassString query = cass_string_init(ss.str().c_str());
CassStatement* statement = cass_statement_new(query, primkeys.size());
for ( ; mit != primkeys.end(); ++mit)
cass_statement_bind_string(statement, i++, cass_string_init(mit->second.second.c_str()));
cass_statement_set_paging_size(statement, 100);
do
{
cass_session_print_query(statement);
CassIterator* iterator;
CassFuture* future = cass_session_execute(session_, statement);
if (cass_future_error_code(future) != 0)
{
CassString message = cass_future_error_message(future);
fprintf(stderr, "Error: %.*s\n", (int)message.length, message.data);
break;
}
result = cass_future_get_result(future);
ccount = cass_result_column_count(result);
vector<string> cnames;
for (i = 0; i < ccount; i++)
cnames.push_back(cass_result_column_name(result, i).data);
iterator = cass_iterator_from_result(result);
ListVector::iterator vit;
while (cass_iterator_next(iterator))
{
const CassRow* row = cass_iterator_get_row(iterator);
for (vit = cnames.begin(); vit != cnames.end(); ++vit)
{
CassString value;
char value_buffer[256];
cass_value_get_string(cass_row_get_column_by_name(row, (*vit).c_str()), &value);
if (value.length == 0 || value.data == NULL)
continue;
memcpy(value_buffer, value.data, value.length);
value_buffer[value.length] = '\0';
retresults[(*vit)].push_back(value_buffer);
}
}
has_more_pages = cass_result_has_more_pages(result);
if (has_more_pages)
cass_statement_set_paging_state(statement, result);
cass_iterator_free(iterator);
cass_result_free(result);
} while (has_more_pages);
}
return retresults;
With this, an initial query string of SELECT id,album,title,artist,data FROM songs; results in a Cassandra query string of SELECT id,album,title,artist,data FROM songs;. However, if I add one more column to the SELECT portion SELECT id,album,title,artist,data,tags FROM songs; the query string in the Cassandra cppdriver library becomes something like: ,ar����,dat�� jOM songX. This results in the following error from Cassandra / library: Error: line 1:49 no viable alternative at character '�'.
I have also tried fewer columns, but with a WHERE clause, and that results in the same problem.
Is this a bug? Or am I building and sending strings to the cppdriver library incorrectly?
You should cass_future_wait() on the execute future before testing the error code.
Unrelated: there are also a couple of things that should be freed (future, statement), but I'm assuming that was omitted to keep this concise.
So, it looks like (for whatever reason) I HAVE to parse out the row key from the results. I checked the example, and I was able to not parse out the row key information and everything still worked. I am not yet entirely sure what is forcing me to do this (compared to the provided paging example), but for others, you need to include the following within the while (cass_iterator_nex(iterator)) block to "magically" fix my code above.
CassUuid key;
char key_buffer[CASS_UUID_STRING_LENGTH];
const CassRow* row = cass_iterator_get_row(iterator);
cass_value_get_uuid(cass_row_get_column(row, 0), key);
cass_uuid_string(key, key_buffer);
This is really a long shot, but since you mentioned the Music Service example, did you possibly download and use cql_collections.zip query strings? If so, the strings (now fixed) had minor syntax errors:
-use music
-CREATE TABLE music.songs ( id uuid PRIMARY KEY, album text, artist text, data blob, reviews list, tags set, title text, venue map
+use music;
+CREATE TABLE music.songs ( id uuid PRIMARY KEY, album text, artist text, data blob, reviews list, tags set, title text, venue map);
AeroBuffalo's code worked for me except that I had to put '&' in front of the second parameter of cass_value_get_uuid() function. It required reference type.
cass_value_get_uuid(cass_row_get_column(row, 0), &key);
Related
I'm learning Avro & C++ (both together :) ) and what i'm trying to do is:
Load a schema.
Map the schema fields to key / value paris of name & type.
Then iterate the avro data according to the mapping.
From what I've found and did, I'm extracting the schema from the Avro data file, and handles it as GenricDatum.
When I try to iterate it's fields - I get the name ok, but the field type is null, where I would expect to get the actual type. any help would be appreciated.
My code:
const char *avroFilePathCstr = avroFilePath.c_str();
avro::DataFileReader<avro::GenericDatum> reader(avroFilePathCstr);
auto dataSchema = reader.dataSchema();
avro::GenericDatum datum(dataSchema);
ProcessAvroSchema(dataSchema);
void ProcessAvroSchema(avro::GenericDatum schema) {
const avro::GenericRecord& schemaRecord = schema.value<avro::GenericRecord>();
for(unsigned int i = 0 ; i < schemaRecord.fieldCount(); i++)
{
avro::GenericDatum fieldDatum = schemaRecord.fieldAt(i);
cout << "SCHEMA:: fieldName: " << schemaRecord.schema() -> nameAt(i) << " , fieldType: " << fieldDatum.type() << "\n";
}
}
I have been picking my brain for a while trying to figure this one out.
The problem I am having is that the function I am using in Oracle returns a BLOB. It's a list of items that are concatenated together using ||.
From the research I have done,
In the QSQLQuery docs it says "Stored procedures that uses the return statement to return values, or return multiple result sets, are not fully supported. For specific details see SQL Database Drivers." - which leads me to believe I may need to switch to a different codebase if Qt cannot handle it yet.
The documentation for the QOCI driver mentions this "Binary Large Objects (BLOBs) can be read and written, but be aware that this process may require a lot of memory. You should use a forward only query to select LOB fields (see QSqlQuery::setForwardOnly())."
I did set
query.setForwardOnly(true);
Before I prepared or executed the statement.
However, I still get this error
QSqlError("6502", "Unable to execute statement", "ORA-06502: PL/SQL: numeric or value error: character string buffer too small\nORA-06512: at line 55\n")
I had to scrub the code a bit, I hope this is still helpful to give context to what i'm trying to accomplish
temp_clob clob;
name varchar2(183) := ?;
start varchar2(16) := ?;
end varchar2(16) := ?;
count integer := ?;
return_val named_redacted_table_object; -- Sorry had to remove this, it's a table with more Date, Varchar, etc
begin
dbms_lob.createtemporary(temp_clob, true);
return_val := package_name.function_name (
set_name => name,
start_time => to_date(start, 'yyyy-mm-dd hh24:mi'),
end_time => to_date(end, 'yyyy-mm-dd hh24:mi'),
max_count => count);
-- In here was a loop that would break apart the removed table object and make it into strings along the following lines
-- '' || return_val(i).name || return_val(i).value || etc
-- and would store these into the CLOB / temp_clob
? := temp_clob;
end;
I could not get something as simple as this to work
begin
? := 'test123';
end;
With the assumption I could at least read this string in Qt.
Here is my code for Qt
QString name = "test";
QSqlQuery query(db);
query.setForwardOnly(true);
query.prepare(sql);
QString test_sql_text = ui->comboBox_test_text->currentText();
qDebug() << name;
query.bindValue(0, name);
query.bindValue(1, "2003-03-14 00:00");
query.bindValue(2, "2005-03-14 23:00");
query.bindValue(3, 2);
query.bindValue(4, QString(""), QSql::Out);
bool query_executed_ok = query.exec();
qDebug() << "did it execute?" << query_executed_ok;
// qDebug() << query.executedQuery();
qDebug() << query.boundValues();
qDebug() << query.lastError();
QSqlRecord rec = query.record();
qDebug() << rec;
int record_count = rec.count();
qDebug() << "Records: " << record_count;
while (query.next()) {
for(int i=0;i<record_count;i++) {
qDebug() << query.isValid() << " - " << rec.fieldName(i) << " " << query.value(i).toString();
}
}
The error posted appears to be from within the Oracle code; ORA.... You have stripped so much it's hard to see what is actually happening, especially the are where the error apparently occurred. But perhaps using Oracle supplied code that is specifically designed to handle CLOBs. Instead of
'' || return_val(i).name ...
Try
dbms_lob.append(temp_clob, to_clob(return_val(i).name))
begin
? := 'test123';
end;
Bind variables are used to assign values to variables. You define your variable in your pl/sql code and assign a value to it at runtime by using a bind variable. In that case pl/sql code will compile correctly.
In your code the bind variable is used to replace the pl/sql variable, not the value, which will fail. Your pl/sql block cannot be compiled because it cannot resolve the "?".
A valid use of bind variables would be
BEGIN
l_xyz := ?;
END;
where you assign the value test123 at runtime.
It took some fiddling, and I realize I gave fairly cryptic code. So thank you to belayer and Koen for taking a shot at my mess.
What I was able to determine and get working for anyone else running into this:
Let me start off by saying I am not sure if this is a bug, or if i'm doing something in a way that was not intended by the designers of QSqlQuery (The class for handling SQL calls).
The call would work in SQL developer and I would see the intended CLOB with all characters. I was unable to get DBMS_Output to work, however, I saw this post saying to reserve space on the string before binding it to the query.
It solves my issue and shows the result in the debug window. However, it presents a new problem. What if the string becomes larger than my hard coded reserve value?
Here's the updated code for that
query.prepare(sql);
QString name= ui->comboBox_name->currentText();
qDebug() << project;
query.bindValue(":name", project);
query.bindValue(":start_date", "2005-03-14 00:00");
query.bindValue(":end_date", "2006-03-14 23:00");
query.bindValue(":max_count", 3);
QString testStr ="*****";
//testStr.truncate(0); //Maybe this works too?
testStr.reserve( 1000000 ); // This did it for sure
qDebug() << testStr.capacity();
query.bindValue(":result_clob", testStr, QSql::Out);
bool query_executed_ok = query.exec();
qDebug() << "did it execute?" << query_executed_ok;
if (query_executed_ok) {
testStr = query.boundValue(":result_clob").toString();
qDebug() << testStr;
} else {
qDebug() << "string is empty";
}
I got the idea to do this, from THIS post.
I have been trying to load an SQL database into a datatable in C++, however; it doesn't seem to want to work. The connection is working though, as DataReader works. Here is my code
void importDatabase() {
SqlConnection con;
SqlDataAdapter^ da;
SqlCommand cmd;
DataTable^ dt;
int count = 1;
try {
con.ConnectionString = "Data Source=MYNAME\\SQLEXPRESS;Initial Catalog=VinylRecords;Integrated Security=True";
cmd.CommandText = "SELECT * FROM Records";
cmd.Connection = %con;
con.Open();
da = gcnew SqlDataAdapter(%cmd);
dt = gcnew DataTable("Records");
Console::Write(da->ToString());
da->Fill(dt);
for (int i = 0; i < dt->Rows->Count - 1; i++) {
String^ value_string;
value_string = dt->Rows[i]->ToString();
Console::WriteLine(dt->Rows[i]->ToString());
count++;
}
cout << "There are " << count << " many records";
}
catch (Exception^ ex) {
Console::WriteLine(ex->ToString());
}
}
Please note, that I slightly altered the source name to post here, but only the first part.
What is wrong with my code?
So, the problem is here:
dt->Rows[i]->ToString()
Rows[i] is a Row object. And the Row class's ToString() method always prints out the fully qualified typename, which is what you are seeing. So this is technically working just fine. What you will need to do to get something useful is: you will need to access a specific column in that row and get it's value, then output that.
Something along the lines of:
foreach (DataRow dr in dt.Rows)
{
Console.Write(dr.Field<int>("ColumnOne"));
Console.Write(" | ");
Console.WriteLine(dr.Field<string>("ColumnTwo"));
}
I am not entirely sure on the syntax for accessing a specific cell inside of a DataTable when using C++\CLI. So I have provided the C# equivalent to explain why it is you were getting output of managed type names (e.g. "System.Data.DataRow") instead of the info inside of the Row's columns.
Also, I noticed you tagged this question with "mysql", but you are using the ADO.NET System.Data.SqlClient namespace. The SqlDataReader and SqlDataAdapter classes only work with TSQL (Microsoft's SQL Server databases) If you are actually connecting to a mysql database you will want to use the System.Data.OdbcDataAdapter class. You can read a little more here: https://msdn.microsoft.com/en-us/library/ms254931.aspx
Is it possible to obtain the string equivalent of protobuf enums in C++?
e.g.:
The following is the message description:
package MyPackage;
message MyMessage
{
enum RequestType
{
Login = 0;
Logout = 1;
}
optional RequestType requestType = 1;
}
In my code I wish to do something like this:
MyMessage::RequestType requestType = MyMessage::RequestType::Login;
// requestTypeString will be "Login"
std::string requestTypeString = ProtobufEnumToString(requestType);
The EnumDescriptor and EnumValueDescriptor classes can be used for this kind of manipulation, and the
the generated .pb.h and .pb.cc names are easy enough to read, so you can look through them to get details on the functions they offer.
In this particular case, the following should work (untested):
std::string requestTypeString = MyMessage_RequestType_Name(requestType);
See the answer of Josh Kelley, use the EnumDescriptor and EnumValueDescriptor.
The EnumDescriptor documentation says:
To get a EnumDescriptor
To get the EnumDescriptor for a generated enum type, call
TypeName_descriptor(). Use DescriptorPool to construct your own
descriptors.
To get the string value, use FindValueByNumber(int number)
const EnumValueDescriptor * EnumDescriptor::FindValueByNumber(int number) const
Looks up a value by number.
Returns NULL if no such value exists. If multiple values have this >number,the first one defined is returned.
Example, get the protobuf enum:
enum UserStatus {
AWAY = 0;
ONLINE = 1;
OFFLINE = 2;
}
The code to read the string name from a value and the value from a string name:
const google::protobuf::EnumDescriptor *descriptor = UserStatus_descriptor();
std::string name = descriptor->FindValueByNumber(UserStatus::ONLINE)->name();
int number = descriptor->FindValueByName("ONLINE")->number();
std::cout << "Enum name: " << name << std::endl;
std::cout << "Enum number: " << number << std::endl;
I'm using pqlib with postgresql version 9.1.11
I have the following code
const char *spid = std::to_string(pid).c_str();
PGresult *res;
const char *paramValues[2] = {u->getID().c_str(), spid};
std::string table;
table = table.append("public.\"").append(Constants::USER_PATTERNS_TABLE).append("\"");
std::string param_name_pid = Constants::RELATION_TABLE_PATTERN_ID;
std::string param_name_uid = Constants::RELATION_TABLE_USER_ID;
std::string command = Constants::INSERT_COMMAND + table + " (" + param_name_uid + ", " + param_name_pid + ") VALUES ($1, $2::int)";
std::cout << "command: " << command << std::endl;
res = PQexecParams(conn, command.c_str(), 2, NULL, paramValues, NULL, NULL,0);
Where
INSERT_COMMAND = "INSERT INTO " (string)
USER_PATTERN_TABLE = "User_Patterns" (string)
RELATION_TABLE_PATTERN_ID = "pattern_id" (string)
RELATION_TABLE_USER_ID = "user_id" (string)
pid = an int
u->getID() = a string
conn = the connection to the db
The table "User_Patterns" is defined as
CREATE TABLE "User_Patterns"(
user_id TEXT references public."User" (id) ON UPDATE CASCADE ON DELETE CASCADE
,pattern_id BIGSERIAL references public."Pattern" (id) ON UPDATE CASCADE
,CONSTRAINT user_patterns_pkey PRIMARY KEY (user_id,pattern_id) -- explicit pk
)WITH (
OIDS=FALSE
);
I already have a user and a pattern loaded into their respective tables.
The command generated is :
INSERT INTO public."User_Patterns" (user_id, pattern_id) VALUES ($1, $2::int)
I also tried with $2, $2::bigint, $2::int4
The problem is:
I receive the error :
ERROR: invalid input syntax for integer: "public.""
I already use PQexecParams to store users and patterns, the only difference is that they all have text/xml fields (the only int field on patterns is a serial one and I don't store that value myself) but because the user_patterns is a relation table I need to store and int for the pattern_id.
I already read the docs for pqlib and saw the examples, both are useless.
The problem is in the lines:
const char *spid = std::to_string(pid).c_str();
const char *paramValues[2] = {u->getID().c_str(), spid};
std::to_string(pid) creates temporary string and .c_str() returns a pointer to an internal representation of this string, which is destroyed at the end of the line, resulting in a dead pointer. You may also see answer to the question
stringstream::str copy lifetime