I have a for-loop and i'm creating a new instance of a struct on the stack each time. This struct just contains 2 variables - 2 char arrays of 64 bytes.
The code is below:
for (std::map<std::string, std::string>::iterator iter = m_mDevices.begin(); iter != m_mDevices.end(); ++iter)
{
Structs::SDeviceDetails sRecord;
if (false == GenerateDeviceCacheRecord(iter->first, iter->second, sRecord)) // could just pass iter in?
{
// Failed to create cache record
return false;
}
}
The really strange thing i am seeing in the debugger, is everytime i loop round, i am seeing the same value in sRecord's buffers. i.e. sRecord.m_strUsername and sRecord.m_strPassword is getting "written over" as opposed to being a newly created struct.
If sRecord.m_strUsername was "abc" on the first loop round, then after the GenerateDeviceCacheRecord function (which just modifies sRecord), sRecord.m_strUsername might be "HIc", where c is the character off the first loop! I'm obviously expecting "abc" and "HI", not "abc" and "HIc". Does anyone know what might be going on here?
Thanks
Extra code:
namespace Constants
{
static const int64 MAX_HOSTNAME_BUFFER = 64;
static const int64 MAX_ILA_BUFFER = 64;
};
struct SDeviceRecordDetails
{
char m_strHostname[Constants::MAX_HOSTNAME_BUFFER];
char m_strILA[Constants::MAX_ILA_BUFFER];
};
bool GenerateDeviceCacheRecord(std::string strHostname, std::string strILA, Structs::SDeviceRecordDetails& sRecord)
{
// Convert strings to char arrays to store in the authentication cache manager records
if (strHostname.length() > Constants::MAX_HOSTNAME_BUFFER)
return false;
if (strILA.length() > Constants::MAX_ILA_BUFFER)
return false;
std::copy(strHostname.begin(), strHostname.end(), sRecord.m_strHostname);
std::copy(strILA.begin(), strILA.end(), sRecord.m_strILA);
return true;
}
//! #brief Devices retrieved from XML file
std::map<std::string, std::string> m_mDevicesAuthenticated;
So. I appreciate that you tried to get closer to a better question. So I'm going to take some next steps with you.
What you posted wasn't really a mcve.
Here's a mcve for your problem:
#include <iostream>
#include <cstdint>
#include <map>
#include <string>
#include <algorithm>
namespace Constants
{
static const int64_t MAX_HOSTNAME_BUFFER = 64;
static const int64_t MAX_ILA_BUFFER = 64;
};
struct SDeviceRecordDetails
{
char m_strHostname[Constants::MAX_HOSTNAME_BUFFER];
char m_strILA[Constants::MAX_ILA_BUFFER];
};
bool GenerateDeviceCacheRecord(std::string strHostname, std::string strILA, SDeviceRecordDetails& sRecord)
{
// Convert strings to char arrays to store in the authentication cache manager records
if (strHostname.length() > Constants::MAX_HOSTNAME_BUFFER)
return false;
if (strILA.length() > Constants::MAX_ILA_BUFFER)
return false;
std::copy(strHostname.begin(), strHostname.end(), sRecord.m_strHostname);
std::copy(strILA.begin(), strILA.end(), sRecord.m_strILA);
return true;
}
std::map<std::string, std::string> m_mDevices;
int main() {
m_mDevices["hello"] = "foo";
m_mDevices["buzz"] = "bear";
for (std::map<std::string, std::string>::iterator iter = m_mDevices.begin(); iter != m_mDevices.end(); ++iter) {
SDeviceRecordDetails sRecord;
const bool result = GenerateDeviceCacheRecord(iter->first, iter->second, sRecord);
if (result == false)
std::cout << "Failed\n";
else
std::cout << sRecord.m_strHostname << " " << sRecord.m_strILA << "\n";
}
}
Things to note:
I can take this as is (instead of two code blocks in your question) and throw it at a compiler.
I included the proper #include lines.
There were namespaces in your type names that weren't represented in your code.
m_mDevicesAuthenticated != m_mDevices.
You didn't include anything that actually had any output.
What is actually in m_mDevices? This is really important to include!
Among other small corrections I had to apply to the code to get it to build.
What did this code do?
This code almost produces the correct output. It has an error, in that the strings that are written to sRecord are not null terminated.
Because of how compilers generate code, and that you don't explicitly clear sRecord each loop, it's likely that this is the root cause of your problem.
Let's fix that:
Instead of:
std::copy(strHostname.begin(), strHostname.end(), sRecord.m_strHostname);
std::copy(strILA.begin(), strILA.end(), sRecord.m_strILA);
Let'd do:
snprintf(sRecord.m_strHostname, Constants::MAX_HOSTNAME_BUFFER, "%s", strHostname.c_str());
snprintf(sRecord.m_strILA, Constants::MAX_ILA_BUFFER, "%s", strILA.c_str());
Or perhaps you are concerned about what sRecord starts each loop with:
In this case, sRecord is not initialized at the beginning of each loop. The compiler is free to have junk data in the struct for optimization purposes.
It happens that most compilers will place each iteration of the struct in that exact same spot in memory. This means that the junk data in the struct could be the data from the previous iteration. Or some other junk depending on how the compiler optimizations function.
You could fix this by initializing the struct to contain explicit data:
SDeviceRecordDetails sRecord = {};
What does all of this look like:
The finished code, with all the bug fixes looks like:
#include <iostream>
#include <cstdint>
#include <map>
#include <string>
#include <algorithm>
namespace Constants
{
static const int64_t MAX_HOSTNAME_BUFFER = 64;
static const int64_t MAX_ILA_BUFFER = 64;
};
struct SDeviceRecordDetails
{
char m_strHostname[Constants::MAX_HOSTNAME_BUFFER];
char m_strILA[Constants::MAX_ILA_BUFFER];
};
bool GenerateDeviceCacheRecord(std::string strHostname, std::string strILA, SDeviceRecordDetails& sRecord)
{
// Convert strings to char arrays to store in the authentication cache manager records
if (strHostname.length() > Constants::MAX_HOSTNAME_BUFFER)
return false;
if (strILA.length() > Constants::MAX_ILA_BUFFER)
return false;
snprintf(sRecord.m_strHostname, Constants::MAX_HOSTNAME_BUFFER, "%s", strHostname.c_str());
snprintf(sRecord.m_strILA, Constants::MAX_ILA_BUFFER, "%s", strILA.c_str());
return true;
}
std::map<std::string, std::string> m_mDevices;
int main() {
m_mDevices["hello"] = "foo";
m_mDevices["buzz"] = "bear";
m_mDevices["zed"] = "zoo";
for (std::map<std::string, std::string>::iterator iter = m_mDevices.begin(); iter != m_mDevices.end(); ++iter) {
SDeviceRecordDetails sRecord = {};
const bool result = GenerateDeviceCacheRecord(iter->first, iter->second, sRecord);
if (result == false)
std::cout << "Failed\n";
else
std::cout << sRecord.m_strHostname << " " << sRecord.m_strILA << "\n";
}
}
And outputs:
buzz bear
hello foo
zed zoo
Which looks correct to my eyes.
I don't see any initialisation here. You're seeing whatever happened to be at that place in memory before, which for you, today, happens to be the previous contents of those data members.
Related
I'm trying to figure out why using the merge operator for a large number of keys with rocksdb is very slow.
My program uses a simple associative merge operator (based on upstream StringAppendOperator) that concatenates values using a delimiter for a given key. It takes a very long time to merge all the keys and for the program to finish running.
PS: I built rocksdb from source - latest master. I'm not sure if I'm missing something very obvious.
Here's a minimally reproducible example with about 5 million keys - number of keys can be adjusted by changing the limit of the for loop. Thank you in advance!
#include <filesystem>
#include <iostream>
#include <utility>
#include <rocksdb/db.h>
#include "rocksdb/merge_operator.h"
// Based on: https://github.com/facebook/rocksdb/blob/main/utilities/merge_operators/string_append/stringappend.h#L13
class StringAppendOperator : public rocksdb::AssociativeMergeOperator
{
public:
// Constructor: specify delimiter
explicit StringAppendOperator(std::string delim) : delim_(std::move(delim)) {};
bool Merge(const rocksdb::Slice &key, const rocksdb::Slice *existing_value,
const rocksdb::Slice &value, std::string *new_value,
rocksdb::Logger *logger) const override;
static const char *kClassName() { return "StringAppendOperator"; }
static const char *kNickName() { return "stringappend"; }
[[nodiscard]] const char *Name() const override { return kClassName(); }
[[nodiscard]] const char *NickName() const override { return kNickName(); }
private:
std::string delim_;// The delimiter is inserted between elements
};
// Implementation for the merge operation (concatenates two strings)
bool StringAppendOperator::Merge(const rocksdb::Slice & /*key*/,
const rocksdb::Slice *existing_value,
const rocksdb::Slice &value, std::string *new_value,
rocksdb::Logger * /*logger*/) const
{
// Clear the *new_value for writing.
assert(new_value);
new_value->clear();
if (!existing_value)
{
// No existing_value. Set *new_value = value
new_value->assign(value.data(), value.size());
}
else
{
// Generic append (existing_value != null).
// Reserve *new_value to correct size, and apply concatenation.
new_value->reserve(existing_value->size() + delim_.size() + value.size());
new_value->assign(existing_value->data(), existing_value->size());
new_value->append(delim_);
new_value->append(value.data(), value.size());
std::cout << "Merging " << value.data() << "\n";
}
return true;
}
int main()
{
rocksdb::Options options;
options.create_if_missing = true;
options.merge_operator.reset(new StringAppendOperator(","));
# tried a variety of settings
options.max_background_compactions = 16;
options.max_background_flushes = 16;
options.max_background_jobs = 16;
options.max_subcompactions = 16;
rocksdb::DB *db{};
auto s = rocksdb::DB::Open(options, "/tmp/test", &db);
assert(s.ok());
rocksdb::WriteBatch wb;
for (uint64_t i = 0; i < 2500000; i++)
{
wb.Merge("a:b", std::to_string(i));
wb.Merge("c:d", std::to_string(i));
}
db->Write(rocksdb::WriteOptions(), &wb);
db->Flush(rocksdb::FlushOptions());
rocksdb::ReadOptions read_options;
rocksdb::Iterator *it = db->NewIterator(read_options);
for (it->SeekToFirst(); it->Valid(); it->Next())
{
std::cout << it->key().ToString() << " --> " << it->value().ToString() << "\n";
}
delete it;
delete db;
std::filesystem::remove_all("/tmp/test");
return 0;
}
Shared your question in the Speedb Hive, on Discord.
The reply is from Hilik, our o-founder and chief scientist.
'Merge operators are very useful to get a quick write response time, alas reads require reading the original and applying by order the merges . This operation may be very expensive esp with strings that needed to be copied and appended on each merge. The simplest way to resolve this is to use read modify write eventually . Doing this at the application level is possible but may be problematic (if two threads can do this operation concurrently) . We are thinking of ways to resolve this during the compaction and are willing to work with you on a PR...'
Hope this helps. Join the discord server to participate in this discussion and many other interesting and related topics.
Here is the link to the discussion about your topic
const char *attribute[] =
{"abc","efg","hij","lmn","opq","rst","uvw","Xyz"};
want to find Boolean and location of the "lmn" in the above array.
This example will show you how to get both a bool and an index back.
Demo here : https://onlinegdb.com/vHHJ9QG1M
#include <array>
#include <limits>
#include <algorithm>
#include <iostream>
#include <string_view>
// make a struct to be able to return two (readable) values from function
// I almost never use std::pair it results in hard to read code.
// where you have to check the semantics of first/second over and over again.
struct is_attribute_result_t
{
// conversion to bool so result can be directly used in "if's"
constexpr operator bool() const
{
return is_attribute;
}
bool is_attribute{false};
std::size_t index{std::numeric_limits<std::size_t>::max()};
};
// make a std::array that is usable at compile time
// use string_view because it implements operator== for the whole string (const char* doesn't)
constexpr std::array<std::string_view,8> attributes = {"abc","efg","hij","lmn","opq","rst","uvw","xyz"};
// make a function that can be evaluated at compile time
// so no std::find, just use availability of (constexpr) operator== on string_view
constexpr is_attribute_result_t test_attribute(const std::string_view& attribute)
{
is_attribute_result_t result;
for(std::size_t n = 0; n < attributes.size(); ++n)
{
if(attribute == attributes[n])
{
result.is_attribute = true;
result.index = n;
return result;
}
}
return result;
}
int main()
{
// nice thing is you can now also check at compile time.
static_assert(test_attribute("abc"));
static_assert(test_attribute("abc").index == 0ul);
static_assert(test_attribute("lmn"));
static_assert(test_attribute("lmn").index == 3ul);
static_assert(!test_attribute("123"));
// and ofcourse still use the function at runtime too
if ( auto result = test_attribute("lmn"))
{
std::cout << "lmn is an attribute, and is found at index = " << result.index << "\n";
}
return 0;
}
class Person
{
public:
string fname;
string lname;
string occupation;
string gender;
int age;
};
int main()
{
Person bc;
bc.fname = "Bevelry";
bc.lname = "Crusher";
bc.gender = "female";
bc.occupation = "Doctor, USS 1701-D";
bc.age = 40;
cout << bc.all << "\n"; //Something like this?
}
Is it possible for me to print every variable of an object without specifying them by myself? And is it possible for me to make a select list of variables, something like an array of variables, and then print them?
EDIT: i accidently put the cout in the class, fixed now
Is it possible for me to print every variable of an object without specifying them by myself?
No.
And is it possible for me to make a select list of variables, something like an array of variables, and then print them?
Not automatically, but you could create a collection of std::anys and add some decoding for it yourself.
Example:
#include <any>
#include <functional>
#include <iostream>
#include <iterator>
#include <string>
#include <vector>
// decode and print one std::any
void decode_one(std::ostream& os, const std::any& a) {
if(auto s = std::any_cast<std::reference_wrapper<std::string>>(&a)) os << s->get();
else if(auto i = std::any_cast<std::reference_wrapper<int>>(&a)) os << *i;
else if(auto d = std::any_cast<std::reference_wrapper<double>>(&a)) os << *d;
// add more types to decode here
else os << "<unknown type>";
}
// a custom ostream operator to decode a vector of anys:
std::ostream& operator<<(std::ostream& os, const std::vector<std::any>& va) {
auto begin = va.begin();
auto end = va.end();
if(begin != end) {
decode_one(os, *begin);
for(std::advance(begin, 1); begin != end; std::advance(begin, 1)) {
os << ',';
decode_one(os, *begin);
}
}
return os;
}
int main() {
int a = 10;
std::string b = "Hello world";
double c = 3.14159;
std::vector<std::any> va{
std::ref(a),
std::ref(b),
std::ref(c)
};
c *= 2.; // just to show that the vector holds a reference
std::cout << va << '\n'; // print the variables in the vector
}
Output:
10,Hello world,6.28318
It's not possible. In C++ there isn't anything that does what you want.
At one side one can easily say that this is not possible in C++.
However, at the other side, one can add that this situation happens quite often when people are making object oriented computer programs, and in Java there is the toString() method for this. So, you might actually write your own toString() method (based on the way it is done in Java), and work with this.
Good luck
I ran into a little problem programming something, I've looked around but I didn't seem to find out the answer.
I'll spare you useless code.
Here are the declarations:
struct one {
std::string string1, string2;
bool boolean;
};
struct two {
std::string string3, string4;
bool boolean;
};
void function(uint first_parameter, one **first, two **second);
And here is what the main looks like:
int main()
{
one *passes;
two *users;
//...
passes = new one[size_one]();
users = new two[size_two]();
//Filling the arrays...
std::thread t[PARTS];
for (int start = 0; start < PARTS; start++)
t[start] = std::thread(function, first_parameter, &passes, &users);
for (int i = 0; i < PARTS; i++)
t[i].join();
}
Whenever I try to access an element of one of my structures (allocated on the free store) in my thread function, (I typically would access it like so: (*first)[0].string1[0]) I do not get the string1 I normally can access in main. Aren't the std::strings located in the free store?
check your function to have valid types of parameters.
this example bellow works nice.
#include <iostream>
typedef struct _one
{
std::string first_str;
std::string second_str;
bool check;
} one;
void do_something(one** passes)
{
one* original = *passes;
one first = original[0];
std::cout << first.first_str; // -> hello
// the same
std::cout << (*passes)[0].first_str;
}
int main()
{
one* passes = nullptr;
passes = new one[10];
passes[0].first_str = "hello";
do_something(&passes);
}
I have come up with the following structure to declare various formats if messages that are to be received from the network:
#include <stdint.h>
#include <iostream>
#include <string.h>
template<int T>
struct uint
{
static uint<T> create(uint64_t value)
{
uint<T> r = {value};
return r;
}
uint(uint64_t value)
{
v = value;
}
uint()
{}
uint<T>& operator =(uint64_t value)
{
v = value;
return *this;
}
operator uint64_t() const
{
return (uint64_t)v;
}
unsigned long long v:T;
}__attribute__((packed));
example:
typedef uint<5> second_t;
suppose one of the message formats (which are auto-generated via some process) is like this:
struct seconds
{
char _type;
second_t _second;
} __attribute__((packed));
Now suppose I would like to populate an instance of the above messahe using a string:
int main()
{
seconds ii;
const char *i = "123456";
// memset, memcpy,sprintf... ??? what to use here?
std::cout << ii._type << " " << ii._second << std::endl;
}
Given a stream 123456, I expect the instance of the seconds (ii) structure to have char ii._type = '1' and integer ii._second = 23456. But I dont know how to do that. Do you have a clue how i can do that? and do you have any suggestion how to improve the basic structure?
thanks
You have a number of easier and more reliable options available that require almost no work.
check out google protocol buffers (platform independent message serialisation and deserialisation): https://developers.google.com/protocol-buffers/
or boost::serialization - (probably faster, but not platform-independant) http://www.boost.org/doc/libs/1_58_0/libs/serialization/doc/index.html