I have a memory location of 128 bytes. I try to fill the memory with data starting from 1...127.
I need to write a code which get two parameter like offset , data type. Based on the arguments I need to convert the data on the memory to the specific datatype mentioned.
say for example
unsigned char *pointer = (unsigned char *)malloc(sizeof(unsigned char) * 128);
printf("\n\n loading some default values...");
for (unsigned int i = 0; i < 128; i++) {
pointer[i] = i + 1;
}
convertTo(3,efloat);
convertTo(100,edword);
void convertTo(uint8_t offset, enum datatype){
switch(datatype)
{
case efloat:
//// conversion code here..
break;
case edword:
//// conversion code here..
break;
case eint:
//// conversion code here..
break;
}
}
I tried using many methods like atoi, atof, strtod, strtol, etc.., but nothing gives me correct value. Say if I give offset as 2, eint(16-bit) which should take value 2,3 and give 515
Here is a generic version of what you want which wraps the type to convert to and the offset into a single struct. While the template code is more complicated, the usage is IMHO, much cleaner. Additionally, the long switch statement has been removed (at the expense of some less readable template code).
// Use an alias for the type to convert to (for demonstration purposes)
using NewType = short;
// Struct which wraps both the offset and the type after conversion "neatly"
template <typename ConversionType>
struct Converter {
// Define a constructor so that the instances of
// the converter can be created easily (see main)
Converter(size_t offset) : Offset(offset) {}
// This provides access to the type to convert to
using Type = ConversionType;
size_t Offset;
};
// Note: The use of the typename keyword here is to let the compiler know that
// ConverterHelper::Type is a type
template <typename ConverterHelper>
typename ConverterHelper::Type convertTo(char* Array, ConverterHelper ConvHelper) {
// This converts the bytes in the array to the new type
typename ConverterHelper::Type* ConvertedVar =
reinterpret_cast<typename ConverterHelper::Type*>(Array + ConvHelper.Offset);
// Return the value of the reinterpreted bytes
return *ConvertedVar;
}
int main()
{
char ExampleArray[8] = {0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08};
// Create a new NewType (short) using bytes 1 and 2 in ExampleArray
NewType x = convertTo(ExampleArray, Converter<NewType>(1));
}
On the machine I used to test this, x had a value of 770, as John suggested it might.
If you remove the alias NewType and use the actual type you wish to convert to, the intention of convertTo is, again IMHO, very clear.
Here is a live demo Coliru Demo. Just change the type alias NewType to see the output for different types.
Try *reinterpret_cast<uint16_t*>(pointer + offset). Of course, what you will get depends on the endianess of your system. 0x02 0x03 might be interpreted as 0x0203 (515) or 0x0302 (770).
Related
Related SO questions:
variable size struct
string-based generator
Sadly neither one (or other similar ones) provide the solution I'm looking for.
Background
USB descriptors are (generally) byte-array structures. A "string descriptor" is defined as an array of bytes, that begins with a standard "header" of 2 bytes, followed by a string of UNICODE (16-bit) characters.
For example a USB string descriptor of value "AB" would have the following sequence of bytes:
0x06 0x03 0x41 0x00 0x42 0x00
where 0x06 is the total size of the descriptor (including the header), 0x03 is its "type" (defined by the standard)
Current (unsatisfactory) approach:
// other types omitted for clarity
enum UsbDescriptorType: uint8_t { USB_DESCR_STRING = 0x03 };
struct UsbDescrStd {
uint8_t bLength;
UsbDescriptorType bDescriptorType;
};
template<size_t N>
struct UsbDescrString final: UsbDescrStd {
char str[N * 2];
constexpr UsbDescrString(const char s[N]) noexcept
: UsbDescrStd{sizeof(*this), UsbDescriptorType::USB_DESCR_STRING}
, str {}
{
for(size_t i = 0; i < N; ++i)
str[i * 2] = s[i];
}
};
Below are the examples of its usage and short comments on why they are "not good enough" for me:
// requires size information
constexpr UsbDescrString<9> uds9{"Descr str"};
// string duplication
constexpr UsbDescrString<sizeof("Descr str")-1> udsa{"Descr str"};
// requires an explicit string storage
constexpr auto UsbDescrStrTxt{"Descr str"};
constexpr UsbDescrString<sizeof(UsbDescrStrTxt)-1> udsa2{UsbDescrStrTxt};
// ugly use of a macro
#define MAKE_UDS(name, s) UsbDescrString<sizeof(s)-1> name{s}
constexpr MAKE_UDS(udsm, "Descr str");
"String argument to template" is explicitly prohibited as of C++20, cutting that solution off as well.
What I'm trying to achieve
Ideally I'd love to be able to write code like the following:
constexpr UsbDescrString uds{"Descr str"}; // or a similar "terse" approach
It is simple, terse, error-resistant, and to the point. And I need help writing my UsbDescrString in a way that allows me to create compile-time objects without unnecessary code bloat.
Adding a CTAD to UsbDescrString should be enough
template<size_t N>
struct UsbDescrString final: UsbDescrStd {
char str[N * 2];
constexpr UsbDescrString(const char (&s)[N+1]) noexcept
: UsbDescrStd{sizeof(*this), UsbDescriptorType::USB_DESCR_STRING}
, str {}
{
for(size_t i = 0; i < N; ++i)
str[i * 2] = s[i];
}
};
template<size_t N>
UsbDescrString(const char (&)[N]) -> UsbDescrString<N-1>;
Note that in order to prevent array to pointer decay, const char (&) needs to be used as the constructor parameter.
Demo
"String argument to template" is explicitly prohibited as of C++20,
cutting that solution off as well.
However, thanks to P0732, with the help of some helper classes such as basic_fixed_string, now in C++20 you can
template<fixed_string>
struct UsbDescrString final: UsbDescrStd;
constexpr UsbDescrString<"Descr str"> uds9;
The c++/cli template below is working, but it seems like there ought to be a way to generalize the template further or add a helper that can create the template instances at compile time.
Was thinking something like http://en.cppreference.com/w/cpp/utility/integer_sequence might work, but need some assistance with the helper / implementer function.
Simplified main to demonstrate the desired syntax vs what is currently used:
int main(array<String^>^ args) {
// the actual number of possible char lengths is sparse (generally)
// but the API allows for 1-1024
List<int>^ varList = gcnew List<int>();
varList->Add(40);
varList->Add(80);
varList->Add(128);
SortedList<int, List<String^>^>^ allStrings = gcnew SortedList<int, List<String^>^>();
// want something like this, but the compiler complains that
// the template is invalid expectes compile-time constant expressions
for each(int key in varList) {
allStrings->Add(key, UpdateTest<key>());
}
// this works, but has 1024 lines of case N:
for each(int key in varList) {
switch (key) {
case 1: allStrings->Add(key, UpdateTest<1>());
case 2: allStrings->Add(key, UpdateTest<2>());
case 3: allStrings->Add(key, UpdateTest<3>());
//... all 1024 possible char[N] sizes...
case 1024: allStrings->Add(key, UpdateTest<1024>());
}
}
}
Template works with the 1024 switch case N: calls. Is there a way to have a helper / implementer instantiate all 1024 without the cases?
template <std::size_t N> List<String^>^ APIwrapper::UpdateTest() {
typedef char CHARX[N]; // N valid for 1 to 1024
CHARX vals[MAXFIELDS];
// NATIVE C++ VendorAPI(short, void*) vals is the address of the word aligned destination data
int count = VendorAPI(N, &vals);
List<String^>^ retList = gcnew List<String^>();
for (int c = 0; c < count; c++) {
CHARX temp;
strncpy(temp, vals[c], N); // \0 terminated char arrays
String^ x = gcnew String(temp);
retList->Add(x->Substring(0, N)->Trim());
}
return retList;
}
Following answer works for c++14, I don't know if it's compatible with cli.
You can use templates to generate a std::array of function pointers:
using UpdateTestPtr = decltype(&UpdateTest<0>);
// Struct used to generate the array's content.
template<typename Sequence>
struct UpdateTestArrayImpl;
// Specialization used to get the values in the integer sequence.
template<std::size_t... indices>
struct UpdateTestArrayImpl<std::integer_sequence<std::size_t,indices...>> {
static constexpr std::array<UpdateTestPtr,sizeof...(indices)> value{UpdateTest<indices>...};
};
// Factorise sequence creation
template<std::size_t N>
using UpdateTestArray = UpdateTestArrayImpl<std::make_index_sequence<N>>;
static constexpr std::size_t N = 512;
// The array is needed at runtime. Create a normal (not constexpr) instance.
static std::array<UpdateTestPtr,N> functionArray = UpdateTestArray<N>::value;
And turn the switch/case into an array lookup:
for each(int key in varList) {
allStrings->Add(key, functionArray[key]());
}
Live demo
Some compilers might produce a "template instantiation depth" error, depending on how make_integer_sequence is implemented. The max limit of recursion depth can normally be increased with compiler options.
Don't do this with templates.
You're using templates to turn a single large block of memory (CHARX vals[MAXFIELDS];) into lots of individual strings. Instead, do that yourself.
List<String^>^ APIwrapper::UpdateTest(size_t size)
{
char* vals = new char[size * MAXFIELDS];
// NATIVE C++ VendorAPI(short, void*) vals is the address of the word aligned destination data
int count = VendorAPI(size, vals);
List<String^>^ retList = gcnew List<String^>();
char temp[1025]; // max size + 1.
for (int c = 0; c < count; c++)
{
// Instead of relying on the compiler to know that [1] should be 10 bytes in,
// [2] should be 20 bytes in, etc, do that yourself.
strncpy(temp, &vals[size * c], size);
temp[size] = '\0'; // safety
String^ x = gcnew String(temp);
retList->Add(x->Trim());
}
delete [] vals;
return retList;
}
Now, having said that: This is an unusual API. Effectively, you're passing a block of memory, and a stride for the placement of each string. That's somewhat strange, but also only some data can be retrieved with each stride value??? That's very strange. I would check if the vendor's API will still respond properly if 1024 is always used as the stride.
Today I talked to a friend about the differences between statically and dynamically typed languages (more info about the difference between static and dynamic typed languages in this SO question). After that, I was wondering what kind of trick can be used in C++ to emulate such dynamic behavior.
In C++, as in other statically typed languages, the variable type is specified at compile time. For example, let's say I have to read from a file a big amount of numbers, which are in the majority of the cases quite small, small enough to fit in an unsigned short type. Here comes the tricky thing, a small amount of these values are much bigger, bigger enough to need an unsigned long long to be stored.
Since I assume I'm going to do calculations with all of them I want all of them stored in the same container in consecutive positions of memory in the same order than I read them from the input file.. The naive approach would be to store them in a vector of type unsigned long long, but this means having typically up to 4 times extra space of what is actually needed (unsigned short 2 bytes, unsigned long long 8 bytes).
In dynamically typed languages, the type of a variable is interpreted at runtime and coerced to a type where it fits. How can I achieve something similar in C++?
My first idea is to do that by pointers, depending on its size I will store the number with the appropriate type. This has the obvious drawback of having to also store the pointer, but since I assume I'm going to store them in the heap anyway, I don't think it matters.
I'm totally sure that many of you can give me way better solutions than this ...
#include <iostream>
#include <vector>
#include <limits>
#include <sstream>
#include <fstream>
int main() {
std::ifstream f ("input_file");
if (f.is_open()) {
std::vector<void*> v;
unsigned long long int num;
while(f >> num) {
if (num > std::numeric_limits<unsigned short>::max()) {
v.push_back(new unsigned long long int(num));
}
else {
v.push_back(new unsigned short(num));
}
}
for (auto i: v) {
delete i;
}
f.close();
}
}
Edit 1:
The question is not about saving memory, I know in dynamically typed languages the necessary space to store the numbers in the example is going to be way more than in C++, but the question is not about that, it's about emulating a dynamically typed language with some c++ mechanism.
Options include...
Discriminated union
The code specifies a set of distinct, supported types T0, T1, T2, T3..., and - conceptually - creates a management type to
struct X
{
enum { F0, F1, F2, F3... } type_;
union { T0 t0_; T1 t1_; T2 t2_; T3 t3_; ... };
};
Because there are limitations on the types that can be placed into unions, and if they're bypassed using placement-new care needs to be taken to ensure adequate alignment and correct destructor invocation, a generalised implementation becomes more complicated, and it's normally better to use boost::variant<>. Note that the type_ field requires some space, the union will be at least as large as the largest of sizeof t0_, sizeof t1_..., and padding may be required.
std::type_info
It's also possible to have a templated constructor and assignment operator that call typeid and record the std::type_info, allowing future operations like "recover-the-value-if-it's-of-a-specific-type". The easiest way to pick up this behaviour is to use boost::any.
Run-time polymorphism
You can create a base type with virtual destructor and whatever functions you need (e.g. virtual void output(std::ostream&)), then derive a class for each of short and long long. Store pointers to the base class.
Custom solutions
In your particular scenario, you've only got a few large numbers: you could do something like reserve one of the short values to be a sentinel indicating that the actual value at this position can be recreated by bitwise shifting and ORing of the following 4 values. For example...
10 299 32767 0 0 192 3929 38
...could encode:
10
299
// 32767 is a sentinel indicating next 4 values encode long long
(0 << 48) + (0 << 32) + (192 << 16) + 3929
38
The concept here is similar to UTF-8 encoding for international character sets. This will be very space efficient, but it suits forward iteration, not random access indexing a la [123].
You could create a class for storing dynamic values:
enum class dyn_type {
none_type,
integer_type,
fp_type,
string_type,
boolean_type,
array_type,
// ...
};
class dyn {
dyn_type type_ = dyn_type::none_type;
// Unrestricted union:
union {
std::int64_t integer_value_;
double fp_value_;
std::string string_value_;
bool boolean_value_;
std::vector<dyn> array_value_;
};
public:
// Constructors
dyn()
{
type_ = dyn_type::none_type;
}
dyn(std::nullptr_t) : dyn() {}
dyn(bool value)
{
type_ = dyn_type::boolean_type;
boolean_value_ = value;
}
dyn(std::int32_t value)
{
type_ = dyn_type::integer_type;
integer_value_ = value;
}
dyn(std::int64_t value)
{
type_ = dyn_type::integer_type;
integer_value_ = value;
}
dyn(double value)
{
type_ = dyn_type::fp_type;
fp_value_ = value;
}
dyn(const char* value)
{
type_ = dyn_type::string_type;
new (&string_value_) std::string(value);
}
dyn(std::string const& value)
{
type_ = dyn_type::string_type;
new (&string_value_) std::string(value);
}
dyn(std::string&& value)
{
type_ = dyn_type::string_type;
new (&string_value_) std::string(std::move(value));
}
// ....
// Clear
void clear()
{
switch(type_) {
case dyn_type::string_type:
string_value_.std::string::~string();
break;
//...
}
type_ = dyn_type::none_type;
}
~dyn()
{
this->clear();
}
// Copy:
dyn(dyn const&);
dyn& operator=(dyn const&);
// Move:
dyn(dyn&&);
dyn& operator=(dyn&&);
// Assign:
dyn& operator=(std::nullptr_t);
dyn& operator=(std::int64_t);
dyn& operator=(double);
dyn& operator=(bool);
// Operators:
dyn operator+(dyn const&) const;
dyn& operator+=(dyn const&);
// ...
// Query
dyn_type type() const { return type_; }
std::string& string_value()
{
assert(type_ == dyn_type::string_type);
return string_value_;
}
// ....
// Conversion
explicit operator bool() const
{
switch(type_) {
case dyn_type::none_type:
return true;
case dyn_type::integer_type:
return integer_value_ != 0;
case dyn_type::fp_type:
return fp_value_ != 0.0;
case dyn_type::boolean_type:
return boolean_value_;
// ...
}
}
// ...
};
Used with:
std::vector<dyn> xs;
xs.push_back(3);
xs.push_back(2.0);
xs.push_back("foo");
xs.push_back(false);
An easy way to get dynamic language behavior in C++ is to use a dynamic language engine, e.g. for Javascript.
Or, for example, the Boost library provides an interface to Python.
Possibly that will deal with a collection of numbers in a more efficient way than you could do yourself, but still it's extremely inefficient compared to just using an appropriate single common type in C++.
The normal way of dynamic typing in C++ is a boost::variant or a boost::any.
But in many cases you don't want to do that. C++ is a great statically typed language and it's just not your best use case to try to force it to be dynamically typed (especially not to save memory use). Use an actual dynamically typed language instead as it is very likely better optimized (and easier to read) for that use case.
I'm getting a string containing raw binary data which needs to be converted to integers. The Problem is these values are not always in the same order and do not always appear. So the format of the binary data gets described in a config file and the type of the values read from the binary data is not known at compile time.
I'm thinking of a solution similar to this:
enum BinaryType {
TYPE_UINT16,
TYPE_UNIT32,
TYPE_INT32
};
long convert(BinaryType t, std::stringstream ss) {
long return_value;
switch(t) {
case TYPE_UINT16:
unsigned short us_value;
ss.read(&us_value, sizeof(unsigned short));
return_value = short;
break;
case TYPE_UINT32:
unsigned int ui_value;
ss.read(&ui_value, sizeof(unsigned int));
return_value = ui_value;
break;
case TYPE_INT32:
signed int si_value;
ss.read(&si_value, sizeof(signed int));
return_value = si_value;
break;
}
return return_value;
}
The goal is to output these values in decimal.
My Questions are:
This code is very repetitive. Is there a simpler solution? (Templates?)
should I make use of the standard types like signed int if the value needs to be 32 bit? What to use instead? Endianness?
A simple solution: define a base class for converters:
class Converter {
public:
virtual int_64 convert(std::stringstream& ss) = 0;
}
Next define a concrete converter for each binary type. Have a map/array mapping from binary types identifiers to your converters, e.g.:
Converter* converters[MAX_BINARY_TYPES];
converters[TYPE_UINT16] = new ConverterUINT16;
...
Now, you can use it like this (variables defined like in your function convert):
cout << converters[t]->convert(ss)
For portability, instead of basic types like int, long, etc, you should use int32_t, int64_t which are guaranteed to be the same on all systems.
Of course, if your code is meant to deal with different endianness, you need to deal with it explicitly. For the above example code you can have two different converters' sets, one for little endian data decoding, another for big endian. Another thing you can do is to write a wrapper class for std::stringstream, let's call it StringStream, which defines functions for reading int32, uint32, etc., and swaps the bytes if the endianness is different than the architecture of the system your code is running on. You can make the class a template and instantiate it with one of the two:
class SameByteOrder {
public:
template<typename T> static void swap(T &) {}
};
class OtherByteOrder {
public:
template<typename T> static void swap(T &o)
{
char *p = reinterpret_cast<char *>(&o);
size_t size = sizeof(T);
for (size_t i=0; i < size / 2; ++i)
std::swap(p[i], p[size - i - 1]);
}
};
then use the swap function inside your StringStream's functions to swap (or not) the bytes.
I have a lump of binary data in the form of const std::vector<unsigned char>, and want to be able to extract individual fields from that, such as 4 bytes for an integer, 1 for a boolean, etc. This needs to be, as far as possible, both efficient and simple. eg. It should be able to read the data in place without needing to copy it (eg. into a string or array). And it should be able to read one field at a time, like a parser, since the lump of data does not have a fixed format. I already know how to determine what type of field to read in each case - the problem is getting a usable interface on top of an std::vector for doing this.
However I can't find a simple way to get this data into an easily usable form that gives me useful read functionality. eg. std::basic_istringstream<unsigned char> gives me a reading interface, but it seems like I need to copy the data into a temporary std::basic_string<unsigned char> first, which is not idea for bigger blocks of data.
Maybe there is some way I can use a streambuf in this situation to read the data in place, but it would appear that I'd need to derive my own streambuf class to do that.
It occurs to me that I can probably just use sscanf on the vector's data(), and that would seem to be both more succinct and more efficient than the C++ standard library alternatives. EDIT: Having been reminded that sscanf doesn't do what I wrongly thought it did, I actually don't know a clean way to do this in C or C++. But am I missing something, and if so, what?
You have access to the data in a vector through its operator[]. A vector's data is guranteed to be stored in a single contiguous array, and [] returns a reference to a member of that array. You may use that reference directly, or through a memcpy.
std::vector<unsigned char> v;
...
byteField = v[12];
memcpy(&intField, &v[13], sizeof intField);
memcpy(charArray, &v[20], lengthOfCharArray);
EDIT 1:
If you want something "more convenient" that that, you could try:
template <class T>
ReadFromVector(T& t, std::size_t offset,
const std::vector<unsigned char>& v) {
memcpy(&t, &v[offset], sizeof(T));
}
Usage would be:
std::vector<unsigned char> v;
...
char c;
int i;
uint64_t ull;
ReadFromVector(c, 17, v);
ReadFromVector(i, 99, v);
ReadFromVector(ull, 43, v);
EDIT 2:
struct Reader {
const std::vector<unsigned char>& v;
std::size_t offset;
Reader(const std::vector<unsigned char>& v) : v(v), offset() {}
template <class T>
Reader& operator>>(T&t) {
memcpy(&t, &v[offset], sizeof t);
offset += sizeof t;
return *this;
}
void operator+=(int i) { offset += i };
char *getStringPointer() { return &v[offset]; }
};
Usage:
std::vector<unsigned char> v;
Reader r(v);
int i; uint64_t ull;
r >> i >> ull;
char *companyName = r.getStringPointer();
r += strlen(companyName);
If your vector stores binary data, you can't use sscanf or similar, they work on text.
For converting a byte for a bool is simple enough
bool b = my_vec[10];
For extracting an unsigned int that's stored in big endian order (assuming your ints are 32 bits):
unsigned int i = my_vec[10] << 24 | my_vec[11] << 16 | my_vec[12] << 8 | my_vec[13];
A 16 bit unsigned short would be similar:
unsigned short s = my_vec[10] << 8 | my_vec[11];ยจ
If you can afford the Qt dependency, QByteArray has the fromRawData() named constructor, which wraps existing data buffers in a QByteArray without copying the data. With that byte array, you can the feed a QTextStream.
I'm not aware of any such function in the standard streams library (short of implementing your own streambuf, of course), but I'd love to be proved wrong :)
You can use a struct that describes the data you are trying to extract. You can move data from your vector into the struct like this:
struct MyData {
int intVal;
bool boolVal;
char[15] stringVal;
} __attribute__((__packed__));
// assuming all extracted types are prefixed with a one byte indicator.
// Also assumes "vec" is your populated vector
int pos = 0;
while (pos < vec.size()-1) {
switch(vec[pos++]) {
case 0: { // handle int
int intValue;
memcpy(&vec[pos], &intValue, sizeof(int));
pos += sizeof(int);
// do something with handled value
break;
}
case 1: { // handle double
double doubleValue;
memcpy(&vec[pos], &doubleValue, sizeof(double));
pos += sizeof(double);
// do something with handled value
break;
}
case 2: { // handle MyData
struct MyData data;
memcpy(&vec[pos], &data, sizeof(struct MyData));
pos += sizeof(struct MyData);
// do something with handled value
break;
}
default: {
// ERROR: unknown type indicator
break;
}
}
}
Use a for loop to iterate over the vector and use bitwise operators to access each bit group. For example, to access the upper four bits of the first usigned char in your vector:
int myInt = vec[0] & 0xF0;
To read the fifth bit from the right, right after the chunk we just read:
bool myBool = vec[0] & 0x08;
The three least significant (lowest) bits can be accesed like so:
int myInt2 = vec[0] & 0x07;
You can then repeat this process (using a for loop) for every element in your vector.