Get a char* from an ostream without copying - c++

I have an ostream and data has been written to it. Now I want that data in the form of a char array. Is there a way to get the char buffer and its size without copying all of the bytes? I mean, I know I can use ostringstream and call str().c_str() on it but that produces a temporary copy.

I guess this is what you're looking for - a stream buffer that returns a pointer to its buffer:
#include <iostream>
#include <vector>
#include <string>
class raw_buffer : public std::streambuf
{
public:
raw_buffer(std::ostream& os, int buf_size = 256);
int_type overflow(int_type c) override;
std::streamsize showmanyc() override;
std::streamsize xsputn(const char_type*, std::streamsize) override;
int sync() override;
bool flush();
std::string const& str() const;
private:
std::ostream& os_;
std::vector<char> buffer;
std::string aux;
};
Now str() is simple. It returns a pointer to the underlying buffer of the auxillary buffer:
std::string const& raw_buffer::str() const
{
return aux;
}
The rest of the functions are the usual implementations for a stream buffer. showmanyc() should return the size of the auxiliary buffer (aux is just a running total of the entire buffer, buffer on the other hand is the size specified at construction).
For example, here is overflow(), which should update both buffers at same time but still treat buffer as the primary buffer:
raw_buffer::int_type raw_buffer::overflow(raw_buffer::int_type c) override
{
if (os_ && !traits_type::eq_int_type(c, traits_type::eof()))
{
aux += *this->pptr() = traits_type::to_char_type(c);
this->pbump(1);
if (flush())
{
this->pbump(-(this->pptr() - this->pbase()));
this->setp(this->buffer.data(),
this->buffer.data() + this->buffer.size());
return c;
}
}
return traits_type::eof();
}
flush() is used to copy the contents of buffer to the stream (os_), and sync() should be overrided to call flush() too.
xsputn also needs to be overrided to write to aux as well:
std::streamsize raw_buffer::xsputn(const raw_buffer::char_type* str, std::streamsize count) override
{
for (int i = 0; i < count; ++i)
{
if (traits_type::eq_int_type(this->sputc(str[i]), traits_type::eof()))
return i;
else
aux += str[i];
}
return count;
}
Now we can put this together with a customized stream:
class raw_ostream : private virtual raw_buffer
, public std::ostream
{
public:
raw_ostream(std::ostream& os) : raw_buffer(os)
, std::ostream(this)
{ }
std::string const& str() const
{
return this->raw_buffer::str();
}
std::streamsize count()
{
return this->str().size();
}
};
It can be used like this:
int main()
{
raw_ostream rostr(std::cout);
rostr << "Hello, World " << 123 << true << false;
auto& buf = rostr.str();
std::cout << buf;
}

Related

Arduino split char* based on delimiter to value

thanks for your time, so I have a Char* from mqtt
I want to break this down into 3 seperate values
Char* mqttvalue
//Input Would be like the below for example.
mqttvalue = (255,200,230);
// I would like to split the values into the below.
int 1 = 255
int 2 = 200
int 3 = 230
I've tried strtok with no luck. probably something really dumb but some guidance would help.
Thank you
Edit, what I tried.
//Dummy Value for testing
Split("255,240,230");
//Split Value
void Split(char* e) {
String v[3];
char *p;
int i = 0;
p = strtok(e, ",");
while(p && i < 3)
{
v[i] = p;
p = strtok(NULL, ",");
Serial.println(p);
++i;
};
Serial.println(v[0]);
Serial.println(v[1]);
Serial.println(v[2]);
}
Updated the code to the below from a string to char* its all now working.
//Split Value
void Split(char* e) {
char* v[3];
char *p;
int i = 0;
p = strtok(rgb, ",");
while(p && i < 3)
{
v[i] = p;
p = strtok(NULL, ",");
i++;
};
Serial.println(v[0]);
Serial.println(v[1]);
Serial.println(v[2]);
};
As this is quite often asked question and I'd propose more like using wrapper class inheriting Stream interface on C string (and it can be initialized from Arduino String object too).
However usage with Arduino String class is tricky as the original String shouldn't be altered during usage of StringStream, but it's possible to reinitialize it again. Using rvalue (String literal, passed into the class) is forbidden by using non const reference as the parameter of setData method and constructor.
However it's not tested much, so there might be some mistakes. The copy/move constructor and assigment is ommited (and it shouldn't be :D), also using operator=(String&) and operator=(const char*) would be more intuitive interface for it.
class StringStream : public Stream
{
public:
StringStream()
{
setTimeout(1);
}
StringStream(const char * str)
{
setData(str);
}
StringStream(const char * begin, const char * end)
{
setData(begin, end);
}
explicit StringStream(String & view) // cannot be String literal (rvalue) and it gets invalidated if you change original String
{
setData(view);
}
////////////////////////////////////
inline void setData(const char * begin, const char * end)
{
m_start = begin;
m_end = end;
setTimeout(1);
}
inline void setData(const char * begin)
{
setData(begin, begin + strlen(begin));
}
inline void setData(String & view)
{
setData(view.c_str(), m_start + view.length());
}
//////////////////////////////////
// Stream Interface:
virtual int available() override
{
return m_end - m_start;
}
virtual int read() override
{
if (m_start < m_end)
{
return *(m_start++);
}
return -1;
}
virtual int peek() override
{
if (m_start < m_end)
{
return *m_start;
}
return -1;
}
virtual size_t write(uint8_t) override {
return 0;
}
protected:
const char * m_start{0};
const char * m_end{0};
};
And the test program would be like:
void setup() {
Serial.begin(115200);
}
void loop() {
Serial.println("Output:");
StringStream test{" 144, 7899, -5478"};
Serial.println(test.parseInt());
Serial.println(test.parseInt());
Serial.println(test.parseInt());
test.setData("1 2");
Serial.println(test.parseInt());
Serial.println(test.parseInt());
delay(2000);
}

No output from using array in class

I get a blank output. I'm a newbie and have been struggling on this for some time.
I have gotten 0 errors by the compiler.
Also what can be improved on this?
How can I get the length of const char* as an int instead of size_t without having to use static_cast.
#include <iostream>
#include <cassert>
class String
{
private:
char* Str_Buffer{};
int Str_Size{};
public:
String(const char* string = " ")
: Str_Size{ static_cast<int>(strlen(string)) }
{
Str_Buffer = new char[Str_Size];
}
String& operator=(const String& string)
{
if (this == &string)
return *this;
delete[] Str_Buffer;
Str_Size = string.Str_Size;
if (string.Str_Buffer)
{
Str_Buffer = new char[Str_Size];
for (int index{ 0 }; index < Str_Size; ++index)
Str_Buffer[index] = string.Str_Buffer[index];
}
return *this;
}
char& operator[](const int index)
{
assert(index >= 0);
assert(index < Str_Size);
return Str_Buffer[index];
}
friend std::ostream& operator<<(std::ostream& out, const String& string)
{
out << string.Str_Buffer;
return out;
}
~String()
{
delete[] Str_Buffer;
}
};
int main()
{
String word("Hello world!");
std::cout << word;
return 0;
}
I get a blank output.
You don't fill your String::Str_Buffer with meaningful data in the constructor. You could use std::strcpy() from <cstring> to do that. std::strlen() is also declared in that header file. To use std::strcpy() the memory pointed to by String::Str_Buffer needs to be one char bigger than the string you want to copy there because strings in C and C++ are zero-terminated ('\0').
How can I get the length of const char* as an int instead of size_t without having to use static_cast.
Why would you want an int? Sizes of objects in C++ are measured with values of type std::size_t (defined in several headers but when in doubt include <cstddef>). std::size_t is guaranteed to be big enough to handle all object sizes. It is for example the return type of std::strlen() and the sizeof-operator.
Your assignment operator is not exception-safe:
String& operator=(const String& string)
{
// ...
delete[] Str_Buffer; // the old state is now gone
Str_Size = string.Str_Size;
if (string.Str_Buffer)
{
Str_Buffer = new char[Str_Size]; // when new[] throws, the object
// will be in an undefined state
// ...
Possible but not elegant solution:
String& operator=(const String& string)
{
char *temp = new[string.Str_Size];
// copy string.Str_Buffer to temp
delete[] Str_Buffer;
Str_Buffer = temp;
Str_Size string.Str_Size
return *this;
}
See Copy-and-Swap for an better solution.
Resource Management
Please familiarize yourself with The Rule of Five and the Copy-and-Swap Idiom.
A starting point for a class that manages a string could look like that:
#include <cassert> // assert()
#include <cstddef> // std::size_t
#include <cstring> // std::strlen(), std::strcpy()
#include <utility> // std::swap(), std::exchange()
#include <iostream>
class string_t
{
size_t length = 0;
char *data = nullptr;
public:
string_t() = default;
string_t(char const *str)
: length { str ? std::strlen(str) : 0 },
data { new char[length + 1]{} }
{
str && std::strcpy(data, str);
}
string_t(string_t const &other) // copy constructor
: length { other.length },
data { new char[length + 1]{} }
{
other.data && std::strcpy(data, other.data);
}
string_t(string_t &&other) // move constructor
: length { std::exchange(other.length, 0) }, // steal others resources and
data { std::exchange(other.data, nullptr) } // give other a state it's
{} // destructor can work with
string_t& operator=(string_t other) // assignment operator
{ // mind: other gets copied
std::swap(length, other.length); // steal other's resources
std::swap(data, other.data); // other's destructor will
} // take care of ours.
~string_t() { delete[] data; }
std::size_t get_length() const { return length; }
char& operator[](std::size_t index)
{
assert(index < length);
return data[index];
}
// stream-insertion operator:
friend std::ostream& operator<<(std::ostream &os, string_t const &str)
{
return os << (str.data ? str.data : "");
}
};
int main()
{
string_t foo{ "Hello!" }; // char const* constructor
std::cout << foo << '\n';
string_t bar{ foo }; // copy constructor
std::cout << bar << '\n';
string_t qux{ string_t{ "World!" } }; // move constructor (from a temporary)
std::cout << qux << '\n';
bar = qux; // assignment operator
std::cout << bar << '\n';
}
First of all, you need to include for strlen. You get a blank output because the constructor does not write the input string to Str_Buffer. You may use std::copy to copy the memory to the allocated buffer.
You have to use static cast, because strlen returns std::size_t. Just change the type of Str_Size to std::size_t to get rid of the static cast.
Also take a look at the rule of five. Defining a move and copy constuctor will improve performace of your code.
See a working version of your code below:
#include <iostream>
#include <cassert>
#include <cstring>
#include <algorithm>
class String
{
private:
char* Str_Buffer;
std::size_t Str_Size;
public:
String(const char* string = " ")
: Str_Size{ strlen(string) }
{
Str_Buffer = new char[Str_Size];
std::copy(string, string + Str_Size, Str_Buffer);
}
String(const String& other)
: Str_Size(other.Str_Size)
{
Str_Buffer = new char[Str_Size];
std::copy(other.Str_Buffer, other.Str_Buffer + Str_Size, Str_Buffer);
}
String(String && other)
{
*this = std::move(other);
}
String& operator=(const String& string)
{
if (this == &string)
return *this;
delete[] Str_Buffer;
Str_Size = string.Str_Size;
if (string.Str_Buffer)
{
Str_Buffer = new char[Str_Size];
for (std::size_t index = 0; index < Str_Size; ++index)
Str_Buffer[index] = string.Str_Buffer[index];
}
return *this;
}
char& operator[](const int index)
{
assert(index >= 0);
assert(index < Str_Size);
return Str_Buffer[index];
}
friend std::ostream& operator<<(std::ostream& out, const String& string)
{
out << string.Str_Buffer;
return out;
}
~String()
{
delete[] Str_Buffer;
}
};
int main()
{
String word("Hello world!");
std::cout << word;
return 0;
}

c++ compress with libzpaq to char* buffer

I have a std::vector<short> and would like to compress (and later decompress) with the libzpaq from https://github.com/zpaq/zpaq/ to something like char* buffer.
However I don't get the concept of this Reader and Writer class mentioned in the header file. How do I put my std::vector in to get a compressed buffer out?
Currently I have something like the following code.
#include <vector>
#include <string>
#include <stdio.h>
#include "libzpaq.h"
struct writer: public libzpaq::Writer {
void put(int c) {
}
};
struct reader: public libzpaq::Reader {
int get() {
}
};
void libzpaq::error(const char* msg) {
fprintf(stderr, "Oops: %s\n", msg);
exit(1);
}
int main() {
short a[] = {2,5,8,2,4,2,2,2,6,5,4,3,4,2,2};
std::vector<short> v(a, a+15);
char* buffer;
reader in;
writer out;
libzpaq::compress(&in, &out, "5");
}
And I wan't to compress the vector v into buffer. (And later decompress it again.)
But I don't understand the concept of the Reader and Writer struct/class.
The docu (http://mattmahoney.net/dc/libzpaq.3.html) also mentions the functions virtual int read(char* buf, int n) and virtual void write(const char* buf, int n) for the Reader and Writer. How can I cast a std::vector<short> to char* buf end get the length in n bytes of this buf?
Edit 1: I found a class StringBuffer in libzpaq.h line 1376. But something like
buffer = reinterpret_cast<char*> (&v[0]);
length = sizeof(short)*v.size();
libzpaq::StringBuffer inString, outString;
inString.read(buffer, length);
libzpaq::compress(&inString, &outString, "5");
std::cout << "size outstring: " << outString.size() << std::endl;
std::cout << "size instring: " << inString.size() << std::endl;
always gives me
size outstring: 0
size instring: 0
Even if I try it with a much larger vector v of some thousend random elements.
With Reader you provide byte by byte access to the data you want to compress. So with std::vector<short> it would look like this.
struct reader : public libzpaq::Reader {
reader(const std::vector<short>& v) :
m_v(v),
m_offset(0) {
}
int get() {
if (m_offset < m_v.size() * sizeof (short)) {
return *((char*) m_v.data() + m_offset++);
} else {
return -1;
}
}
int m_offset;
std::vector<short> m_v;
};
Writer should collect output data of the Reader. If you want to collect it in char array I could recommend to do it like this.
struct writer : public libzpaq::Writer {
void put(int c) {
m_buffer.push_back(c);
}
int size() {
m_buffer.size();
}
void copy_to(char* dst) {
memcpy(dst, m_buffer.data(), m_buffer.size());
}
std::vector<char> m_buffer;
};
Then call it:
writer w;
reader r(v1);
libzpaq::compress(&r, &w, "5");
char* buffer = new char[w.size()];
w.copy_to(buffer);
If you want to use StringBuffer then you should write some data to buffer, before read, that why it returns 0. Look at example:
char* buffer = reinterpret_cast<char*> (&v[0]);
int length = sizeof (short)*v.size();
libzpaq::StringBuffer in, out1, out2;
// fill buffer with source data
in.write(buffer, length);
// compress to out1
libzpaq::compress(&in, &out1, "5");
// decompress out1 to out2
libzpaq::decompress(&out1, &out2);
// check result
short* b = (short*)out2.data();
for(int i = 0; i < 15; ++i) {
std::cout << b[i] << std::endl;
}

Redirect and timestamp std::cout

I need to redirect an ofstream to a file and timestamp every line that's printed.
(It's part of a logging system).
I have a working class that manages to do just that but it refuses to flush the file when std::endl is emmited. I'd apreciate any help on that.
(If there is a simpler way to do this do tell).
#include <iostream>
#include <streambuf>
#include <fstream>
#include <sys/time.h>
#include <cstring>
#include <memory>
class TimeStampBuf: public std::streambuf {
public:
explicit TimeStampBuf(std::streambuf* dest) :
_dest(dest),
_isAtStartOfLine(true),
_owner( NULL) {
}
explicit TimeStampBuf(std::ostream& dest) :
_dest(dest.rdbuf()),
_isAtStartOfLine(true),
_owner(&dest) {
_owner->rdbuf(this);
}
virtual ~TimeStampBuf() {
if (_owner != NULL) {
_owner->rdbuf(_dest);
}
}
protected:
virtual int overflow(int ch) {
if (_isAtStartOfLine) {
char timebuff[30];
timeval curTime;
gettimeofday(&curTime, NULL);
strftime(timebuff, sizeof(timebuff), "%Y-%m-%d %H:%M:%S:",
localtime(&curTime.tv_sec));
sprintf(timebuff + strlen(timebuff), "%03u\t",
(unsigned int) curTime.tv_usec / 1000);
_dest->sputn(timebuff, strlen(timebuff));
}
_isAtStartOfLine = ch == '\n';
return _dest->sputc(ch);
}
private:
std::streambuf *_dest;
bool _isAtStartOfLine;
std::ostream *_owner;
};
class OutputRedirectAndStamp {
public:
OutputRedirectAndStamp(std::string file, std::ostream &s = std::cout, std::ios::openmode mode = std::ios::out){
_s=&s;
_file=file;
if(_file.size()){
_mode=mode;
_buf.open(file.c_str(),mode);
_coutBuf = s.rdbuf((std::streambuf*)&_buf);
}
_tsb.reset(new TimeStampBuf(s));
}
void reopen(void){
_tsb.reset();
if(_file.size()){
_s->rdbuf(_coutBuf); //reset to previous output
_buf.close();
_buf.open(_file.c_str(),_mode);
_coutBuf = _s->rdbuf((std::streambuf*)&_buf);
}
_tsb.reset(new TimeStampBuf(*_s));
}
~OutputRedirectAndStamp() {
_tsb.reset();
if(_file.size()){
_s->rdbuf(_coutBuf); //reset to previous output
}
}
private:
std::string _file;
std::ios::openmode _mode;
std::ostream *_s;
std::filebuf _buf;
std::streambuf *_coutBuf;
std::unique_ptr<TimeStampBuf> _tsb;
};
int main() //example main
{
std::unique_ptr<OutputRedirectAndStamp> a;
a.reset(new OutputRedirectAndStamp("test.txt",std::cout,std::ios::app | std::ios::out));
std::cout<<"this is written to file"<<2<<std::endl;
a->reopen();
std::cout<<"this is written to file also"<<std::endl;
a.reset();
std::cout<<"this is written to stdout"<<std::endl;
return 0;
}
When you flush an std::ostream, the stream buffer's pubsync() is called which in turn calls the virtual function sync(). If you don't override sync() it will just claim that it succeeded by returning 0. From your sync() override you should just call the held stream buffer's pubsync():
int TimeStampBuf::sync() {
return _dest->pubsync();
}

QDataStream on external memory?

I'd like serialize QVector into char* array. I do this by the following code:
QVector<int> in;
...
QByteArray bytes;
QDataStream stream(&bytes, QIODevice::WriteOnly);
stream << in;
std::copy(bytes.constData(), bytes.constData() + bytes.size(), out);
I guarantee that out is large enough. Due to the fact that this code is called extremely often I would like to avoid this unnecessary std::copy operation and make either QByteArray or QDataStream work on preallocated user memory pointed by out. Is that possible? Any bight ideas?
UPDATE: QByteArray::fromRawData() doesn't match the needs cause it does not allow to change char* buffer it was created on, in other words, QByteArray performs deep copy on first modification of such created instance.
As they say. This ensures that the raw data array itself will never be modified by QByteArray.
SOLUTION: The solution proposed by #skyhisi does perfectly match my needs. The complete code is the following.
SimpleBuffer.hpp
#pragma once
#include <QtCore/QIODevice>
class SimpleBuffer : public QIODevice {
Q_OBJECT
Q_DISABLE_COPY(SimpleBuffer)
public:
SimpleBuffer(char* const begin, const char* const end) :
_begin(begin),
_end(end){}
virtual bool atEnd() const {
return _end == _begin;
}
virtual bool isSequential() const {
return true;
}
protected:
virtual qint64 readData(char*, qint64) {
return -1;
}
virtual qint64 writeData(const char* const data, const qint64 maxSize) {
const qint64 space = _end - _begin;
const qint64 toWrite = qMin(maxSize, space);
memcpy(_begin, data, size_t(toWrite));
_begin += toWrite;
return toWrite;
}
private:
char* _begin;
const char* const _end;
};
main.cpp
#include "SimpleBuffer.hpp"
#include <QtCore/QVector>
#include <QtCore/QDataStream>
#include <QtCore/QByteArray>
int main(int, char**) {
QVector<int> src;
src << 3 << 7 << 13 << 42 << 100500;
const size_t dataSize = sizeof(quint32) + src.size() * sizeof(int);
char* const data = new char[dataSize];
// prepare stream and write out the src vector
{
SimpleBuffer simpleBuffer(data, data + dataSize);
simpleBuffer.open(QIODevice::WriteOnly);
QDataStream os(&simpleBuffer);
os << src;
}
// read vector with QByteArray
QVector<int> dst;
{
const QByteArray byteArray = QByteArray::fromRawData((char*)data, dataSize);
QDataStream is(byteArray);
is >> dst;
}
delete [] data;
// check we've read exactly what we wrote
Q_ASSERT(src == dst);
return 0;
}
I think you may need to implement a QIODevice, you could make a very simple sequential device quite easily. Here's one I've quickly thrown together, I haven't checked it works (feel free to get it working and edit the post).
class SimpleBuffer : public QIODevice
{
Q_OBJECT
public:
SimpleBuffer(char* begin, char* end):mBegin(begin),mEnd(end){}
virtual bool atEnd() const {return mEnd == mBegin; }
virtual bool isSequential() const { return true; }
protected:
virtual qint64 readData(char*, qint64) { return -1; }
virtual qint64 writeData(const char* data, qint64 maxSize)
{
const qint64 space = mEnd - mBegin;
const qint64 toWrite = qMin(maxSize, space);
memcpy(mBegin, data, size_t(toWrite));
mBegin += toWrite;
return toWrite;
}
private:
char* mBegin;
char* mEnd;
Q_DISABLE_COPY(SimpleBuffer)
};
Maybe fromRawData works:
QByteArray QByteArray::fromRawData ( const char * data, int size ) [static]
Using it something like :
char* out=new char[enoughbytes]; // preallocate at suitable scope
QVector<int> in;
QByteArray ba=QByteArray::fromRawData(out,enoughbytes);
QDataStream stream(&ba,QIODevice::WriteOnly);
stream << in;
Note that QDataStream adds some of it's own data at the start of the data (not much though), so remember to preallocate a bit more for that, as well as for whatever additional data QVector serializes.
Why not use QBuffer?
QByteArray myBuffer;
myBuffer.reserve(10000); // no re-allocation
QBuffer buffer(&myBuffer);
buffer.open(QIODevice::WriteOnly);
QDataStream out(&buffer);
out << QApplication::palette();