I am in process of replacing statements like:
auto pszOutBuffer = new char[dwSize];
ReadDataFromHttp(pszOutBuffer, dwSize);
if(dwSize>100)
ParseHttpData(pszOutBuffer);
...
delete []pszOutBuffer;
TO:
auto OutBufferPtr = make_unique<char[]>(dwSize);
auto pszOutBuffer = OutBufferPtr.get();
ReadDataFromHttp(pszOutBuffer, dwSize);
if(dwSize>100)
ParseHttpData(pszOutBuffer);
...
So that I get advantage of smart pointer unique_ptr. I would like to keep the variable pszOutBuffer as is so less changes in Git commits appear, and unique_ptr.get() isn't required to be repeated.
To make this simple, and less verbose to read; I thought of writing a macro MAKE_UNIQUE_PTR(type,size) which will be a single statement (not two as shown above). But such macro will not be able to have unqiue_ptr as well as pszOutBuffer as in:
auto pszOutBuffer = MAKE_UNIQUE_PTR(char, dwSize);
I may think of writing a function template, but then... how to keep unique_ptr after function (MAKE_UNIQUE_PTR) returns?
EDIT:
With this hypothetical macro/function, the code would simply be:
auto pszOutBuffer = MAKE_UNIQUE_PTR(char, dwSize);
ReadDataFromHttp(pszOutBuffer, dwSize);
if(dwSize>100)
ParseHttpData(pszOutBuffer);
...
With these advantages:
unique_ptr still controls the life time of buffer.
Raw-style pointer is still in place without chaging them to unique_ptr::get() calls.
Hence can safely delete delete[] calls from multiple code paths.
NOTE that the very first code is the code I've in hand. No macro, no unique_ptr- just legacy code having new and delete. And yeah... those Hungarian notation variables.
Unfortunately, std::make_unique<T[N]>() is not supported for reasons you can see in the original proposal. However, nothing stops you from crafting your own make_unique for arrays (e.g., make_unique_array) as below:
template<typename T>
std::enable_if_t<std::is_array<T>::value, std::unique_ptr<T>>
make_unique_array(std::size_t const n) {
using RT = std::remove_extent_t<T>;
return std::unique_ptr<T>(new RT[n]);
}
As you are already using macro you can write one which creates both. It's ugly but does its job.
#define SmartMacro(pszOutBuffer, dwSize) \
auto pszOutBuffer##ptr = make_unique<char[]>(dwSize); \
auto pszOutBuffer = pszOutBuffer##ptr.get(); \
memset(pszOutBuffer, 0, dwSize);
// Usage
SmartMacro(buffer, 10);
// 'buffer' is the raw pointer
// 'bufferptr' is the unique pointer
template<class T, class D>
struct smart_unique:std::unique_ptr<T,D> {
using std::unique_ptr<T,D>::unique_ptr;
operator T*()const{return this->get();}
operator T const*()const{return this->get();}
};
template<class T, class D>
struct make_smart_unique_t {
template<class...Ts>
smart_unique<T, D> operator()(Ts&&...ts)const{
return smart_unique<T,D>( new T(std::forward<Ts>(ts)...); );
}
template<class T0, class...Ts>
smart_unique<T, D> operator()(std::initializer_list<T0> il, Ts&&...ts)const{
return smart_unique<T,D>( new T(il, std::forward<Ts>(ts)...); );
}
};
template<class T, class D>
struct make_smart_unique_t<T[], D> {
smart_unique<T[], D> operator[](std::size_t N)const{
return smart_unique<T[],D>( new T[N]; );
}
template<class...Ts>
smart_unique<T[], D> operator()(Ts&&...ts)const{
return smart_unique<T[],D>( new T[sizeof...(Ts)]{std::forward<Ts>(ts)...}; );
}
};
template<class T, class D=std::default_delete<T>>
constexpr make_smart_unique_t<T,D> make_smart_unique{};
this should support:
auto pszOutBuffer = make_smart_unique<char[]>[dwSize];
ReadDataFromHttp(pszOutBuffer, dwSize);
if(dwSize>100)
ParseHttpData(pszOutBuffer);
as well as:
auto pszDataBuffer = make_smart_unique<int[]>(1,2,3,4,5);
and
auto pszDataBuffer = make_smart_unique<int>();
no macro magic needed.
The design here is simple: make_smart_unique<scalar> is similar to make_unique, but it returns a smart_unique instead (which implicitly casts to T*: be very careful!)
make_smart_unique<Array[]> has two different ways to invoke it. With [N] is creates an array of the passed in size: with (args...) it creates an array of the size of the number of elements, and constructs each one from each arg.
Related
InputManager* input = new InputManager(new int[]{ SDLK_UP, SDLK_DOWN, SDLK_LEFT, SDLK_RIGHT });
I wanna pass this array of keys (or a pointer to it) but i need the InputManager's constructor to know its size. Since its known at compile time how would i get it?
Online i found this template
template <int N>
InputManager::InputManager(int (&keyArray)[N]) {
this->maxKeys = N;
this->keys = keyArray;
}
But i get an error that the arguments dont match.
I need any possible solution where i dont need to manually write the length. So macros, templates or anything else is accepted.
Don't use so many pointers. It's not healthy and you're likely to either leak memory, or try to free memory that you can't (if you didn't pass a newly-allocated array). Here's a minimal modification of your code:
class InputManager {
public:
template <int N>
InputManager(int (&keyArray)[N])
: maxKeys(N), keys(std::make_unique<int[]>(N))
{
std::copy_n(keyArray, N, keys.get());
}
private:
std::size_t maxKeys;
std::unique_ptr<int[]> keys;
};
int main() {
int keys[] = { 1, 5, 4, 7, 2 };
InputManager input {keys};
}
This compiles (GodBolt).
Notes:
This way, you don't need to write a custom destructor; although you would need to write an assignment operator which copies data; and a copy constructor.
It's probably better to just use an std::vector internally (or std::array if you know the input size beforehand), and take any span or range of keys in the constructor.
Wrap creating of this array into function using variadic template.
If you want only return a pointer to array:
template<class ... enums>
int* makeArray (enums ... e) {
return new int[sizeof...(e)]{e...};
}
When you want to know size of returned array you can return pair:
template<class ... enums>
std::pair< int*, size_t > makeArray2(enums ... e) {
constexpr size_t N = sizeof...(e);
return std::make_pair( new int[N]{e...}, N);
}
I am using a library function that expects an array of pointers (void**) at some point, it works more or less like this.
void* args[] = { &var_a, &var_b, &var_c, ... };
someFunction(args);
After using it once, I would like to call the same function again, so what I do is create another variable like:
void* args_2[] = { &var_d, &var_e, &var_f, ... };
someFunction(args_2);
And so on ...
I would like to find a way to recycle the args symbol, so I don't have to do args_2, args_3, args_4 every time I call it; but when I try to reassign it like:
args = { &var_d, &var_e, &var_f, ... };
I get the following:
error: assigning to an array from an initializer list
I understand the error but I don't know how to avoid it or coerce this thing into the intended array of pointers type.
I know they are two different languages, but I am looking for a solution that works in both C and C++.
Why use a local variable at all, if you only need it once to call someFunction?
using args = void *[];
someFunction(args{&a, &b});
someFunction(args{&a, &b, &c});
Alternatively, C++ify this a bit more with a wrapper:
template <class... T>
decltype(auto) someFunction(T *... args) {
void *args_[] { args... };
return someFunction(args_);
}
someFunction(&a, &b);
someFunction(&a, &b, &c);
You could make use of compound literals as below with pointer to pointer.
void** args = (void *[]){ &var_a, &var_b, &var_c, ... };
args = (void *[]){ &var_d, &var_e, &var_f, ... };
I'm trying to write an async logger which accepts variadic arguments that are then strung together using a variadic stringer and then pushed onto a single producer single consumer queue.
I'm stuck in my enqueue function part of my Log struct which looks as follows:
template <typename T>
std::string Log::stringer(T const & t){
return boost::lexical_cast<std::string>(t);
}
template<typename T, typename ... Args>
std::string Log::stringer(T const & t, Args const & ... args){
return stringer(t) + stringer(args...);
}
template<typename T, typename ... Args>
void Log::enqueue(T & t, Args & ... args){
boost::function<std::string()> f
= boost::bind(&Log::stringer<T &, Args & ...>,this,
boost::ref(t),
boost::forward<Args>(args)...);
/// the above statement fails to compile though if i use 'auto f' it works ->
/// but then it is unclear to me what the signature of f really is ?
// at this point i would like to post the functor f onto my asio::io_service,
// but not able to cause it's not clear to me what the type of f is.
// I think it should be of type boost::function<std::string()>
}
Inside main(), I call
Log t_log;
t_log.enqueue("hello"," world");
My suggestion for the function you ask about:
template <typename T, typename... Args> void enqueue(T &t, Args const&... args) {
this->io_service->post([=]{
auto s = stringer(t, args...);
//std::fprintf(stderr, "%s\n", s.c_str());
});
}
This works with GCC and Clang (GCC 4.9 or later because of a known issue with captured variadic packs).
But really, I'd reconsider the design at hand, and certainly start a lot simpler until you know what areas deserve further optimization.
Questionables
There are many things I don't understand about this code:
Why are the arguments being taken by non-const reference
Why are you subsequently using std::forward<> on them (you already now the value category, and it's not going to change)
Why are you passing the stringization to an io_service?
the queue is going to introduce locking (kind of refuting the lockfree queue) and
the stringization is going to have its result ignored...
Why would you use boost::function here? This incurs a (another) dynamic allocation and an indirect dispatch... Just post f
Why are the arguments bound by reference in the first place? If you're going to process the arguments on a different thread, this leads to Undefined Behaviour. E.g. imagine the caller doing
char const msg[] = "my message"; // perhaps some sprintf output
l.enqueue(cat.c_str(), msg);
The c_str() is stale after the enqueue returned and msg goes out of scope soon, or gets overwritten with other data.
Why are you using bind approaches when you clearly have c++11 support (because you used std::forward<> and attributes)?
Why are you using a lockfree queue (do anticipate to be constantly logging at max CPU? In that case, logging is the core functionality of you application and you should probably think this through a bit (a lot) more rigorously (e.g. write into preallocated alternating buffers and decide on max backlog etc).
In all other cases, you probably want at most 1 single thread running on a lockfree queue. This would likely already be overkill (spinning a thread constantly is expensive). Instead, you could gracefully fallback to yields/synchronization if there's nothing to do n cycles.
You can just bind to a shared_ptr. This is a lot safer and more convenient than binding to .get()
In my sample below I've just removed the need for scoped_ptrs by not allocating everything from the heap (why was that?). (You can use boost::optional<work> if you needed work.)
The explicit memory-order load/stores give me bad vibes too. The way they're written would make sense only if exactly two threads are involved in the flag, but this is in no way apparent to me at the moment (threads are created all around).
On most platforms there will be no difference, and in light of the above, the presence of explicit memory ordering stands out as a clear code smell
The same thing applies to the attempts to forcibly inline certain functions. You can trust your compiler and you should probably refrain from second guessing it until you know you have a bottleneck caused by suboptimal generated code
Since you intend to give threads thread affinity, do use thread locals. Either use GCC/MSVC extensions in C++03 (__thread) or use c++11 thread_local, e.g. in pop()
thread_local std::string s;
s.reserve(1000);
s.resize(0);
This enormously reduces the number of allocations (at the cost of making pop() non-reentrant, which is not required.
I later noticed this pop() is limited to a single thread
What is the use of having that lockfree queue if all you do is ... spinlock manually around it?
void push(std::string const &s) {
while (std::atomic_flag_test_and_set_explicit(&this->lock, std::memory_order_acquire))
;
while (!this->q->push(s))
;
std::atomic_flag_clear_explicit(&this->lock, std::memory_order_release);
}
Cleanup Suggestion
Live On Coliru
#include <boost/iostreams/device/array.hpp>
#include <boost/iostreams/stream.hpp>
#include <boost/atomic.hpp>
#include <boost/lockfree/spsc_queue.hpp>
#include <boost/thread/thread.hpp>
/*
* safe for use from a single thread only
*/
template <unsigned line_maxchars = 1000>
class Log {
public:
Log(std::string const &logFileName, int32_t queueSize)
: fp(stderr), // std::fopen(logFileName.c_str(),"w")
_shutdown(false),
_thread(&Log::pop, this),
_queue(queueSize)
{ }
void pop() {
std::string s;
s.reserve(line_maxchars);
struct timeval ts;
while (!_shutdown) {
while (_queue.pop(s)) {
gettimeofday(&ts, NULL);
std::fprintf(fp, "%li.%06li %s\n", ts.tv_sec, ts.tv_usec, s.c_str());
}
std::fflush(fp); // RECONSIDER HERE?
}
while (_queue.pop(s)) {
gettimeofday(&ts, NULL);
std::fprintf(fp, "%li.%06li %s\n", ts.tv_sec, ts.tv_usec, s.c_str());
}
}
template <typename S, typename T> void stringer(S& stream, T const &t) {
stream << t;
}
template <typename S, typename T, typename... Args>
void stringer(S& stream, T const &t, Args const &... args) {
stringer(stream, t);
stringer(stream, args...);
}
template <typename T, typename... Args> void enqueue(T &t, Args const&... args) {
thread_local char buffer[line_maxchars] = {};
boost::iostreams::array_sink as(buffer);
boost::iostreams::stream<boost::iostreams::array_sink> stream(as);
stringer(stream, t, args...);
auto output = as.output_sequence();
push(std::string(output.first, output.second));
}
void push(std::string const &s) {
while (!_queue.push(s));
}
~Log() {
_shutdown = true;
_thread.join();
assert(_queue.empty());
std::fflush(fp);
std::fclose(fp);
fp = NULL;
}
private:
FILE *fp;
boost::atomic_bool _shutdown;
boost::thread _thread;
boost::lockfree::spsc_queue<std::string> _queue;
};
#include <chrono>
#include <iostream>
int main() {
using namespace std::chrono;
auto start = high_resolution_clock::now();
{
Log<> l("/tmp/junk.log", 1024);
for (int64_t i = 0; i < 10; ++i) {
l.enqueue("hello ", i, " world");
}
}
std::cout << duration_cast<microseconds>(high_resolution_clock::now() - start).count() << "μs\n";
}
As you can see, I've reduced the code by a third. I've documented the fact that it's only safe for use from a single thread.
Asio is gone. Lexical cast is gone. Things have meaningful names. No more memory order fiddling. No more thread affinity fiddling. No more inline envy. No more tedious string allocations.
The things that you'd likely benefit the most from is
make the array_sinks/buffers pooled and stored in the queue by reference
not flush on every log
I have a T* addressing a buffer with len elements of type T. I need this data in the form of an std::vector<T>, for certain reasons. As far as I can tell, I cannot construct a vector which uses my buffer as its internal storage. Why is that?
Notes:
Please don't suggest I use iterators - I know that's usually the way around such issues.
I don't mind that the vector having to copy data around if it's resized later.
This question especially baffles me now that C++ has move semantics. If we can pull an object's storage from under its feet, why not be able to shove in our own?
You can.
You write about std::vector<T>, but std::vector takes two template arguments, not just one. The second template argument specifies the allocator type to use, and vector's constructors have overloads that allow passing in a custom instance of that allocator type.
So all you need to do is write an allocator that uses your own internal buffer where possible, and falls back to asking the default allocator when your own internal buffer is full.
The default allocator cannot possibly hope to handle it, since it would have no clue on which bits of memory can be freed and which cannot.
A sample stateful allocator with an internal buffer containing already-constructed elements that should not be overwritten by the vector, including a demonstration of a big gotcha:
struct my_allocator_state {
void *buf;
std::size_t len;
bool bufused;
const std::type_info *type;
};
template <typename T>
struct my_allocator {
typedef T value_type;
my_allocator(T *buf, std::size_t len)
: def(), state(std::make_shared<my_allocator_state, my_allocator_state>({ buf, len, false, &typeid(T) })) { }
template <std::size_t N>
my_allocator(T(&buf)[N])
: def(), state(std::make_shared<my_allocator_state, my_allocator_state>({ buf, N, false, &typeid(T) })) { }
template <typename U>
friend struct my_allocator;
template <typename U>
my_allocator(my_allocator<U> other)
: def(), state(other.state) { }
T *allocate(std::size_t n)
{
if (!state->bufused && n == state->len && typeid(T) == *state->type)
{
state->bufused = true;
return static_cast<T *>(state->buf);
}
else
return def.allocate(n);
}
void deallocate(T *p, std::size_t n)
{
if (p == state->buf)
state->bufused = false;
else
def.deallocate(p, n);
}
template <typename...Args>
void construct(T *c, Args... args)
{
if (!in_buffer(c))
def.construct(c, std::forward<Args>(args)...);
}
void destroy(T *c)
{
if (!in_buffer(c))
def.destroy(c);
}
friend bool operator==(const my_allocator &a, const my_allocator &b) {
return a.state == b.state;
}
friend bool operator!=(const my_allocator &a, const my_allocator &b) {
return a.state != b.state;
}
private:
std::allocator<T> def;
std::shared_ptr<my_allocator_state> state;
bool in_buffer(T *p) {
return *state->type == typeid(T)
&& points_into_buffer(p, static_cast<T *>(state->buf), state->len);
}
};
int main()
{
int buf [] = { 1, 2, 3, 4 };
std::vector<int, my_allocator<int>> v(sizeof buf / sizeof *buf, {}, buf);
v.resize(3);
v.push_back(5);
v.push_back(6);
for (auto &i : v) std::cout << i << std::endl;
}
Output:
1
2
3
4
6
The push_back of 5 fits into the old buffer, so construction is bypassed. When 6 is added, new memory is allocated, and everything starts acting as normal. You could avoid that problem by adding a method to your allocator to indicate that from that point onward, construction should not be bypassed any longer.
points_into_buffer turned out to be the hardest part to write, and I've omitted that from my answer. The intended semantics should be obvious from how I'm using it. Please see my question here for a portable implementation in my answer there, or if your implementation allows it, use one of the simpler versions in that other question.
By the way, I'm not really happy with how some implementations use rebind in such ways that there is no avoiding storing run-time type info along with the state, but if your implementation doesn't need that, you could make it a bit simpler by making the state a template class (or a nested class) too.
The short answer is that a vector can't use your buffer because it wasn't designed that way.
It makes sense, too. If a vector doesn't allocate its own memory, how does it resize the buffer when more items are added? It allocates a new buffer, but what does it do with the old one? Same applies to moving - if the vector doesn't control its own buffer, how can it give control of this buffer to another instance?
These days - you no longer need to wrap a T* in an std::vector, you can wrap it with an std::span (in C++20; before that - use gsl::span). A span offers you all the convenience of a standard library container - in fact, basically all relevant features of std::vector excluding changes to the size - with a very thin wrapper class. That's what you want to use, really.
For more on spans, read: What is a "span" and when should I use one?
I want to use std::vector for dynamically allocating memory. The scenario is:
int neededLength = computeLength(); // some logic here
// this will allocate the buffer
std::vector<TCHAR> buffer( neededLength );
// call a function that accepts TCHAR* and the number of elements
callFunction( &(buffer[0]), buffer.size() );
The code above works, but this &(buffer[0]) looks ugly. Is there a more elegant way to achieve the same?
It's really odd that nobody know this!!!
in C++11 you could use:
buffer.data()
it could get the address of the vector
I have test it:
vector<char>buffer;
buffer.push_back('w');
buffer.push_back('h');
buffer.push_back('a');
buffer.push_back('t');
buffer.push_back('\0');
char buf2[10];
memcpy(buf2,buffer.data(),10);
Specification here.
Well, you can remove one set of parens:
&buffer[0]
but that is the common, idiomatic way of doing it. If it really offends you, I suppose you could use a template - something like:
template <typename T>
T * StartOf( std::vector <T> & v ) {
return &v[0];
}
Actually, the main problem with &buffer[0] (note the absence of parantheses) isn't that it isn't really pretty. (That's subjective anyway. I remember finding buffer.begin(), buffer.end() not pretty at all, when I first learned to use the STL.)
The main problem is that it invokes undefined behavior whenever buffer is empty -- and most code never checks for that. That's why I put these into my toolbox:
template <class T, class TAl>
inline T* begin_ptr(std::vector<T,TAl>& v)
{return v.empty() ? NULL : &v[0];}
template <class T, class TAl>
inline const T* begin_ptr(const std::vector<T,TAl>& v)
{return v.empty() ? NULL : &v[0];}
template <class T, class TAl>
inline T* end_ptr(std::vector<T,TAl>& v)
{return v.empty() ? NULL : (begin_ptr(v) + v.size());}
template <class T, class TAl>
inline const T* end_ptr(const std::vector<T,TAl>& v)
{return v.empty() ? NULL : (begin_ptr(v) + v.size());}
Using these, you can write your code as
callFunction( begin_ptr(buffer), buffer.size() );
Whether begin_ptr(buffer) is prettier than &buffer[0] is left for you to decide. However, given that NULL should be checked for every pointer function argument, it definitely is more safe.
but this &(buffer[0]) looks ugly
It’s the normal way. You can omit the parentheses, though:
&buffer[0]
No.
Try &(buffer.front()), but it's not much prettier :)
Elegant way would be to change callFunction or to write wrapper for it as follows:
// legacy function
void callFunction( TCHAR* buf, int buf_size)
{
// some code
}
// helpful template
void callFunction( std::vector<TCHAR>::iterator begin_it, std::vector<TCHAR>::iterator end_it )
{
callFunction( &*begin_it, std::distance( begin_it, end_it ) );
}
// somewhere in the code
int neededLength = computeLength();
std::vector<TCHAR> buffer( neededLength );
callFunction( buffer.begin(), buffer.end() );
You could even make wrapper for all such functions (with different types, not only TCHAR):
template<typename T>
void callFunction( T begin_it, typename std::vector<typename T::value_type>::iterator end_it )
{
callFunction( &*begin_it, std::distance( begin_it, end_it ) );
}
Type T will be properly deduced (as std::vector<sometype>) and you'll be able still write callFunction( buffer.begin(), buffer.end() );.
Note that you cannot declare template function as void callFunction( typename std::vector<typename T::value_type>::iterator begin_it, typename std::vector<typename T::value_type>::iterator end_it ) as someone proposed recently as an edit to this answer, because in that case you will get the deducion error.
The reason it looks ugly is because you're at the borderline of nice and clean C++ style code and nice and clean C style code. The C++ code uses iterators, the C code uses pointers and sizes.
You could create some glue to circumvent these problems:
template< typename at_Container, typename at_Function >
void for_container( at_Container& c, at_Function f ) {
f( &c[0], c.size() );
}
and call it in the client code.
void afunction( int* p, size_t n ) {
for( int* p = ap; p != ap+n; ++p ) {
printf( "%d ", *p );
}
}
void clientcode() {
std::vector<int> ints(30,3);
for_container( ints, afunction );
}
For functions like these, I use a utility class, SizedPtr<T> that basically holds a pointer and an element count. A set of converter functions creates the SizedPtr<T> from different inputs. So the call changes to:
vector<TCHAR> foo;
callFunction(sizedptr(foo));
One could even add an implicit std::vector constructor to SizedPtr, but I wanted to avoid this dependency.
This helps only if callFunction is under your control. It is a pleasure to work with, if you work with different vector types in one application and you want to consolidate. If you generally work with std::vector, it's mostly pointless.
Roughly:
template<typename T>
class SizedPtr
{
T * m_ptr;
size_t m_size;
public:
SizedPtr(T* p, size_t size) : ... {}
T * ptr() { return m_ptr; }
size_t size() const { return m_size; }
// index access, STL container interface, Sub-Sequence, ...
}
The idea behind this is to separate the operation - manipulating a contiguous sequence of elements - from the storage (std::vector). It's similar to what STL does with iterators, but avoids template infection.
As already said, no.
The reason is that &buffer[0] is the only way guarantied by the standard to get the adresse of the vector buffer.
If you're using std::vector just for its RAII properties (so that it will free the memory for you), and you don't actually need it to resize or anything, you might be better off using a Boost scoped_array
boost::scoped_array<TCHAR> buffer( new TCHAR[neededLength] );
callFunction( buffer.get(), neededLength );
scoped_array will call delete[] on the array when it goes out of scope.