linux to windows C++ byte array - c++

I have to replicate the following Java functionality in C++ to get data from Linux to Windows. Is Winsock2 the best way to go?.
Also, any reference code to suggest?
TIA,
B
import java.nio.ByteBuffer;
public class MessageXdr {
private ByteBuffer buffer;
private int size;
// taille max corps de message
private static final int T_MAX_CORPS_MSG = 16384;
public MessageXdr() {
buffer = ByteBuffer.allocate(4 * T_MAX_CORPS_MSG);
size =0;
}
public MessageXdr(byte[] array)
{
ByteBuffer tmpBuffer = ByteBuffer.wrap(array);
buffer = tmpBuffer.asReadOnlyBuffer();
size = array.length;
}
public int getSize()
{
return size;
}
public int getPosition()
{
return buffer.position();
}
public byte[] getArray()
{
return buffer.array();
}
public void resetBuffer()
{
size = 0;
buffer.rewind();
}
public int readInt()
{
int retour = buffer.getInt();
return retour;
}
public long readUnsignedInt()
{
ByteBuffer tmp = ByteBuffer.allocate(8);
tmp.putInt(0);
tmp.putInt(buffer.getInt());
return tmp.getLong(0);
}
public float readFloat()
{
float retour = buffer.getFloat();
return retour;
}
public void writeInt(int v)
{
buffer.putInt(v);
size+=4;
}
public void writeFloat(float v)
{
buffer.putFloat(v);
size+=4;
}
}

If you are allowed to use the MFC classes (CSocket), it might be closer to the code you have in Java.
http://msdn.microsoft.com/en-us/library/wxzt95kb(VS.80).aspx
Otherwise, Winsock2 is fine (the MFC classes just use that in their implementation).

I haven't worked with it yet, but when it comes to marshalling more complex data structures i would look into boost for the serialization part.
For the actual data transmission, winsock2 is the basic socket api in windows, all other api's are built on it (well, don't know about Windows 7) .But again, looking into boost could provide you with something platform independent you don't have to figure out twice. But from my experience, sockets are complex beasts, so you will have to figure out a lot anyway...
And avoid the CSocket from MFC, that's the worst implementation ever. (Even if some say that they fixed some of it's misbehaviours, it's just not worth it.)

Strict byte arrays don't need any translation from linux to windows or other systems. If you are dealing with integers and floats however...
Personally I would use Poco::BinaryWriter and Poco::BinaryReader
http://pocoproject.org/docs/Poco.BinaryWriter.html
using namespace Poco;
using namespace std;
std::ofstream myFile("path", ios::in | ios::binary);
BinaryWriter writer(myFile, BIG_ENDIAN_BYTE_ORDER);
writer << 10.0f;
writer << 10000;
//etc etc
myFile.close();
Now to read
std::ifstream myFile("path", ios::in | ios::binary);
BinaryReader reader(myFile, BIG_ENDIAN_BYTE_ORDER);
int intVariable;
float floatVariable;
reader >> floatVariable;
reader >> intVariable;
//etc etc
myFile.close();

Related

How to wait for data on a COM port

I am using .Net framework's SerialPort class (System.IO.Ports) and am trying to wait to receive data. I've looked around and the WaitCommEvent function seems to be what people suggest using, however it expects a HANDLE as the first parameter. I would like to know either how to get a handle from the SerialPort or a different way to wait for data compatible with the SerialPort class.
The SerialPort class has a DataReceived event:
Indicates that data has been received through a port represented by the SerialPort object.
The documentation provides the following C++/CLI example:
#using <System.dll>
using namespace System;
using namespace System::IO::Ports;
ref class PortDataReceived
{
public:
static void Main()
{
SerialPort^ mySerialPort = gcnew SerialPort("COM1");
mySerialPort->BaudRate = 9600;
mySerialPort->Parity = Parity::None;
mySerialPort->StopBits = StopBits::One;
mySerialPort->DataBits = 8;
mySerialPort->Handshake = Handshake::None;
mySerialPort->RtsEnable = true;
mySerialPort->DataReceived += gcnew SerialDataReceivedEventHandler(DataReceivedHandler);
mySerialPort->Open();
Console::WriteLine("Press any key to continue...");
Console::WriteLine();
Console::ReadKey();
mySerialPort->Close();
}
private:
static void DataReceivedHandler(
Object^ sender,
SerialDataReceivedEventArgs^ e)
{
SerialPort^ sp = (SerialPort^)sender;
String^ indata = sp->ReadExisting();
Console::WriteLine("Data Received:");
Console::Write(indata);
}
};
int main()
{
PortDataReceived::Main();
}

c/c++ get large size data like 180 array from another class in stm32

I have an 32-bit ARM Cortex M4 (the processor in Pixhawk) to write two classes, each one is one threading in Pixhawk codebase setting.
The first one is LidarScanner, which dealing with incoming serial data and generates "obstacle situation". The second one is Algorithm, which handle "obstacle situation" and take some planning strategy. Here are my solution right now, use the reference function LidarScanner::updateObstacle(uint8_t (&array)[181]) to update "obstacle situation" which is 181 size array.
LidarScanner.cpp:
class LidarScanner{
private:
struct{
bool available = false;
int AngleArr[181];
int RangeArr[181];
bool isObstacle[181] = {}; //1: unsafe; 0:safe;
}scan;
......
public:
LidarScanner();
//main function
void update()
{
while(hal.uartE->available()) //incoming serial data is available
{
decode_data(); //decode serial data into three kind data: Range, Angle and Period_flag
if(complete_scan()) //determine if the lidarscanner one period is completed
{
scan.available = false;
checkObstacle(); //check obstacle situation and store safety in isObstacle[181]
scan.available = true;
}
}
}
//for another API recall
void updateObstacle(uint8_t (&array)[181])
{
for(int i=0; i<=181; i++)
{
array[i]=scan.isObstacle[i];
}
}
//for another API recall
bool ScanAvailable() const { return scan.available; }
......
}
Algorithm.cpp:
class Algorithm{
private:
uint8_t Obatcle_Value[181] = {};
class LidarScanner& _lidarscanner;
......
public:
Algorithm(class LidarScanner& _lidarscanner);
//main funcation
void update()
{
if (hal.uartE->available() && _lidarscanner.ScanAvailable())
{
//Update obstacle situation into Algorithm phase and do more planning strategy
_lidarscanner.updateObstacle(Obatcle_Value);
}
}
......
}`
Usually, it works fine. But I want to improve the performances so that I want to know what's the most effective way to do that. thanks!!!!
The most efficient way to copy data is to use the DMA.
DMAx_Channelx->CNDTR = size;
DMAx_Channelx->CPAR = (uint32_t)&source;
DMAx_Channelx->CMAR = (uint32_t)&destination;
DMAx_Channelx->CCR = (0<<DMA_CCR_MSIZE_Pos) | (0<<DMA_CCR_PSIZE_Pos)
| DMA_CCR_MINC | DMA_CCR_PINC | DMA_CCR_MEM2MEM ;
while(!(DMAx->ISR & DMA_ISR_TCIFx ));
AN4031 Using the DMA controller.

How to wrap UTF-8 encoded C++ std::strings with Swig in C#?

My question is nearly identical to this question, except that the linked question deals with char*, whereas I'm using std::string in my code. Like the linked question, I'm also using C# as my target language.
I have a class written in C++:
class MyClass
{
public:
const std::string get_value() const; // returns utf8-string
void set_value(const std::string &value); // sets utf8-string
private:
// ...
};
And this get's wrapped by SWIG in C# as follows:
public class MyClass
{
public string get_value();
public void set_value(string value);
}
SWIG does everything for me, except that it doesn't make an utf8 to utf16 string conversion during the calls to MyClass. My strings come through fine if they are representable in ASCII, but if I try passing a string with non-ascii characters in a round-trip through "set_value" and "get_value", I end up with unintelligible characters.
How can I make SWIG wrap UTF-8 encoded C++ strings in C#? n.b. I'm using std::string, not std::wstring, and not char*.
There's a partial solution on the SWIG sourceforge site, but it deals with char* not std::string, and it uses a (configurable) fixed length buffer.
With the help (read: genius!) of David Jeske in the linked Code Project article, I have finally been able to answer this question.
You'll need this class (from David Jeske's code) in your C# library.
public class UTF8Marshaler : ICustomMarshaler {
static UTF8Marshaler static_instance;
public IntPtr MarshalManagedToNative(object managedObj) {
if (managedObj == null)
return IntPtr.Zero;
if (!(managedObj is string))
throw new MarshalDirectiveException(
"UTF8Marshaler must be used on a string.");
// not null terminated
byte[] strbuf = Encoding.UTF8.GetBytes((string)managedObj);
IntPtr buffer = Marshal.AllocHGlobal(strbuf.Length + 1);
Marshal.Copy(strbuf, 0, buffer, strbuf.Length);
// write the terminating null
Marshal.WriteByte(buffer + strbuf.Length, 0);
return buffer;
}
public unsafe object MarshalNativeToManaged(IntPtr pNativeData) {
byte* walk = (byte*)pNativeData;
// find the end of the string
while (*walk != 0) {
walk++;
}
int length = (int)(walk - (byte*)pNativeData);
// should not be null terminated
byte[] strbuf = new byte[length];
// skip the trailing null
Marshal.Copy((IntPtr)pNativeData, strbuf, 0, length);
string data = Encoding.UTF8.GetString(strbuf);
return data;
}
public void CleanUpNativeData(IntPtr pNativeData) {
Marshal.FreeHGlobal(pNativeData);
}
public void CleanUpManagedData(object managedObj) {
}
public int GetNativeDataSize() {
return -1;
}
public static ICustomMarshaler GetInstance(string cookie) {
if (static_instance == null) {
return static_instance = new UTF8Marshaler();
}
return static_instance;
}
}
Then, in Swig's "std_string.i", on line 24 replace this line:
%typemap(imtype) string "string"
with this line:
%typemap(imtype, inattributes="[MarshalAs(UnmanagedType.CustomMarshaler, MarshalTypeRef = typeof(UTF8Marshaler))]", outattributes="[return: MarshalAs(UnmanagedType.CustomMarshaler, MarshalTypeRef = typeof(UTF8Marshaler))]") string "string"
and on line 61, replace this line:
%typemap(imtype) const string & "string"
with this line:
%typemap(imtype, inattributes="[MarshalAs(UnmanagedType.CustomMarshaler, MarshalTypeRef = typeof(UTF8Marshaler))]", outattributes="[return: MarshalAs(UnmanagedType.CustomMarshaler, MarshalTypeRef = typeof(UTF8Marshaler))]") string & "string"
Lo and behold, everything works. Read the linked article for a good understanding of how this works.

std::fstream with multiple buffers?

You can specify one buffer for your file stream like that:
char buf[BUFFER_SIZE];
std::ofstream file("file", std::ios_base::binary | std::ios_base::out);
if (file.is_open())
{
file.rdbuf()->pubsetbuf(buf, BUFFER_SIZE);
file << "abcd";
}
What I want to do now, is using more than just one buffer:
char* buf[] = { new char[BUFFER_SIZE], new char[BUFFER_SIZE], new char[BUFFER_SIZE], };
Is it possible without creating a custom derivation of std::streambuf?
EDIT:
I think I need to explain what I want to do in more detail. Please consider the following situation:
- The file(s) I want to read won't fit into memory
- The file while be accessed by some kind of a binary jump search
So, if you split the file into logical pages of a specific size, then I would like to provide multiple buffers which are representing specific pages. This would increase performance when a file location is read and the related page is already in a buffer.
I gather from the comment that you want to do a kind of scatter-gather I/O. I'm pretty sure there's no support for that in the C++ standard I/O streams library, so you'll have to roll your own.
If you want to do this efficiently, you can use OS support for scatter-gather. E.g., POSIX/Unix-like systems have writev for this purpose.
There's nothing like this provided by Standard. However, depending on your platform, you can use Memory Mapped Files, which provide the same functionality. Windows and Linux both provide them.
I will take a look at boost::iostreams::mapped_file, but I think my requirement is much simpler. I've created a custom class derived from basic_filebuf.
template<typename char_type>
class basic_filemultibuf : public std::basic_filebuf<char_type/*, std::char_traits<char_type>*/>
{
private:
char_type** m_buffers;
std::ptrdiff_t m_buffer_count,
m_curent_buffer;
std::streamsize m_buffer_size;
protected:
virtual int_type overflow(int_type meta = traits_type::eof())
{
if (this->m_buffer_count > 0)
{
if (this->m_curent_buffer == this->m_buffer_count)
this->m_curent_buffer = 0;
this->basic_filebuf::setbuf(this->m_buffers[this->m_curent_buffer++], this->m_buffer_size);
}
return this->basic_filebuf::overflow(meta);
}
public:
basic_filemultibuf(basic_filebuf const& other)
: basic_filebuf(other),
m_buffers(NULL),
m_buffer_count(0),
m_curent_buffer(-1),
m_buffer_size(0)
{
}
basic_filemultibuf(basic_filemultibuf const& other)
: basic_filebuf(other),
m_buffers(other.m_buffers),
m_buffer_count(other.m_buffer_count),
m_curent_buffer(other.m_curent_buffer),
m_buffer_size(other.m_buffer_size)
{
}
basic_filemultibuf(FILE* f = NULL)
: basic_filemultibuf(basic_filebuf(f))
{
}
basic_filemultibuf* pubsetbuf(char** buffers, std::ptrdiff_t buffer_count, std::streamsize buffer_size)
{
if ((this->m_buffers = buffers) != NULL)
{
this->m_buffer_count = buffer_count;
this->m_buffer_size = buffer_size;
this->m_curent_buffer = 0;
}
else
{
this->m_buffer_count = 0;
this->m_buffer_size = 0;
this->m_curent_buffer = -1;
}
this->basic_filebuf::setbuf(NULL, 0);
return this;
}
};
Example usage:
typedef basic_filemultibuf<char> filemultibuf;
std::fstream file("file", std::ios_base::binary | std::ios_base::in | std::ios_base::out);
char** buffers = new char*[2];
for (int i = 0; i < n; ++i)
buffers[i] = new char[4096];
filemultibuf multibuf(*file.rdbuf());
multibuf.pubsetbuf(buffers, 2, 4096);
file.set_rdbuf(&multibuf);
//
// do awesome stuff with file ...
//
for (int i = 0; i < n; ++i)
delete[] buffers[i];
That's pretty much it. The only thing I would really like to do is offer this functionally for other streambufs, because the usage of multiple buffers should not be restricted to filebuf. But it seems to me it isn't possible without rewriting the file specific functions.
What do you think about that?

Keeping the downloaded torrent in memory rather than file libtorrent

Working with Rasterbar libtorrent I dont want the downloaded data to sit on my hard drive rather a pipe or variable or something Soft so I can redirect it to somewhere else, Mysql, or even trash if it is not what I want, is there anyway of doing this in preferably python binding if not in C++ using Libtorrent?
EDIT:--> I like to point out this is a libtorrent question not a Linux file handling or Python file handling question. I need to tell libtorrent to instead of save the file traditionally in a normal file save it to my python pipe or variable or etc.
You can do this by implementing your own storage class to use with libtorrent. Unfortunately this is not possible to do in python, but you can do it in c++. The documentation for it is a bit scarce and can be found here.
Here's a simple example of how to do this by storing all the data in RAM:
struct temp_storage : storage_interface
{
temp_storage(file_storage const& fs) : m_files(fs) {}
virtual bool initialize(bool allocate_files) { return false; }
virtual bool has_any_file() { return false; }
virtual int read(char* buf, int slot, int offset, int size)
{
std::map<int, std::vector<char> >::const_iterator i = m_file_data.find(slot);
if (i == m_file_data.end()) return 0;
int available = i->second.size() - offset;
if (available <= 0) return 0;
if (available > size) available = size;
memcpy(buf, &i->second[offset], available);
return available;
}
virtual int write(const char* buf, int slot, int offset, int size)
{
std::vector<char>& data = m_file_data[slot];
if (data.size() < offset + size) data.resize(offset + size);
std::memcpy(&data[offset], buf, size);
return size;
}
virtual bool rename_file(int file, std::string const& new_name) { assert(false); return false; }
virtual bool move_storage(std::string const& save_path) { return false; }
virtual bool verify_resume_data(lazy_entry const& rd, error_code& error) { return false; }
virtual bool write_resume_data(entry& rd) const { return false; }
virtual bool move_slot(int src_slot, int dst_slot) { assert(false); return false; }
virtual bool swap_slots(int slot1, int slot2) { assert(false); return false; }
virtual bool swap_slots3(int slot1, int slot2, int slot3) { assert(false); return false; }
virtual size_type physical_offset(int slot, int offset) { return slot * m_files.piece_length() + offset; };
virtual sha1_hash hash_for_slot(int slot, partial_hash& ph, int piece_size)
{
int left = piece_size - ph.offset;
TORRENT_ASSERT(left >= 0);
if (left > 0)
{
std::vector<char>& data = m_file_data[slot];
// if there are padding files, those blocks will be considered
// completed even though they haven't been written to the storage.
// in this case, just extend the piece buffer to its full size
// and fill it with zeroes.
if (data.size() < piece_size) data.resize(piece_size, 0);
ph.h.update(&data[ph.offset], left);
}
return ph.h.final();
}
virtual bool release_files() { return false; }
virtual bool delete_files() { return false; }
std::map<int, std::vector<char> > m_file_data;
file_storage m_files;
};
You'd also need a constructor function to pass in through the add_torrent_params struct when adding a torrent:
storage_interface* temp_storage_constructor(
file_storage const& fs, file_storage const* mapped
, std::string const& path, file_pool& fp
, std::vector<boost::uint8_t> const& prio)
{
return new temp_storage(fs);
}
From this point it should be fairly straight forward to store it in a MySQL database or any other back-end.
If you're on Linux, you could torrent into a tmpfs mount; this will avoid writing to disk. That said, this obviously means you're storing large files in RAM; make sure you have enough memory to deal with this.
Note also that most Linux distributions have a tmpfs mount at /dev/shm, so you could simply point libtorrent to a file there.
I've implemented a torrent client in Go just for this purpose. I wanted to able to handle and control the data directly, for use in writing torrentfs, and to have storage backends to S3 and various databases.
It would be trivial to plug in an in-memory storage backend to this client.
Try giving the library a cStringIO "file handle" instead of a real file handle. That works for most python libraries.