I want to create an array of packed ciphertexts using the BFV homomorphic encryption scheme in Microsft SEAL, C++.
If I have 8192 slots in one packed BFV ciphertext, I want to create an array of 10 such ciphertexts to perform various operations on them.
I am not very good with assigning memory and using pointers which would probably be part of the solution but I am not sure. Could anyone please help me with this? Thank you so much!
I tried using pointers but since "Ciphertext" is a unique datatype to Microsoft SEAL I came across a few errors which probably means I didnt do it right.
Related
So basically, what I need is to have 28,800 values which can be accessed by an index and can all be set to true or false. Using an array of bools or integers isn't an option, because the size needs to be set during rumtime using a parameter. Using a vector is way too slow and memory intensive. I'm new to C++ and therefore don't have any idea on how to solve this, can anyone help?
EDIT: Thank you to all who commented! Like I said, Im new to C++ programming and your answers really helped me to understand the functionality behind vectors.
So, after everyone said that vector isn't slow I checked again and it turns out that my program was running so slow because of another bug I had while filling the vector. But especially midor's and Some programmer dude's answers helped me to make the program run a bit faster than before, so thanks!
Using a vector is way too slow and memory intensive.
C++ specializes std::vector<bool> so it uses only as much memory as it needs. One bit per "flag" (+ bookkeeping overhead of course).
You could only optimize that implementation if you knew its size a priori (which you don't according to your question), or if you know that the bitmap will only contain very few set bits (e.g. 1bit in a 50'000, but you'd need to measure if a more complex implementation would be worth it). For sparse bitmaps an std::unordered_set<std::uint32_t> that stores the set bits could be an option.
But 28'800 is a very small number, so don't waste your time on optimizations. You won't get any benefits from it.
This if my first post, I hope I'll meet the standards...
I'm translating into c++ (at which I'm quite new) a program originally written in MATLAB for reasons of efficiency. The piece of code I am actually working on resumes accesses to various indexes of a vector (matrix) in one step. For example, if M1 is a matrix of size, let's say, 10x15, the program would define a new one as follows:
idxs1 = [1 2 3];
idxs2 = [1 2 3 4 5];
M2 = M1 (idxs1 , idxs2);
resulting M2 as a matrix of size 3x5. Now, I guess what MATLAB actually does is access one by one the various places of M1 given by the indexes and then construct M2 by rearranging the many contents acquired, all very efficiently.
My question is, how can I reproduce such mechanism in c++? As far as I know there is no direct way to access in a row various indexes of an array, and the for loop I'm using seems rather cumbersome. Maybe there's some intelligent way to do it without demanding 'too much' processor time? Also, for the sake of educational purposes, I would be grateful if someone could explain what MATLAB actually does when such operation is performed.
Thanks in advance and sorry for the eventual inconveniences!
P.S: Just in case it adds anything to the question, I'm working with MEX files to link both languages.
P.S2: By the way, I found some related questions but regarding other languages:
python: Access multiple elements of an array
perl: How can I use an array slice to access several elements of an array simultaneously?
c: How to access multiple elements of array in one go?
"Armadillo is a high quality C++ linear algebra library, aiming towards a good balance between speed and ease of use
Useful for algorithm development directly in C++, or quick conversion of research code into production environments; the syntax (API) is deliberately similar to Matlab"
Link: http://arma.sourceforge.net/
Math program data structures can be some of the most elaborate out there. I can't even figure out what the 3rd line of your example actually does, so I can't even guess how MATLAB implements anything.
What I can tell you is, that one line of MATLAB is almost certainly hiding a ton of operations. If you want to recreate it you just need to make a utility function with a couple of for-loops that copy over all the correct indices one-by-one. Ultimately, this can't be too much different than one MATLAB does.
If you have a ton of matrix operations you need to support and you're on a large project, you might want to look into finding a C++ matrixes library. I don't have one to recommend, but Boost is a popular C++ library for many purposes including matrixes. (You can also make your own data structure, but not recommended for a novice.)
What MATLAB exactly does is left unspecified, and might very well differ from case to case depending on the indices, and even for a given set of indices it could differ from machine to machine. So let's not speculate.
In particular, it's left unspecified whether MATLAB physically copies M1. Such a copy can be faked, which saves time. The technique is known as "copy on write".
In C++, this would be possible as well, but harder. Furthermore, none of the existing container classes supports it.
If you're going to copy the elements, the CPU won't be the bottleneck. Instead, the memory bus will limit you. This is particularly the case when the indices aren't contiguous. For a 3x5 matrix, the time will be dominated by overhead - contiguity doesn't matter yet.
I'm using mpi::boost::gather to implement one algorithm which need to gather different size vectors at each node. I found it can be done by using
MPI_gather. However, any suggestions on achieve it by boost library? Just wanna to make the code consistently. Thanks!
Hello as a c++ programmer, the lack of pointers really annoys more than wrappers would help me.
I want to receive a packet without my data being truncated should i use a byte array and sequentially stack the data? or should i use a memorystream?
the point is my data comes in structure like this for example
char Signature[8]
DWORD Length
char Data[Length]
in c++ i used a vector to store the incoming data in a growing buffer untill the Length was reach.
Whats the right way to do it in VB.net?
When i needed to transfer numbers i transfer them in its binary representation cause its more efficient and in c++ is just a cast away.
I found in vb.net the need to use a binaryreader in order to cast its data to a Uint32.
I am stuck between all this wrappers I feel they make the work even more complicated.
The only reason i want to use vb is because the ease to integrate with GUI objects which are a real pain when dealing with MFC.
I am open to suggestions, would even consider to move to a different IDE.
Thanks in advance.
I'm currently working on a project that requires working with gigabytes of scientific data sets. The data sets are in the form of very large arrays (30,000 elements) of integers and floating point numbers. The problem here is that they are too large too fit into memory, so I need an on disk solution for storing and working with them. To make this problem even more fun, I am restricted to using a 32-bit architecture (as this is for work) and I need to try to maximize performance for this solution.
So far, I've worked with HDF5, which worked okay, but I found it a little too complicated to work with. So, I thought the next best thing would be to try a NoSQL database, but I couldn't find a good way to store the arrays in the database short of casting them to character arrays and storing them like that, which caused a lot of bad pointer headaches.
So, I'd like to know what you guys recommend. Maybe you have a less painful way of working with HDF5 while at the same time maximizing performance. Or maybe you know of a NoSQL database that works well for storing this type of data. Or maybe I'm going in the totally wrong direction with this and you'd like to smack some sense into me.
Anyway, I'd appreciate any words of wisdom you guys can offer me :)
Smack some sense into yourself and use a production-grade library such as HDF5. So you found it too complicated, but did you find its high-level APIs ?
If you don't like that answer, try one of the emerging array databases such as SciDB, rasdaman or MonetDB. I suspect though, that if you have baulked at HDF5 you'll baulk at any of these.
In my view, and experience, it is worth the effort to learn how to properly use a tool such as HDF5 if you are going to be working with large scientific data sets for any length of time. If you pick up a tool such as a NoSQL database, which was not designed for the task at hand, then, while it may initially be easier to use, eventually (before very long would be my guess) it will lack features you need or want and you will find yourself having to program around its deficiencies.
Pick one of the right tools for the job and learn how to use it properly.
Assuming your data sets really are large enough to merit (e.g., instead of 30,000 elements, a 30,000x30,000 array of doubles), you might want to consider STXXL. It provides interfaces that are intended to (and largely succeed at) imitate those of the collections in the C++ standard library, but are intended to work with data too large to fit in memory.
I have been working on scientific computing for years, and I think HDF5 or NetCDF is a good data format for you to work with. It can provide efficient parallel read/wirte, which is important for dealing with big data.
An alternate solution is to use array database, like SciDB, MonetDB, or RasDaMan. However, it will be kinda painful if you try to load HDF5 data into an array database. I once tried to load HDF5 data into SciDB, but it requires a series of data transformations. You need to know if you will query the data often or not. If not often, then the time-consuming loading may be unworthy.
You may be interested in this paper.
It can allow you to query the HDF5 data directly by using SQL.