What is the recommended method(s) for accessing data from a previous or upcoming block of data from a bulkio input stream? Edit: I am using c++.
For example, if I wanted to perform convolution on a stream of incoming data where each calculation is dependent on some number of values forward/backward in the stream how would I reach into the "next" block of data or "previous" block of data to perform the convolution calculations at the block boundaries? Or temporarily store this information somewhere within the component so it can be used from one block to the next?
Or a simpler example, if I send a repeating vector of 8 octet values into my component, I want the component to simply flip from 0 to 1 or vice versa whenever a 1 is received (dependent on last index of the previous block of data to calculate the first index of the next block of data).
Desired:
in: [0,0,0,1,0,0,0] [0,0,0,1,0,0,0] [0,0,0,1,0,0,0] ->
out: [1,1,1,0,0,0,0] [0,0,0,1,1,1,1] [1,1,1,0,0,0,0]
What I have been able to achieve:
in: [0,0,0,1,0,0,0] [0,0,0,1,0,0,0] [0,0,0,1,0,0,0] ->
out: [1,1,1,0,0,0,0] [1,1,1,0,0,0,0] [1,1,1,0,0,0,0]
I've thought to store the relevant information from the previously processed block in a variable somewhere inside the component code serviceFunction() although I haven't found a way to do this without the value being reinitialized (is each block of data a new call to serviceFunction()?).
Alternatively, I thought to make a read-only property to hold the values I care about but I suspect there may be a better approach I am unaware of.
Thanks,
-Mark
If you need to retain some number of samples between reads, BulkIO input streams support data overlapping. You provide both a number of samples to read, and a number of samples to consume (which is necessarily less than or equal to the read size), and the next read will start at the first sample that was not consumed. Refer to the REDHAWK manual section on the BulkIO Stream API (5.7.3 in the 2.0.8 manual) for further details. You can also maintain history by simply keeping the last data block read as a member variable of your component class.
Your simple example suggests a state machine rather than the contents of the last read. In general, if there is information you want to store between iterations of your serviceFunction(), you are free to add your own member variables to your component class (e.g., "MyComponent_i"). It is not necessary to declare a property to add members. Only the base class (e.g., "MyComponent_base") is intended to be regenerated as you modify the properties or ports of the component, so any changes you make to the component class are preserved.
The training data is read from two .npy files. Say, train_set is regarded as X, and train_label is regarded as Y. Therefore, it is not a multiple input case. My task requires to augment the image patches in different manner. So how to define different Image Generator for different patches? Although there could be a lot of patches, I use 3 patches as an example:
for patch1:
datagen = ImageDataGenerator(rotation_range=20)
for patch2:
datagen = ImageDataGenerator(rotation_range=40)
for patch3:
datagen = ImageDataGenerator(rotation_range=60)
How to apply different generators on different patches, and how may I use the model.fit(...) or model.fit_generator(...) for the described scenario?
Also, Is there a way to rotate the image by a particular degree instead of a range?
Thanks!
I didn't do it myself, but I think one approach is to use the first datagen and pass the first group of training data with fit_generator and with the selected number of epochs. Then, save weight and use the second datagen and the second group with fit_generator. You also need to set initial_epoch and also need to load the weights. To generalize the question, what you need to do is to resume training with the second datagen. Please see https://keras.io/getting-started/faq/#how-can-i-save-a-keras-model.
Well straight to the point, I have two arrays say oldArray[SIZE] and newArray[SIZE]. I want to find the difference between the each element of both arrays eg:
oldArray[0]-newArray[0] =
oldArray[1]-newArray[1] =
oldArray[2]-newArray[2] =
:
:
oldArray[SIZE]-newArray[SIZE] =
If the difference is zero no worries but if the diff is >0 store the data along with index. What the best way to store. I want to send this difference data to the client over network. Only ways that I am aware of is using a vector or a dynamic array. I'd really appreciate help with this.
Update: oldArray[] and newArra[] are two image frames of a video sequence which have depth values for each pixel, I want to compute the difference between the two frames and send only the difference over the network and on the other end I will again reconstruct the image frame, data is integer range from 0 to 1024. Hope this helps
I'd go for a std::map<int,std::pair<T,T>> where key is the index in question, and the std::pair contains the old value in first and the new value in second. No entries for equal first and second.
As for your edit a std::map<int,int> where key is the index, and value is the difference might be sufficient to keep your bitmaps synchronized.
How to serialize that properly over the network is a different kettle of fish.
I have a pipe of two channels A and B, processing looks like than A --> operation --> B. The inputs are put into A and the results are read from B.
I'd like to have a channel C which on put! would put the value to A and on new value emitted from B would emit that value too. So it would wrap the A and B channels into one channels. The read port of A would be C's read port, and the write port of B would be the write port for C.
How to do that simply? Thank you for suggestions.
If I understand your requirements, and I'm not sure I do, there are two possible solutions I can think of.
The first is the pipline command, which will allow you to connect in and out channels and setup processing in the middle, which can be further paralleled to improve performance.
However, from your description, I think you might find alts! a better solution. this will allow you to define multiple input channels where you can take data from one or the other, process the data and then put it into an output channel.
Gidday mate,
Am I correct in understanding you want channel C to be both an input and output? I'm not sure this is possible with "vanilla" channel operations; as soon as B puts onto C, then A would pick it up again. You really need separate input and output channels.
What are you trying to solve that needs this structure? Perhaps it could be solved another way?
I need to export data from 3 maps to preferably a single CSV and would like to be able to do so without simply making a column for every possible key (there may be up to 65024 of them).
The output would be a CSV containing the value at each of the keys at each timestep (may be several hundred thousand).
Anyone got any ideas?
Reduce the granularity by categorizing your keys into groups and store them with one timestep per row. Then you can plot one datapoint per line.
Let me know if you need clarification, i'd need some more info.