Related
I've recently trained a darknet yolov4 model to detect 2 objects, converted it to tensorflow and then onnx using the following tutorial.
https://github.com/onnx/models/blob/master/vision/object_detection_segmentation/yolov4/dependencies/Conversion.ipynb
I ended up with a model with the following input and output layer dimensions
How can I determine the shape of the three output layers that have the unknown numbers?
I need them so I can use the model in ML.Net.
This is easy:
unk__2241 is batch size, so its unknown for now.
After CSPDarknet53, your output shape is (unk_2241,13,13,512) as reduce factor is 32. Then after SPP you have (unk_2241, 13, 13, 2048) with kernel size = [1,3,5,13].
You have 3 heads for detecting large, medium and small objects on image and 3 anchors per each of size of object. In this 3 heads yolo uses 3 feature maps which comes from modified PANet and this feature maps are (unk_2241,52,52,256), (unk_2241,26,26,512), (unk_2241,13,13,1024) as reduce factor is 8, 16, 32.
Then before each yolo layer, there is convolution for making final feature map and its shape is (unk_2241, 52, 52, 21), (unk_2241, 26, 26, 21) , (unk_2241, 13, 13, 21), and kernel shape is (256, (class number + coords of bbox + confidence)*anchor number for each size, 1, 1) ->(256, 21, 1, 1), (512, 21, 1, 1), (1024, 21, 1, 1).
And in yolo head you will have (unk_2241, 52, 52, 3, 7), (unk_2241, 26, 26, 3, 7), (unk_2241, 13, 13, 3, 7), where yolo head is dividing last axis of input by number of anchor, and number of class + coords of bbox + confidence.
As the result:
unk__2241 = unk__2242 = unk__2245 = unk__2248 -> They all batch sizes.
YOLO head output shapes: (unk_2242, 52, 52, 3, 7), (unk_2245, 26, 26, 3, 7), (unk_2248, 13, 13, 3, 7).
Batch size could be size of all dataset.
Suppose two vectors:
std::vector<int> v1{1, 10, 18, 41, 6};
std::vector<int> v2{-12, 107, 14, -3, 5};
How to replace the i-th element of v1 with v2? For example
v1[2]<-v2
leads to
v1={1, 10, -12, 107, 14, -3, 5 , 41, 6}
The simple solution is:
v1[2] = v2[0]
v1.insert(v1.begin() + 3, v2.begin() + 1, v2.end())
Is there any faster (lower time complexity) method? Is it possible to perform these operations as a single statement?
Thanks for your help.
I'm discovering Halide and got some success with a pipeline doing various
transformations. Most of these are based on the examples within the sources (color-transformations, various filters, hist-eq).
My next step needs to process the image in blocks. In a more general form,
partially-overlapping blocks.
Examples
Input:
[ 1, 2, 3, 4, 5, 6, 7, 8,
9, 10, 11, 12, 13, 14, 15, 16,
17, 18, 19, 20, 21, 22, 23, 24,
25, 26, 27, 28, 29, 30, 31, 32]
Non-overlapping blocks:
Size: 2x4
[ 1, 2, 3, 4,
9, 10, 11, 12]
[ 5, 6, 7, 8,
13, 14, 15, 16]
[ 17, 18, 19, 20,
25, 26, 27, 28]
[ 21, 22, 23, 24,
29, 30, 31, 32]
Overlapping blocks:
Size: 2x4 with 50% overlap (both axes)
[ 1, 2, 3, 4,
9, 10, 11, 12]
[ 3, 4, 5, 6,
11, 12, 13, 14]
[ 5, 6, 7, 8,
13, 14, 15, 16]
-
[ 9, 10, 11, 12,
17, 18, 19, 20]
[11, 12, 13, 14,
19, 20, 21, 22]
...
I suspect there should be a nice way to express these, as those are also quite common
in many algorithms (e.g. macroblocks).
What i checked out
I tried to gather ideas from the tutorial and example apps and found the following,
which seem somewhat connected to what i want to implement:
Halide tutorial lesson 6: Realizing Funcs over arbitrary domains
// We start by creating an image that represents that rectangle
Image<int> shifted(5, 7); // In the constructor we tell it the size
shifted.set_min(100, 50); // Then we tell it the top-left corner
The problem i have: how to generalize this to multiple shifted domains without looping?
Halide tutorial lesson 9: Multi-pass Funcs, update definitions, and reductions
Here RDom is introduced which looks nice to create a block-view
Most examples using RDom seem to be sliding-window like approaches where there are no jumps
Target
So in general i'm asking how to implement a block-based view which can then be processed by
other steps.
It would be nice if the approach will be general enough to realize both, overlapping & no overlapping
Somehow generating the top-left indices first?
In my case, the image-dimension is known at compile-time which simplifies this
But i still would like some compact form which is nice to work with from Halide's perspective (no handcoded stuff like those examples with small filter-boxes)
The approach used might be depending on the output per block, which is a scalar in my case
Maybe someone can give me some ideas and/or some examples (which would be very helpful).
I'm sorry for not providing code, as i don't think i could produce anything helpful.
Edit: Solution
After dsharlet's answer and some tiny debugging/discussion here, the following very simplified self-containing code works (assuming an 1-channel 64x128 input like this one i created).
#include "Halide.h"
#include "Halide/tools/halide_image_io.h"
#include <iostream>
int main(int argc, char **argv) {
Halide::Buffer<uint8_t> input = Halide::Tools::load_image("TestImages/block_example.png");
// This is a simple example assuming an input of 64x128
std::cout << "dim 0: " << input.width() << std::endl;
std::cout << "dim 1: " << input.height() << std::endl;
// The "outer" (block) and "inner" (pixel) indices that describe a pixel in a tile.
Halide::Var xo, yo, xi, yi, x, y;
// The distance between the start of each tile in the input.
int tile_stride_x = 32;
int tile_stride_y = 64;
int tile_size_x = 32;
int tile_size_y = 64;
Halide::Func tiled_f;
tiled_f(xi, yi, xo, yo) = input(xo * tile_stride_x + xi, yo * tile_stride_y + yi);
Halide::RDom tile_dom(0, tile_size_x, 0, tile_size_y);
Halide::Func tile_means;
tile_means(xo, yo) = sum(Halide::cast<uint32_t>(tiled_f(tile_dom.x, tile_dom.y, xo, yo))) / (tile_size_x * tile_size_y);
Halide::Func output;
output(xo, yo) = Halide::cast<uint8_t>(tile_means(xo, yo));
Halide::Buffer<uint8_t> output_(2, 2);
output.realize(output_);
Halide::Tools::save_image(output_, "block_based_stuff.png");
}
Here's an example that breaks a Func into blocks of abitrary stride and size:
Func f = ... // The thing being blocked
// The "outer" (block) and "inner" (pixel) indices that describe a pixel in a tile.
Var xo, yo, xi, yi;
// The distance between the start of each tile in the input.
int tile_stride_x, tile_stride_y;
Func tiled_f;
tiled_f(xi, yi, xo, yo) = f(xo * tile_stride_x + xi, yo * tile_stride_y + yi);
Func tiled_output;
tiled_output(xi, yi, xo, yo) = ... // Your tiled processing here
To compute some reduction (like statistics) on each block, you can do the following:
RDom tile_dom(0, tile_size_x, 0, tile_size_y);
Func tile_means;
tile_means(xo, yo) = sum(tiled_output(tile_dom.x, tile_dom.y, xo, yo)) / (tile_size_x * tile_size_y);
To flatten the tiles back into a result is a bit tricky. It probably depends on your method of combining the results in overlapped areas. If you want to add up the overlapping tiles, the simplest way is probably to use an RDom:
RDom tiles_dom(
0, tile_size_x,
0, tile_size_y,
min_tile_xo, extent_tile_xo,
min_tile_yo, extent_tile_yo);
Func output;
Expr output_x = tiles_dom[2] * tile_stride_x + tiles_dom[0];
Expr output_y = tiles_dom[3] * tile_stride_y + tiles_dom[1];
output(x, y) = 0;
output(output_x, output_y) += tiled_output(tiles_dom[0], tiles_dom[1], tiles_dom[2], tiles_dom[3]);
Note that in the above two blocks of code, tile_stride_x and tile_size_x are independent parameters, allowing for any tile size and overlap.
In both of your examples, tile_size_x = 4, and tile_size_y = 2. To get non-overlapping tiles, set the tile strides equal to the tile size. To get 50% overlapping tiles, set tile_stride_x = 2, and tile_stride_y = 1.
A useful schedule for an algorithm like this is:
// Compute tiles as needed by the output.
tiled_output.compute_at(output, tile_dom[2]);
// or
tiled_output.compute_at(tile_means, xo);
There are other options, like using a pure func (no update/RDom) that uses the mod operator to figure out tile inner and outer indices. However, this approach can be difficult to schedule efficiently with overlapping tiles (depending on the processing you do at each tile). I use the RDom approach when this problem comes up.
Note that with the RDom approach, you have to supply the bounds of the tile indices you want computed (min_tile_xo, extent_tile_xo, ...), which can be tricky for overlapped tiles.
I have just started my journey with Wolfram Mathematica and I want to implement a simple genetic algorithm. The construction of the data is given and I have to start with such rows/columns.
Here is what I have:
chromosome := RandomSample[CharacterRange["A", "G"], 7]
chromosomeList = Table[chromosome, 7] // MatrixForm
This gives me a matrix, where every row represents a chromosome:
yPos = Flatten[Position[chromosomeList, #], 1] & /# {"A", "B", "C",
"D", "E", "F", "G"};
yPos = yPos[[All, 3 ;; 21 ;; 3]] // Transpose
Now every column represents a letter (From A to G) and every row it's index in every chromosome:
Here is a given efficiency matrix, where very row represents different letter (From A to G) and every column gives the value that should be applied on the particular position:
efficiencyMatrix = {
{34, 31, 20, 27, 24, 24, 18},
{14, 14, 22, 34, 26, 19, 22},
{22, 16, 21, 27, 35, 25, 30},
{17, 21, 24, 16, 31, 22, 20},
{17, 29, 22, 31, 18, 19, 26},
{26, 29, 37, 34, 37, 20, 21},
{30, 28, 37, 28, 29, 23, 19}}
What I want to do is to create a matrix with values that correspond to the letter and it's position. I have done it like that:
values = Transpose[{ efficiencyMatrix[[1, yPos[[1]]]],
efficiencyMatrix[[2, yPos[[2]]]],
efficiencyMatrix[[3, yPos[[3]]]],
efficiencyMatrix[[4, yPos[[4]]]],
efficiencyMatrix[[5, yPos[[5]]]],
efficiencyMatrix[[6, yPos[[6]]]],
efficiencyMatrix[[7, yPos[[7]]]]}]
How can I write it in more elegant way?
You can apply a list of functions to some variable using the function Through, which is helpful when applying Position multiple times. Because Position[patt][expr] == Position[expr, patt], we can do
Through[ (Position /# CharacterRange["A","C"])[{"B", "C", "A"}] ]
to get {3, 1, 2}.
Position can also operate on lists, so we can simplify finding ypos by doing
Transpose#Map[Last, Through[(Position /# characters)[chromosomeList]], {2}]
where characters is the relevant output of CharacterRange.
We can also simplify dealing with ranges of integers by mapping over the Range function, so in total we end up with
characters = CharacterRange["A","G"]
efficiencies = ...
chromosomes = ...
ypos = Transpose#Map[Last, Through[(Position /# characters)[chromosomes]], {2}];
efficiencies[[#, ypos[[#]]]]& /# Range[Length[characters]] //Transpose ]
I was playing with integer array hashing and the different ways to go from a representation to the other. I ended up with the following:
void main(string[] args) {
import std.algorithm, std.array, std.conv, std.stdio, std.digest.md;
union hashU {
ubyte[] hashA;
int[] hashN;
};
hashU a;
auto md5 = new MD5Digest();
a.hashN = [1, 2, 3, 4, 5];
/* Using an union, no actual data conversion */
md5.put( a.hashA );
auto hash = md5.finish();
writeln(hash);
// [253, 255, 63, 4, 193, 99, 182, 232, 28, 231, 57, 107, 18, 254, 75, 175]
/* Using a cast... Doesn't match any of the other representations */
md5.put( cast(ubyte[])(a.hashN) );
hash = md5.finish();
writeln(hash);
// [254, 5, 74, 210, 231, 185, 139, 238, 103, 63, 159, 242, 45, 80, 240, 12]
/* Using .to! to convert from array to array */
md5.put( a.hashN.to!(ubyte[]) );
hash = md5.finish();
writeln(hash);
// [124, 253, 208, 120, 137, 179, 41, 93, 106, 85, 9, 20, 171, 53, 224, 104]
/* This matches the previous transformation */
md5.put( a.hashN.map!(x => x.to!ubyte).array );
hash = md5.finish();
writeln(hash);
// [124, 253, 208, 120, 137, 179, 41, 93, 106, 85, 9, 20, 171, 53, 224, 104]
}
My question is the following: what does the cast do? I'd have expected it to do either the same as .to! or the union trick, but it doesn't seem so.
I think Colin Grogan has it right, but his wording is a little confusing.
Using the union, the array is simply reinterpreted, no calculation/computation happens at all. The pointer and length of the int[] are reinterpreted to refer to ubyte elements. Before: 5 ints, after: 5 ubytes.
The cast is a little smarter than that: It adjusts the length of the array so that it refers to the same memory as before. Before: 5 ints, after: 20 ubytes (5*int.sizeof/ubyte.sizeof = 5*4/1 = 20).
Both the union and the cast reinterpret the bytes of the ints as ubytes. That is, an int value 1 will result in 4 ubytes: 0,0,0,1, or 1,0,0,0 depending on endianess.
The to variants convert every single element to the new element type. Before: 5 ints, after: 5 ubytes with the same values as the ints. If one of the ints couldn't be converted to a ubyte, to would throw an exception.
Printing the elements after the various conversions might help clarifying what happens where:
void main()
{
import std.algorithm, std.array, std.conv, std.stdio;
union hashU
{
ubyte[] hashA;
int[] hashN;
}
hashU a;
a.hashN = [1, 2, 3, 4, 5];
writeln( a.hashA ); /* union -> [1, 0, 0, 0, 2] (depends on endianess) */
writeln( cast(ubyte[])(a.hashN) );
/* cast -> [1, 0, 0, 0, 2, 0, 0, 0, 3, 0, 0, 0, 4, 0, 0, 0, 5, 0, 0, 0]
(depends on endianess) */
writeln( a.hashN.to!(ubyte[]) ); /* `to` -> [1, 2, 3, 4, 5] */
}
As far as I know, cast simply tells the compiler to start talking to the chunk of memory that you've cast as the type you've cast it to.
The last two options in your example actually convert the numbers to the new type, and so do some actual work. Which is why they end up being the same values.
The issue in your first two examples is that the int[] is larger in memory than the ubyte[]. (4 bytes per element vs 1 byte per element)
I've edited your first two methods:
/* Using an union, no actual data conversion */
md5.put( a.hashA );
auto hash = md5.finish();
writefln("Hash of: %s -> %s", a.hashA, hash);
// Hash of: [1, 0, 0, 0, 2] -> [253, 255, 63, 4, 193, 99, 182, 232, 28, 231, 57, 107, 18, 254, 75, 175]
// notice 5 bytes in first array
/* Using a cast... Doesn't match any of the other representations */
md5.put( cast(ubyte[])(a.hashN) );
hash = md5.finish();
writefln("Hash of: %s -> %s", cast(ubyte[])(a.hashN), hash);
// Hash of: [1, 0, 0, 0, 2, 0, 0, 0, 3, 0, 0, 0, 4, 0, 0, 0, 5, 0, 0, 0] -> [254, 5, 74, 210, 231, 185, 139, 238, 103, 63, 159, 242, 45, 80, 240, 12]
// notice 20 bytes (4 x 5 bytes) in first array
So in the first, you're reading the length of ubyte[]'s bytes. In the second your casting the length of int[]'s length to ubyte[].
Edit: Prob wasn't clear. A union is pretty dumb, it simply stores all the values in the same memory. When you go to read any of them out, it will only read X bits of that memory, depending on the length of the type you're reading.
So because you're reading the int[] THEN casting it, it reads all 20 of the bytes and casts them to a ubyte[]. Which is of course different than just reading the 5 bytes of the ubyte[] variable.
I think I made sense there :)
cast is a D operator used when developer needs to perform an explicit type conversion.