Bounding Box Issue - Prediction using ML.Net - ml.net

Question

I am using ML.net to consume Tensorflow object detection model exported from Azure custom vision. As prediction output, I am supposed to get "detected class", "detected score" and "bounding box". I am getting one class and one score but the bounding box is an array of 256 float rows. I am supposed to get only 4 values as bounding box - x,y,h,w but I am getting 256 rows against a single detected class. I am using Ml.Net 1.5.4 from nugget package store in Visual studio

Answer 1

I believe the array float output is flatten (256 / 4 = 64) so there are 64 detections in total. A List<float[4]> output would be easier to understand and interrogate but this isn't happening. Thus, you need to split the 256 array into chunks of 4.
To correctly capture the ONNX outputs for a Custom Vision ONNX exported model requires the following class:
public class ONNXOutput
{
[ColumnName("detected_boxes")]
public float[] DetectedBoxesCoords { get; set; }
[ColumnName("detected_classes")]
public long[] DetectedClasses { get; set; }
[ColumnName("detected_scores")]
public float[] DetectedScores { get; set; }
}
DetectedClasses and DetectedScores are separate outputs from the model and need to be retrieved independently to DetectedBoxes.
This website provides more information https://mlnet-workshop.azurewebsites.net/category/ONNX