Building a Timeline from Lossy Time Stamps - c++

Question

I'm working with a product from Velodyne called the VLP-16 (the manual is available from their website with details) and I'm trying to build a timeline of the data it sends. The data it sends comes through a UDP transmission (UDP packets may appear out of order) and each packet is time-stamped with a 32-bit microsecond value. The microsecond value is synced with UTC time. This means that the timestamp will wrap around back to zero after each hour in UTC time. Since UDP packets may technically appear out of order, it is difficult to know what hour a packet may belong to.
Here's a snippet of code that generally describes the problem at hand:
struct LidarPacket
{
uint32_t microsecond;
/* other data */
};
struct LidarTimelineEntry
{
uint32_t hour;
LidarPacket packet;
};
using LidarTimeline = std::vector<LidarTimelineEntry>;
void InsertAndSort(LidarTimeline& timeline, uint32_t hour, const LidarPacket&);
void OnLidarPacket(LidarTimeline &timeline, LidarPacket& newestPacket)
{
/* Where to insert 'newestPacket'? */
}
The simplest approach would be to assume that the packets come in order.
void OnLidarPacket(LidarTimeline &timeline, LidarPacket& newestPacket)
{
if (timeline.empty()) {
timeline.emplace_back(LidarTimelineEntry{0, newestPacket});
return;
}
auto &lastEntry = timeline.back();
if (newestPacket.microsecond < lastEntry.packet.microsecond) {
InsertAndSort(timeline, lastEntry.hour + 1, newestPacket);
} else {
InsertAndSort(timeline, lastEntry.hour, newestPacket);
}
}
This approach will fail if even one packet is out of order though. A slightly more robust way is to also check to see if the wrap occurs near the end of the hour.
bool NearEndOfHour(const LidarPacket& lidarPacket)
{
const uint32_t packetDuration = 1344; // <- approximate duration of one packet.
const uint32_t oneHour = 3600000000; // <- one hour in microseconds
return (lidarPacket.microsecond < packetDuration) || (lidarPacket.microsecond > (oneHour - packetDuration));
}
void OnLidarPacket(LidarTimeline &timeline, LidarPacket& newestPacket)
{
if (timeline.empty()) {
timeline.emplace_back(LidarTimelineEntry{0, newestPacket});
return;
}
auto &lastEntry = timeline.back();
if ((newestPacket.microsecond < lastEntry.packet.microsecond) && NearEndOfHour(lastEntry.packet)) {
InsertAndSort(timeline, lastEntry.hour + 1, newestPacket);
} else {
InsertAndSort(timeline, lastEntry.hour, newestPacket);
}
}
But it's difficult to tell if this is really going to cut it. What is the best way to build a multi-hour timeline from microsecond-stamped data coming from a UDP stream?
Doesn't have to be answered in C++

Answer 1

The approach I ended up going with is the following:
If the timeline is empty, add the time point to the timeline with hour=0
Otherwise, create three possible time points. One using the hour of the last time point, one with an hour before the hour of the last time point, and the last an hour after the last time point.
Compute the absolute differences of all these time points with the last time point (using a 64-bit integer).
Select the hour which yields the smallest absolute difference with the last time point.
This approach passed the following tests:
uint32_t inputTimes[] = { 0, 750, 250 };
// gets sorted to:
uint32_t outputTimes[] = { 0, 250, 750 };
int32_t outputHours[] = { 0, 0, 0 };
uint32_t inputTimes[] = { 1500, oneHour - 500, 250 };
// gets sorted to:
uint32_t outputTimes[] = { oneHour - 500, 250, 1500 };
int32_t outputHours[] = { -1, 0, 0 };
uint32_t inputTimes[] = { oneHour - 500, 1500, 250 };
// gets sorted to:
uint32_t outputTimes[] = { oneHour - 500, 250, 1500 };
int32_t outputHours[] = { 0, 1, 1 };