I am doing reinforcement learning with a deep neural network with the ray/rllib framework. I am writing out the events of TensorFlow. Sometimes the redis-server failes and the training will stop. After restarting, a new tfevent-file will be created. This results in many different colors in tensorboard.
Is there a way to merge these files or tell TensorBoard to give them the same color in the plot?
Here is an example output of multiple graphs, which shows that for one experiment multiple colors are used by tensorboard.
Tensorboard is able to plot from several event files for the same experiment.
One simple solution here is to place the event files that you want merged in the same directory and watch the curves being merged automatically.
Related
I am trying to clean some of the noise in the point clouds that I obtain using a Intel RealSense D435 camera. In particular, I have noticed that the edges in these point clouds tend to be smoothed out to curves which is making segementation based on normals next to impossible.
I have done some research and found that the Intel Realsense SDK does include some post processing algorithms that aim to preserve edges and clean noise (see here). The examples here, however, all seem to be for modifying the RealSense SDK using c++ and I don't see how I would implement them into ROS.
My question is: is it possible to implement these post processing algorithms using ROS? Currently, I start a camera stream using the rs_camera.launch file, but I don't see any parameters I can set here to enable these post-processing methods. Is there a separate node that I can run for this post processing?
It seems that you have to create your own filtering node.
You can have this launch file running a node that publishes a sensor_msgs::PointCloud2 and then create another one that will convert this message type to rs2::pointcloud and use this in your filters as in the rs-post-processing tutorial. After the filtering, you should convert the rs2::pointcloud back to sensor_msgs::PointCloud2.
Later on, it might be interesting to take a look at nodelets to improve your filtering pipeline, so you can separate the filters in different nodelets and use them in other cases as well.
In that sense, ROS is just a way for you to send/receive messages over different nodes.
I am working on a pytorch project, where I’m using a webcam video stream. An object detector is used to find objects within the frame and each box is given an id by a tracker. Then, I want to analyse each bounding box with a CNN-LSTM and classify it (binary classification) based on the previous frame sequence of that box (for the last 5 frames). I want the program to run a close to real-time as possible.
Currently I am stuck with the CNN-LSTM part of my problem - the detector and tracker are working quite well already.
I am a little bit clueless on how to approach this task. Here are the questions I have:
1) How does inferencing work in this case? Do I have to save np arrays for each bounding box containing the last 5 frames, then add the current frame and delete the oldest one? Then use the model for each bounding box that is in the current frame. This way sounds very slow and inefficient. Is there a faster or easier way?
2) Do you have any tipps for creating the dataset? I have a couple of videos with bounding boxes and labels. Should I loop through the videos and save save each frame sequence for each bounding box in a new folder, together with a csv that contains the label? I have never worked with an CNN-LSTM, so I don’t know how to load the data for training.
3) Would it be possible to use the extracted features of the CNN in parallel? As mentioned above, The extracted features should be used by the LSTM for a binary classification problem. The classification is only needed for the current frame. I would like to use an additional classifier (8 classes) based on the extracted CNN features, also only for the current frame. For this classifier, the LSTM is not needed.
Since my explaining propably is very confusing, the following image hopefully helps with understanding what I want to build:
Architecture
This is the architecture I want to use. Is this possible using Pytorch? So far, I only worked with CNNs and LSTM seperately. Any help is apprechiated :)
I would like to use the stand-alone tensorboard projector and have multiple datasets (same data points, different times).
I want the projector to load multiple datasets, apply PCA and then animate the points in time through the "checkpoints". (changing the point coordinates in the 2D/3D space in an animated fashion)
Where can I start to dig in the projector code in order to load multiple datasets and change the rendered points coordinates after initial render?
With still nearly zero experience with Tensorboard, I recently searched for a similar feature.
Loading multiple datasets in TensorboardX is no problem by calling add_embedding multiple times with different global_step parameter. It seems that for transitions there is no build-in solution.
One alternative I suggest is hypertools. It supports multiple dimension reduction methods, as well as animations.
I would appreciate any update on this topic as I am still searching for alternatives.
I have a large (10-100GB) data file of 16-bit integer data, which represents a time series from a data acquisition device. I would like to write a piece of python code that scans through it, plotting a moving window of a few seconds of this data. Ideally, I would like this to be as continuous as possible.
The data is sampled at 4MHz, so to plot a few seconds of data involves plotting ~10 million data points on a graph. Unfortunately I cannot really downsample since the features I want to see are sparse in the file.
matplotlib is not really designed to do this. It is technically possible, and I have a semi-working matplotlib solution which allows me to plot any particular time window, but it's far too slow and cumbersome to do a continuous scan of incrementally changing data - redrawing the figure takes several seconds, which is far too long.
Can anyone suggest a python package or approach do doing this?
PyQtGraph is faster than Matplotlib but I don't know if it can plot 10 million points a second. It does also include multiple methods to down-sample your data so one of them might still be useful to you. Note that it requires Qt and PyQt.
Still, you have between 5e9 and 5e10 data samples. If you can simultaneously plot
10 million of them, this still means making between 500 and 5000 plots. Are you really going to inspect them all visually? You might consider to implement some kind of feature detection.
Something that has worked for me in a similar problem (time varying heat-maps) was to run a batch job of producing several thousands such plots over night, saving each as a separate image. At 10s a figure, you can produce 3600 in 10h. You can then simply scan through the images which could provide you with the insight you're looking for.
What kind of debugging is available for image processing/computer vision/computer graphics applications in C++? What do you use to track errors/partial results of your method?
What I have found so far is just one tool for online and one for offline debugging:
bmd: attaches to a running process and enables you to view a block of memory as an image
imdebug: enables printf-style of debugging
Both are quite outdated and not really what I would expect.
What would seem useful for offline debugging would be some style of image logging, lets say a set of commands which enable you to write images together with text (probably in the form of HTML, maybe hierarchical), easy to switch off at both compile and run time, and the least obtrusive it can get.
The output could look like this (output from our simple tool):
http://tsh.plankton.tk/htmldebug/d8egf100-RF-SVM-RBF_AC-LINEAR_DB.html
Are you aware of some code that goes in this direction?
I would be grateful for any hints.
Coming from a ray tracing perspective, maybe some of those visual methods are also useful to you (it is one of my plans to write a short paper about such techniques):
Surface Normal Visualization. Helps to find surface discontinuities. (no image handy, the look is very much reminiscent of normal maps)
color <- rgb (normal.x+0.5, normal.y+0.5, normal.z+0.5)
Distance Visualization. Helps to find surface discontinuities and errors in finding a nearest point. (image taken from an abandoned ray tracer of mine)
color <- (intersection.z-min)/range, ...
Bounding Volume Traversal Visualization. Helps visualizing a bounding volume hierarchy or other hierarchical structures, and helps to see the traversal hotspots, like a code profiler (e.g. Kd-trees). (tbp of http://ompf.org/forum coined the term Kd-vision).
color <- number_of_traversal_steps/f
Bounding Box Visualization (image from picogen or so, some years ago). Helps to verify the partitioning.
color <- const
Stereo. Maybe useful in your case as for the real stereographic appearance. I must admit I never used this for debugging, but when I think about it, it could prove really useful when implementing new types of 3d-primitives and -trees (image from gladius, which was an attempt to unify realtime and non-realtime ray tracing)
You just render two images with slightly shifted position, focusing on some point
Hit-or-not visualization. May help to find epsilon errors. (image taken from metatrace)
if (hit) color = const_a;
else color = const_b
Some hybrid of several techniques.
Linear interpolation: lerp(debug_a, debug_b)
Interlacing: if(y%2==0) debug_a else debug_b
Any combination of ideas, for example the color-tone from Bounding Box Visualization, but with actual scene-intersection and lighting applied
You may find some more glitches and debugging imagery on http://phresnel.org , http://phresnel.deviantart.com , http://picogen.deviantart.com , and maybe http://greenhybrid.deviantart.com (an old account).
Generally, I prefer to dump bytearray of currently processed image as raw data triplets and run Imagemagick to create png from it with number e.g img01.png. In this way i can trace the algorithms very easy. Imagemagick is run from the function in the program using system call. This make possible do debug without using any external libs for image formats.
Another option, if you are using Qt is to work with QImage and use img.save("img01.png") from time to time like a printf is used for debugging.
it's a bit primitive compared to what you are looking for, but i have done what you suggested in your OP using standard logging and by writing image files. typically, the logging and signal export processes and staging exist in unit tests.
signals are given identifiers (often input filename), which may be augmented (often process name or stage).
for development of processors, it's quite handy.
adding html for messages would be simple. in that context, you could produce viewable html output easily - you would not need to generate any html, just use html template files and then insert the messages.
i would just do it myself (as i've done multiple times already for multiple signal types) if you get no good referrals.
In Qt Creator you can watch image modification while stepping through the code in the normal C++ debugger, see e.g. http://labs.qt.nokia.com/2010/04/22/peek-and-poke-vol-3/