In computer vision fields, What is the differences between appearance features and semantic features ?
Those are generally vague terms that depend on the context and the author, and can be used interchangeably. But usually, I think that the term "appearance features" describes the geometric/photometric attributes of a primitive, for example its texture, its surface normal or curvature, etc. These properties can usually be captured in a more or less reliable manner by some local statistics (e.g. traditional SIFT-like descriptors). They tend to be mostly local.
Semantic features, on the other hand, tend to usually be more "high-level", in the sense that they can't be inferred in a trivial manner from local statistics. They describe the meaning of an entity with respect to other entities in the scene, so they are usually tied to the segmentation of the scene into objects that don't only make sense mathematically, but are recognizable as physical objects. For example, a semantic feature for a set of pixels in an image would be something like "trees with red leaves". According to Wikipedia, In ontology engineering, the term semantics refers to the meaning of concepts, properties, and relationships that formally represent real-world entities. I think this is coherent with the explanation given above.
Related
For many high performance applications, such as game engines or financial software, considerations of cache coherency, memory layout, and cache misses are crucial for maintaining smooth performance. As the C++ standard has evolved, especially with the introduction of Move Semantics and C++14, it has become less clear when to draw the line of pass by value vs. pass by reference for mathematical POD based classes.
Consider the common POD Vector3 class:
class Vector3
{
public:
float32 x;
float32 y;
float32 z;
// Implementation Functions below (all non-virtual)...
}
This is the most commonly used math structure in game development. It is a non-virtual, 12 byte size class, even in 64 bit since we are explicitly using IEEE float32, which uses 4 bytes per float. My question is as follows - What is the general best practice guideline to use when deciding to pass POD mathematical classes by value or by reference for high performance applications?
Some things for consideration when answering this question:
It is safe to assume the default constructor does not initialize any values
It is safe to assume no arrays beyond 1D are used for any POD math structures
Clearly most people pass 4-8 byte POD constants by value, so there doesn't seem to be much debate there
What happens when this Vector is a class member variable vs a local variable on the stack? If pass by reference is used, then it would use the memory address of the variable on the class vs a memory address of something local on the stack. Does this use-case matter? Could this difference where PBR is used result in more cache misses?
What about the case where SIMD is used or not used?
What about move semantic compiler optimizations? I have noticed that when switching to C++14, the compiler will often use move semantics when chain function calls are made passing the same vector by value, especially when it is const. I observed this by perusing the assembly breakdown
When using pass by value and pass by reference with these math structures, does const make a much impact on compiler optimizations? See the above point
Given the above, what is a good guideline for when to use pass by value vs pass by reference with modern C++ compilers (C++14 and above) to minimize cache misses and promote cache coherency? At what point might someone say this POD math structure is too large for pass by value, such as a 4v4 affine transform matrix, which is 64 bytes in size assuming use of float32. Does the Vector, or rather any small POD math structure, declared on the stack vs. being referenced as a member variable matter when making this decision?
I am hoping someone can provide some analysis and insight to where a good modern guideline for best practices can be established for the above situation. I believe the line has become more blurry as for when to use PBV vs PBR for POD classes as the C++ standard has evolved, especially in regard to minimizing cache misses.
I see the question title is on the choice of pass-by-value vs. pass-by-reference, though it sounds like what you are after more broadly is the best practice to efficiently passing around 3D vectors and other common PODs. Passing data is fundamental and intertwined with programming paradigm, so there isn't a consensus on the best way to do it. Besides performance, there are considerations to weigh like code readability, flexibility, and portability to decide which approach to favor in a given application.
That said, in recent years, "data-oriented design" has become a popular alternative to object-oriented programming, especially in video game development. The essential idea is to think about the program in terms of data it needs to process, and how all that data can be organized in memory for good cache locality and computation performance. There was a great talk about it at CppCon 2014: "Data-Oriented Design and C++" by Mike Acton.
With your Vector3 example for instance, it is often the case that a program has not just one but many 3D vectors that are all processed the same way, say, all undergo the same geometric transformation. Data-oriented design suggests it is then a good idea to lay the vectors out in contiguously in memory and that they are all transformed together in a batch operation. This improves caching and creates opportunities to leverage SIMD instructions. You could implement this example with the Eigen C++ linear algebra library. The vectors can be represented using a Eigen::Matrix<float, 3, Eigen::Dynamic> of shape 3xN to store N vectors, then manipulated using Eigen's SIMD-accelerated operations.
I come across these two papers from Zhang, et. al (2008) and Wu & Nevatia (2007). One of them classified the paper as local data association based, and the other is global data association based.
After reading the two papers, I assume that by local data association, they used local shape features to the result of foreground extraction such as edgelet features, and boosting edgelet based, and by global data association, they only used common (or should I say non-specific?) features, such as position, scale, appearance, and frame index. Could anyone make sure that my understanding is right? Also, is there any literature that I should read to enhance my comprehension on this context?
Thank you for your attention. I'm looking forward to your reply.
Think of it as local referring to your neighborhood. You know various things around there i.e. what you have learned over time. But if you move to a new neighbourhood you know a little about it. But you still know some things like the new neighbourhood would have a parking space for each house and things like that.
In the same way when you use specific features like some special shapes that are local data association because a shape of an apple would not fit a banana. But if you used some other feature you might be able to generalize your result over multiple inputs which is a case of global data association.
I am porting some graph code out of Python (networkx) and into C++ (BGL). In my Python code, the vertices and edges of the graph are client-defined objects implementing an established interface; I go on to call a bunch of methods on them. Everything is fine.
Naively, it would seem that BGL is meant to support a similar design pattern with "bundled properties." These basically allow one to define custom types for the vertices and edges by passing certain template parameters:
adjacency_list<OutEdgeList, VertexList,
Directed, VertexProperties,
EdgeProperties, GraphProperties,
EdgeList>
The custom vertex and edge types here are given by VertexProperties and EdgeProperties.
In working on this port I've noticed a couple of things that make me think that maybe BGL's bundled properties interface is really only meant to support (more-or-less) immutable types:
Edge and vertex "descriptors"
If you put something into the graph, you get back a "descriptor" that you have to use to reference it from there on. There are descriptors for edges and vertices, and they are the "keys" -- implemented as immutables -- to the graph. So if you have a vertex and you want to find neighboring vertices, you have to (a) get the current vertex descriptor, (b) use BGL methods to find the descriptors of its neighbors, and finally (c) reference each of the neighbors via their respective descriptors.
The net result of all this bookkeeping is an apparent need for additional containers -- std::map, say -- to provide reverse-lookup from vertices and edges (again, types VertexProperty and EdgeProperty) to their descriptors.
"BGL isn't meant to store pointers."
I spotted this claim in a similar SO question but haven't been able to verify it in the Boost docs anywhere. From the ensuing discussion I am left to speculate that the constraint may actually be a bit stronger: "BGL isn't meant to directly reference the heap." This doesn't seem entirely reasonable, though, as the container types are configurable (OutEdgeList and VertexList, above) and default standard things like vectors.
I'm a BGL n00b and am having trouble understanding what's intended with bundled properties. (And, frankly, I feel a bit overwhelmed by the programming model in general -- "properties", "concepts", "traits", "descriptors", AHHHH!) Questions:
Do BGL graphs efficiently support complex and possibly heap-bound types for VertexProperty and EdgeProperty? Or are these meant to be lightweight containers for immutable data?
If the former, is there any way to get around all the descriptor bookkeeping?
If the latter, what's the "right way" to deal with large, complicated things that we might want to stick in a BGL graph?
Do BGL graphs efficiently support complex and possibly heap-bound types for VertexProperty and EdgeProperty? Or are these meant to be lightweight containers for immutable data?
Of course. You can do it anyway you want. Point in case: you could make the bundle hold an (immutable or mutable) pointer to a heap allocated "complex" type that is - of course - entirely mutable. Now,
I'd suggest using your preferred ownership adaptor (unique_ptr, scoped_ptr, shared_ptr, what not). Look at std::string: it too is "heap based", but you didn't ever worry about using it in a property bundle, did you?
If the former, is there any way to get around all the descriptor bookkeeping?
There is not strictly any "descriptor bookkeeping". There might be depending on the graph model. But in general, descriptor is really an abstraction of the underlying container iterator model (not that this could be iterators across multiple containers, e.g. the edges(EdgeListgraph) as opposed to out_edges(v, IncidenceGraph) for adjacency_list<>.
The point is to decouple the logic from assumptions about the storage model. Even with some pretty unsightly void* passing-around, the compiler should compile it down to the same code as direct iterator access. In this sense the "bookkeeping" is usually perceptual, and you are likely picking up on mental load of having the extra conceptual layer.
If the latter, what's the "right way" to deal with large, complicated things that we might want to stick in a BGL graph?
Oops. I think I accidentally addressed this under 1. The absolute simplest thing to use might be refcounted sharing. Your specific situation might allow for more efficient solutions.
CAPITA SELECTA
"BGL isn't meant to store pointers."
Well, not as bundles, perhaps. Even here, it depends solely on how you manage the ownership/lifetime of the pointees.
I think the linked answer is excellent. It resonates with what I said above.
So if you have a vertex and you want to find neighboring vertices, you have to (a) get the current vertex descriptor, (b) use BGL methods to find the descriptors of its neighbors, and finally (c) reference each of the neighbors via their respective descriptors.
Most BGL algorithms rely on the presence of vertex_index_t property (or require one to be specified as a parameter) to make sure these are low-cost operations. In fact, if you use vecS then the vertex index is the index into the vertex vector, so reverse and forward lookups are pretty simple. (You can always look at the generated assembly with optimizations enabled to see whether there are any surprises).
This answer might be inspirational: Boost Graph Library: possible to combine Bundled Properties with Interior Properties?
TL;DR / Summary
I have a feeling coming from Python you might be understandably underestimating C++ optimizing compilers in template-heavy generic library code.
What bubbles up when you sift through the layers of generic machinery in practice evaporates at compile time. Of course, the downside of this is that can be hard to see through the abstractions. But it also means that power and abstraction level aren't compromised.
BGL is one library that allows you to switch to radically different graph models, storage layout etc. with very few code changes. The goal here wasn't "ease of use" or "do what I mean" (use Java or python for that).
The goal was to make the choice for C++ /not/ remove all agility by hard-coding every single implementation detail into your entire code base. Instead, work at the level of library concepts and benefit from the freedom you retain to experiment/change your approach as requirements evolve.
I'm trying to write a program to smooth discrete digitized data for use in a motion simulator. The data will be provided as a set of t, x(t) points and is intended to be used to create cyclic motion; thus the smoothed data must be not only continuous over the range of t values but also between the two endpoints. In addition, data provided will most likely be of significantly lower resolution than required and thus significant interpolation will take place.
I've looked at various techniques such as Gauss-Newton and Levenberg–Marquardt curve fits, but these assume that an objective function is known beforehand (and it will not be). Unfortunately the users of said motion simulator may not be able to choose an appropriate function (due to their differing backgrounds). Finally, the code must be usable on a non-proprietary, cross-platform (and preferably compiled) language which can run on embedded platforms (most likely Linux on ARM) - this precludes the use of Maple (which provides a generic 'fit' routine that selects an appropriate objective function), Matlab (similar IIRC) or other math-related languages. I should say that I'm predisposed to C++ due to experience.
Some typical data can be found on pages here.
What technique would be useful for this?
It would likely be simpler and more adaptive to different data sets to apply Digital Signal Processing (DSP) techniques for rate conversion by upsampling and interpolation. The C++ SPUC library may help you here - it supports several interpolation filters.
I implemented a generic cubic spline fitting function that can be applied to any dimensional euclidean and quaternion data, which might fit (no pun intended) your purpose. I don't know how well the fitting compares to other algorithms though since only the input data keys are considered as potential spline key placement, but you can have a look at it here: http://sourceforge.net/p/spinxengine/code/HEAD/tree/sxp_src/core/math/parametric.inl (fit_spline function).
For creating cyclic motion you should be able to replicate the keys before and after the fitted data sequence and modify the function so that it forces creation of a key at end of the cycle. This is relatively simple function so it shouldn't be a big effort to modify it like that. You need to also setup a spline fitting constraint to define in what kind of conditions new splines are created (e.g. what's distance tolerance from fitted data to input data, etc.)
Cheers, Jarkko
This may be a weird question, but I need to know if the way OpenGL aliases the edges of drawn geometry (polygons, lines, etc) is "consistant", meaning it looks the same across different platforms/hardware.
Aliasing in this case is the sharp edges that "anti-aliasing" aims to remove. I want aliasing however, and would like to use it for animation techniques that need the aliasing effect to be pixel perfect (it's for sprite-type effects). Is this feasible, or is it impossible to predict how it'll alias across different computers?
No it is not. To quote the spec (any version, Appendix A: Invariance):
The OpenGL specification is not pixel exact. It therefore does not guarantee an exact match between images produced by different GL implementations. However, the specification does specify exact matches, in some cases, for images produced by the same implementation.
So there might variation. Also consider that some graphics drivers allow the user to force antialiasing, overriding the application's settings.