I've been searching for a C/C++ library that does symbolic differantation and integrals of polynoms, but haven't found one that suits my needs.
I'm afraid that the problem is that I'm not using the correct terminology.
The problem is this :
given a polynom p, I would like to look at the function
f(p) = integral of (p')^2 from a to b
And generate partial derivatives for f with respect to p's coefficients.
Theoretically, there should be no problem here as we are dealing with polynoms, but I haven't found something that can keep the connection between the original coefficients and the modified polynom.
Does anyone know if there are libraries that can do such things, or am I better of creating my own?
Have you tried to use http://www.fadbad.com/fadbad.html ? It's quite useful.
I would write my own derivative class. There are books available meanwhile which document how to do this. But assuming you know the math rules, it is rather trivial.
Using such a derivative class you can then write a template function to generate your polynomial and the derivative and the square and the integral while keeping track of the derivatives vs. the coefficients. The problem is that you may carry around a lot of derivatives which are always zero. To avoid this is rather complicated.
A normal derivative class would contain a value and an array of derivative values.
There may be a constructor to create an independent variable by value and index -- initializing the value by the passed value and all derivatives to zero except the one matching the index to 1.
Then you write operators and functions for everything you need -- which is not much assuming you're only dealing with polynomials.
Related
I'm using the Eigen::LevenbergMarquardt solver for a model fitting application. The functor I'm providing to it includes functions to compute the error vector and Jacobian. These functions contain a lot of similar code and some costly calculations are repeated duplicated.
The prototype of the () operator used to compute the error vector includes what appears to be an optional pointer to a Jacobian matrix. If the Eigen::LevenbergMarquardt solver can be setup to compute the error vector and Jacobian at the same time in relevant cases it would really speed up my algorithm.
int operator()(const Eigen::VectorXf& z, Eigen::VectorXf& fvec, Eigen::MatrixXf* _j = 0)
I have not found any documentation describing this _j parameter or how it can be used. Checking its value during my code shows it is always a NULL pointer.
Does anyone know what this parameter is used for and if it's possible to compute the error vector and Jacobian simultaneously when both are needed?
Not sure about this particular solver but these kind of parameters are commonly only used whenever the solver needs it. Maybe it is just an extension point for the future. Looking at the source code, the LM solver never calls the functor with that parameter.
I think a better approach in your case would be to cache redundant parts of the computation within your functor. Maybe just keep a copy of the input vector and do a quick memcmp before doing the computation. Not ideal but since the interface has no way of telling you when the inputs change, that's probably the most robust option you have.
I am using Eigen to do some linear algebra computations in my code. However, all of mathematical formulas are based on the fact that indexing starts from 1. So, each time that I want to implement them in the code, I have to check if my indexing in the code is consistent with them or not. I was wondering that if it is possible to tell Eigen to start the indexing from 1 instead of 0.
Indexing operations in Eigen allow, in addition to indexing with integers, indexing with symbolic indices. You should be able implement your own custom symbolic index, derived from Eigen::symbolic::BaseExpr, that could be used as a 1-based index API, where its eval_impl method simply subtracts 1 from its arg. E.g.:
template<typename Arg0>
class MyIndexExpr : public BaseExpr<MyIndexExpr<Arg0> >
{
public:
MyIndexExpr(const Arg0& arg0) : m_arg0(arg0) {}
template<typename T>
Index eval_impl(const T& values) const { return m_arg0.eval_impl(values) - 1; }
protected:
Arg0 m_arg0;
};
To use this in product code, however, would most likely be a very bad idea, likely to lead to confusion, possible bugs as well as an extra non-motivated run-time overhead for any indexing. As you are coding in C++, you might want to stick to its zero-based indexing practice. Maybe you could consider symbolic indexing for when writing tests for your formulas, but using integer zero-based indexing for your product code.
The answer is "not really".
Of course, as #πάνταῥεῖ suggested, you could write a wrapper or inherit from Eigen types and overwrite the indexing operators accordingly. Alternatively, you could implement a custom Index type which, when converted to Eigen::Index, will subtract 1.
But both approaches are error-prone and will likely rather increase confusion, especially if you miss some relevant parts. Also it will extremely confuse any C++ programmer looking at your code, as 0-based indexing is the most natural way in C/C++ (and many languages whose syntax is derived from them, like Java, C#, ...)
Finally, (as also suggested by #dfri) if you code in C++ get used to 0-based indexing, this will save you a lot of trouble in the long run.
I'm new to C++ and I think a good way for me to jump in is to build some basic models that I've built in other languages. I want to start with just Linear Regression solved using first order methods. So here's how I want things to be organized (in pseudocode).
class LinearRegression
LinearRegression:
tol = <a supplied tolerance or defaulted to 1e-5>
max_ite = <a supplied max iter or default to 1k>
fit(X, y):
// model learns weights specific to this data set
_gradient(X, y):
// compute the gradient
score(X,y):
// model uses weights learned from fit to compute accuracy of
// y_predicted to actual y
My question is when I use fit, score and gradient methods I don't actually need to pass around the arrays (X and y) or even store them anywhere so I want to use a reference or a pointer to those structures. My problem is that if the method accepts a pointer to a 2D array I need to supply the second dimension size ahead of time or use templating. If I use templating I now have something like this for every method that accepts a 2D array
template<std::size_t rows, std::size_t cols>
void fit(double (&X)[rows][cols], double &y){...}
It seems there likely a better way. I want my regression class to work with any size input. How is this done in industry? I know in some situations the array is just flattened into row or column major format where just a pointer to the first element is passed but I don't have enough experience to know what people use in C++.
You wrote a quite a few points in your question, so here are some points addressing them:
Contemporary C++ discourages working directly with heap-allocated data that you need to manually allocate or deallocate. You can use, e.g., std::vector<double> to represent vectors, and std::vector<std::vector<double>> to represent matrices. Even better would be to use a matrix class, preferably one that is already in mainstream use.
Once you use such a class, you can easily get the dimension at runtime. With std::vector, for example, you can use the size() method. Other classes have other methods. Check the documentation for the one you choose.
You probably really don't want to use templates for the dimensions.
a. If you do so, you will need to recompile each time you get a different input. Your code will be duplicated (by the compiler) to the number of different dimensions you simultaneously use. Lots of bad stuff, with little gain (in this case). There's no real drawback to getting the dimension at runtime from the class.
b. Templates (in your setting) are fitting for the type of the matrix (e.g., is it a matrix of doubles or floats), or possibly the number of dimesions (e.g., for specifying tensors).
Your regressor doesn't need to store the matrix and/or vector. Pass them by const reference. Your interface looks like that of sklearn. If you like, check the source code there. The result of calling fit just causes the class object to store the parameter corresponding to the prediction vector β. It doesn't copy or store the input matrix and/or vector.
I have a dataset from custom abstract objects and a custom distance function. Is there any good SVM libraries that allows me to train on my custom objects (not 2d points) and my custom distance function?
I searched the answers in this similar stackoverflow question, but none of them allows me to use custom objects and distance functions.
First things first.
SVM does not work on distance functions, it only accepts dot products. So your distance function (actually similarity, but usually 1-distance is similarity) has to:
be symmetric s(a,b)=s(b,a)
be positive definite s(a,a)>=0, s(a,a)=0 <=> a=0
be linear in first argument s(ka, b) = k s(a,b) and s(a+b,c) = s(a,c) + s(b,c)
This can be tricky to check, as you actually ask "is there a function from my objects to some vector space, phi such that s(phi(x), phi(y))" is a dot-product, thus leading to definition of so called kernel, K(x,y)=s(phi(x), phi(y)). If your objects are themselves elements of vector space, then sometimes it is enough to put phi(x)=x thus K=s, but it is not true in general.
Once you have this kind of similarity nearly any SVM library (for example libSVM) works with providing Gram matrix. Which is simply defined as
G_ij = K(x_i, x_j)
Thus requiring O(N^2) memory and time. Consequently it does not matter what are your objects, as SVM only works on pairwise dot-products, nothing more.
If you look appropriate mathematical tools to show this property, what can be done is to look for kernel learning from similarity. These methods are able to create valid kernel which behaves similarly to your similarity.
Check out the following:
MLPack: a lightweight library that provides lots of functionality.
DLib: a very popular toolkit that is used both in industry and academia.
Apart from these, you can also use Python packages, but import them from C++.
I'm trying to build a polynomial function generator, so that it takes a vector (arbitrary size) as argument, and generates a polynomial function I can use later.
for instance,
poly_gen(vector<int> power_index)
returns a function (or by other method) in forms of (that i can call with another function)
y(k)=a0+ a1*n+ a2*n^2 + a3*n^3 + ... + ak*n^k
where a0,a1....ak are stored in the vector- power_index
and later I can call it with
int calc_poly(int n)
and this calc_poly can return me a number, calculated by using the polynomial expression generated by poly_gen()
PS:
I don't know how to search this question by key words.
function,construction, generator, pointer, functor...
didn't give me the desired results.
thank you all!
You can't generate functions at runtime in C++, so you're going to have to go with a functor.
You can create an object that stores the coefficients given by power_index in some manner (perhaps a direct copy), and give it an operator() (int n) operator that will take the coefficients and calculate the value of the polynomial (Horner's rule?). Then you can pass that object around freely.
So, you need a constructor, an internal representation of the coefficients, and an operator() that does the actual calculation. Should be simple enough.
There was a nice "rosetta stone" question on (almost) this very problem a while back.
There are several C++ answers there: one using boost::lambda, one using a more conventional approach, and one using MPL (and also a C++0x version, which IMHO would be the ideal solution if your compiler supports it). Obviously the simple quadratic there will need generalizing to arbitrary numbers of powers but that's simple enough compared with getting your head around the function object concept.