I have been struggling with efficiently adding RMSE output for my premade estimator model like you get when using keras' train function. I was eyeing with using add_metrics, but I am not even sure if you can use it for premade estimators and if yes how? Meaning how do I need to code the metric_fn?
The way google uses via calling predict and transforming this into a np.array takes ages for me.
I am happy receiving any idea on how to make this work.
Thanks in advance!
Related
I am currently using Tensorflow 2.0 with a simple CNN, i am initializing the first layer with some handcrafted filters that i would like to visualize during the learning process.
In the histogram part of tensorboard i only see the first kernel of the layer but i would like to see all of them. Is there an easy way to do this?
Thanks in advance
Creating a small function that does this on the displaycallback during the epoch end is the way i solved it, is not the cleanest , and would be nice if someone can correct it :)
class DisplayCallback(tf.keras.callbacks.Callback):
def on_epoch_end(self, epoch, logs=None):
variables_names = [v.name for v in model.trainable_variables]
with file_writer_cm.as_default():
for i in range(model.layers[0].get_weights()[0].shape[3]):
tf.summary.histogram(variables_names[0].split('/')[0]+"/kernel_"+str(i), model.layers[0].get_weights()[0][:,:,:,i], step=epoch)
I have used libsvm in Matlab to create an SVM model. I can't create the model in the code where I do the prediction so I need to save the model and use it later. I want to use that model in my C++ code to make predictions. I know how to predict in matlab itself using svmpredict, but I want to save the model created in matlab and use it in C++ for predictions. Fist of all, is it possible? If so how do I save the model in matlab and call it back in C++?
One option is to save the parameters learned by the model in a csv file. The returned model from svmtrain is a struct. One of the elements of this struct are the model parameters. You could then read this into your C++ file.
However, this seems redundant because libSVM is already written in C. Hence, the predict function being called is being called in C.
If all you need is being able to predict values in your C++ code, one thing you can do is extracting the model parameters in matlab and use it in predictions in your C++ code.
You may already know that you can manually do the predictions by substituting the required values and predicting based on the sign.
This answer has information about what parameters to extract in the case of RBF kernel and how you can make predictions.
Is tf.py_func allowed at online prediction time?
If yes any examples of how to use it?
Does the answer change if I need to install additional pip packages?
My use-case: I work with text, I need to do word stemming (using porter stemmer), I know how to do it using python, tensorflow doesn't have Ops for that. I would like to use the same text processing at training and prediction time - thus I would like to encode it all into a tensorflow graph.
https://www.tensorflow.org/api_docs/python/tf/py_func comes with known limitations and I would like to know if it will work during training and online prediction before I invest more time into it.
Thanks
Unfortunately, no. Py_func can not be restored from a saved model. However, since your use case involves pre-processing, just invoke the py_func explicitly in all three (train, eval, serving) input functions. This won't work if the py_func is in the middle of your graph, but for stemming, it should work just fine.
I've trained a little dataset of car with inception.
Now, I Have a meta file, a ckpt file then a pbtxt file.
And now I want to know how to make prediction with it..
I tried to use freeze_graph.py but it ask an output_node_names parameter and I absolutely don't know what it could be.
If you know how I could use my ckpt/meta/pbtxt to do prediction or how to freeze my graph with freeze_graph to use classify.py I would be very thanksful!
Thanks in advance!
I need to do a 3-fold cross validation using Joaquim's SVM light. Cross Validation and SVM are new things to me and I don't know if I'm doing it right. What have I done so far? I converted my data in 3 files that I called fold1.txt fold2.txt fold3.txt with my features in this following model:
1 numberofthefeature:1 numberofthefeature:1 ...
And I also did a file called words.txt with my tokens where the number of the lines are my numberofthefeature. Did I do everything right?
So, now I have to do the 3-fold cross-validation, but I don't know how to do it with Joaquim's SVM light. I don't know to make the svm light learn and classify using the three files and choose which ones I'm going to use as a test and a train. Do I have to do a script or a program to do it?
Thanks to everybody
Thiago
I am gonna assume that you are doing text-mining as you are referring to Thorsten Joachims. Anyways, here is a set of tutorial videos on text classification, with x-validation:
http://vancouverdata.blogspot.ca/2010/11/text-analytics-with-rapidminer-part-5.html