I have a network model defined using slim library in python tensorflow.
Now I want to port it to c++. However it seems to me that tensorflow.contrib.slim is a python only library. How could I use it in my c++ software?
Basically I have something like:
self.conv1 = slim.convolution2d( \
inputs=self.imageIn,num_outputs=32,\
kernel_size=[8,8],stride=[4,4],padding='VALID', \
biases_initializer=None,scope=myScope+'_conv1')
How could I implement this in c++ tensorflow?
At the moment I'm trying to replace slim implementation with plain tensorflow, and then going c++. However replacing
self.conv1 = slim.convolution2d( \
inputs=self.imageIn,num_outputs=32,\
kernel_size=[8,8],stride=[1,4,4,1],padding='VALID', \
biases_initializer=None, weights_initializer=_initializer, scope=myScope+'_conv1')
with plain tf
with tf.variable_scope(myScope+'_conv1'):
weights = tf.get_variable("weights",[8, 8, 3, 32],
initializer=_initializer, dtype=tf.float32)
self.conv1 = tf.nn.conv2d(self.imageIn, weights, [1, 4, 4, 1], padding='VALID')
in a working model produces a mess: nothing works anymore. What am I forgetting?
Thanks a lot
Related
Sorry that this question is a bit vague but that's the problem, I've been pretty much shooting in the dark due to being unable to find ANY information on this specific area.
I'm hosting AUAudioUnits on OSX. That all works fine. I can find em, load em, instantiate em and use em. No probs.
But it seems there are more options buried somewhere and I've no idea where to even look to configure this.
So... the problem: I have a specific AU (Superior Drummer 3 in case it's relevant) that comes in 2 flavours: stereo and 16 channel. It seems that both come from the same component (I assume via AUAudioUnit.registerSubclass) and it has a configurationDictionary that seems to contain the information for the 16 channel version (configurationDictionary dump here):
configs: ["SupportedChannelLayoutTags": {
Output = (
6619138,
6684674,
6750210,
6946818
);
}, "HasCustomView": 1, "BusCountWritable": <__NSArrayI 0x600000cd3210>(
0,
0,
0
)
, "ChannelConfigurations": <__NSSingleObjectArrayI 0x600000006e00>(
<__NSArrayI 0x600000290620>(
0,
2
)
)
, "InitialOutputs": <__NSArrayI 0x600003025ef0>(
2,
2,
2,
2,
2,
2,
2,
2,
2,
2,
2,
2,
2,
2,
2,
2
)
]
But I have absolutely no idea where to go from here and am utterly stumped. How is one supposed to extract that information from the dictionary (given that I have no idea of the type information - and bearing in mind I've no interest in hacking it, I want to know the proper flags etc to do this) and how is one supposed to reconfigure the subclass with that info?
Any help at all would be so appreciated I cannot tell you. Even just a pointer in the right direction.
Cheers
I have trained some models using tensorflow 1.5.1 and I have the checkpoints for those models (including .ckpt and .meta files). Now I want to do inference in c++ using those files.
In python, I would do the following to save and load the graph and the checkpoints.
for saving:
images = tf.placeholder(...) // the input layer
//the graph def
output = tf.nn.softmax(net) // the output layer
tf.add_to_collection('images', images)
tf.add_to_collection('output', output)
for inference i restore the graph and the checkpoint then restore the input and output layers from collections like so:
meta_file = './models/last-100.meta'
ckpt_file = './models/last-100'
with tf.Session() as sess:
saver = tf.train.import_meta_graph(meta_file)
saver.restore(sess, ckpt_file)
images = tf.get_collection('images')
output = tf.get_collection('output')
outputTensors = sess.run(output, feed_dict={images: np.array(an_image)})
now assuming that I did the saving in python as usual, how can I do inference and restore in c++ with simple code like in python?
I have found examples and tutorials but for tensorflow versions 0.7 0.12 and the same code doesn't work for version 1.5. I found no tutorials for restoring models using c++ API on tensorflow website.
For the sake of this thread. I will rephrase my comment into an answer.
Posting a full example would require either a CMake setup or putting the file into a specific directory to run bazel. As I do favor the first way and it would burst all limits on this post to cover all parts I would like to redirect to a complete implementation in C99, C++, GO without Bazel which I tested for TF > v1.5.
Loading a graph in C++ is not much more difficult than in Python, given you compiled TensorFlow already from source.
Start by creating a MWE, which creates a very dump network graph is always a good idea to figure out how things work:
import tensorflow as tf
x = tf.placeholder(tf.float32, shape=[1, 2], name='input')
output = tf.identity(tf.layers.dense(x, 1), name='output')
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
saver = tf.train.Saver(tf.global_variables())
saver.save(sess, './exported/my_model')
There are probably tons of answers here on SO about this part. So I just let it stay here without further explanation.
Loading in Python
Before doing stuff in other languages, we can try to do it in python properly -- in the sense: we just need to rewrite it in C++.
Even restoring is very easy in python like:
import tensorflow as tf
with tf.Session() as sess:
# load the computation graph
loader = tf.train.import_meta_graph('./exported/my_model.meta')
sess.run(tf.global_variables_initializer())
loader = loader.restore(sess, './exported/my_model')
x = tf.get_default_graph().get_tensor_by_name('input:0')
output = tf.get_default_graph().get_tensor_by_name('output:0')
it is not helpful as most of these API endpoints do not exists in the C++ API (yet?). An alternative version would be
import tensorflow as tf
with tf.Session() as sess:
metaGraph = tf.train.import_meta_graph('./exported/my_model.meta')
restore_op_name = metaGraph.as_saver_def().restore_op_name
restore_op = tf.get_default_graph().get_operation_by_name(restore_op_name)
filename_tensor_name = metaGraph.as_saver_def().filename_tensor_name
sess.run(restore_op, {filename_tensor_name: './exported/my_model'})
x = tf.get_default_graph().get_tensor_by_name('input:0')
output = tf.get_default_graph().get_tensor_by_name('output:0')
Hang on. You can always use print(dir(object)) to get the properties like restore_op_name, ... .
Restoring a model is an operation in TensorFlow like every other operation. We just call this operation and providing the path (a string-tensor) as an input. We can even write our own restore operation
def restore(sess, metaGraph, fn):
restore_op_name = metaGraph.as_saver_def().restore_op_name # u'save/restore_all'
restore_op = tf.get_default_graph().get_operation_by_name(restore_op_name)
filename_tensor_name = metaGraph.as_saver_def().filename_tensor_name # u'save/Const'
sess.run(restore_op, {filename_tensor_name: fn})
Even this looks strange, it now greatly helps to do the same stuff in C++.
Loading in C++
Starting with the usual stuff
#include <tensorflow/core/public/session.h>
#include <tensorflow/core/public/session_options.h>
#include <tensorflow/core/protobuf/meta_graph.pb.h>
#include <string>
#include <iostream>
typedef std::vector<std::pair<std::string, tensorflow::Tensor>> tensor_dict;
int main(int argc, char const *argv[]) {
const std::string graph_fn = "./exported/my_model.meta";
const std::string checkpoint_fn = "./exported/my_model";
// prepare session
tensorflow::Session *sess;
tensorflow::SessionOptions options;
TF_CHECK_OK(tensorflow::NewSession(options, &sess));
// here we will put our loading of the graph and weights
return 0;
}
You should be able to compile this by either put it in the TensorFlow repo and use bazel or simply follow the instructions here to use CMake.
We need to create such a meta_graph created by tf.train.import_meta_graph. This can be done by
tensorflow::MetaGraphDef graph_def;
TF_CHECK_OK(ReadBinaryProto(tensorflow::Env::Default(), graph_fn, &graph_def));
In C++ reading a graph from file is not the same as importing a graph in Python. We need to create this graph in a session by
TF_CHECK_OK(sess->Create(graph_def.graph_def()));
By looking at the strange python restore function above:
restore_op_name = metaGraph.as_saver_def().restore_op_name
restore_op = tf.get_default_graph().get_operation_by_name(restore_op_name)
filename_tensor_name = metaGraph.as_saver_def().filename_tensor_name
we can code the equivalent piece in C++
const std::string restore_op_name = graph_def.saver_def().restore_op_name()
const std::string filename_tensor_name = graph_def.saver_def().filename_tensor_name()
Having this in place, we just run the operation by
sess->Run(feed_dict, // inputs
{}, // output_tensor_names (we do not need them)
{restore_op}, // target_node_names
nullptr) // outputs (there are no outputs this time)
Creating the feed_dict is probably a post on its own and this answer is already long enough. It does only cover the most important stuff. I would like to redirect to a complete implementation in C99, C++, GO without Bazel which I tested for TF > v1.5. This is not that hard -- it just can get very long in the case of the plain C version.
The following code (taken from - https://github.com/dennybritz/tf-rnn/blob/master/bidirectional_rnn.ipynb)
import tensorflow as tf
import numpy as np
tf.reset_default_graph()
# Create input data
X = np.random.randn(2, 10, 8)
# The second example is of length 6
X[1,6:] = 0
X_lengths = [10, 6]
cell = tf.contrib.rnn.LSTMCell(num_units=64, state_is_tuple=True)
outputs, states = tf.nn.bidirectional_dynamic_rnn(
cell_fw=cell,
cell_bw=cell,
dtype=tf.float64,
sequence_length=X_lengths,
inputs=X)
output_fw, output_bw = outputs
states_fw, states_bw = states
is giving the following error for
tensorflow - 1.1 for both 2.7 and 3.5
ValueError: Attempt to reuse RNNCell <tensorflow.contrib.rnn.python.ops.core_rnn_cell_impl.LSTMCell object at 0x10ce0c2b0>
with a different variable scope than its first use. First use of cell was with scope
'bidirectional_rnn/fw/lstm_cell', this attempt is with scope 'bidirectional_rnn/bw/lstm_cell'.
Please create a new instance of the cell if you would like it to use a different set of weights.
If before you were using: MultiRNNCell([LSTMCell(...)] * num_layers), change to:
MultiRNNCell([LSTMCell(...) for _ in range(num_layers)]). If before you were using the same cell
instance as both the forward and reverse cell of a bidirectional RNN, simply create two instances
(one for forward, one for reverse). In May 2017, we will start transitioning this cell's behavior to use
existing stored weights, if any, when it is called with scope=None (which can lead to silent model degradation,
so this error will remain until then.)
But it is working in
tensorflow - 1.0.1 for python 3.5 (did not test on python - 2.7)
I tried with multiple code examples I found online but
tf.nn.bidirectional_dynamic_rnn
is giving the same error with tensorflow - 1.1
Is there a bug in tensorflow 1.1 or am i just missing something?
Sorry you ran into this. I can confirm that the error appears in 1.1 (docker run -it gcr.io/tensorflow/tensorflow:1.1.0 python) but not in 1.2 RC0 (docker run -it gcr.io/tensorflow/tensorflow:1.2.0-rc0 python).
So it looks like either 1.2-rc0 or 1.0.1 are your options for the moment.
I am using python's pil library to display images. Now I have a sequence of frames to display as a video content. I have a np.array that contains the RGB values of all the frames.
Could not find a method similar to Mathlabs implay to display these frames.
I can use imshow in a loop, but thats would be very slow as I need to mention framerate.
Matplotlib animations work well, and is easy to use. For reasonable size images they typically run at 30fps, or around that. Matplotlib 1.1+ has a nice new animation interface: here are some examples and a tutorial.
Older versions of matplotlib aren't to hard to animate either (you basically just set the data directly and refresh the plot) but the animation depends a bit more on the backend, so you need to look for an appropriate example.
For a specific example, if images is your list of matplotlib images that you want to animate, you can simply do:
animation.ArtistAnimation(fig, images, interval=50, blit=True, repeat_delay=1000)
This, btw, is taken from this example, if you want to also see the code that generates test images. The code to animate is simply the line above.
I have implemented a handy script that just suits your need. Try it out here
An example to show lists of images in a directory will be
import os
import glob
from scipy.misc import imread
img_dir = 'YOUR-IMAGE-DIRECTORY'
img_files = glob.glob(os.path.join(video_dir, '*.jpg'))
def redraw_fn(f, axes):
img_file = img_files[f]
img = imread(img_file)
if not redraw_fn.initialized:
redraw_fn.im = axes.imshow(img, animated=True)
redraw_fn.initialized = True
else:
redraw_fn.im.set_array(img)
redraw_fn.initialized = False
videofig(len(img_files), redraw_fn, play_fps=30)
If you happen to already have a working OpenCV install built with OpenEXR – which if you don’t, it’s at least as much of an irritating time-sink to rebuild OpenCV as e.g. compiling SciPy from source in the first place – but so if the library is already there and working, you can use the Python bindings to quickly† view images with only a little bit of boilerplate. From this example:
import OpenEXR, Imath, cv
filename = "GoldenGate.exr"
exrimage = OpenEXR.InputFile(filename)
dw = exrimage.header()['dataWindow']
(width, height) = (dw.max.x - dw.min.x + 1, dw.max.y - dw.min.y + 1)
def fromstr(s):
mat = cv.CreateMat(height, width, cv.CV_32FC1)
cv.SetData(mat, s)
return mat
pt = Imath.PixelType(Imath.PixelType.FLOAT)
(r, g, b) = [fromstr(s) for s in exrimage.channels("RGB", pt)]
bgr = cv.CreateMat(height, width, cv.CV_32FC3)
cv.Merge(b, g, r, None, bgr)
cv.ShowImage(filename, bgr)
cv.WaitKey()
I believe the OpenCV matrix type implements the python interfaces for memoryview et al – don’t be scared off by those objects as they’re NumPy arrays with different socks on, if you will.
†) quickly, w/r/t both the developer sense of speed: you can use this stuff immediately instead of building SciPy addons or mucking about with the Python array view C interface; but also in the real sense, as everything that comprises the aforementioned stuff – the OpenCV matrix structs, their related Python C API underpinnings, the OpenEXR format, and the stock implementation of the interface to same – have been raked over the optimization coals for years, largely by notable and grant-backed squadrons of specialist scholar-nerds who know what they are doing in this arena.
so I decided I would put my few R functions into a package and I'm reading/learning Writing R Extension.
it obviously complains about an amount of things I'm not doing right.
after enough googling, I'm firing a few questions here, this one is about testing style: I am using RUnit and I like having tests as close possible to the code being tested. this way I won't forget about the tests and I use the tests as part of the technical documentation.
for example:
fillInTheBlanks <- function(S) {
## NA in S are replaced with observed values
## accepts a vector possibly holding NA values and returns a vector
## where all observed values are carried forward and the first is
## carried backward. cfr na.locf from zoo library.
L <- !is.na(S)
c(S[L][1], S[L])[1 + cumsum(L)]
}
test.fillInTheBlanks <- function() {
checkEquals(fillInTheBlanks(c(1, NA, NA, 2, 3, NA, 4)), c(1, 1, 1, 2, 3, 3, 4))
checkEquals(fillInTheBlanks(c(1, 2, 3, 4)), c(1, 2, 3, 4))
checkEquals(fillInTheBlanks(c(NA, NA, 2, 3, NA, 4)), c(2, 2, 2, 3, 3, 4))
}
but R CMD check issues NOTE lines, like this one:
test.fillInTheBlanks: no visible global function definition for
‘checkEquals’
and it complains about me not documenting the test functions.
I don't really want to add documentation for the test functions and I definitely would prefer not having to add a dependency to the RUnit package.
how do you think I should look at this issue?
Where are you putting your unit tests? You may not want to put them into the R directory. A more standard approach is to put them under inst\unitTests. Have a look at this R-wiki page regarding the configuration.
Alternatively, you can specify what files will be exported in your NAMESPACE, and by extension, what functions should and should not be documented.
Beyond that, ideally you should have your tests run when R CMD CHECK is called; that's part of the design. In which case, you should create a test script to call your tests in a separate tests directory. And you will need to load the RUnit package in that script (but you don't need to make it a dependency of your package).
Edit 1:
Regarding your failure because it can't find the checkEquals function: I would change you function to be like this:
test.fillInTheBlanks <- function() {
require(RUnit)
checkEquals(fillInTheBlanks(c(1, NA, NA, 2, 3, NA, 4)), c(1, 1, 1, 2, 3, 3, 4))
checkEquals(fillInTheBlanks(c(1, 2, 3, 4)), c(1, 2, 3, 4))
checkEquals(fillInTheBlanks(c(NA, NA, 2, 3, NA, 4)), c(2, 2, 2, 3, 3, 4))
}
That way the package is loaded when the function is called or it will inform the user that the package is required.
Edit 2:
From "Writing R Extensions":
Note that all user-level objects in a package should be documented; if a package pkg contains user-level objects which are for “internal” use only, it should provide a file pkg-internal.Rd which documents all such objects, and clearly states that these are not meant to be called by the user. See e.g. the sources for package grid in the R distribution for an example. Note that packages which use internal objects extensively should hide those objects in a name space, when they do not need to be documented (see Package name spaces).
You can use the pkg-internal.Rd file as one option, but if you intend on having many hidden objects, this is usually handled in the declarations in the NAMESPACE.
Did you load the RUnit package?
Your best bet is probably to look at a package containing existing code using RUnit.