Try using inception v3 model in ML.net but failed. Keep telling me that the data format is not correct for "DecodeJpeg". I tried different data formats(image path, image, data bytes) and input name like "DecodeJpeg/contents" but they didn't work.
Please see code in bellow. I transferred it from using Inception v1 model.
var pipeline = mlContext.Transforms.LoadImages("DecodeJpeg", imageFolder, "ImagePath")
.Append(mlContext.Transforms.ResizeImages("DecodeJpeg", 224, 224, "DecodeJpeg"))
.Append(mlContext.Transforms.ExtractPixels("DecodeJpeg", "DecodeJpeg", interleavePixelColors: true,
offsetImage: 117))
.Append(mlContext.Model.LoadTensorFlowModel(inceptionModelPath).ScoreTensorFlowModel("softmax/logits", "DecodeJpeg", true)
Related
I have a tensorflow model deployed on Vertex AI of Google Cloud. The model definition is:
item_model = tf.keras.Sequential([
tf.keras.layers.StringLookup(
vocabulary=item_vocab, mask_token=None),
tf.keras.layers.Embedding(len(item_vocab) + 1, embedding_dim)
])
user_model = tf.keras.Sequential([
tf.keras.layers.StringLookup(
vocabulary=user_vocab, mask_token=None),
# We add an additional embedding to account for unknown tokens.
tf.keras.layers.Embedding(len(user_vocab) + 1, embedding_dim)
])
class NCF_model(tf.keras.Model):
def __init__(self,user_model, item_model):
super(NCF_model, self).__init__()
# define all layers in init
self.user_model = user_model
self.item_model = item_model
self.concat_layer = tf.keras.layers.Concatenate()
self.feed_forward_1 = tf.keras.layers.Dense(32,activation= 'relu')
self.feed_forward_2 = tf.keras.layers.Dense(64,activation= 'relu')
self.final = tf.keras.layers.Dense(1,activation= 'sigmoid')
def call(self, inputs ,training=False):
user_id , item_id = inputs[:,0], inputs[:,1]
x = self.user_model(user_id)
y = self.item_model(item_id)
x = self.concat_layer([x,y])
x = self.feed_forward_1(x)
x = self.feed_forward_2(x)
x = self.final(x)
return x
The model has two string inputs and it outputs a probability value.
When I use the following input in the batch prediction file, I get an empty prediction file.
Sample of csv input file:
userid,itemid
yuu,190767
yuu,364
yuu,154828
yuu,72998
yuu,130618
yuu,183979
yuu,588
When I use a jsonl file with the following input.
{"input":["yuu", "190767"]}
I get the following error.
('Post request fails. Cannot get predictions. Error: Exceeded retries: Non-OK result 400 ({\n "error": "Failed to process element: 0 key: input of \'instances\' list. Error: INVALID_ARGUMENT: JSON object: does not have named input: input"\n}) from server, retry=3.', 1)
What seems to be going wrong with these inputs?
After a bit of experimenting, I found out what was wrong with the batch prediction input. In the csv file, the item column was being interpreted as an integer whereas the model has a string as an input. I'm not sure why there was no output at all in that case and I couldn't find the logs for the batch prediction.
The correct format for jsonlines was:
["user1", "item1"]
["user2", "item2"]
["user3", "item3"]
The one I used assumed the input was a named layer, 'input'. In all of this, I found the documentation of google cloud to be lacking.
I have a google-cloud-ml model that I can run prediction by passing a 3 dimensional array of float32...
{ 'instances' [ { 'input' : '[ [ [ 0.0 ], [ 0.5 ], [ 0.8 ] ] ... ] ]' } ] }
However this is not an efficient format to transmit images, so I'd like to pass base64 encoded png or jpeg. This document talks about doing that, but what is not clear is what the entire json object looks like. Does the { 'b64' : 'x0welkja...' } go in place of the '[ [ [ 0.0 ], [ 0.5 ], [ 0.8 ] ] ... ] ]', leaving the enclosing 'instances' and 'input' the same? Or some other structure? Or does the tensorflow model have to be trained on base64?
The TensorFlow model does not have to be trained on base64 data. Leave your training graph as is. However, when exporting the model, you'll need to export a model that can accept PNG or jpeg (or possibly raw, if it's small) data. Then, when you export the model, you'll need to be sure to use a name for the output that ends in _bytes. This signals to CloudML Engine that you will be sending base64 encoded data. Putting it all together would like something like this:
from tensorflow.contrib.saved_model.python.saved_model import utils
# Shape of [None] means we can have a batch of images.
image = tf.placeholder(shape = [None], dtype = tf.string)
# Decode the image.
decoded = tf.image.decode_jpeg(image, channels=3)
# Do the rest of the processing.
scores = build_model(decoded)
# The input name needs to have "_bytes" suffix.
inputs = { 'image_bytes': image }
outputs = { 'scores': scores }
utils.simple_save(session, export_dir, inputs, outputs)
The request you send will look something like this:
{
"instances": [{
"b64": "x0welkja..."
}]
}
If you just want an efficient way to send images to a model (and not necessarily base-64 encode it), I would suggest uploading your images(s) to Google Cloud Storage and then having your model read off GCS. This way, you are not limited by image size and you can take advantage of multi-part, multithreaded, resumable uploads etc. that the GCS API provides.
TensorFlow's tf.read_file will directly off GCS. Here's an example of a serving input_fn that will do this. Your request to CMLE would send it an image URL (gs://bucket/some/path/to/image.jpg)
def read_and_preprocess(filename, augment=False):
# decode the image file starting from the filename
# end up with pixel values that are in the -1, 1 range
image_contents = tf.read_file(filename)
image = tf.image.decode_jpeg(image_contents, channels=NUM_CHANNELS)
image = tf.image.convert_image_dtype(image, dtype=tf.float32) # 0-1
image = tf.expand_dims(image, 0) # resize_bilinear needs batches
image = tf.image.resize_bilinear(image, [HEIGHT, WIDTH], align_corners=False)
#image = tf.image.per_image_whitening(image) # useful if mean not important
image = tf.subtract(image, 0.5)
image = tf.multiply(image, 2.0) # -1 to 1
return image
def serving_input_fn():
inputs = {'imageurl': tf.placeholder(tf.string, shape=())}
filename = tf.squeeze(inputs['imageurl']) # make it a scalar
image = read_and_preprocess(filename)
# make the outer dimension unknown (and not 1)
image = tf.placeholder_with_default(image, shape=[None, HEIGHT, WIDTH, NUM_CHANNELS])
features = {'image' : image}
return tf.estimator.export.ServingInputReceiver(features, inputs)
Your training code will train off actual images, just as in rhaertel80's suggestion above. See https://github.com/GoogleCloudPlatform/training-data-analyst/blob/master/courses/machine_learning/deepdive/08_image/flowersmodel/trainer/task.py#L27 for what the training/evaluation input functions would look like.
I was trying to use #Lak's answer (thanks Lak) to get online predictions for multiple instances in one json file, but kept getting the following error (I had two instances in my test json, hence the shape [2]):
input filename tensor must be scalar but had shape [2]
The problem is that ML engine apparently batches all the instances together and passes them to the serving inpur receiver function, but #Lak's sample code assumes the input is a single instance (it indeed works fine if you have a single instance in your json). I altered the code so that it can process a batch of inputs. I hope it will help someone:
def read_and_preprocess(filename):
image_contents = tf.read_file(filename)
image = tf.image.decode_image(image_contents, channels=NUM_CHANNELS)
image = tf.image.convert_image_dtype(image, dtype=tf.float32) # 0-1
return image
def serving_input_fn():
inputs = {'imageurl': tf.placeholder(tf.string, shape=(None))}
filename = inputs['imageurl']
image = tf.map_fn(read_and_preprocess, filename, dtype=tf.float32)
# make the outer dimension unknown (and not 1)
image = tf.placeholder_with_default(image, shape=[None, HEIGHT, WIDTH, NUM_CHANNELS])
features = {'image': image}
return tf.estimator.export.ServingInputReceiver(features, inputs)
The key changes are that 1) you don't squeeze the input tensor (that would cause trouble in the special case when your json contains only one instance) and, 2) use tf.map_fn to apply the read_and_preprocess function to a batch of input image urls.
I am rying to use tensorboard embeddings page in order to visualize the results of word2vec. After debugging, digging of lots of codes i came to a point that tensorboard successfully runs, reads the confguration file, reads the tsv files but now the embeddings page does not show data.
( the page is opened , i can see the menus , items etc) this is my config file:
embeddings {
tensor_name: 'word_embedding'
metadata_path: 'c:\data\metadata.tsv'
tensor_path: 'c:\data\tensors2.tsv'
}
What can be the problem?
The tensor file originally is 1gb. in size, if i try that file , the app crashes becasue of the memory. So i copy and paste 1 or 2 pages of the original file into tensor2.tsv and use this file. May be this is the problem. May be i need to create more data by copy/ paste.
thx
tolga
Try following code snippet to get visualized word embedding in tensorboard. Open tensorboard with logdir, check localhost:6006 for viewing your embedding.
tensorboard --logdir="visual/1"
# code
fname = "word2vec_model_1000"
model = gensim.models.keyedvectors.KeyedVectors.load(fname)
# project part of vocab, max of 100 dimension
max = 1000
w2v = np.zeros((max,100))
with open("prefix_metadata.tsv", 'w+') as file_metadata:
for i,word in enumerate(model.wv.index2word[:max]):
w2v[i] = model.wv[word]
file_metadata.write(word + '\n')
# define the model without training
sess = tf.InteractiveSession()
with tf.device("/cpu:0"):
embedding = tf.Variable(w2v, trainable=False, name='prefix_embedding')
tf.global_variables_initializer().run()
path = 'visual/1'
saver = tf.train.Saver()
writer = tf.summary.FileWriter(path, sess.graph)
# adding into projector
config = projector.ProjectorConfig()
embed= config.embeddings.add()
embed.tensor_name = 'prefix_embedding'
embed.metadata_path = 'prefix_metadata.tsv'
# Specify the width and height of a single thumbnail.
projector.visualize_embeddings(writer, config)
saver.save(sess, path+'/prefix_model.ckpt', global_step=max)
In summary
File.writeFile() creates a PNG file of 0 bytes when trying to write a Blob made from base64 data.
In my application, I am trying to create a file that consists of base64 data stored in the db. The rendered equivalent of the data is a small anti-aliased graph curve in black on a transparent background (never more that 300 x 320 pixels) that has previously been created and stored from a canvas element. I have independently verified that the stored base64 data is indeed correct by rendering it at one of various base64 encoders/decoders available online.
Output from "Ionic Info"
--------------------------------
Your system information:
Cordova CLI: 6.3.1
Gulp version: CLI version 3.9.1
Gulp local:
Ionic Framework Version: 2.0.0-rc.2
Ionic CLI Version: 2.1.1
Ionic App Lib Version: 2.1.1
Ionic App Scripts Version: 0.0.39
OS:
Node Version: v6.7.0
--------------------------------
The development platform is Windows 10, and I've been testing directly on a Samsung Galaxy S7 and S4 so far.
I know that the base64 data has to be converted into binary data (as a Blob) first, as File does not yet support writing base64 directly in to an image file. I found various techniques with which to do this, and the code which seems to suit my needs the most (and reflects a similar way I would have done it in java is illustrated below):
Main code from constructor:
this.platform.ready().then(() => {
this.graphDataService.getDataItem(this.job.id).then((data) =>{
console.log("getpic:");
let imgWithMeta = data.split(",")
// base64 data
let imgData = imgWithMeta[1].trim();
// content type
let imgType = imgWithMeta[0].trim().split(";")[0].split(":")[1];
console.log("imgData:",imgData);
console.log("imgMeta:",imgType);
console.log("aftergetpic:");
// this.fs is correctly set to cordova.file.externalDataDirectory
let folderpath = this.fs;
let filename = "dotd_test.png";
File.resolveLocalFilesystemUrl(this.fs).then( (dirEntry) => {
console.log("resolved dir with:", dirEntry);
this.savebase64AsImageFile(dirEntry.nativeURL,filename,imgData,imgType);
});
});
});
Helper to convert base64 to Blob:
// convert base64 to Blob
b64toBlob(b64Data, contentType, sliceSize) {
//console.log("data packet:",b64Data);
//console.log("content type:",contentType);
//console.log("slice size:",sliceSize);
let byteCharacters = atob(b64Data);
let byteArrays = [];
for (let offset = 0; offset < byteCharacters.length; offset += sliceSize) {
let slice = byteCharacters.slice(offset, offset + sliceSize);
let byteNumbers = new Array(slice.length);
for (let i = 0; i < slice.length; i++) {
byteNumbers[i] = slice.charCodeAt(i);
}
let byteArray = new Uint8Array(byteNumbers);
byteArrays.push(byteArray);
}
console.log("size of bytearray before blobbing:", byteArrays.length);
console.log("blob content type:", contentType);
let blob = new Blob(byteArrays, {type: contentType});
// alternative way WITHOUT chunking the base64 data
// let blob = new Blob([atob(b64Data)], {type: contentType});
return blob;
}
save the image with File.writeFile()
// save the image with File.writeFile()
savebase64AsImageFile(folderpath,filename,content,contentType){
// Convert the base64 string in a Blob
let data:Blob = this.b64toBlob(content,contentType,512);
console.log("file location attempt is:",folderpath + filename);
File.writeFile(
folderpath,
filename,
data,
{replace: true}
).then(
_ => console.log("write complete")
).catch(
err => console.log("file create failed:",err);
);
}
I have tried dozens of different decoding techniques, but the effect is the same. However, if I hardcode simple text data into the writeFile() section, like so:
File.writeFile(
folderpath,
"test.txt",
"the quick brown fox jumps over the lazy dog",
{replace: true}
)
A text file IS created correctly in the expected location with the text string above in it.
However, I've noticed that whether the file is the 0 bytes PNG, or the working text file above, in both cases the ".then()" consequence clause of the File Promise never fires.
Additionally, I swapped the above method and used the Ionic 2 native Base64-To-Gallery library to create the images, which worked without a problem. However, having the images in the user's picture gallery or camera roll is not an option for me as I do not wish to risk a user's own pictures while marshalling / packing / transmitting / deleting the data-rendered images. The images should be created and managed as part of the app.
User marcus-robinson seems to have experienced a similar issue outlined here, but it was across all file types, and not just binary types as seems to be the case here. Also, the issue seems to have been closed:
https://github.com/driftyco/ionic/issues/5638
Anybody experiencing something similar, or possibly spot some error I might have caused? I've tried dozens of alternatives but none seem to work.
I had similar behaviour saving media files which worked perfectly on iOS. Nonetheless, I had the issue of 0 bytes file creation on some Android devices in release build (dev build works perfectly). After very long search, I followed the following solution
I moved the polyfills.js script tag to the top of the index.html in the ionic project before the cordova.js tag. This re-ordering somehow the issue is resolved.
So the order should look like:
<script src="build/polyfills.js"></script>
<script type="text/javascript" src="cordova.js"></script>
Works on ionic 3 and ionic 4.
The credits go to 1
I got that working with most of your code:
this.file.writeFile(this.file.cacheDirectory, "currentCached.jpeg", this.b64toBlob(src, "image/jpg", 512) ,{replace: true})
The only difference i had was:
let byteCharacters = atob(b64Data.replace(/^data:image\/(png|jpeg|jpg);base64,/, ''));
instead of your
let byteCharacters = atob(b64Data);
Note: I did not use other trimming etc. like those techniques you used in your constructor class.
I'm training my model using TensorFlow in C++. Python is used only for constructing the graph. So is there a way to save and restore the graph and its state purely in C++? I know about the Python class tf.train.Saver but as far as I understand it does not exist in C++.
The tf.train.Saver class currently exists only in Python, but (i) it is built from TensorFlow ops that you can run from C++, and (ii) it exposes the Saver.as_saver_def() method that lets you get a SaverDef protocol buffer with the names of ops that you must run to save or restore a model.
In Python, you can get the names of the save and restore ops as follows:
saver = tf.train.Saver(...)
saver_def = saver.as_saver_def()
# The name of the tensor you must feed with a filename when saving/restoring.
print saver_def.filename_tensor_name
# The name of the target operation you must run when restoring.
print saver_def.restore_op_name
# The name of the target operation you must run when saving.
print saver_def.save_tensor_name
In C++ to restore from a checkpoint, you call Session::Run(), feeding in the name of the checkpoint file as saver_def.filename_tensor_name, with a target op of saver_def.restore_op_name. To save another checkpoint, you call Session::Run(), again feeding in the name of the checkpoint file as saver_def.filename_tensor_name, and fetching the value of saver_def.save_tensor_name.
The recent TensorFlow version includes some helper functions to do the same in C++ without Python. These are generate from the ProtoBuf in the pip-package (${HOME}/.local/lib/python2.7/site-packages/tensorflow/include/tensorflow/core/protobuf/saver.pb.h).
// save
tensorflow::Tensor checkpointPathTensor(tensorflow::DT_STRING, tensorflow::TensorShape());
checkpointPathTensor.scalar<std::string>()() = "some/path";
tensor_dict feed_dict = {{graph_def.saver_def().filename_tensor_name(), checkpointPathTensor}};
status = sess->Run(feed_dict, {}, {graph_def.saver_def().save_tensor_name()}, nullptr);
// restore
tensorflow::Tensor checkpointPathTensor(tensorflow::DT_STRING, tensorflow::TensorShape());
checkpointPathTensor.scalar<std::string>()() = "some/path";
tensor_dict feed_dict = {{graph_def.saver_def().filename_tensor_name(), checkpointPathTensor}};
status = sess->Run(feed_dict, {}, {graph_def.saver_def().restore_op_name()}, nullptr);
This is based on the undocumented python-way (more details) of restoring a model
def restore(sess, metaGraph, fn):
restore_op_name = metaGraph.as_saver_def().restore_op_name # u'save/restore_all'
restore_op = tf.get_default_graph().get_operation_by_name(restore_op_name)
filename_tensor_name = metaGraph.as_saver_def().filename_tensor_name # u'save/Const'
sess.run(restore_op, {filename_tensor_name: fn})
For a working and complete version see here.