ML Engine Online Prediction - Unexpected tensor name: values - google-cloud-platform

Question

I get the following error when trying to make an online prediction on my ML Engine model.
The key "values" is not correct. (See error on image.)
enter image description here
I already tested with RAW image data : {"image_bytes":{"b64": base64.b64encode(jpeg_data)}}
& Converted the data to a numpy array.
Currently I have the following code:
from googleapiclient import discovery
import base64
import os
from PIL import Image
import json
import numpy as np
os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = "/Users/jacob/Desktop/******"
def predict_json(project, model, instances, version=None):
"""Send json data to a deployed model for prediction.
Args:
project (str): project where the Cloud ML Engine Model is deployed.
model (str): model name.
instances ([Mapping[str: Any]]): Keys should be the names of Tensors
your deployed model expects as inputs. Values should be datatypes
convertible to Tensors, or (potentially nested) lists of datatypes
convertible to tensors.
version: str, version of the model to target.
Returns:
Mapping[str: any]: dictionary of prediction results defined by the
model.
"""
# Create the ML Engine service object.
# To authenticate set the environment variable
# GOOGLE_APPLICATION_CREDENTIALS=<path_to_service_account_file>
service = discovery.build('ml', 'v1')
name = 'projects/{}/models/{}'.format(project, model)
if version is not None:
name += '/versions/{}'.format(version)
response = service.projects().predict(
name=name,
body={'instances': instances}
).execute()
if 'error' in response:
raise RuntimeError(response['error'])
return response['predictions']
savepath = 'upload/11277229_F.jpg'
img = Image.open('test/01011000/11277229_F.jpg')
test = img.resize((299, 299))
test.save(savepath)
img1 = open(savepath, "rb").read()
def load_image(filename):
with open(filename) as f:
return np.array(f.read())
predict_json('image-recognition-25***08', 'm500_200_waug', [{"values": str(base64.b64encode(img1).decode("utf-8")), "key": '87'}], 'v1')

Answer 1

The error message itself indicates (as you point out in the question), that the key "values" is not one of the inputs specified in the model. To inspect the model's input, use saved_model_cli show --all --dir=/path/to/model. That will show you a list of the names of the inputs. You'll need to use the correct name.
That said, it appears there is another issue. It's not clear from the question what type of input your model is expecting, though it's likely one of two things:
A matrix of integers or floats
A byte string with the raw image file
contents.
The exact solution will depend on which of the above your exported model is using. saved_model_cli will help here, based on the type and shape of the input. It will either be DT_FLOAT32 (or some other int/float type) and [NONE, 299, 299, CHANNELS] or DT_STRING and [NONE], respectively.
If your model is type (1), then you will need to send a matrix of ints/floats (which does not use base64 encoding):
predict_json('image-recognition-25***08', 'm500_200_waug', [{CORRECT_INPUT_NAME: load_image(savepath).tolist(), "key": '87'}], 'v1')
Note the use of tolist to convert the numpy array to a list of lists.
In the case of type (2), you need to tell the service you have some base64 data by adding in {"b64": ...}:
predict_json('image-recognition-25***08', 'm500_200_waug', [{CORRECT_INPUT_NAME: {"b64": str(base64.b64encode(img1).decode("utf-8"))}, "key": '87'}], 'v1')
All of this, of course, depends on using the correct name for CORRECT_INPUT_NAME.
One final note, I'm assuming your model actually does have key as an additional inputs since you included it in your request; again, that can all be verified against the output of saved_model_cli show.

Answer 2

I used to get this errors too. If anyone comes across this error, and using gcloud.
Tensors are automatically called csv_rows. For example this works for me now
"instances": [{
"csv_row": "STRING,7,4.02611534,9,14,0.66700000,0.17600000,0.00000000,0.00000000,1299.76500000,57",
"key": "0"
}]