ML-Engine Local Predict - Failed to Run Model - google-cloud-ml

Question

gcloud ml-engine local predict fails
First I identified the required input.json structure with saved_model_cli show --all --dir saved_model/
Response:
MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:
signature_def['serving_default']:
The given SavedModel SignatureDef contains the following input(s):
inputs['inputs'] tensor_info:
dtype: DT_UINT8
shape: (-1, -1, -1, 3)
name: image_tensor:0
....
From this I formatted my input.json for gcloud ml-engine local predict as:
{"inputs": {"b64": "ENCODED"}}
...
Finally, I ran gcloud ml-engine local predict --model-dir saved_model/ --json-instances=PATH-TO-INPUTS.json
Response:
ERROR: (gcloud.ml-engine.local.predict) /usr/local/lib/python2.7/dist-packages/requests/__init__.py:83:
2018-10-03 09:20:06.598090: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
ERROR:root:Exception during running the graph: invalid literal for long() with base 10: '\xff\xd8\xff\xdb'
Traceback (most recent call last):
File "/usr/lib/google-cloud-sdk/lib/googlecloudsdk/command_lib/ml_engine/local_predict.py", line 172, in <module>
main()
File "/usr/lib/google-cloud-sdk/lib/googlecloudsdk/command_lib/ml_engine/local_predict.py", line 167, in main
signature_name=args.signature_name)
File "/usr/lib/google-cloud-sdk/lib/third_party/ml_sdk/cloud/ml/prediction/prediction_lib.py", line 106, in local_predict
predictions = model.predict(instances, signature_name=signature_name)
File "/usr/lib/google-cloud-sdk/lib/third_party/ml_sdk/cloud/ml/prediction/prediction_utils.py", line 233, in predict
preprocessed, stats=stats, **kwargs)
File "/usr/lib/google-cloud-sdk/lib/third_party/ml_sdk/cloud/ml/prediction/frameworks/tf_prediction_lib.py", line 350, in predict
"Exception during running the graph: " + str(e))
cloud.ml.prediction.prediction_utils.PredictionError: Failed to run the provided model: Exception during running the graph: invalid literal for long() with base 10: '\xff\xd8\xff\xdb' (Error code: 2)
Any help overcoming this obstacle would be great. I have not been able to identify a solution from research online thus far. Thank you!

Answer 1

The data in input.json does not match the shape of inputs['inputs']. You have not provided enough information about what the various dimensions represent, but I suspect this is NHWC (batch size x height x width x channels) encoding of an image.
I also suspect these are supposed to be raw pixel values. In which case, you should not be base64 encoding the values, i.e., you should send data like this:
{"inputs": [[[0, 0, 0], ... [0, 0, 0]]]}
That being said, you should consider sending the image as a byte string and decoding the image in the graph. More information about various approaches can be found here:
https://stackoverflow.com/a/46222990/1399222
It seems you are using "Raw Tensor Encoded as JSON" and I recommend "Compressed Imaged Data", or the slightly simpler "Tensors Packed as Byte Strings"