Google AI platform custom prediction routines with multiple inputs, how to read json inputs - google-cloud-platform

Question

For creating a custom prediction routine with a Keras (Tensorflow 2.1) model, I am having trouble figuring out what form the json inputs are coming in as, and how to read them in the predictor class for multiple inputs. All of the custom prediction routine examples in the documentation use simple flat single-input lists. If for example we send in our inputs as:
{"instances": [
{
"event_type_input": [1, 2, 20],
"event_dwelltime_input": [1.368, 0.017, 0.0],
"rf_input": [1.2, -2.8]},
{
"event_type_input": [14, 40, 20],
"event_dwelltime_input": [1.758, 13.392, 0.0],
"rf_input": [1.29, -2.87]}
]}
How should we ingest the incoming json in our predictor class?
class MyPredictor(object):
def __init__(self, model):
self.model = model
def predict(self, instances, **kwargs):
inputs = np.array(instances)
# The above example from the docs is wrong for multiple inputs
# What should our inputs be to get the inputs in the right shape
# for our keras model?
outputs = self.model.predict(inputs)
return outputs.tolist()
Our json inputs to google ai platform are a list of dictionaries. However, for a keras model, our inputs need to be in different shape, like the following:
inputs = {
"event_type_input": np.array([[1, 2, 20], [14, 40, 20]]),
"event_dwelltime_input": np.array([[1.368, 0.017, 0.0], [1.758, 13.392, 0.0]])
"rf_input": np.array([[1.2, -2.8], [1.29, -2.87]]}
model.predict(inputs)
Am I right that the thing to do then is just reshape the instances? The only confusion is that if using the tensorflow framework (instead of a custom prediction routine), it handles predicting on the json input fine, and I thought that all the tensorflow framework is doing is calling the .predict method on the instances (unless indeed there is some under-the-hood reshaping of the data. I couldn't find a source to find out what is exactly happening)
Main question: How should we write our predictor class to take in the instances such that we can run the model.predict method on it?

Answer 1

I would suggest creating a new Keras Model and exporting it.
Create a separate Input layer for each of the three inputs to the new Model (with the name of the Input being the name in your JSON struct). Then, in this Model, reshape the inputs, and borrow the weights/structure from your trained model, and export the new model. Something like this:
trained_model = keras.models.load_model(...) # trained model
input1 = keras.Input(..., name='event_type_input')
input2 = keras.Input(..., name='event_dwelltime_input')
input3 = keras.Input(..., name='rf_input')
export_inputs = keras.concatenate([input1, input2, input3])
reshaped_inputs = keras.layers.Lambda(...) # reshape to what the first hidden layer of your trained model expects
layer1 = trained_model.get_layer(index=1)(reshaped_inputs)
layer2 = trained_model.get_layer(index=2)(layer1) # etc. ...
...
exportModel = keras.Model(export_inputs, export_output)
exportModel.save(...)