I want to do linear regression with aws sagemaker. Where i have trained my model with some values and it's predicting values as per inputs. but sometimes it predicts value out of range as in i am predicting percentage which can't go less than 0 and more than 100. how can i restrict it here:
sess = sagemaker.Session()
linear =
sagemaker.estimator.Estimator(containers[boto3.Session().region_name],
role,
train_instance_count=1,
train_instance_type='ml.c4.xlarge',
output_path='s3://{}/{}/output'.format(bucket, prefix),
sagemaker_session=sess)
linear.set_hyperparameters(feature_dim=5,
mini_batch_size=100,
predictor_type='regressor',
epochs=10,
num_models=32,
loss='absolute_loss')
linear.fit({'train': s3_train_data, 'validation': s3_validation_data})
how can i make my model not to predict values out of range : [0,100].
Yes you can. You can implement the output_fn to "brick wall" your output. SageMaker would call the output_fn after the model returns the value to do any post-processing of the result.
This can be done by creating a separate python file, specify the output_fn method there.
Provide this python file when instantiating your Estimator.
something like
sess = sagemaker.Session()
linear =
sagemaker.estimator.Estimator(containers[boto3.Session().region_name],
role,
train_instance_count=1,
train_instance_type='ml.c4.xlarge',
output_path='s3://{}/{}/output'.format(bucket, prefix),
sagemaker_session=sess)
linear.set_hyperparameters(feature_dim=5,
mini_batch_size=100,
predictor_type='regressor',
epochs=10,
num_models=32,
loss='absolute_loss',
entry_point = 'entry.py'
)
linear.fit({'train': s3_train_data, 'validation': s3_validation_data})
Your entry.py could look something like
def output_fn(data, accepts):
"""
Args:
data: A result from TensorFlow Serving
accepts: The Amazon SageMaker InvokeEndpoint Accept value. The content type the response object should be
serialized to.
Returns:
object: The serialized object that will be send to back to the client.
"""
Implement the logic to "brick wall" here.
return data.outputs['outputs'].string_val
Related
I have a requirement where I want to perform processing on sentences encoded by sentence_transformers model and then add some logic to get the final results and I want to do it on serverless inference endpoint.
The code should be something like this:
df = pd.read_csv(filename)
model = SentenceTransformer('sentence-transformers/all-MiniLM-L6-v2')
output = model.encode(df['query'])
vec_result.append(output)
cos_scores = util.pytorch_cos_sim(vec_result, vec_result)
final_result = postporcess(df, cos_scores)
return final_result
Is it doable using inference endpoint?
Let me explain my problem:
I have to update the code of a notebook that used version 1.x of sagemaker to make a batch prediction from an xgboost endpoint that has been generated in aws SageMaker.
After defining a dataframe called ordered_data, when trying to run this:
def batch_predict(data, xgb_predictor, rows=500):
split_array = np.array_split(data, int(data.shape[0] / float(rows) + 1))
predicates = ''
for array in split_array:
new_predictions = xgb_predictor.predictor.predict(array).decode('utf-8').
predictions = predictions + '\n' + predictions_new
predictions = predictions.replace('\n', ',')
predictions = predictions.replace(',,', ',')
return np.fromstring(predictions[1:], sep=',')
def get_predictions(sorted_data, xgb_predictor):
xgb_predictor.content_type = 'text/csv'.
xgb_predictor.serializer = csv_serializer
xgb_predictor.deserializer = None
#predictions = batch_predict(ordered_data.as_matrix(), xgb_predictor) # get the scores for each piece of data
predictions = batch_predict(ordered_data.values, xgb_predictor)
predictions = pd.DataFrame(predictions, columns=['score'])
return predictions
xgb_predictor = sagemaker.predictor.RealTimePredictor(endpoint_name='sagemaker-xgboost-2023-01-18')
predictions = get_predictions(sorted_data, xgb_predictor)
predictions2 = pd.concat([predictions, raw_data[[['order_id']]]], axis=1).
I've checked the documentation of sagemaker v2, and tried to update many things, and also I've run the code !sagemaker-upgrade-v2 --in-file file.ipynb --out-file file2.ipynb
but nothing works.
I get several errors like:
'content_type' property of object 'deprecated_class..DeprecatedClass' has no setter.
If I delete the line where I define content_type, I get: AttributeError: 'NoneType' object has no attribute 'ACCEPT'.
and so on.
I need to update all this code but I don't know how.
SageMaker RealTimePredictor class has serializer and deserializer parameters in Python SDK V2, behavior for serialization of input data and deserialization of result data can be configured through initializer arguments.
Note: The csv_serializer, json_serializer, npy_serializer, csv_deserializer, json_deserializer, and numpy_deserializer objects have been deprecated in v2
serializer=CSVSerializer(),
deserializer=JSONDeserializer()
I have deployed a the universal_sentence_encoder_large_3 to an aws sagemaker. When I am attempting to predict with the deployed model I get Failed precondition: Table not initialized. as an error. I have included the part where I save my model below:
import tensorflow as tf
import tensorflow_hub as hub
import numpy as np
def tfhub_to_savedmodel(model_name, export_path):
model_path = '{}/{}/00000001'.format(export_path, model_name)
tfhub_uri = 'http://tfhub.dev/google/universal-sentence-encoder-large/3'
with tf.Session() as sess:
module = hub.Module(tfhub_uri)
sess.run([tf.global_variables_initializer(), tf.tables_initializer()])
input_params = module.get_input_info_dict()
dtype = input_params['text'].dtype
shape = input_params['text'].get_shape()
# define the model inputs
inputs = {'text': tf.placeholder(dtype, shape, 'text')}
output = module(inputs['text'])
outputs = {
'vector': output,
}
# export the model
tf.saved_model.simple_save(
sess,
model_path,
inputs=inputs,
outputs=outputs)
return model_path
I have seen other people ask this problem but no solution has been ever posted. It seems to be a common problem with tensorflow_hub sentence encoders
I was running into this exact issue earlier this week while trying to modify this example Sagemaker notebook. Particularly the part where serving the model. That is, running predictor.predict() on the Sagemaker Tensorflow Estimator.
The solution outlined in the issue worked perfectly for me- https://github.com/awslabs/amazon-sagemaker-examples/issues/773#issuecomment-509433290
I think it's just because tf.tables_initializer() only runs for training but it needs to be specified through the legacy_init_op if you want to run it during prediction.
I am implementing a tfx pipeline, similar to the chicago taxi example
The prediction of the pushed model returns {"predictions": []}.
How do I debug this issue?
I can see logs of the predictions being made. But because it returns an empty array the status code is 200 and there is no usefull information on what went wrong. I expect the prediction request data isn't passed correctly to the estimator.
The chicago example uses this as their serving receiver and that works. I assume it should also work for my example
def _example_serving_receiver_fn(transform_output, schema):
"""Build the serving in inputs.
Args:
transform_output: directory in which the tf-transform model was written
during the preprocessing step.
schema: the schema of the input data.
Returns:
Tensorflow graph which parses examples, applying tf-transform to them.
"""
raw_feature_spec = _get_raw_feature_spec(schema)
raw_feature_spec.pop(_LABEL_KEY)
raw_input_fn = tf.estimator.export.build_parsing_serving_input_receiver_fn(
raw_feature_spec, default_batch_size=None)
serving_input_receiver = raw_input_fn()
transformed_features = transform_output.transform_raw_features(
serving_input_receiver.features)
return tf.estimator.export.ServingInputReceiver(
transformed_features, serving_input_receiver.receiver_tensors)
The main difference is that I only expect 1 input: a string of programming languages separated by '|': 'java|python'.
I then split that string up in my preprocessing function and make it into a multi one hot encoded array of shape 500 (I have exactly 500 options)
It could also be the case that the prediction isn't being correctly transformed by tf transform. (tf transform is part of the tfx pipeline and runs correctly)
request: {"instances": ["javascript|python"]}
response: {"predictions": []}
expected response: {"predictions": [520]} (its a regression model)
I get the following error when trying to make an online prediction on my ML Engine model.
The key "values" is not correct. (See error on image.)
enter image description here
I already tested with RAW image data : {"image_bytes":{"b64": base64.b64encode(jpeg_data)}}
& Converted the data to a numpy array.
Currently I have the following code:
from googleapiclient import discovery
import base64
import os
from PIL import Image
import json
import numpy as np
os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = "/Users/jacob/Desktop/******"
def predict_json(project, model, instances, version=None):
"""Send json data to a deployed model for prediction.
Args:
project (str): project where the Cloud ML Engine Model is deployed.
model (str): model name.
instances ([Mapping[str: Any]]): Keys should be the names of Tensors
your deployed model expects as inputs. Values should be datatypes
convertible to Tensors, or (potentially nested) lists of datatypes
convertible to tensors.
version: str, version of the model to target.
Returns:
Mapping[str: any]: dictionary of prediction results defined by the
model.
"""
# Create the ML Engine service object.
# To authenticate set the environment variable
# GOOGLE_APPLICATION_CREDENTIALS=<path_to_service_account_file>
service = discovery.build('ml', 'v1')
name = 'projects/{}/models/{}'.format(project, model)
if version is not None:
name += '/versions/{}'.format(version)
response = service.projects().predict(
name=name,
body={'instances': instances}
).execute()
if 'error' in response:
raise RuntimeError(response['error'])
return response['predictions']
savepath = 'upload/11277229_F.jpg'
img = Image.open('test/01011000/11277229_F.jpg')
test = img.resize((299, 299))
test.save(savepath)
img1 = open(savepath, "rb").read()
def load_image(filename):
with open(filename) as f:
return np.array(f.read())
predict_json('image-recognition-25***08', 'm500_200_waug', [{"values": str(base64.b64encode(img1).decode("utf-8")), "key": '87'}], 'v1')
The error message itself indicates (as you point out in the question), that the key "values" is not one of the inputs specified in the model. To inspect the model's input, use saved_model_cli show --all --dir=/path/to/model. That will show you a list of the names of the inputs. You'll need to use the correct name.
That said, it appears there is another issue. It's not clear from the question what type of input your model is expecting, though it's likely one of two things:
A matrix of integers or floats
A byte string with the raw image file
contents.
The exact solution will depend on which of the above your exported model is using. saved_model_cli will help here, based on the type and shape of the input. It will either be DT_FLOAT32 (or some other int/float type) and [NONE, 299, 299, CHANNELS] or DT_STRING and [NONE], respectively.
If your model is type (1), then you will need to send a matrix of ints/floats (which does not use base64 encoding):
predict_json('image-recognition-25***08', 'm500_200_waug', [{CORRECT_INPUT_NAME: load_image(savepath).tolist(), "key": '87'}], 'v1')
Note the use of tolist to convert the numpy array to a list of lists.
In the case of type (2), you need to tell the service you have some base64 data by adding in {"b64": ...}:
predict_json('image-recognition-25***08', 'm500_200_waug', [{CORRECT_INPUT_NAME: {"b64": str(base64.b64encode(img1).decode("utf-8"))}, "key": '87'}], 'v1')
All of this, of course, depends on using the correct name for CORRECT_INPUT_NAME.
One final note, I'm assuming your model actually does have key as an additional inputs since you included it in your request; again, that can all be verified against the output of saved_model_cli show.
I used to get this errors too. If anyone comes across this error, and using gcloud.
Tensors are automatically called csv_rows. For example this works for me now
"instances": [{
"csv_row": "STRING,7,4.02611534,9,14,0.66700000,0.17600000,0.00000000,0.00000000,1299.76500000,57",
"key": "0"
}]