I am requesting online prediction for a trained model (model created with linear learner algorithm) and getting "error": "Prediction failed: unknown error."
This is my first ML model in Google AI platform. Model training was successful, training data, validation data and test data all look good in the output folder. But when I try to test the model by passing the input JSON I get this error. I have looked for similar other posts but couldn't find the solution to get a successful prediction.
metadata.json in the artifact folder looks like
{
"feature_columns": {
"col_0": {
"mapping": {
"0": 0,
"1": 1,
"10": 10,
"2": 2,
"3": 3,
"4": 4,
"5": 5,
"6": 6,
"7": 7,
"8": 8,
"9": 9
},
"mode": "0",
"num_category": 11,
"treatment": "identity",
"type": "categorical"
},
"col_1": {
"mapping": {
"0": 0,
"1": 1,
"10": 10,
"2": 2,
"3": 3,
"4": 4,
"5": 5,
"6": 6,
"7": 7,
"8": 8,
"9": 9
},
"mode": "4",
"num_category": 11,
"treatment": "identity",
"type": "categorical"
}
},
"target_algorithm": "TensorFlow",
"target_column": {
"type": "regression"
}
}
The input JSON that I am passing for testing prediction is
{ "instances": [5,5] }
The model is expected to sum the 2 input features and give a result of 10
Can you please advise where the mistake is?
If you are using gcloud to send a file, do:
{"col_0": "5", "col_1": "5" }
If you are sending a bunch of instances through some other client, do:
{
"instances": [
{"col_0": "5", "col_1": "5" },
{"col_0": "3", "col_1": "2" }
]
}
Lak's answer is good and does the job.
Although my input data was different I got the same error, despite successful local predictions. Here are two additional things that helped me.
Run !gcloud ai-platform predict --help to learn more about how your input should be formatted and which flag to use when making the call.
Inspect the model with !saved_model_cli show --dir ${YOUR_LOCAL_MODEL_PATH} --all to check the names of the inputs. Verify that they are in fact (in your case) inputs[col_0] and inputs[col_1].
Using the "TEST & USE" interface you mentioned above (and in this SO answer) allows quicker experimentation.
Related
I've got a bunch of JSON records that I want to add to an Amazon Redshift instance from S3, via Kinesis Firehose. It's several hundred files, give or take, that have 1,000 or so records each, and each file looks like the below sample. For my purposes, I don't care about the info entry, at least for now. I have a working Kinesis Firehose service that can update my Redshift DB with the sample stock ticker data, so that part is OK. My questions are (and hopefully this shouldn't actually be split into two different posts):
This is in large part a learning exercise, so if it's overkill for what I'm trying to do, that's OK. If there's a reason it's actually a bad idea, let me know.
If I want to just ignore the info field, do I have to use a Lambda to strip it, or is there a way to do that without one? If so, are there any tricks that wouldn't be the same as writing a script to process from a regular textfile? As I'm typing this I realize I could probably just put info in the DB and never touch it, but if there's a reason not to do that, or a cleaner way than that, I'd appreciate hearing it.
When I have individual manufacturers with a set of features, and there could be dozens of features per manufacturer, does it make sense to make a separate DB table for features, or am I coming at it from a Python dict/Perl hash perspective that doesn't make sense for a SQL DB when I need to tie them back together later?
Sample:
{
"info": {
"generated_on": "2022-08-09 19:25:34",
"version": "v1"
},
"manufacturer": [
{
"name": "Audi",
"id": 1,
"num_features": 2,
"features": [
{
"name": "seat heaters",
"standard": "N",
"cost": 100
},
{
"name": "A/C",
"standard": "Y",
"cost": 0
}
]
},
{
"name": "BMW",
"id": 2,
"num_features": 3,
"features": [
{
"name": "seat heaters",
"standard": "Y",
"cost": 0
},
{
"name": "backup camera",
"standard": "N",
"cost": 500
},
{
"name": "A/C",
"standard": "Y",
"cost": 0
}
]
}
]
}
I have a Django app that requests data from an external API and my goal is to convert that data which is returned as list/dictionary format into a new REST API with a Geojson format.
I came across django-rest-framework-gis but I don't know if I could use it without having a Model. But if so, how?
I think the best way is to use the python library geojson
pip install geojson
If you do not have a Model like in geodjango you have to explicitly describe the geometry from the data you have.
from geojson import Point, Feature, FeatureCollection
data = [
{
"id": 1,
"address": "742 Evergreen Terrace",
"city": "Springfield",
"lon": -123.02,
"lat": 44.04
},
{
"id": 2,
"address": "111 Spring Terrace",
"city": "New Mexico",
"lon": -124.02,
"lat": 45.04
}
]
def to_geojson(entries):
features = []
for entry in entries:
point = Point((entry["lon"], entry["lat"]))
del entry["lon"]
del entry["lat"]
feature = Feature(geometry=point, properties=entry)
features.append(feature)
return FeatureCollection(features)
if __name__ == '__main__':
my_geojson = to_geojson(data)
print(my_geojson)
Create the point geometry from lon, lat (Could also be another geometry type)
Create a feature with the created geometry and add the dictionary as properties. Note that I deleted lon, lat entries from the dictionary to not show up as properties.
Create A feature collection from multiple features
Result:
{"features": [{"geometry": {"coordinates": [-123.02, 44.04], "type":
"Point"}, "properties": {"address": "742 Evergreen Terrace", "city":
"Springfield", "id": 1}, "type": "Feature"}, {"geometry":
{"coordinates": [-124.02, 45.04], "type": "Point"}, "properties":
{"address": "111 Spring Terrace", "city": "New Mexico", "id": 2},
"type": "Feature"}], "type": "FeatureCollection"}
More Info here: Documentation Geojson Library
Hi I am trying to receive one last message per user. I have filtered out messages but I am getting many messages for one user. How can I get last message of user.
queryset=MessageModel.objects.filter(
Q(sUser=self.request.user) | Q(rUser=self.request.user)
).order_by('-id')
return queryset
I am getting this list from filtering.
[
{
"message": "Hi b do you want to buy? 2",
"sUser": 1,
"rUser": 15,
"sent": true,
"pk": 8,
"user": {
"pk": 15
}
},
{
"message": "a to c",
"sUser": 1,
"rUser": 16,
"sent": true,
"pk": 7,
"user": {
"pk": 16
}
},
{
"message": "No thanks not buying this time",
"sUser": 15,
"rUser": 1,
"sent": false,
"pk": 6,
"user": {
"pk": 15
}
},
{
"message": "c to a",
"sUser": 16,
"rUser": 1,
"sent": false,
"pk": 4,
"user": {
"pk": 16
}
},
{
"message": "Hi b do you want to buy?",
"sUser": 1,
"rUser": 15,
"sent": true,
"pk": 1,
"user": {
"pk": 15
}
}
]
I want to get message with pk 8 and 7 only because it is the last message to user 15 and 16 but I am getting all messages for 15 and 16.
Ok I guess you are going for type like facebook messenger's dashboard. First get the related message of the user
relatedMessage=MessageModel.objects.filter(
Q(sUser=self.request.user) | Q(rUser=self.request.user)
).order_by('-id')
Add the first message to a list
convoList.append(relatedMessage.latest('id')) #since first message is unique
Then find out the friend from each one so that you can distinct who exactly the friend is (either sender or receiver which may be you or friend, since we are filtering message sender or receiver being you or your friend).
friend = message.sUser
if message.sUser==self.request.user:
friend=message.rUser
Great! You found which one your friend is.
Now for each message in relatedMessage check if message of you friend exist in your convoList or not. If it does not exist then add it to the convolist. Finally create serializer and return its data or queryset whichever you prefer.
This is preferred because it just makes one query to the database hence creating calculating part in the server. If you had first got related user from database then gone for again filtering message, it would be much costly due to huge query twice in database.
you have to
queryset=MessageModel.objects.filter(
Q(sUser=self.request.user) | Q(rUser=self.request.user)
).order_by('-id')
if queryset.exists():
return queryset[0]
else:
return None
Hope this will work for you
I'm not sure the Q expressions in your original query match what you asked for, but let's assume you want the most recent received message for every user and that you can depend on the ordering of id such that the higher the id the more recent.
If you are using a database backend that supports DISTINCT ON you can do the following:
most_recent_messages = MessageModel.objects.all().order_by('rUser', '-id').distinct('rUser')
You can add a filter instead of all if you want to filter the queryset further to particular users for example.
In cloudsearch (using IMDB data) I added two dynamic fields to the existing indexing options
I then got the entire JSON IMDB data from AWS and added the fields location_t and month_i.
[{
"fields": {
"rating": 7.4,
"genres": ["Comedy", "Drama"],
"plot": "A New Jersey guy dedicated to his family, friends, and church, develops unrealistic expectations from watching porn and works to find happiness and intimacy with his potential true love.",
"release_date": "2013-01-18T00:00:00Z",
"title": "Don Jon",
"rank": 1,
"running_time_secs": 5400,
"directors": ["Joseph Gordon-Levitt"],
"image_url": "http://ia.media-imdb.com/images/M/MV5BMTQxNTc3NDM2MF5BMl5BanBnXkFtZTcwNzQ5NTQ3OQ##._V1_SX400_.jpg",
"year": 2013,
"actors": ["Joseph Gordon-Levitt", "Scarlett Johansson", "Julianne Moore"]
},
"type": "add",
"id": "tt2229499",
"location_t": "Berlin",
"month_i": 8},...
When I uploaded the JSON, it ignored the dynamic fields?
I did it anyway and I tried searching but to no avail. Anyone no what I am doing wrong. I followed the instructions to the book in dynamic fields
You need to have your dynamic fields as a part of your fields array, see below:
[{
"fields": {
"rating": 7.4,
"genres": ["Comedy", "Drama"],
"plot": "A New Jersey guy dedicated to his family, friends, and church, develops unrealistic expectations from watching porn and works to find happiness and intimacy with his potential true love.",
"release_date": "2013-01-18T00:00:00Z",
"title": "Don Jon",
"rank": 1,
"running_time_secs": 5400,
"directors": ["Joseph Gordon-Levitt"],
"image_url": "http://ia.media-imdb.com/images/M/MV5BMTQxNTc3NDM2MF5BMl5BanBnXkFtZTcwNzQ5NTQ3OQ##._V1_SX400_.jpg",
"year": 2013,
"actors": ["Joseph Gordon-Levitt", "Scarlett Johansson", "Julianne Moore"],
"location_t": "Berlin",
"month_i": 8
},
"type": "add",
"id": "tt2229499",},...
I have a fixture with multiple models that I'm using for testing. It
works for the basic models, but fails to create the entities for the
models with relationships. Is this a known limitation of app-engine-patch or am I missing
something? I'm using JSON for the fixture file.
I'm creating the fixture file with 'manage.py dumpdata --format=json >> file.json'
Here are the models involved:
class BibleBook(db.Model):
name = db.StringProperty(required=True)
description = db.TextProperty(required=True)
class Task(db.Model):
name = db.StringProperty(required=True)
description = db.TextProperty(required=True)
energy = db.IntegerProperty(default=1)
focus = db.IntegerProperty(default=0)
empathy = db.IntegerProperty(default=0)
denarii = db.IntegerProperty(default=0)
talents = db.IntegerProperty(default=0)
experience = db.IntegerProperty(default=1)
percent_per_task = db.IntegerProperty(default=5)
bibleBook = db.ReferenceProperty(BibleBook)
level = db.StringProperty(required=True, choices=set(["Catachumen", "Laymen", "Elder"]))
drop_percentage = db.IntegerProperty(default=10)
The json in the fixture file looks like this:
[
{"pk": "ag5sYXctYW5kLWdvc3BlbHIcCxIWbGF3YW5kZ29zcGVsX2JpYmxlYm9vaxgDDA",
"model": "lawandgospel.biblebook",
"fields": {"name": "Luke", "description": "Description"}},
{"pk": "ag5sYXctYW5kLWdvc3BlbHIXCxIRbGF3YW5kZ29zcGVsX3Rhc2sYBQw",
"model": "lawandgospel.task",
"fields": {"empathy": 0, "name": "Study Luke", "level": "Catachumen", "energy": 1,
"focus": 0, "experience": 1, "drop_percentage": 10, "talents": 0,
"bibleBook": "ag5sYXctYW5kLWdvc3BlbHIcCxIWbGF3YW5kZ29zcGVsX2JpYmxlYm9vaxgDDA",
"percent_per_task": 5, "denarii": 0, "description": "The Book of Luke"}}
]
The BibleBook model loads properly, but the Task one does not.
I'm checking this by doing:
books = BibleBook.gql('')
self.assertEquals(books.count(), 1)
tasks = Task.gql('')
self.assertEquals(tasks.count(), 1)
The first test passes, but the second does not.
Thanks,
Brian Yamabe
Thanks, celopes, for asking for the additional code. I decided to play with the json file and fixed the problem by using simple numbers for the pk's. Here's the JSON that fixes the problem for the models and tests I posted:
[
{"pk": "1",
"model": "lawandgospel.biblebook",
"fields": {"name": "Luke", "description": "The Gospel According to St. Luke."}},
{"pk": "2",
"model": "lawandgospel.task",
"fields": {"empathy": 0, "name": "Study the Gospel of Luke", "level": "Catachumen",
"energy": 1, "focus": 0, "experience": 1, "drop_percentage": 10, "talents": 0,
"bibleBook": "1", "percent_per_task": 5, "denarii": 0,
"description": "The Book of Luke"}}
]