I am using a Sqlite database, and flask, marshmallow, sqlalchemy to serve as a web api for a front end project.
I'm using a UUID stored as a blob in the database, and trying to stylise the data when its returned to the calling code.
I can convert the Id when its returned in the model using marshmallow's Function field, but using the HyperLinks, the Id is still being output as a byte array string:
class Item(Base):
__tablename__ = 'Items'
Id = Column(LargeBinary, primary_key=True)
def __repr__(self):
return '<Item {}>'.format(self.Name)
class ItemSchema(ma.Schema):
Id = fields.Function(lambda obj: str(UUID(bytes=obj.Id)))
_links = ma.Hyperlinks(
{"url": ma.URLFor("item_detail", id="<Id>"), "collection": ma.URLFor("items")}
)
class Meta:
fields = ("_links", "Id")
Is there a way to format the < Id > that is output in the links?
i.e
{"url": ma.URLFor("item_detail", id="<str(UUID(bytes=Id))>"), "collection": ma.URLFor("items")}
Here's how it is currently output:
{"_links": {"url": "/api/items/b%27%5Cx86%5Cxacx__%5Cxf9%5Cxc2J%5Cx80a6%5Cxa7%5Cx95%5Cx10%5Cx91%5Cxff%27", "collection": "/api/items"}, "Id": "86ac785f-5ff9-c24a-8061-36a7951091ff"}
I want it to look like:
{"_links": {"url": "/api/items/86ac785f-5ff9-c24a-8061-36a7951091ff", "collection": "/api/items"}, "Id": "86ac785f-5ff9-c24a-8061-36a7951091ff"}
I want the link to use the UUID format, not the byte array.
I found out the way around this. It was with the model set up and not marshmallow.
As the database was/is storing the pk as a blob, I was able to use SQLAlchemy_utils UUID column, setting the binary to true, and it all worked:
from sqlalchemy_utils import UUIDType
from uuid import UUID
....
Id = Column(UUIDType(binary=True), primary_key=True)
Related
I have two Models Site and Cell, every site has multiple Cells.
from django.db import models
from django.contrib.gis.db.models import PointField
class SiteManager(models.Model):
def get_by_natural_key(self, name, state_code, location):
return self.get(name=name, state_code=state_code, location=location)
class Site(models.Model):
name = models.CharField(max_length=10)
state_code = models.PositiveSmallIntegerField()
location = PointField()
objects = SiteManager()
class Meta:
unique_together = [['name', 'state_code', 'location']]
def natural_key(self):
return (self.name, self.state_code, self.location)
class Cell(models.Model):
tech = models.CharField(max_length=5)
azimuth = models.IntegerField()
sector_id = models.CharField(max_length=10)
frequency_band = models.CharField(max_length=15)
power = models.DecimalField(decimal_places=2, max_digits=4)
site = models.ForeignKey(Site, on_delete=models.CASCADE)
def natural_key(self):
return (self.tech, self.azimuth,) + self.site.natural_key()
natural_key.dependencies = ['astmaps.site']
I want to retrieve the complete Cell attributes with the related attributes in the Site model, for me to Serialize the resultant Cell's, into GeoJson data, I can easily Serialize the Site model like:
from django.core.serializers import serialize # GeoJSON Serializer
sites = Site.objects.all()
sitesData = serialize('geojson', sites, geometry_field='location',
fields=('name', 'state_code'))
which gives me a GeoJson featureCollection object like:
{
"type":"FeatureCollection",
"crs":{
"type":"name",
"properties":{
"name":"EPSG:4326"
}
},
"features":[
{
"type":"Feature",
"properties": {
"name":"02101",
"state_code":2
},
"geometry":{
"type":"Point",
"coordinates":[
1.34944,
36.1586
]
}
}
]
}
But when It comes to the Cell model, I can't successfully get the geometry field from the related model always null.
Since the Cell model has the Site model as a related model, I've used the function select_related() to get the related attributes:
cells = Cell.objects.select_related('site').all()
cellsData = serialize('geojson', cells, geometry_field='site_location',
fields=('azimuth', ...))
But the GeoJson Serialize function could not identify the Site model attributes from the cells QuerySet:
{
"type":"FeatureCollection",
"crs":{
"type":"name",
"properties":{
"name":"EPSG:4326"
}
},
"features":[
{
"type":"Feature",
"properties":{
"azimuth":340
},
"geometry":null
},
{
"type":"Feature",
"properties":{
"azimuth":340
},
"geometry":null
},
{
"type":"Feature",
"properties":{
"azimuth":240
},
"geometry":null
}
]
}
I've tested the query returned by Django ORM equivalent directly on the database:
cells = Cell.objects.select_related('site').all()
>>> print(cells.query)
SELECT "app_cell"."id", "app_cell"."tech", "app_cell"."azimuth", "app_cell"."sector_id", "app_cell"."frequency_band", "app_cell"."power", "app_cell"."site_id", "app_cell"."id", "app_site"."name", "app_site"."state_code", "app_site"."location"::bytea FROM "app_cell" INNER JOIN "app_site" ON ("app_cell"."site_id" = "app_site"."id")
Which gives me a correct results (all the attributes or columns of the two models):
I've also used Natural Keys, which is the serialization strategy for foreign keys and other relations (as I've read in the documentation and changed the models accordingly):
cellsData = serialize('geojson', cells, geometry_field='site_location',
fields=('azimuth', ...), use_natural_foreign_keys=True)
But the same result, the Serialize method couldn't identify the Site model attributes.
How can I get all the attributes of multiple related models to get serialized using the GeoJSON Serializer?
I managed to get what I want by using raw sql query plus the json_build_object and ST_AsGeoJSON of PostGIS extention:
from django.db import connection
def sites_list(request):
cursor = connection.cursor()
cursor.execute("""select json_build_object(
'type', 'FeatureCollection',
'features', json_agg(ST_AsGeoJSON(t.*)::json)
)
from ( SELECT "app_cell"."id", "app_cell"."tech", "app_cell"."azimuth", "app_cell"."sector_id",
"app_cell"."frequency_band", "app_cell"."power", "app_cell"."site_id", "app_site"."name"
AS "site_name", "app_site"."state_code", "app_cell"."location"::bytea::geometry AS "site_location"
FROM "app_cell" INNER JOIN "app_site" ON ("app_cell"."site_id" = "app_site"."id")
) as t(id, tech, azimuth, sector_id, frequency_band, power, site_id, site_name, state_code, geom);""")
geojson = cursor.fetchone()
return JsonResponse(geojson[0], safe=False)
I had some problems in the JavaScript side, when using:
cursor.fetchall()
return JsonResponse(geojson, safe=False)
the returned GeoJSON object was surrounded by [[]] double brackets, because cursor.fetchall() return a tuple, So, I've used the cursor.fetchone() to get the geoJSON object surrunded only by single brackets [], and used the the index 0 to of the resulted tuble to get the only tuple content as a string, and finally the JsonResponse function will return that result as a JSON object.
I need to create json to insert data using inside both models, for tags I have created the json but I don't get how to create json for Question model to insert data directly from it.
model.py
class Tag(models.Model):
id = models.AutoField(primary_key=True)
name = models.CharField(max_length=255)
class Question(models.Model):
name = models.CharField(max_length=255)
Tag_name = models.ManyToManyField(Tag)
Tag json look like:
[
{ "name": "a" },
{ "name": "b" }
]
Since Tag_name is many to many field it create 2 tables in sqlite but I want to add data using one json only.How to make a json so that data in both table get inserted ?
Let suppose you have read json file and now you want to store that data to db
json_data = get_json_data() # write code to read json here
tag_list = []
for tag in json_data:
tag_list.append(Tag(**tag))
added_tags = Tag.objects.bulk_create(tag_list)
added_tags = [t.id for t in added_tags]
question_object = # write code to get specific question object in which you want to add this
question_object.Tag_name.add(*added_tags)
You could opt for the builtin management command, ./manage.py loaddata. I think it's easier.
You only need a json fixture for the tag model... the fixture, tags.json, could look like:
[
{
"model":"yourapp.tag",
"pk":1,
"fields":{"name": "a" }
},
{
"model":"yourapp.tag",
"pk":2,
"fields":{"name": "b" }
}
]
Store your tags.json inside a folder called fixtures inside your app. Finally, run ./manage.py loaddata tags.json. It will prepopulate the Tag model. Hope it helps...
I have a very simple model:
class challenge(models.Model):
challenge = models.CharField(max_length=255)
timestamp = models.DateTimeField('date published',null=True,blank=True)
code = models.CharField(max_length=255)
And that model is linked via Tastypie.
By default the timestamp field is empty when you add an entry.
I would like to update the timestamp field with the time that the user first accessed the resource.
For example, if admin added data is:
Challenge: Do something
Timestamp: Null
Code: 123
Then after accessing /api/challenge/1/?format=json to have the output be:
{"challenge": "Do something", "code": "sasdasd", "id": 2, "resource_uri": "/api/challenge/2/", "timestamp": "2015-05-11T12:18:54"}
Is this possible using Tastypie and how?
I am using Django Rest Framework to support the use of Annotator.JS(http://annotatorjs.org/) in the frontend of my web application. The problem is the model I use to store any annotations that a user makes is different to the JSON that Annotator.JS sends from the front end in an AJAX request.
The structure of Annotator.JS' JSON is:
{
"id": "39fc339cf058bd22176771b3e3187329", # unique id (added by backend)
"annotator_schema_version": "v1.0", # schema version: default v1.0
"created": "2011-05-24T18:52:08.036814", # created datetime in iso8601 format (added by backend)
"updated": "2011-05-26T12:17:05.012544", # updated datetime in iso8601 format (added by backend)
"text": "A note I wrote", # content of annotation
"quote": "the text that was annotated", # the annotated text (added by frontend)
"uri": "http://example.com", # URI of annotated document (added by frontend)
"ranges": [ # list of ranges covered by annotation (usually only one entry)
{
"start": "/p[69]/span/span", # (relative) XPath to start element
"end": "/p[70]/span/span", # (relative) XPath to end element
"startOffset": 0, # character offset within start element
"endOffset": 120 # character offset within end element
}
],
"user": "alice", # user id of annotation owner (can also be an object with an 'id' property)
"consumer": "annotateit", # consumer key of backend
"tags": [ "review", "error" ], # list of tags (from Tags plugin)
}
The structure of my Annotation model is:
class Annotation(models.Model):
datapoint = models.ForeignKey('datapoint.Datapoint', related_name='%(class)s_parent_datapoint_relation')
owner = models.ForeignKey('users.User', related_name='%(class)s_creator_relation')
# Key fields from the Annotator JSON Format: http://docs.annotatorjs.org/en/latest/annotation-format.html
annotator_schema_version = models.CharField(max_length=8, blank=True)
text = models.TextField(blank=True)
quote = models.TextField()
uri = models.URLField(blank=True)
range_start = models.CharField(max_length=50, blank=True)
range_end = models.CharField(max_length=50, blank=True)
range_startOffset = models.BigIntegerField()
range_endOffset = models.BigIntegerField()
tags = TaggableManager(blank=True)
How can I create a serializer that can translate from the model structure to the JSON?
P.S. Annotator.JS allows the user to send extra information with the JSON structure noted above so the fact that Datapoint isn't included in the JSON structure isn't an issue. This can be passed along without any problem. Owner would hopefully equal User in the JSON.
Thanks for any help, it is greatly appreciated.
Just using a default ModelSerializer should get you serialization for all the simple fields for free (version, text, quote, uri), just by specifying which fields you want serialized. The other fields look straightforward as well:
To compose the ranges object, you can use a SerializerMethodField which lets you define a custom serializer method (return an array of dictionaries containing values from the range_* attributes on your model). Note that if you also need to be able to deserialize JSON into your model you'll need to define custom fields.
To go from owner -> user_id SlugRelatedField (assuming you can access the user id you want from the User object)
You can also use SlugRelatedField to go from TaggableManager -> ["tag", "values"] (again, assuming you can access the value you want through the model object managed by TaggableManager.
In general, everything you want is described in pretty good detail in the documentation.
I am trying to build a small api with django rest framework but I don't want to map directly the tables with calls (as in the examples).
I have the following database schema:
In models.py:
class ProductType(models.Model):
name = models.CharField(max_length=255, blank=False, null=False, unique=True)
class Product(models.Model):
#staticmethod
def get_accepted_fields(self):
return {'color': 'pink', 'size': 34, 'speed': 0, 'another_prop': ''}
name = models.CharField(max_length=255, blank=False, null=False, unique=True)
class ProductConfig(models.Model):
product_type = models.ForeignKey(ProductType)
product = models.ForeignKey(Product)
# a json field with all kind of fields: eg: {"price": 123, "color": "red"}
value = models.TextField(blank=True)
As you can see, every product can have multiple configurations and the value field is a json with different parameters. The json will be one level only. The configuration will have a flag if is active or not (so, the 1 product will have only 1 active configuration)
So, the data will look for example like this:
store_producttype
=================
1 type1
2 type2
store_product
=============
id name
1 car
store_productconfig
===================
id product_type_id product_id value active
1 2 1 { "color": "red", "size": 34, "speed": 342} 0
2 1 1 { "color": "blue", "size": 36, "speed": 123, "another_prop": "xxx"} 1
What I want to know is how can I get /product/1/ like this:
{
"id": 1,
"name": "car",
"type": "type1",
"color": "blue",
"size": 36,
"speed": 123,
"another_prop": "xxx",
}
and to create a new product posting a json similar with the one above.
The json fields are defined but some of them can miss (eg: "another_prop" in the productconfig.id=1
On update, anyway, it will create a new row in productconfig and it will put inactive=0 on the previous one.
So, every product can have different configuration and I want to go back to a specific configuration back in time in some specific cases). I am not really bound to this data model, so if you have suggentions for improvement I am open to them, but I don't want to have that properties as columns in the table.
The question is, what will be the best way to write the serializers for this model? There is any good example somewhere for a such use case?
Thank you.
Let's take this step by step:
In order to get a JSON like the one you posted, you must first transform your string (productConfig value field) to a dictionary. This can be done by using ast.literal_eval ( see more here).
Then, in your product serializer, you must specify the source for each field, like this:
class ProductSerializer(serializers.ModelSerializer):
color = serializer.Field(source='value_dict.color')
size = serializer.Field(source='value_dict.size')
type = serializer.Field(source='type.name')
class Meta:
model = Product
fields = (
'id',
'color',
'size',
'type',
)
This should work just fine for creating the representation that you want. However, this will not create automatically the product config, because DRF doesn't yet allow nested object creation.
This leads us to the next step:
For creating a product with a configuration from JSON, you must override the post method in your view, and create it yourself. This part shouldn't be so hard, but if you need an example, just ask.
This is more of a suggestion: if the json fields are already defined, wouldn't it be easier to define them as separate fields in your productConfig model?