I am using Django Rest Framework to support the use of Annotator.JS(http://annotatorjs.org/) in the frontend of my web application. The problem is the model I use to store any annotations that a user makes is different to the JSON that Annotator.JS sends from the front end in an AJAX request.
The structure of Annotator.JS' JSON is:
{
"id": "39fc339cf058bd22176771b3e3187329", # unique id (added by backend)
"annotator_schema_version": "v1.0", # schema version: default v1.0
"created": "2011-05-24T18:52:08.036814", # created datetime in iso8601 format (added by backend)
"updated": "2011-05-26T12:17:05.012544", # updated datetime in iso8601 format (added by backend)
"text": "A note I wrote", # content of annotation
"quote": "the text that was annotated", # the annotated text (added by frontend)
"uri": "http://example.com", # URI of annotated document (added by frontend)
"ranges": [ # list of ranges covered by annotation (usually only one entry)
{
"start": "/p[69]/span/span", # (relative) XPath to start element
"end": "/p[70]/span/span", # (relative) XPath to end element
"startOffset": 0, # character offset within start element
"endOffset": 120 # character offset within end element
}
],
"user": "alice", # user id of annotation owner (can also be an object with an 'id' property)
"consumer": "annotateit", # consumer key of backend
"tags": [ "review", "error" ], # list of tags (from Tags plugin)
}
The structure of my Annotation model is:
class Annotation(models.Model):
datapoint = models.ForeignKey('datapoint.Datapoint', related_name='%(class)s_parent_datapoint_relation')
owner = models.ForeignKey('users.User', related_name='%(class)s_creator_relation')
# Key fields from the Annotator JSON Format: http://docs.annotatorjs.org/en/latest/annotation-format.html
annotator_schema_version = models.CharField(max_length=8, blank=True)
text = models.TextField(blank=True)
quote = models.TextField()
uri = models.URLField(blank=True)
range_start = models.CharField(max_length=50, blank=True)
range_end = models.CharField(max_length=50, blank=True)
range_startOffset = models.BigIntegerField()
range_endOffset = models.BigIntegerField()
tags = TaggableManager(blank=True)
How can I create a serializer that can translate from the model structure to the JSON?
P.S. Annotator.JS allows the user to send extra information with the JSON structure noted above so the fact that Datapoint isn't included in the JSON structure isn't an issue. This can be passed along without any problem. Owner would hopefully equal User in the JSON.
Thanks for any help, it is greatly appreciated.
Just using a default ModelSerializer should get you serialization for all the simple fields for free (version, text, quote, uri), just by specifying which fields you want serialized. The other fields look straightforward as well:
To compose the ranges object, you can use a SerializerMethodField which lets you define a custom serializer method (return an array of dictionaries containing values from the range_* attributes on your model). Note that if you also need to be able to deserialize JSON into your model you'll need to define custom fields.
To go from owner -> user_id SlugRelatedField (assuming you can access the user id you want from the User object)
You can also use SlugRelatedField to go from TaggableManager -> ["tag", "values"] (again, assuming you can access the value you want through the model object managed by TaggableManager.
In general, everything you want is described in pretty good detail in the documentation.
Related
I have a model that uses PostgreSQL and has field like this:
class MyModel(models.Model):
json_field = models.JSONField(default=list)
This field contains data like this:
[
{"name": "AAAAA", "product": "11111"},
{"name": "BBBBB", "product": "22222"},
]
Now I want to index by json_field -> product field, because it is being used as identification. Then i want to create GinIndex like this:
class Meta:
indexes = [
GinIndex(name='product_json_idx', fields=['json_field->product'], opclasses=['jsonb_path_ops'])
]
When I try to create migration, I get error like this:
'indexes' refers to the nonexistent field 'json_field->product'.
How to create GinIndex that will be used for child attribute in Json Array?
Please don't use a JSONField [Django-doc] for well-structured data: if the structure is clear, like here where we have a list of objects where each object has a name and a product, it makes more sense to work with extra models, like:
class MyModel(models.Model):
# …
pass
class Product(models.Model):
# …
pass
class Entry(models.Model):
my_model = models.ForeignKey(MyModel, on_delete=models.CASCADE)
name = models.CharField(max_length=255)
product = models.ForeignKey(Product, on_delete=models.CASCADE)
This will automatically add indexes on the ForeignKeys, but will also make querying simpeler and usually more efficient.
While databases like PostgreSQL indeed have put effort into making JSON columns easier to query, aggregate, etc. usually it is still beter to perform database normalization [wiki], especially since it has more means for referential integrity, and a lot of aggregates are simpeler on linear data.
If for example later a product is removed, it will require a lot of work to inspect the JSON blobs to remove that product. This is however a scenario that both Django and PostgreSQL databases cover with ON DELETE triggers and which will likely be more effective and safe when using the Django toolchain for this.
Considering I have a model like:
MyStore = (
id = 1,
name = 'Foobar',
information_as_json = {
'open_at': datetime.now(),
'close_at': datetime.now() + timedelta('+1 day'),
'workers' : {
'Person1' : 'Owner',
'Person2' : 'Boss',
'Person3' : 'Boss',
}
})
Inside Django admin forms, for every field is generated an input, but for the field "information_as_json", I don't want to show it as a string or as JSON. That is because the users who are accessing this store admin page, need to read the field 'information_as_json' easier since no one can edit these values because it is generated in another part of the application.
Is it possible to convert these values to a "div" or a plain text? The contents would be:
This store opens at: {information_as_json.open_at}
This store close at: {information_as_json.close_at}
And for the workers, iterate through keys and values:
for key, value in information_as_json.workers:
Worker {key} has the role: {value}
I'm a beginner at Django, so I'm struggling a little with this part.
Every help would be appreciated :D
I would suggest approaching the model a little differently. Rather than storing the opening and closing hours as JSON they can just be fields directly on the store model. The the workers can be a JSONfield [docs] containing name/role pairs. If you're using PostgreSQL for your database you could even use HStoreField [docs], which might be more appropriate.
Here's how I would write a similar model.
class Store(models.Model):
name = models.CharField(max_length=512, unique=True)
workers = models.JSONField(blank=True, default=dict, editable=False)
closing = models.TimeField(blank=True, null=True, editable=False)
opening = models.TimeField(blank=True, null=True, editable=False)
To display the details in the Django admin we just need to define a property which returns the correct string.
#mark_safe
def details(self):
roles = [
f'{x} has the role: {y}'
for x, y in self.workers.items()
]
return '<br>'.join([
f'This store opens at: {self.opening:%-H:%M}',
f'This store closes at: {self.closing:%-H:%M}',
] + roles)
This method can then be referenced in the ModelAdmin and used like a read-only field.
#admin.register(Store)
class StoreAdmin(admin.ModelAdmin):
list_display = ['name', 'opening', 'closing']
fields = ['name', 'details']
readonly_fields = ['details']
I need to create json to insert data using inside both models, for tags I have created the json but I don't get how to create json for Question model to insert data directly from it.
model.py
class Tag(models.Model):
id = models.AutoField(primary_key=True)
name = models.CharField(max_length=255)
class Question(models.Model):
name = models.CharField(max_length=255)
Tag_name = models.ManyToManyField(Tag)
Tag json look like:
[
{ "name": "a" },
{ "name": "b" }
]
Since Tag_name is many to many field it create 2 tables in sqlite but I want to add data using one json only.How to make a json so that data in both table get inserted ?
Let suppose you have read json file and now you want to store that data to db
json_data = get_json_data() # write code to read json here
tag_list = []
for tag in json_data:
tag_list.append(Tag(**tag))
added_tags = Tag.objects.bulk_create(tag_list)
added_tags = [t.id for t in added_tags]
question_object = # write code to get specific question object in which you want to add this
question_object.Tag_name.add(*added_tags)
You could opt for the builtin management command, ./manage.py loaddata. I think it's easier.
You only need a json fixture for the tag model... the fixture, tags.json, could look like:
[
{
"model":"yourapp.tag",
"pk":1,
"fields":{"name": "a" }
},
{
"model":"yourapp.tag",
"pk":2,
"fields":{"name": "b" }
}
]
Store your tags.json inside a folder called fixtures inside your app. Finally, run ./manage.py loaddata tags.json. It will prepopulate the Tag model. Hope it helps...
I am trying to build a small api with django rest framework but I don't want to map directly the tables with calls (as in the examples).
I have the following database schema:
In models.py:
class ProductType(models.Model):
name = models.CharField(max_length=255, blank=False, null=False, unique=True)
class Product(models.Model):
#staticmethod
def get_accepted_fields(self):
return {'color': 'pink', 'size': 34, 'speed': 0, 'another_prop': ''}
name = models.CharField(max_length=255, blank=False, null=False, unique=True)
class ProductConfig(models.Model):
product_type = models.ForeignKey(ProductType)
product = models.ForeignKey(Product)
# a json field with all kind of fields: eg: {"price": 123, "color": "red"}
value = models.TextField(blank=True)
As you can see, every product can have multiple configurations and the value field is a json with different parameters. The json will be one level only. The configuration will have a flag if is active or not (so, the 1 product will have only 1 active configuration)
So, the data will look for example like this:
store_producttype
=================
1 type1
2 type2
store_product
=============
id name
1 car
store_productconfig
===================
id product_type_id product_id value active
1 2 1 { "color": "red", "size": 34, "speed": 342} 0
2 1 1 { "color": "blue", "size": 36, "speed": 123, "another_prop": "xxx"} 1
What I want to know is how can I get /product/1/ like this:
{
"id": 1,
"name": "car",
"type": "type1",
"color": "blue",
"size": 36,
"speed": 123,
"another_prop": "xxx",
}
and to create a new product posting a json similar with the one above.
The json fields are defined but some of them can miss (eg: "another_prop" in the productconfig.id=1
On update, anyway, it will create a new row in productconfig and it will put inactive=0 on the previous one.
So, every product can have different configuration and I want to go back to a specific configuration back in time in some specific cases). I am not really bound to this data model, so if you have suggentions for improvement I am open to them, but I don't want to have that properties as columns in the table.
The question is, what will be the best way to write the serializers for this model? There is any good example somewhere for a such use case?
Thank you.
Let's take this step by step:
In order to get a JSON like the one you posted, you must first transform your string (productConfig value field) to a dictionary. This can be done by using ast.literal_eval ( see more here).
Then, in your product serializer, you must specify the source for each field, like this:
class ProductSerializer(serializers.ModelSerializer):
color = serializer.Field(source='value_dict.color')
size = serializer.Field(source='value_dict.size')
type = serializer.Field(source='type.name')
class Meta:
model = Product
fields = (
'id',
'color',
'size',
'type',
)
This should work just fine for creating the representation that you want. However, this will not create automatically the product config, because DRF doesn't yet allow nested object creation.
This leads us to the next step:
For creating a product with a configuration from JSON, you must override the post method in your view, and create it yourself. This part shouldn't be so hard, but if you need an example, just ask.
This is more of a suggestion: if the json fields are already defined, wouldn't it be easier to define them as separate fields in your productConfig model?
I am trying to implement filtering in one of my data APIs built on Django(non-rel, mongodb-engine) + Tastypie(non-rel). I want to implement filtering on a sub-document inside my main document. The main documents in mongo looks like
{"user_name": "abc", "email": "abc#domain.com", "safety" :{"intra": True, "inter": False, "external": ['a', 'b']}}
The sub-document contains 2 boolean fields and list field. In mongo I can easily query on sub-documents and their fields however I am not able to implement it at the API level.The django model looks like this
from djangotoolbox.fields import DictField, EmbeddedModelField
class UserSafety(models.Model):
user_name = models.CharField()
email = models.EmailField()
safety = DictField() <------- What to use here!
The corresponding Tasty-pie resource looks like
from tastypie_nonrel.fields import DictField, EmbeddedModelField
class UserSafetyResource(ModelResource):
user_name = field.CharField(attribute='user_name')
email = fields.CharField(attribute='email')
safety = DictField(attribute='safety') <----What to use here!
# This part i want to add
class Meta:
filtering = ['safety.intra': ALL, 'safety.inter': ALL]
Eventually I would like it to work this like in an HTTP request
GET http://<server>/my_endpoint/usersafety?safety.intra=true
Any ideas how this can be achieved? May be using either the embedded type fields?