I want populate my database with initial data (dummy data). I have DateTimeFields in many models and for the DateTimeFields it would be good to have a variable in my fixtures files (json, yaml, xml). E.g.: now() + 7h, now() + 6d + 8h + 30m
https://docs.djangoproject.com/en/3.2/howto/initial-data/ :
[
{
"model": "myapp.person",
"pk": 1,
"fields": {
"first_name": "John",
"last_name": "Lennon",
"appointment": now(),
}
},
]
Is there a solution for it?
I found a nice solution. I combined the following very nice python packages FactoryBoy + django-dynamic-fixtures + Faker. Works fine for me. I hope it will hep someone
Related
I have a Django app that requests data from an external API and my goal is to convert that data which is returned as list/dictionary format into a new REST API with a Geojson format.
I came across django-rest-framework-gis but I don't know if I could use it without having a Model. But if so, how?
I think the best way is to use the python library geojson
pip install geojson
If you do not have a Model like in geodjango you have to explicitly describe the geometry from the data you have.
from geojson import Point, Feature, FeatureCollection
data = [
{
"id": 1,
"address": "742 Evergreen Terrace",
"city": "Springfield",
"lon": -123.02,
"lat": 44.04
},
{
"id": 2,
"address": "111 Spring Terrace",
"city": "New Mexico",
"lon": -124.02,
"lat": 45.04
}
]
def to_geojson(entries):
features = []
for entry in entries:
point = Point((entry["lon"], entry["lat"]))
del entry["lon"]
del entry["lat"]
feature = Feature(geometry=point, properties=entry)
features.append(feature)
return FeatureCollection(features)
if __name__ == '__main__':
my_geojson = to_geojson(data)
print(my_geojson)
Create the point geometry from lon, lat (Could also be another geometry type)
Create a feature with the created geometry and add the dictionary as properties. Note that I deleted lon, lat entries from the dictionary to not show up as properties.
Create A feature collection from multiple features
Result:
{"features": [{"geometry": {"coordinates": [-123.02, 44.04], "type":
"Point"}, "properties": {"address": "742 Evergreen Terrace", "city":
"Springfield", "id": 1}, "type": "Feature"}, {"geometry":
{"coordinates": [-124.02, 45.04], "type": "Point"}, "properties":
{"address": "111 Spring Terrace", "city": "New Mexico", "id": 2},
"type": "Feature"}], "type": "FeatureCollection"}
More Info here: Documentation Geojson Library
I have a Cloudant database with objects that use the following format:
{
"_id": "0ea1ac7d5ef28860abc7030444515c4c",
"_rev": "1-362058dda0b8680a818b38e9c68c5389",
"text": "text-data",
"time-data": "1452988105",
"time-text": "3:48 PM - 16 Jan 2016",
"link": "http://url/to/website"
}
I want to fetch objects where the text attribute is distinct. There will be objects with duplicate text and I want Cloudant to handle removing them from a query.
How do I go about creating a MapReduce view that will do this for me? I'm completely new to MapReduce and I'm having difficulty understanding the relationship between the map and reduce functions. I tried tinkering with the built-in COUNT function and writing my own view, but they've failed catastrophically, haha.
Anyways, would it be easier to just delete the duplicates? If so, how do I do that?
While I'm trying to study this and find ELI5s, would anyone help me out? Thanks in advance! I appreciate it.
I'm not sure a MapReduce view is what you are looking for. A MapReduce view will essentially allow you to get the text and the number of docs with that same text, but you really won't be able to get the rest of the fields in the doc (because MapReduce has no idea which doc to return when multiple docs match the text). Here is a sample MapReduce view:
{
"_id": "_design/textObjects",
"views": {
"by_text": {
"map": "function (doc) { if (doc.text) { emit(doc.text, 1); }}",
"reduce": "_count"
}
},
"language": "javascript"
}
What this is doing:
The Map part of the Map Reduce takes each doc and maps it into a doc that looks like this:
{"key":"text-data", "value":1}
So, if you had 7 docs, 2 where text="text-data" and 5 where text="other-text-data" the data would look like this:
{"key":"text-data", "value":1}
{"key":"text-data", "value":1}
{"key":"other-text-data", "value":1}
{"key":"other-text-data", "value":1}
{"key":"other-text-data", "value":1}
{"key":"other-text-data", "value":1}
{"key":"other-text-data", "value":1}
The reduce part of the MapReduce ("reduce": "_count") groups the docs above by the key and returns the count:
{"key":"text-data","value":2},
{"key":"other-text-data","value":5}
You can query this view on your Cloudant instance:
https://<yourcloudantinstance>/<databasename>
/_design/textObjects
/_view/by_text?group=true
This will result in something similar to the following:
{"rows":[
{"key":"text-data","value":2},
{"key":"other-text-data","value":5}
]}
If this is not what you are looking for, but rather you are just looking to keep the latest info for a specific text value then you can simply find an existing document that matches that text and update it with new values:
Add an index on text:
{
"index": {
"fields": [
"text"
]
},
"type": "json"
}
Whenever you add a new document find the document with that same exact text:
{
"selector": {
"text": "text-value"
},
"fields": [
"_id",
"text"
]
}
If it exists update it. If not then insert a new document.
Finally, if you want to keep multiple docs with the same text value, but just want to be able to query the latest you could do something like this:
Add a property called latest or similar to your docs.
Add an index on text and latest:
{
"index": {
"fields": [
"text",
"latest"
]
},
"type": "json"
}
Whenever you add a new document find the document with that same exact text where latest == true:
{
"selector": {
"text": "text-value",
"latest" : true
},
"fields": [
"_id",
"text",
"latest"
]
}
Set latest = false on the existing document (if one exists)
Insert the new document with latest = true
This query will find the latest doc for all text values:
{
"selector": {
"text": {"$gt":null}
"latest" : true
},
"fields": [
"_id",
"text",
"latest"
]
}
I am using the method described in this question to load intial users for my django project.
This saves user permissions, where one permission record might look like this:
{
"pk": 56,
"model": "auth.permission",
"fields": {
"codename": "change_somemodel",
"name": "Can change some model",
"content_type": 19
}
And a user record:
{
"pk": 2,
"model": "auth.user",
"fields": {
"username": "some_user",
"first_name": "",
"last_name": "",
"is_active": true,
"is_superuser": false,
"is_staff": true,
"last_login": "2011-09-20 06:36:54",
"groups": [],
"user_permissions": [
10,
11,
19,
20,
21,
1,
2,
56,
...,
],
"password": "sha1$e4f29$fcec7f8bb930d98abdaaa3c0020220f413c4c1f5",
"email": "",
"date_joined": "2011-03-15 06:01:41"
}
Is there any potential for the content type foreign key to change on a future installation? How about if models or apps added? For example, let's say I add a model to my core app, then I have some reusable apps which are listed after that in settings.py, will those have a different content_type_id on a new installation? Can I include the content_type table in my initial data, or is that likely to cause other issues?
If this isn't a reliable method to load multiple initial users into the database, what are the alternatives?
Check Natural Keys
. Use -n w/ dumpdata, like ./manage.py dumpdata -n auth.User
In practice, it may be that you also have to exclude contenttypes to make this work. Additionally, if you have any user or group data, you'll need to dump those as well. See this thread for more info on excluding contenttypes.
For my app, I had to use the following command for loaddata or test to work with the fixtures generated by manage.py dumpdata. Additionally, you should probably use --indent N option to make sure that you get a human readable file (where N is the number of spaces for indentation)
./manage.py dumpdata -n --indent 4 -e contenttypes appName auth.Group auth.User > initial_data.json
I'm on a page which sends a request to server side for a list of comments of current topic.
like this
localhost:8000/getComments/567 , while 567 is the topic id.
then, in my django view, like this:
def getProjectComments(request, pId):
format = 'json'
mimetype = 'application/javascript' #'application/xml' for 'xml' format
comments = PrjComment.objects.filter(prj=pId)
data = serializers.serialize(format, comments)
return HttpResponse(data,mimetype)
Now , the question is ,
when I try to use jQuery.parseJSON(data) at the browser side.
[
{
"pk": 10,
"model": "app.prjcomment",
"fields":
{
"status": 1,
"time_Commented": "2011-12-11 17:23:56",
"prj": 1,
"content": "my comment 1",
"user": 25
}
},
{
"pk": 9,
"model": "app.prjcomment",
"fields": {
"status": 1,
"time_Commented": "2011-12-11 17:23:51",
"prj": 1,
"content": "comment \u4e00\u4e2a",
"user": 33
}
} ..
I need to use some detail information of user object. (user is a Foreign Key in model PrjComment)
such as user.first_name to display on the comment list.
but here it is only an id for user.("user": 33)
How can I do that? Anyone who can kind help?
Thank you very much
the User is the Django auth_user.
The easy solution would be to specify the values you need:
comments = PrjComment.objects.filter(prj=pId) \
.values('status','time_commented','prj','content','user__id',
'user__username','user__first_name')
Update:
To access your userprofile information use the table name as defined in your AUTH_PROFILE_MODULE setting. In my case the table is called userprofile, and I'd access the data like this:
values('user__userprofile__moreinfo')
where moreinfo is the field you're interested in
I'm trying to create a REST web service that exposes the following Django model:
class Person(models.Model):
uid = models.AutoField(primary_key=True)
name = models.CharField(max_length=40)
latitude = models.CharField(max_length=20)
longitude = models.CharField(max_length=20)
speed = models.CharField(max_length=10)
date = models.DateTimeField(default=datetime.datetime.now)
def __unicode__(self):
return self.name
Here's how I thought about it so far:
Get all Persons
URL: http://localhost/api/persons/
Method: GET
Querystring:
startlat=
endlat=
startlng=
endlng=
Used for getting the Persons that are within the specified coordinate range.
page=
Used for getting the specified page of the response (if the response contains multiple pages).
Returns:
200 OK & JSON
404 Not Found
Example:
Request:
GET http://localhost/api/persons/?startlat=10&endlat=15&startlng=30&endlng=60
Response:
{
"persons":
[
{ "href": "1" },
{ "href": "2" },
{ "href": "3" },
...
{ "href": "100" }
],
"next": "http://localhost/api/persons/?startlat=10&endlat=15&startlng=30&endlng=60&page=2"
}
Get info on a specified Person
URL: http://localhost/api/persons/[id]
Method: GET
Returns:
200 OK & JSON
404 Not Found
Example:
Request:
http://localhost/api/persons/5/
Response:
{
"uid": "5",
"name": "John Smith",
"coordinates": {
"latitude":"14.43432",
"longitude":"56.4322"
},
"speed": "12.6",
"updated": "July 17, 2009, 8:46 a.m."
}
How correct is my attempt so far? Any suggestions are highly appreciated.
{ "href": "1" },
1 is hardly a valid URL. You should use full URLs. Google for HATEOAS.
Also, remember to send a relevant Content-Type header. You may want to make up your own mime-type to describe the format. This gives you the option to later change the content-type (Eg. change the format after publishing). See Versioning REST Web Services
I think query parameters could be simpler and clearer. This would make the URI more readable and would allow more flexibility for future extensions:
GET http://localhost/api/persons/?latitude=10:15&longitude=30:60
You may want to enable these in the future:
GET http://localhost/api/persons/?latitude=10&longitude=60&within=5km
Seems REST-cool. Even i worked on same kind of thing, few days earlier.
The only change, i would love to do in it, is the direct link to the person details. And also some details (like name here) to identify the person, and aid me in decision to navigate further. Like...
{
"persons":
[
{ "name": "John Smith", "href": "http://localhost/api/persons/1/" },
{ "name": "Mark Henry", "href": "http://localhost/api/persons/2/" },
{ "name": "Bruce Wayne", "href": "http://localhost/api/persons/3/" },
...
{ "name": "Karl Lewis", "href": "http://localhost/api/persons/100/" }
],
"next": "http://localhost/api/persons/?startlat=10&endlat=15&startlng=30&endlng=60&page=2"
}
This way, i am giving everything, to present data as,
John
Smith
Mark
Henry
Bruce
Wayne
...
Karl Lewis
Next Page
It's ok to provide shorthand URIs in your JSON responses if you provide some templating system. Like giving a base URI as something like http://whatever.com/persons/{id}/ and then providing IDs. Then with python you can just do a format call on the string. You don't ever want to make the programmer actually look at and understand the meaning of the URIs, which isn't necessary when you use templates.
You might want to take a look at pre-existing REST middleware. I know they saved me a lot of time. I'm using http://code.google.com/p/django-rest-interface/. And a snippet of the urls.py
json_achievement_resource = Collection(
queryset = Achievement.objects.all(),
permitted_methods = ('GET',),
responder = JSONResponder(paginate_by = 10)
)
urlpatterns += patterns('',
url(r'^api/ach(?:ievement)?/(.*?)/json$', json_achievement_resource),
)