Neo4j and Django - django

Is ist possible to use neomodel to make models in django?
How do I have to integrate neo4j in django? I'm using Python 3, so neo4django isn't really an option.
I'm new to both of them and at the moment I'm a little confused...
Thanks a lot! :3

Hey neomodel supports python 3 out of the box you can use it with or without django checkout the documentation here: http://neomodel.readthedocs.org/en/latest/

NEOMODEL_NEO4J_BOLT_URL = bolt://neo4j:password#localhost:7687
NEOMODEL_SIGNALS = True
NEOMODEL_FORCE_TIMEZONE = False
NEOMODEL_ENCRYPTED_CONNECTION = True
NEOMODEL_MAX_POOL_SIZE = 50
You need to add the above lines in your settings.py file to use Neo4j.

Hey I know this was asked 8 years ago, but now there's a tool called neomodel which serves this exact purpose. It can also be combined with django-neomodel which allows for neomodel to be easily integrated into your django project.
Using django-neomodel, all you need to do is specify the URL of the database in your settings.py file like so:
NEOMODEL_NEO4J_BOLT_URL = 'bolt://{username}:{password}#{HOSTorIP}'
You can easily create models in your models.py file. Here are some examples for this neomodel documentation:
from neomodel import (StructuredNode, StringProperty,
UniqueIdProperty, IntegerProperty,
RelationshipTo)
class Country(StructuredNode):
code = StringProperty(unique_index=True, required=True)
class Person(StructuredNode):
uid = UniqueIdProperty()
name = StringProperty(unique_index=True)
age = IntegerProperty(index=True, default=0)
# traverse outgoing IS_FROM relations, inflate to Country objects
country = RelationshipTo(Country, 'IS_FROM')
Then you can run python manage.py install_labels to do the equivalent of running migrations, or python manage.py clear_neo4j to clear all nodes from the database.
Nodes can be created like so:
from models import Person
john = Person(name="john", age=23).save()
frank = Person(name="frank", age=50).save()
canada = Country(code=5).save()
And relationships like so:
john.country.connect(canada)
And nodes/relationships can be retrieved as follows:
frank = Person.nodes.get(name='frank')
frank.age += 1
frank.save()
franks_country = frank.country
print(franks_country)
# {'code': 5}

Related

I want to upload a xml file into a PostgreSQL using Django

I am new to Django and my current task is to upload a xml file with 16 fields and more than 60000 rows to a database in PostgreSQL. I used Django to connect to the Database and was able to create a table in the database.
I also used XML Etree to parse the xml file. I am having trouble storing the data in the table that I created in the sql database.
This is the code that I used to parse:
import xml.etree.ElementTree as ET
def saveXML2db():
my_file = "C:/Users/Adithyas/myproject/scripts/supplier_lookup.xml"
tree = ET.parse(my_file)
root = tree.getroot()
cols = ["organization", "code", "name"]
rows = []
for i in root:
organization = i.find("organization").text
code = i.find("code").text
name = i.find("name").text
x = rows.append([organization, code, name])
data = """INSERT INTO records(organization,code,name) VALUES(%s,%s,%s)"""
x.save()
saveXML2db()
the code runs without any error, but I am unable to store the data into the table in the SQL database.
So I figured out the answer to my question and I wish to share this with you guys.
This is how I imported a xml file to PostgreSQL database using Django ORM:
First, I created a virtual environment to work with:
open command prompt in the folder you wish to run the project
py -m venv envy
envy\Scripts\activate
our virtual environment is ready to use
then,
pip install django
pip install psycopg2
django-admin startproject projectq
cd projectq
py manage.py startapp myapp
now both our project and app is created and ready to use
code . #to open Visual code
now go to settings.py in 'projectq' and add 'myapp' to INSTALLED_APPS:
INSTALLED_APPS = [
'myapp',#add myapp to the installed apps
]
now to connect our project to PostgreSQL database we have to make some changes in the DATABASES in settings.py as well:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'projectq',
'USER': 'postgres',
'PASSWORD': '1234',
}
}
change dbsqlite to the name of the database that you are using, add name of your Database, username and password
now the connection is established.
we move on to the next step
go to models.py to create our table in PostgreSQL to store our xml data:
from django.db import models
# Create your models here.
class Record(models.Model):
po_organization = models.IntegerField()
code = models.CharField(max_length = 100)
name = models.CharField(max_length=100)
address_1 = models.CharField(max_length=100 , null = True)
address_2 = models.CharField(max_length=100, null = True)
If your data has null values it's best to add null = True, to avoid errors
py manage.py makemigrations
py manage.py migrate
now the table we created should appear on the PostgreSQL database
next step is to parse our xml file and to import it to the table we created.
For that we will use Django ORM queries
open terminal in our visual code in models.py
activate virtual environment again
to use ORM query:
py manage.py shell
now add these codes to the interactive console:
>>>from myapp.models import Record
>>>import xml.etree.ElementTree as ET
>>>def data2db():
...file_dir = 'supplier_lookup.xml'
...data = ET.parse(file_dir)
...root = data.findall('record')
...for i in root:
... organization = i.find('organization').text
... code = i.find('code').text
... name = i.find('name').text
... address_1 = i.find('address_1').text
... address_2 = i.find('address_2').text
... x = Record.objects.create(organization=organization, code=code,
... name=name, address_1=address_1, address_2=address_2)
... x.save()
...
>>>data2db()
That's It. The data should be loaded into the database now.
Hope this helps.
Have you checked any python/PostgreSQL examples? Your code should have something like this (untested):
import psycopg2
def storeXmlToPostgres(xmldata):
with psycopg2.connect(host="dbhost", database="dbname", user="username", password="password") as conn:
sql = "INSERT INTO records(organization,code,name) VALUES(%s,%s,%s)"
cur = conn.cursor()
for i in xmldata:
organization = i.find("organization").text
code = i.find("code").text
name = i.find("name").text
cur.execute(sql, [organization, code, name])

Mongoengine : How to update specific fields of an existing document?

I have an existing mongo document which has been exposed over a REST API. The API request will contain certain fields from the document which either needs to be updated with new values or insert new values in them if the field is null. How to perform the update on fields of an existing mongoengine document? I'm using marshmallow-mongoengine for serialization on flask.
The problem that I'm facing is that if a certain field is missing in the request payload, on calling update with the remaining fields as kwargs leads to setting the missing fields as None. How can update or insert only the fields given in the payload?
Joseph's answer is OK. But another answer wont hurt eh!
Here's how i updated my document using flask-mongoengine
Actual code :
Game.objects(id = _id).update(
set__kickoff = request_json.get('kickoff'),
set__gameid = request_json.get('gameid'),
set__home_team = request_json.get('home_team'),
set__away_team = request_json.get('away_team'),
set__home_win = request_json.get('home_win'),
set__draw = request_json.get('draw'),
set__away_win = request_json.get('away_win'),
set__sport = request_json.get('sport')
)
Game class :
import datetime
flask_mongoengine import BaseQuerySet, MongoEngine
db = MongoEngine()
class Game(db.Document):
kickoff = db.DateTimeField(required=True)
added_on = db.DateTimeField(default=datetime.datetime.utcnow)
gameid = db.FloatField(required=True)
home_team = db.StringField(required=True)
home_win = db.FloatField(required=True)
draw = db.FloatField(required=True)
away_win = db.FloatField(required=True)
away_team = db.StringField(required=True)
sport = db.StringField(required=True)
meta = {
'collection':'games',
'queryset_class': BaseQuerySet
}
PS : Remember to indent the code in python
Additionally I noticed you tagged Marshmallow in your question. Here, a sample derived from their official git repo here
First we need a Mongoengine Document:
import mongoengine as me
class Task(me.EmbeddedDocument):
content = me.StringField(required=True)
priority = me.IntField(default=1)
class User(me.Document):
name = me.StringField()
password = me.StringField(required=True)
email = me.StringField()
tasks = me.ListField(me.EmbeddedDocumentField(Task))
Great ! Now it's time for the Marshmallow Schema. To keep things DRY, we use marshmallow-mongoengine to do the mapping:
import marshmallow_mongoengine as ma
class UserSchema(ma.ModelSchema):
class Meta:
model = User
Finally it's time to use our schema to load/dump documents:First let's create a document
user_schema = UserSchema()
u, errors = user_schema.load({"name": "John Doe", "email":
"jdoe#example.com", "password": "123456","tasks": [{"content": "Find a
proper password"}]})
u.save()
If the document already exists, we can update it using update
u
u2, errors = user_schema.update(u, {"name": "Jacques Faite"})
>>> u2.name
"Jacques Faite"
If you only want to update one single document you can use the save method. That's what I do. If a document already exists, it updates fields instead of creating a new document.
car = Car.objects(pk=car_id) # return a queryset
if car:
car = car.get(pk=car_id) # return an object from queryset
car.make = requestData['make']
car.model = requestData['model']
car.mileage = requestData['mileage']
car.save()
If you want to update many documents then, I recommend checking out the atomic updates section of the user guide.
Something like~
Car.objects(param="param to filter by").update(set__param=newParam)
"set" followed by two underscores is a modifier. There are more modifiers available in the guide I linked above.

Problems Using Django 1.11 with Sql-Server database view

I am trying to use Django (django 1.11.4) to read data from a SQL-Server view (sql server 2012 - I use sql_server.pyodbc [aka django-pyodbc] for this), and nothing seems to work.
Here's my model:
class NumUsersAddedPerWeek(models.Model):
id = models.BigIntegerField(primary_key=True)
year = models.IntegerField('Year')
week = models.IntegerField('Week')
num_added = models.IntegerField('Number of Users Added')
if not settings.RUNNING_UNITTESTS:
class Meta:
managed = False
db_table = 'num_users_added_per_week'
and here's how the database view is created:
create view num_users_added_per_week
as
select row_number() over(order by datepart(year, created_at), datepart(week, created_at)) as 'id',
datepart(year, created_at) as 'year', datepart(week, created_at) as 'week', count(*) as 'num_added'
from [<database name>].[dbo].[<table name>]
where status = 'active' and created_at is not null
group by datepart(year, created_at), datepart(week, created_at)
The view works just fine by itself (e.g., running 'select * from num_users_added_per_week' runs just fine (and very quickly)...
I used the following django command (i.e., 'action') to try 3 different ways of attempting to pull data via the model, and none of them worked (although, judging from other posts, these approaches seemed to work with previous versions of django) :(:
from django.core.management.base import BaseCommand, CommandError
from <project name>.models import NumUsersAddedPerWeek
from django.db import connection
class Command(BaseCommand):
def handle(self, *args, **options):
# attempt # 1 ...
num_users_info = NumUsersAddedPerWeek.objects.all()
info = num_users_info.first()
for info in num_users_info:
print(info)
# attempt # 2 ...
cursor = connection.cursor()
cursor.execute('select * from num_users_added_per_week')
result = cursor.fetchall()
# attempt # 3 ...
num_users_info = NumUsersAddedPerWeek.objects.raw('select * from num_users_added_per_week')
for info in num_users_info:
print(info)
Each of the 3 different approaches gives me the same error: "('42S02', "[42S02] [Microsoft][ODBC SQL Server Driver][SQL Server]Invalid object name 'num_users_added_per_week'. (208) (SQLExecDirectW)")"
Please note: my migrations are running just fine - adding class Meta: managed = False is crucial with latest versions of Django in situations where you do not want migrations to create / update / delete your sql table structure...
I figured it out - I have a custom Database Router (in settings.DATABASE_ROUTERS) that I had not properly added this to (I am doing this because the project has multiple databases - see Multi-DB to see why and how to do this). (So boneheaded bug on my part)
But here's what I found out: It turns out all three of the methods I used should work, if you have 1 database in your project. If you have multiple databases then you can query the database through your model object (e.g., <Model Name>.objects.all()) or through raw sql, but you have to specify the raw sql via your model (e.g., <Model Name>.objects.raw(<select * from <view name>)) - otherwise your Database Router will not know which database to use.

Merging models, Django 1.11

After upgrading from Django 1.8 to 1.11 I've been looking at a means of merging some records - some models have multiple entries with the same name field, for example. There's an answer here that would appear to have what I would need:
https://stackoverflow.com/a/41291137/1195207
I tried it with models like this:
class GeneralType(models.Model):
#...
domains = models.ManyToManyField(Domain, blank=True)
#...
class Domain(models.Model):
name = models.TextField(blank=False)
#...
...where Domain has various records with duplicate names. But, it fails at the point indicated:
def merge(primary_object, alias_objects=list(), keep_old=False):
"""
Use this function to merge model objects (i.e. Users, Organizations, Polls,
etc.) and migrate all of the related fields from the alias objects to the
primary object. This does not look at GenericForeignKeys.
Usage:
from django.contrib.auth.models import User
primary_user = User.objects.get(email='good_email#example.com')
duplicate_user = User.objects.get(email='good_email+duplicate#example.com')
merge(primary_user, duplicate_user)
"""
# ...snip....
for alias_object in alias_objects:
for related_object in alias_object._meta.related_objects:
related_name = related_object.get_accessor_name()
if related_object.field.many_to_one:
#...snip...
elif related_object.field.one_to_one:
#...snip...
elif related_object.field.many_to_many:
related_name = related_name or related_object.field.name
for obj in getattr(alias_object, related_name).all():
getattr(obj, related_name).remove(alias_object) # <- fails here
getattr(obj, related_name).add(primary_object)
The problem is apparently that 'GeneralType' object has no attribute 'generaltype_set'. Adding a related_name to GeneralType doesn't fix this - the script fails in the same manner but quoting the name I've now given it. I'm not quite sure what Django is up to here so any suggestions would be welcome.
Edit:
In a Django shell I can successfully reference GeneralType from Domain, so it's something about the script above that I'm not getting. Example:
>>> d = Domain.objects.first()
>>> d
<Domain: 16s RNA>
>>> d.generaltype_set
<django.db.models.fields.related_descriptors.ManyRelatedManager object at 0x11175ba90>
>>> d.generaltype_set.first()
<GeneralType: Greengenes>
>>> getattr(d,'generaltype_set')
<django.db.models.fields.related_descriptors.ManyRelatedManager object at 0x10aa38250>
I managed to come up with a workaround. It seems that everything would function if I referenced generaltype.domains in the getattr(obj, related_name) part of the script, so I modified it as follows just before the line marked as failing in the question above:
if obj.__class__.__name__ == 'GeneralType':
related_name = 'domains'
Everything ran as it should after that, it seems.

How to create GIN index in Django migration

In Django, since version 1.11 we have a class for PostgreSQL GinIndex (https://docs.djangoproject.com/en/1.11/ref/contrib/postgres/indexes/). I'd like to create a migration that constructs such index on a VectorSearchField I added to one of my tables. So far, I've tried to simply add db_index=True to the VectorSearchField, but that fails, because it tries to create a B-Tree index (I think) and the VectorSearchField values are too long.
I managed to create the index I want by running a migrations.RunSQL() migration with:
CREATE INDEX entryline_sv_index ON entryline USING GIN (sv);
However, I guess, since there is a special GinIndex class in Django, maybe there is a way to create such index without executing raw SQL?
Here's a model class:
import django.contrib.postgres.search as pg_search
class EntryLine(models.Model):
speaker = models.CharField(max_length=512, db_index=True)
text = models.TextField()
sv = pg_search.SearchVectorField(null=True) # I want a GIN index on this field.
Any idea how to properly create an index for sv field in a migration? Or is executing the CREATE INDEX ... query the best way?
Haven't yet had a chance to migrate my old manual CREATE INDEX codes to the new system introduced in 1.11 but my understanding is
from django.contrib.postgres.indexes import GinIndex
import django.contrib.postgres.search as pg_search
class EntryLine(models.Model):
speaker = models.CharField(max_length=512, db_index=True)
text = models.TextField()
sv = pg_search.SearchVectorField(null=True)
class Meta:
indexes = [GinIndex(fields=['sv'])]
Is what's required. Raw SQL CREATE INDEX statements need not be used any more.