error rebuilding the index using django-elasticsearch-dsl - django

I am trying to connect to a existing index in my local elastic search engine.
I am using django-elasticsearch-dsl package. I followed this tutorial to do that. https://github.com/sabricot/django-elasticsearch-dsl
please note that,I already developed my django app[website] with mysql database. I have some unstructured text data indexed in elastic-search. I want to develop this 'advanced search' HTML page for users querying data frpm elasticsearch.
I followed almost everything upto
python manage.py search_index --rebuild but once I excute it it asked Are you sure you want to delete the 'website_data_discovery' indexes? [n/Y]: when I say n it will be aborted.
when I say Y it gives a lengthy error saying django.db.utils.ProgrammingError: Table 'crdc.website_data_discovery' doesn't exist
This is my file structure.
crdc
website
->__pycache__
-> media
->migrations
->static
->templates
->__init__.py
->admin.py
->apps.py
->documnets.py
->forms.py
->models.py
->tests.py
->urls.py
->views.py
manage.py
This is my settings.py/crdc.
###more codes here
INSTALLED_APPS = [
'website.apps.WebsiteConfig',
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'django_elasticsearch_dsl',
]
ELASTICSEARCH_DSL={
'default': {
'hosts': 'localhost:9200'
},
}
###more codes here
This is documents.py/website
from django_elasticsearch_dsl import DocType, Index
from .models import Data_Discovery
data_discovery = Index('website_data_discovery')
data_discovery.settings( number_of_shards = 5, number_of_replicas = 1,)
#data_discovery.doc_type
class Data_DiscoveryDocument(DocType):
class Meta:
model = Data_Discovery
fields = ['extracted_text', 'source_type']
this is models.py/website
from django.db import models
class WebsiteRepository(models.Model):
##### mysql model
class Documents(models.Model):
#### mysql model
class Data_Deposite(models.Model):
#### mysql model
class Data_Discovery(models.Model):
## Elasticsearch model
Any help would be greatly appreciated. Thanks

I know this is late. But, I thought I should answer my own question, so that anyone face the same problem, can figure it out.
I created another App for Elastic search component.Yes, I created another App under the same project. And I followed this tutorial https://pypi.org/project/django-elasticsearch-dsl/
Then I called Elasticsearch view functions at views.py from the website App. It works perfectly now.
I am not a expert in Django or Elastic Search. So, I can not explain why it is not working as the previous attempt. I hope this helps to anyone who comes to this kind of situation.

Related

Error after adding Django channels to installed_apps in settings file

I am just trying out django channels so I created a virtual environment and installed django, drf and channels. It threw error asking for visual c++ build tools which got resolved after installing it. Then I created a channels project and an app. Then just for testing I added a sample model as below and registered it with the admin. It compiled well and also I was able to see the model in the admin page.
My Model Calss
from django.db import models
# Create your models here.
class College(models.Model):
name = models.CharField(max_length=50)
address = models.CharField(max_length=150)
objects = models.Manager()
def __str__(self):
return self.name
My admin.py
from django.contrib import admin
from .models import College
# Register your models here.
admin.site.register(College)
Now the Problem
I added channels to the INSTALLED_APPS list in the settings.py file like below,
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'channelApp',
'channels',
]
Now when I try to run the server using the runserver command I get the following error
ModuleNotFoundError: No module named 'win32api'
LookupError: No installed app with label 'admin'.
I have been searching but failed to find any suitable answer. Kindly help me out.
Thanks in advance.
Just after I posted this I stumbled onto a SO post
Issue after installing django channels
Just to reiterate in short, this is an open bug and the work around is to install the following package
pip install pypiwin32
After installing you may have to close and reopen the editor for the changes to reflect. And the error is resolved.

django.core.exceptions.ImproperlyConfigured: Application labels aren't unique, duplicates: thumbnail

I am developing a django web project that uses the following packages/applications:
sorl-thumbnail
django-oscar
Here is a snippet of my settings.py file:
INSTALLED_APPS = [
'registration', #should be immediately above 'django.contrib.auth'
'django.contrib.auth',
# ...
'zinnia',
'zinnia_tinymce',
'sorl.thumbnail',
'embed_video',
# ...
'django.contrib.flatpages',
'compressor',
'widget_tweaks',
] + get_core_apps()
When I comment out sorl.thumbnail, I am able to run the development server using manage.py runserver. However, if I uncomment the sorl.thumbnail line and try to run the development server, it throws an exception:
django.core.exceptions.ImproperlyConfigured: Application labels aren't unique, duplicates: thumbnail
Now, I am aware that a similar question exists on this site, however, following the instructions in the accepted solution, i.e.:
create a sol_thumbnail folder in same directory as the manage.py script
create sorl_thumbnail/apps.py (see below)
modify myproject/mysite/___init____.py (see below)
sorl-thumbnail/apps.py
from django.apps import AppConfig
class SorlthumbnailConfig(AppConfig):
name = 'sorl-thumbnail'
label = 'sorl.thumbnail'
myproject/mysite/_init _.py
default_app_config = 'sorl-thumbnail.apps.SorlthumbnailConfig'
Why is the fix above not working, and how do I resolve this issue?
BTW: I am using django-1.10
I went though the same problem of duplicated applications, and following exactly the similar question I solved my problem.
The problem with your solution is that you have added default_app_config = 'sorl-thumbnail.apps.SorlthumbnailConfig' to myproject/mysite/___init____.py, but you should have added to myproject/sorl-thumbnail/___init____.py.

RuntimeError: Model class django.contrib.sites.models.Site doesn't declare an explicit app_label and isn't in an application in INSTALLED_APPS

I am building an application with Django Rest Framework and AngularJs. I am using Django-rest-auth for my authentication purposes, although, I have not been able to set it up. Anyway, I am trying to set up this app with my project. I realized I need to install django-rest-auth-registration to get it running, so I followed this documentation to do the following things:
I ran the commands
pip install django-rest-auth
and
pip install django-allauth
Any my settings.py looks like this:
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
# 3rd party apps
'rest_framework',
'rest_framework.authtoken',
'rest_auth',
'allauth',
'allauth.account',
'rest_auth.registration',
# My app
'myapp',
]
I have also added the authentication backends, context_processors, and the proper urls.
However, when I try to migrate, my terminal throws the following error:
RuntimeError: Model class django.contrib.sites.models.Site doesn't
declare an explicit app_label and isn't in an application in
INSTALLED_APPS.
Why do I get this error, and how do I solve it to migrate my project? Thanks!
The fix
Just add Django's Sites framework to your apps and set SITE_ID to 1 in your settings.
INSTALLED_APPS = [
...
'django.contrib.sites',
]
SITE_ID = 1
Why does this happen?
Django's Sites Framework is a contributed module bundled with the core library that allows for the use of a single Django application/codebase with different sites (that can use different databases, logic in views, etc). The SITE_ID setting, as stated in the docs, "is used so that application data can hook into specific sites and a single database can manage content for multiple sites."
In this particular case AllAuth requires the Sites Framework in order to function properly. Many other third-party libraries are built to safely handle cases where multiple sites may be present and as such may be best .
I landed on this post via Google search. My problem was running tests that blew up with the error:
RuntimeError: Model class app.taxonomy.models.Term doesn't declare an explicit app_label and isn't in an application in INSTALLED_APPS.
This was running on Python 2.7.x with absolute imports. As mentioned by Colton Hicks in the comments, below, this can also happen with Python 3 (pytest 3.2.3 with Django 1.11.4).
In my tests.py:
from __future__ import absolute_import
[...]
from .models import Demographics, Term
After changing the relative import to an absolute import the problem went away:
from taxonomy.models import Demographics, Term
HTH
Try adding the app_label = 'yourApp' in the models Meta class:
class Meta:
app_label = 'yourApp'
I got the error above. However my problem was the in the urls.py. I was following PyDanny cookiecutter django recipe. My error was to put in the urls.py this line:
url(r'^demo/', include('project.demoapp.urls', namespace='demoapp')),
when I corrected to this:
url(r'^demo/', include('demoapp.urls', namespace='demoapp')),
all was well. I also changed my local apps (I did this first and so the critical error was the url misconfiguration):
LOCAL_APPS = [
# Your stuff: custom apps go here
'demoapp.apps.DemoAppConfig',
]
I have django debug toolbar installed and this was actually causing the/my problem.
INSTALLED_APPS (in settings.py) needs the entry 'django.contrib.sessions'. Make sure to run migrate after adding.
Just add 'django.contrib.sites', to INSTALLED_APPS and set SITE_ID = 1 in your settings.py file.
Upgraded Answer for Django>=4.0 // 2022
Add Django's Sites framework and FlatPages Framework to your INSTALLED_APPS and set SITE_ID in your settings.
INSTALLED_APPS = [
...
'django.contrib.sites',
'django.contrib.flatpages',
]
SITE_ID = 1
Your tests should work like a charm
This error occurred because I had created a new app folder for a subset of sites related to another feature. This needed to be added to my INSTALLED_APPS in settings.py
Django 4.1+ (2023)
After almost an hour digging, what solved for me was this:
INSTALLED_APPS = [
...
'django.contrib.sessions',
]
No need for SITE_ID or additional INSTALLED_APPS entries.
Everything worked as expected after I made a migration
python manage.py migrate
Good luck

nothing happens when running "python manage.py sql polls"

I am trying to get Django running on Google App Engine using Django non-rel. I am following the Django 1.5 tutorial However, when i run:
python manage.py sql polls
nothing is returned. Can anyone suggest why 'manage.py sql' is silently failing?
When I sync the database I get:
$ python manage.py syncdb
Creating tables ...
Installing custom SQL ...
Installing indexes ...
Installed 0 object(s) from 0 fixture(s)
The database is specified in settings.py as follows:
# Activate django-dbindexer for the default database
DATABASES['default'] = {'ENGINE': 'dbindexer', 'TARGET': DATABASES['default']}
AUTOLOAD_SITECONF = 'indexes'
and I have remembered to include 'polls' in the settings.py
INSTALLED_APPS = (
# 'django.contrib.admin',
'django.contrib.contenttypes',
'django.contrib.auth',
'django.contrib.sessions',
'djangotoolbox',
'autoload',
'dbindexer',
'polls',
# djangoappengine should come last, so it can override a few manage.py commands
'djangoappengine',
)
and the models are present in polls/models.py
from django.db import models
class Poll(models.Model):
question = models.CharField(max_length=200)
pub_date = models.DateTimeField('date published')
class Choice(models.Model):
poll = models.ForeignKey(Poll)
choice_text = models.CharField(max_length=200)
votes = models.IntegerField(default=0)
NOTE: If I change settings.py to use a local sqlite3 database, 'manage.py sql polls' behaves as described in the tutorial. Therefore, as far as I can tell, this has nothing to do with:
the /polls directory structure
the /polls/models.py file
the python path
Why do you expect it do anything? GAE is, specifically, a non-relational (NoSQL) datastore. There is quite simply no SQL to produce.
You should be aware that GAE, even with django-non-rel, is quite different from standard Django, and following the Django tutorial is only likely to confuse you.

How do I stop django from syncing certain tables when I do syncdb?

Whenever I run the syncdb command, I'm getting a lot of auth tables created but I'm not including the admin or auth packages. Here are the tables it is creating on its own:
Creating table auth_permission
Creating table auth_group_permissions
Creating table auth_group
Creating table auth_user_user_permissions
Creating table auth_user_groups
Creating table auth_user
Creating table auth_message
I don't need these tables. In my settings.py file, I have commented out the AuthenticationMiddleware, MessageMiddleware and all apps except for django.contrib.sessions and my own personal apps. Is there a setting somewhere that I'm missing so that these tables aren't created?
Here are my INSTALLED_APPS
INSTALLED_APPS = (
'deanproxy.globaltags',
'deanproxy.blog',
'deanproxy.auth',
'deanproxy.twitter',
# 'django.contrib.auth',
# 'django.contrib.contenttypes',
'django.contrib.sessions',
# 'django.contrib.sites',
# 'django.contrib.messages',
#'django.contrib.staticfiles',
# Uncomment the next line to enable the admin:
# 'django.contrib.admin',
# Uncomment the next line to enable admin documentation:
# 'django.contrib.admindocs',
)
EDIT: I Just realized that if I comment out my own app 'deanproxy.auth', that all of these tables are not created. However, my auth app isn't including any of the django auth stuff and it's a very, very simple auth system (just e-mail and password). It appears Django may be getting confused by the name of it...
I found the cause. I had an app called 'auth' which was a very simple auth system and nowhere near as involved as the django auth app. It appears that django was including all of these tables whenever I added 'deanproxy.auth' to my INSTALLED_APPS. Whenever I renamed this app to something else, these tables were not being included anymore. I'm going to try to track down why it does this... however, if anyone else has this issue, just name your app to something other than auth. Even though you're including it from your app, it still thinks it's the django auth system...