I'm using Django and the Haystack module to create a search engine. I want to use ElasticSearch. I have installed it and launched it with:
$ brew install elasticsearch
$ elasticsearch -f -D es.config=/usr/local/Cellar/elasticsearch/0.90.2/config/elasticsearch.yml
My settings seem correct and work:
# Haystack configuration
HAYSTACK_CONNECTIONS = {
'default': {
'ENGINE': 'haystack.backends.elasticsearch_backend.ElasticsearchSearchEngine',
'URL': 'http://127.0.0.1:8000/',
'INDEX_NAME': 'haystack',
},
}
HAYSTACK_SIGNAL_PROCESSOR = 'haystack.signals.RealtimeSignalProcessor'
Here is my search indexes:
from haystack import indexes
from account.models import Profile
class ProfileIndex(indexes.SearchIndex, indexes.Indexable):
text = indexes.CharField(document=True, use_template=True)
first_name = indexes.CharField(model_attr='first_name')
last_name = indexes.CharField(model_attr='last_name')
def get_model(self):
return Profile
and my profile_text.txt:
{{ object.first_name }}
{{ object.last_name }}
Everything seems correct I guess, I follow the documentation and this tutorial.
But now, when I'm triggering:
$ python manage.py rebuild_index
I get this error:
pyelasticsearch.exceptions.InvalidJsonResponseError: <Response [404]>
If someone knows why? :)
Thank you.
You're running the Elastic Search Server on the same port as the Django Server is running on.
Change the port from 8000 to something else, and then it'll work!
Related
When I deploy my django app in azure with sql database, it is showing me The page cannot be displayed because an internal server error has occurred.
But when I deploy it without sql database it works fine. What is the problem?
error log
Most likely causes:
IIS received the request; however, an internal error occurred during the processing of the request. The root cause of this error depends on which module handles the request and what was happening in the worker process when this error occurred.
IIS was not able to access the web.config file for the Web site or application. This can occur if the NTFS permissions are set incorrectly.
IIS was not able to process configuration for the Web site or application.
The authenticated user does not have permission to use this DLL.
The request is mapped to a managed handler but the .NET Extensibility Feature is not installed.
I tried to reproduce your issue but failed. I deploy my django app to azure with sqlserver database by using django-mssql successfully.
Please refer to the steps I did:
Step 1: Add configuration in settings.py
DATABASES = {
'default': {
'NAME': '***',
'ENGINE': 'sqlserver_ado',
'HOST': '***.database.windows.net',
'USER': '***',
'PASSWORD': '***',
'OPTIONS': {
'provider': 'SQLOLEDB',
'use_legacy_date_fields': 'True'
}
}
}
Step 2: Add test code to query results:
from django.db import connection
def my_custom_sql():
with connection.cursor() as cursor:
cursor.execute("SELECT * FROM ***")
row = cursor.fetchone()
return row
Step 3: Install django-mssql package and dependency on KUDU.
Step 4: Deploy django app to azure and access url in browser.
Hope it helps you.
Update Answer:
Firstly , the error as below when you tried to build django with django-mssql because your version of django is too high.
From the django-mssql doc , you could see : Django 1.8 is supported by the current release and your django version is 1.11.8.
Secondly, the django-pyodbc package which you mentioned in your comment just supports django 1.10. You could refer to this doc.
So, I suggest you to use django-pyodbc-azure package which supports django 1.11 by now.
You could follow the tutorial in above doc.
Configuration in settings.py
DATABASES = {
'default': {
'NAME': '***',
'ENGINE': 'sql_server.pyodbc',
'HOST': '***.database.windows.net',
'USER': '***',
'PASSWORD': '***',
'OPTIONS': {
'driver': 'ODBC Driver 13 for SQL Server',
}
}
}
My django version is as same as you and it works for me.
Any concern please let me know.
Update Answer2 :
I just used Visual Studio 2017 python django template.
My view.py :
"""
Definition of views.
"""
from django.shortcuts import render
from django.http import HttpRequest
from django.template import RequestContext
from datetime import datetime
from django.db import connection
def my_custom_sql():
with connection.cursor() as cursor:
cursor.execute("SELECT * FROM dbo.student")
row = cursor.fetchone()
return row
def home(request):
"""Renders the home page."""
assert isinstance(request, HttpRequest)
return render(
request,
'app/index.html',
{
'title':'Home Page',
'year':datetime.now().year,
}
)
def contact(request):
"""Renders the contact page."""
assert isinstance(request, HttpRequest)
return render(
request,
'app/contact.html',
{
'title':'Contact',
'message':'Your contact page.',
'year':datetime.now().year,
}
)
def about(request):
row = my_custom_sql()
"""Renders the about page."""
assert isinstance(request, HttpRequest)
return render(
request,
'app/about.html',
{
'title': row,
'message':'Your application description page.',
'year':datetime.now().year,
}
)
Please note the my_custom_sql() function.
My about.html:
{% extends "app/layout.html" %}
{% block content %}
<h2>{{ title }}.</h2>
<h3>{{ message }}</h3>
<p>Test Something!</p>
{% endblock %}
And the pyodbc settings in the settings.py just refer to my previous update answer.
Please enable Diagnostics Logging for debugging your application and determine the real cause of your application is not working. This blog post may also help you troubleshooting-logging your application.
I'm new in django and trying to connect to my heroku database from my localhosted django app.
I've followed the instructions on heroku website but it's seems django won't understand that i'm trying to access an external database.
i get this error
relation "myAppName_user" does not exist
LINE 1: INSERT INTO "myAppName_user" ("title", "content") VALUES
('Beat...
However i never created or never intented to create a table named myAppName_user
I'm just trying to access the table user in my heroku postgres db but i don't know why it tries with myAppName_user
i just explicitly added myAppName in the INSTALLED_APPS config as django throws an error if it's not the case.
My DATABASE config :
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
}
}
DATABASES['default'] =
dj_database_url.config(default='postgres://xxxxx')
I want to test my database from my localhost. I'm doing this easily with Node JS but i can't find the solution in django.
Any suggestions ?
EDIT
from __future__ import unicode_literals
from django.db import models
class Fruits(models.Model):
title = models.CharField(max_length=100)
content = models.TextField()
def __str__(self):
return self.name
I am getting issue with " raise BulkIndexError while running this python manage.py rebuild_index ?
Here is my haystack configuration in settings.py file
HAYSTACK_CONNECTIONS = {
'default': {
'ENGINE': 'haystack.backends.elasticsearch_backend.ElasticsearchSearchEngine',
'URL': 'http://127.0.0.1:9200/',
'INDEX_NAME': 'haystack',
#'SILENTLY_FAIL': False,
},
}
HAYSTACK_SIGNAL_PROCESSOR = 'haystack.signals.RealtimeSignalProcessor'
Here is my search_indexes.py
class ProductIndex(indexes.SearchIndex, indexes.Indexable):
text = indexes.CharField(document=True, use_template=True)
content_auto = indexes.EdgeNgramField(model_attr='title')
def get_model(self):
return Product
def index_queryset(self, using=None):
return self.get_model().objects.all()
here is my views.py
def search_titles():
products = SearchQuerySet().autocomplete(content_auto=request.POST.get('search_text', ''))
return render_to_response('sea.html', {'products':products})
When I try to indexing the my product model I ran this command
python manage.py rebuild_index
, It has not indexed. It raises
File "/home/Documents/swamy/project/env/local/lib/python2.7/site-packages/elasticsearch/helpers/init.py", line 156, in streaming_bulk
raise BulkIndexError('%i document(s) failed to index.' % len(errors), errors)
elasticsearch.helpers.BulkIndexError: ('500 document(s) failed to index.'
this error.
My model has 21000 products, Can any one help to fix this issue ?
Thanks In Advance !
Elasticsearch changed the way bulk indices are created.
You can use version 1.4.0 which works seamlessly with django-haystack.
Using Django & Haystack with ElasticSearch.
After installing haystack and ES, and Rebuilding Index
./manage.py rebuild_index
WARNING: This will irreparably remove EVERYTHING from your search index in connection 'default'.
Your choices after this are to restore from backups or rebuild via the rebuild_index command.
Are you sure you wish to continue? [y/N] y
Removing all documents from your index because you said so.
All documents removed.
Indexing 1039 <django.utils.functional.__proxy__ object at 0x10ca3ded0>.
AttributeError: 'module' object has no attribute 'ElasticSearchError'
Updating index has the same problem
/manage.py update_index
Indexing 1039 <django.utils.functional.__proxy__ object at 0x10ea49d90>.
AttributeError: 'module' object has no attribute 'ElasticSearchError'
Clear index works fine though ( probably because there is no index )
./manage.py clear_index
WARNING: This will irreparably remove EVERYTHING from your search index in connection 'default'.
Your choices after this are to restore from backups or rebuild via the `rebuild_index` command.
Are you sure you wish to continue? [y/N] y
Removing all documents from your index because you said so.
All documents removed.
Versions
django-haystack==2.0.0-beta
pyelasticsearch==0.5
elasticsearch==0.20.6
localhost:9200 says :
{
"ok" : true,
"status" : 200,
"name" : "Jigsaw",
"version" : {
"number" : "0.20.6",
"snapshot_build" : false
},
"tagline" : "You Know, for Search"
}
Haystack settings :
HAYSTACK_CONNECTIONS = {
'default': {
'ENGINE': 'haystack.backends.elasticsearch_backend.ElasticsearchSearchEngine',
'URL': 'http://127.0.0.1:9200/',
'INDEX_NAME': 'haystack',
},
}
search_indexes.py :
import datetime
import haystack
from haystack import indexes
from app.models import City
class CityIndex(indexes.SearchIndex, indexes.Indexable):
text = indexes.CharField(document=True, use_template=True)
name = indexes.CharField(model_attr='name')
state = indexes.CharField(model_attr='state')
country = indexes.CharField(model_attr='country')
lat = indexes.FloatField(model_attr='latitude')
lon = indexes.FloatField(model_attr='longitude')
alt = indexes.FloatField(model_attr='altitude')
pop = indexes.IntegerField(model_attr='population')
def get_model(self):
return City
Any help - why I am getting error ?
Solved it !
After debugging the process using pdb
./manage.py rebuild_index
At line 222 - in /haystack/backend/elasticsearch_backend.py
Changed
except (requests.RequestException, pyelasticsearch.ElasticSearchError), e:
To
# except (requests.RequestException, pyelasticsearch.ElasticSearchError), e:
except Exception as inst:
import pdb; pdb.set_trace()
I found out the core error was this
'ElasticSearch' object has no attribute 'from_python'.
To which I found solution here - https://github.com/toastdriven/django-haystack/issues/514#issuecomment-4058230
The version of pyelasticsearch I was using was from http://github.com/rhec/pyelasticsearch,
So I installed pyelasticsearch from a fork - http://github.com/toastdriven/pyelasticsearch using :
pip install --upgrade git+https://github.com/toastdriven/pyelasticsearch.git#3bfe1a90eab6c2dfb0989047212f4bc9fb814803#egg=pyelasticsearch
and That fixed it & Index was build !
I am having issues getting mongodb working with Django, my setup is Mac OS X 10.7. I followed the tutorial available here: http://django-mongodb.org/topics/setup.html . I have tried with both virtualenv and without (this is my first time installing django so I shouldn't have any conflicts).
In settings.py
DATABASES = {
'default' : {
'ENGINE' : 'django_mongodb_engine',
'NAME' : 'my_database'
}
}
In firstapp.models.py
from django.db import models
from djangotoolbox.fields import ListField
class Post(models.Model):
title = models.CharField()
text = models.TextField()
tags = ListField()
comments = ListField()
In my shell (python manage.py shell), I try running:
from mydjango.firstapp.models import Post
post = Post.objects.create();
But I keep getting the following: DatabaseError: could not connect to localhost:27017: [Errno 61] Connection refused (full traceback)
If I switch settings.py to the following:
DATABASES = {
'default': {
'ENGINE': 'django_mongodb_engine',
'NAME': 'my_database',
'USER': '',
'PASSWORD': '',
'HOST': 'localhost',
'PORT': '27017',
'SUPPORTS_TRANSACTIONS': False,
},
}
I get this error in the shell: ImproperlyConfigured: port must be an instance of int
Did you setup MongoDB separately? The howto you link doesn't seem to go over the MongoDB configuration. It assumes the database is already running. In any case MongoDB seems down, or is at least listening somewhere else.
The last error ("...instance of int") is just because you specified '27017' (a string) instead of 27017 in the configuration dictionary. But even then it should be equivalent to the first, simpler configuration.
In case MongoDB is running but you still get this error while trying to connect from another machine, it may be due to Firewall running on the MongoDB server.
I was running into this exact same error on CentOS 6.5 running MongoDB 2.6.0. Disabling firewall on the machine resolved the issue for me.
If you are creating models at models.py then there is an example below
models.py
from mongoengine import *
class UserLocation(Document):
message = StringField(required=True, max_length=200)
You dont need the sqlite3 if you are not using it and only the mongodb then you have an optio to comment it
settings.py
# DATABASES = {
# 'default': {
# 'ENGINE': 'django.db.backends.sqlite3',
# 'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
# }
# }
Now in
views.py
from .models import UserLocation
def save_function(request):
msg = "shinto"
# saving the data to the database
data = UserLocation(message = msg)
data.save()
# reading the data
read_data = json.loads(UserLocation.objects().to_json())
print read_data
There is also another method and its very simple (you don't need to create models in models.py)
views.py
from pymongo import MongoClient
client = MongoClient(port=27017)
db = client.testing_db # use a database called testing_db
collection = db.files # inside that db a collection called files
def a_new_function():
fooman = {"name" : "shinto", "age" : 25}
collection.insert(fooman)
# if you need to display only the name "shinto
#cursor = collection.find({"name" : "shinto"})
# if you need to display all then empty
cursor = collection.find({})
for document in cursor:
print(document)
could not connect to localhost:27017 Connection refused or you see 111 is because you haven't either installed or opened mongodb in another terminal
For installation on ubuntu do the following
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 2930ADAE8CAF5059EE73BB4B58712A2291FA4AD5
if Ubuntu 12.04 (deprecated) then in terminal
echo "deb [ arch=amd64 ] https://repo.mongodb.org/apt/ubuntu precise/mongodb-org/3.6 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.6.list
if Ubuntu 14.04 then in terminal
echo "deb [ arch=amd64 ] https://repo.mongodb.org/apt/ubuntu trusty/mongodb-org/3.6 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.6.list
if Ubuntu 16.04 then in terminal
echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.6 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.6.list
After do
sudo apt-get update
sudo apt-get install -y mongodb-org
After installation then start mongodb
sudo service mongod start
To stop mongodb do
sudo service mongod stop
To restart mongodb
sudo service mongod restart
Finally you can use
mongo --host 127.0.0.1:27017
which will solve the issue
Thank You
From your question it seems like you are trying to use Django with Mongodb. In which case why do you need Mongoengine?
The official Mongodb documentation talks about djongo. It works by translating SQL queries into query documents.
You don't need Mongoengine to run it.
All native Django contrib modules (eg. admin, user, session) work without any modification.
MongoEngine requires rewriting Django modules and last I checked, the native admin module didn't run on MongoEngine.
Your existing models run without any ORM translation as well.