Django: custom 404 handler that returns 404 status code - django

The project I'm working on has some data that needs to get passed to every view, so we have a wrapper around render_to_response called master_rtr. Ok.
Now, I need our 404 pages to run through this as well. Per the instructions, I created a custom 404 handler (cleverly called custom_404) that calls master_rtr. Everything looks good, but our tests are failing, because we're receiving back a 200 OK.
So, I'm trying to figure out how to return a 404 status code, instead. There seems to be an HttpResponseNotFound class that's kinda what I want, but I'm not quite sure how to construct all of that nonsense instead of using render_to_response. Or rather, I could probably figure it out, but it seems like their must be an easier way; is there?
The appropriate parts of the code:
def master_rtr(request, template, data = {}):
if request.user.is_authenticated():
# Since we're only grabbing the enrollments to get at the courses,
# doing select_related() will save us from having to hit database for
# every course the user is enrolled in
data['courses'] = \
[e.course for e in \
Enrollment.objects.select_related().filter(user=request.user) \
if e.view]
else:
if "anonCourses" in request.session:
data['courses'] = request.session['anonCourses']
else:
data['courses'] = []
data['THEME'] = settings.THEME
return render_to_response(template, data, context_instance=RequestContext(request))
def custom_404(request):
response = master_rtr(request, '404.html')
response.status_code = 404
return response

The easy way:
def custom_404(request):
response = master_rtr(...)
response.status_code = 404
return response
But I have to ask: why aren't you just using a context processor along with a RequestContext to pass the data to the views?

Just set status_code on the response.

Into your application's views.py add:
# Imports
from django.shortcuts import render
from django.http import HttpResponse
from django.template import Context, loader
##
# Handle 404 Errors
# #param request WSGIRequest list with all HTTP Request
def error404(request):
# 1. Load models for this view
#from idgsupply.models import My404Method
# 2. Generate Content for this view
template = loader.get_template('404.htm')
context = Context({
'message': 'All: %s' % request,
})
# 3. Return Template for this view + Data
return HttpResponse(content=template.render(context), content_type='text/html; charset=utf-8', status=404)
The secret is in the last line: status=404
Hope it helped!

Related

Django SAML logout failing

In my Django project I use python3-saml to login with SSO. The login works like expected but the logout is failing with an error message 'No hostname defined'. I really don't know how to solve this as the only parameter passed to logout is the request and request is missing 'http_host' and 'server_name', read here.
My logout part looks like following:
def get(self, request, pkuser=None):
try:
get_user_model().objects.get(pk=pkuser)
except get_user_model().DoesNotExist:
return redirect('HomePage')
logger = logging.getLogger('iam')
logger.info('IAM logout')
auth = OneLogin_Saml2_Auth(request, custom_base_path=settings.SAML_FOLDER)
logger.info('account logout')
# OneLogin_Saml2_Utils.delete_local_session()
try:
auth.logout(
name_id=request.session['samlNameId'],
session_index=request.session['samlSessionIndex'],
nq=request.session['samlNameIdNameQualifier'],
name_id_format=request.session['samlNameIdFormat'],
spnq=request.session['samlNameIdSPNameQualifier']
)
logger.info('account logout success')
OneLogin_Saml2_Utils.delete_local_session()
logger.info('account logout: deleted local session')
except Exception as e:
logger.info('account logout failed: {}'.format(str(e)))
logout(request)
return redirect('HomePage')
Maybe I'm using the wrong package...? Any help or advice will be appreciated.
I think this is happening because the logout method is missing http_host and server_name.
To fix this issue modify the request object to include the http_host and server_name attributes before calling the logout method.
def get(self, request, pkuser=None):
try:
get_user_model().objects.get(pk=pkuser)
except get_user_model().DoesNotExist:
return redirect('HomePage')
logger = logging.getLogger('iam')
logger.info('IAM logout')
auth = OneLogin_Saml2_Auth(request, custom_base_path=settings.SAML_FOLDER)
logger.info('account logout')
request.http_host = 'your_http_host'
request.server_name = 'your_server_name'
try:
auth.logout(
name_id=request.session['samlNameId'],
session_index=request.session['samlSessionIndex'],
nq=request.session['samlNameIdNameQualifier'],
name_id_format=request.session['samlNameIdFormat'],
spnq=request.session['samlNameIdSPNameQualifier']
)
logger.info('account logout success')
OneLogin_Saml2_Utils.delete_local_session()
logger.info('account logout: deleted local session')
except Exception as e:
logger.info('account logout failed: {}'.format(str(e)))
logout(request)
return redirect('HomePage')
In the handler for your ACS URL you will also be creating an object from OneLogin_Saml2_Auth(), which theoretically is working. Check if the setup for the request to that is different to the setup here.
One thing that stands out between this and my code is that my one has a line prior to the constructor request_annotated = saml2_prepare_request(request) (all the names in this line will likely be different for you other than request, but the format would remain the same) where you are passing request_annotated to the OneLogin_Saml2_Auth constructor rather than request. If so, duplicate that line. The library is expecting that there are some specific things annotated to the request dictionary. The code for this function in my Django is:
def saml2_prepare_request(request):
return {
'http_host': request.META['HTTP_HOST'],
'script_name': request.META['PATH_INFO'],
'server_port': request.META['SERVER_PORT'],
'get_data': request.GET.copy(),
'post_data': request.POST.copy()
}

Django - Get response to return image without saving file to models

I am using the tmdb api to return movie info as well as images.
Steps below of Get logic
Api request is made which provides movie info as well as "backdrop_path"
I then use this path to make another request for the jpg related to that movie.
Blocker
I'm unable to then output that jpg. It currently returns a url path as below.
Views.py
from django.shortcuts import render
from django.views.generic import TemplateView
import requests
import urllib
# Create your views here.
def index(request):
# Query API with user input
if 'movie' in request.GET:
api_key = 'api'
id = request.GET['movie']
url = 'https://api.themoviedb.org/3/search/movie?api_key={}&language=en-US&query={}&include_adult=false'
response = requests.get(url.format(api_key,id))
# successful request
if response.status_code == 200:
# Parse json output for key value pairs
tmdb = response.json()
# save image jpg
backdrop_path = tmdb['results'][0]['backdrop_path']
url = 'https://image.tmdb.org/t/p/original/{}'
gg = urllib.request.urlretrieve(url.format(backdrop_path), 'test.jpg')
context = {
'title': tmdb['results'][0]['original_title'],
'overview': tmdb['results'][0]['overview'],
'release_date': tmdb['results'][0]['release_date'],
'vote_average': tmdb['results'][0]['vote_average'],
'vote_count': tmdb['results'][0]['vote_count'],
'backdrop_path' : tmdb['results'][0]['backdrop_path'],
'jpg' : gg
}
return render(request, 'home.html', {'context': context})
else: # returns homepage if invalid request
return render(request, 'home.html')
else: # Homepage without GET request
return render(request, 'home.html')
urlretrieve doesn't return the image ready to be used, instead it returns a tuple with the local name of the file and the headers (as an HTTPMessage object as you can see in your example), which is what you're seeing.
However, I don't think returning a file in your response is ideal, nor it would work in your scenario. Since you seem to be using templates, what I would do is return the image url and use it in an image HTML tag, like this <img src="{{ jpg }}"/>

Django -- Views MultiValueDictKeyError

I want to return 4 different versions of the homepage
Homepage with search bar. No data present from API
Homepage with search bar. Data present from API
Homepage with search bar. No data present if request doesn't exist in API
Homepage with search bar. No data present if submit button is hit without any data being entered.
Version two, three and four all work.
However version 1, the homepage without a GET request is not returned. Due to:
MultiValueDictKeyError at / 'city'" in the views.py file.
How can this be resolved? Any help will be greatly appreciated
urls.py
from django.urls import path
from . import views
urlpatterns = [
path('', views.index),
]
views.py
from django.shortcuts import render
import requests
def index(request):
# Query API with user input
payload = {'q': request.GET['city'], 'appid': 'API-KEY'}
response = requests.get('http://api.openweathermap.org/data/2.5/weather', params=payload)
# successful request
if response.status_code == 200:
# Parse json output for key value pairs
e = response.json()
context = {
'city_name': e['name'],
'weather':e['weather'][0]['main'],
'description' : e['weather'][0]['description'],
'temp' : e['main']['temp'],
'pressure':e['main']['pressure'],
'humidity':e['main']['humidity'],
'visibility':e['visibility'],
'wind_speed':e['wind']['speed'],
'wind_deg':e['wind']['deg']
}
return render(request, 'index.html', {'context': context})
else: # returns homepage if invalid city name is given in form
return render(request, 'index.html')
Instead of calling request.GET['city'] directly, check if city is set first, like:
if 'city' in request.GET:
payload = {'q': request.GET['city'], 'appid': 'API-KEY'}

How to scrape pages after login

I try to find a way to scrape and parse more pages in the signed in area.
These example links accesible from signed in I want to parse.
#http://example.com/seller/demand/?id=305554
#http://example.com/seller/demand/?id=305553
#http://example.com/seller/demand/?id=305552
#....
I want to create spider that can open each one of these links and then parse them.
I have created another spider which can open and parse only one of them.
When I tried to create "for" or "while" to call more requests with other links it allowed me not because I cannot put more returns into generator, it returns error. I also tried link extractors, but it didn't work for me.
Here is my code:
#!c:/server/www/scrapy
# -*- coding: utf-8 -*-
from scrapy import Spider
from scrapy.selector import Selector
from scrapy.http import FormRequest
from scrapy.http.request import Request
from scrapy.spiders import CrawlSpider, Rule
from array import *
from stack.items import StackItem
from scrapy.linkextractors import LinkExtractor
class Spider3(Spider):
name = "Spider3"
allowed_domains = ["example.com"]
start_urls = ["http://example.com/login"] #this link lead to login page
When I am signed in it returns page with url, that contains "stat", that is why I put here first "if" condition.
When I am signed in, I request one link and call function parse_items.
def parse(self, response):
#when "stat" is in url it means that I just signed in
if "stat" in response.url:
return Request("http://example.com/seller/demand/?id=305554", callback = self.parse_items)
else:
#this succesful login turns me to page, it's url contains "stat"
return [FormRequest.from_response(response,
formdata={'ctl00$ContentPlaceHolder1$lMain$tbLogin': 'my_login', 'ctl00$ContentPlaceHolder1$lMain$tbPass': 'my_password'},callback=self.parse)]
Function parse_items simply parse desired content from one desired page:
def parse_items(self,response):
questions = Selector(response).xpath('//*[#id="ctl00_ContentPlaceHolder1_cRequest_divAll"]/table/tr')
for question in questions:
item = StackItem()
item['name'] = question.xpath('th/text()').extract()[0]
item['value'] = question.xpath('td/text()').extract()[0]
yield item
Can you help me please to update this code to open and parse more than one page in each sessions?
I don't want to sign in over and over for each request.
The session most likely depends on the cookies and scrapy manages that by itself. I.e:
def parse_items(self,response):
questions = Selector(response).xpath('//*[#id="ctl00_ContentPlaceHolder1_cRequest_divAll"]/table/tr')
for question in questions:
item = StackItem()
item['name'] = question.xpath('th/text()').extract()[0]
item['value'] = question.xpath('td/text()').extract()[0]
yield item
next_url = '' # find url to next page in the current page
if next_url:
yield Request(next_url, self.parse_items)
# scrapy will retain the session for the next page if it's managed by cookies
I am currently working on the same problem. I use InitSpider so I can overwrite __init__ and init_request. The first is just for initialisation of custom stuff and the actual magic happens in my init_request:
def init_request(self):
"""This function is called before crawling starts."""
# Do not start a request on error,
# simply return nothing and quit scrapy
if self.abort:
return
# Do a login
if self.login_required:
# Start with login first
return Request(url=self.login_page, callback=self.login)
else:
# Start with pase function
return Request(url=self.base_url, callback=self.parse)
My login looks like this
def login(self, response):
"""Generate a login request."""
self.log('Login called')
return FormRequest.from_response(
response,
formdata=self.login_data,
method=self.login_method,
callback=self.check_login_response
)
self.login_data is a dict with post values.
I am still a beginner with python and scrapy, so I might be doing it the wrong way. Anyway, so far I have produced a working version that can be viewed on github.
HTH:
https://github.com/cytopia/crawlpy

How to set cookie in Django and then render template?

I want to set a cookie inside a view and then have that view render a template. As I understand it, this is the way to set a cookie:
def index(request):
response = HttpResponse('blah')
response.set_cookie('id', 1)
return response
However, I want to set a cookie and then render a template, something like this:
def index(request, template):
response_obj = HttpResponse('blah')
response_obj.set_cookie('id', 1)
return render_to_response(template, response_obj) # <= Doesn't work
The template will contain links that when clicked will execute other views that check for the cookie I'm setting. What's the correct way to do what I showed in the second example above? I understand that I could create a string that contains all the HTML for my template and pass that string as the argument to HttpResponse but that seems really ugly. Isn't there a better way to do this? Thanks.
This is how to do it:
from django.shortcuts import render
def home(request, template):
response = render(request, template) # django.http.HttpResponse
response.set_cookie(key='id', value=1)
return response
The accepted answer sets the cookie before the template is rendered. This works.
response = HttpResponse()
response.set_cookie("cookie_name", "cookie_value")
response.write(template.render(context))
If you just need the cookie value to be set when rendering your template, you could try something like this :
def view(request, template):
# Manually set the value you'll use for rendering
# (request.COOKIES is just a dictionnary)
request.COOKIES['key'] = 'val'
# Render the template with the manually set value
response = render(request, template)
# Actually set the cookie.
response.set_cookie('key', 'val')
return response
response = render(request, 'admin-dashboard.html',{"email":email})
#create cookies
expiry_time = 60 * 60 #in seconds
response.set_cookie("email_cookie",email,expiry_time);
response.set_cookie("classname","The easylearn academy",expiry_time);
def index(request, template):
response = HttpResponse('blah')
response.set_cookie('id', 1)
id = request.COOKIES.get('id')
return render_to_response(template,{'cookie_id':id})