Django: simulate HTTP requests in shell - django

I just learnt that with Rails is possible to simulate HTTP requests in the console with few lines of code.
Check out: http://37signals.com/svn/posts/3176-three-quick-rails-console-tips (section "Dive into your app").
Is there a similar way to do that with Django? Would be handy.

You can use RequestFactory, which allows
inserting a user into the request
inserting an uploaded file into the request
sending specific parameters to the view
and does not require the additional dependency of using requests.
Note that you have to specify both the URL and the view class, so it takes an extra line of code than using requests.
from django.test import RequestFactory
request_factory = RequestFactory()
my_url = '/my_full/url/here' # Replace with your URL -- or use reverse
my_request = request_factory.get(my_url)
response = MyClasBasedView.as_view()(my_request) # Replace with your view
response.render()
print(response)
To set the user of the request, do something like my_request.user = User.objects.get(id=123) before getting the response.
To send parameters to a class-based view, do something like response = MyClasBasedView.as_view()(my_request, parameter_1, parameter_2)
Extended Example
Here's an example of using RequestFactory with these things in combination
HTTP POST (to url url, functional view view, and a data dictionary post_data)
uploading a single file (path file_path, name file_name, and form field value file_key)
assigning a user to the request (user)
passing on kwargs dictionary from the url (url_kwargs)
SimpleUploadedFile helps format the file in a way that is valid for forms.
from django.core.files.uploadedfile import SimpleUploadedFile
from django.test import RequestFactory
request = RequestFactory().post(url, post_data)
with open(file_path, 'rb') as file_ptr:
request.FILES[file_key] = SimpleUploadedFile(file_name, file_ptr.read())
file_ptr.seek(0) # resets the file pointer after the read
if user:
request.user = user
response = view(request, **url_kwargs)
Using RequestFactory from a Python shell
RequestFactory names your server "testserver" by default, which can cause a problem if you're not using it inside test code. You'll see an error like:
DisallowedHost: Invalid HTTP_HOST header: 'testserver'. You may need to add 'testserver' to ALLOWED_HOSTS.
This workaround from #boatcoder's comment shows how to override the default server name to "localhost":
request_factory = RequestFactory(**{"SERVER_NAME": "localhost", "wsgi.url_scheme":"https"}).

How I simulate requests from the python command line is:
Use the excellent requests library
Use the django reverse function
A simple way of simulating requests is:
>>> from django.urls import reverse
>>> import requests
>>> r = requests.get(reverse('app.views.your_view'))
>>> r.text
(prints output)
>>> r.status_code
200
Update: be sure to launch the django shell (via manage.py shell), not a classic python shell.
Update 2: For Django <1.10, change the first line to
from django.core.urlresolvers import reverse

(See tldr; down)
Its an old question,
but just adding an answer, in case someone maybe interested.
Though this might not be the best(or lets say Django) way of doing things.
but you can try doing this way.
Inside your django shell
>>> import requests
>>> r = requests.get('your_full_url_here')
Explanation:
I omitted the reverse(),
explanation being, since reverse() more or less,
finds the url associated to a views.py function,
you may omit the reverse() if you wish to, and put the whole url instead.
For example, if you have a friends app in your django project,
and you want to see the list_all() (in views.py) function in the friends app,
then you may do this.
TLDR;
>>> import requests
>>> url = 'http://localhost:8000/friends/list_all'
>>> r = requests.get(url)

Related

Django: Stripe & POST request

I am currently trying to implement Stripe Connect in my Django project. Stripe documentations states for Standard accounts:
Assuming no error occurred, the last step is to use the provided code
to make a POST request to our access_token_url endpoint to fetch the
user’s Stripe credentials:
curl https://connect.stripe.com/oauth/token \
-d client_secret=sk_test_Dur3X2cOCwyjlf9Nr7OCf3qO \
-d code="{AUTHORIZATION_CODE}" \
-d grant_type=authorization_code
I now wonder how to send a POST request with Django without form & user action (clicking the submit button)?
Since Standard Connect relies on OAuth for its connection flow:
https://stripe.com/docs/connect/standard-accounts#oauth-flow
so you can use an OAuth python library like Rauth, as you mentioned, to handle the flow.
Also please note that Stripe Python library provides an implementation of the OAuth flow here:
https://github.com/stripe/stripe-python/blob/a938c352c4c11c1e6fee064d5ac6e49c590d9ca4/stripe/oauth.py
You can see an example of its usage here:
https://github.com/stripe/stripe-python/blob/f948b8b95b6df5b57c7444a05d6c83c8c5e6a0ac/examples/oauth.py
The example uses Flask not Django but should give you a good idea in terms of its use.
With regards to the advantages of using an existing OAuth implementation as opposed to implementing the calls directly yourself: one advantage I see is that your code would reuse a library that generally covers all different uses cases (e.g better error handling) and is also well tested.
Thanks to #psmvac I could implement it the 'proper' way now using the oAuth of Stripe. Here some reference/example Django code if anyone is trying the same. Obviously, urls.py has to be configured. This is in my views.py:
def stripe_authorize(request):
import stripe
stripe.api_key = ''
stripe.client_id = 'XYZ'
url = stripe.OAuth.authorize_url(scope='read_only')
return redirect(url)
def stripe_callback(request):
import stripe
from django.http import HttpResponse
# import requests
stripe.api_key = 'XYZ'
## import json # ?
code = request.GET.get('code', '')
try:
resp = stripe.OAuth.token(grant_type='authorization_code', code=code)
except stripe.oauth_error.OAuthError as e:
full_response = 'Error: ' + str(e)
return HttpResponse(full_response)
full_response = '''
<p>Success! Account <code>{stripe_user_id}</code> is connected.</p>
<p>Click here to
disconnect the account.</p>
'''.format(stripe_user_id=resp['stripe_user_id'])
return HttpResponse(full_response)
def stripe_deauthorize(request):
from django.http import HttpResponse
import stripe
stripe_user_id = request.GET.get('stripe_user_id', '')
try:
stripe.OAuth.deauthorize(stripe_user_id=stripe_user_id)
except stripe.oauth_error.OAuthError as e:
return 'Error: ' + str(e)
full_response = '''
<p>Success! Account <code>{stripe_user_id}</code> is disconnected.</p>
<p>Click here to restart the OAuth flow.</p>
'''.format(stripe_user_id=stripe_user_id)
return HttpResponse(full_response)

Reset cookies in Scrapy without disabling them

I use CrawlSpider to crawl a website. The website detects my spider using cookies. If I disable them, it also detects that I am a robot. So how to use new cookies in each request.
My spider is pretty simple :
# -*- coding: utf-8 -*-
import scrapy
import requests
from scrapy.spiders import CrawlSpider, Rule
from scrapy.linkextractors import LinkExtractor
class ExampleSpider(CrawlSpider):
name = 'example'
allowed_domains = ['www.example.com']
start_urls = ['http://www.example.com/items']
rules = (
Rule(LinkExtractor(allow=('/items/.'),deny=('sendMessage')), follow=True),
Rule(LinkExtractor(allow=('/item/[a-z\+]+\-[0-9]+') ,deny=('sendMessage')), callback='parse_item', follow=False),
)
def parse_item(self, response):
#parsing the page et yielding data
PS: I'm using tor to change ip every x seconds.
You can set cookies for each Request via cookies parameter. It's a bit more complicated when using CrawlSpider class, because it generates requests for you based on rules. However, you can add process_request parameter to your Rule. From the documentation:
process_request is a callable, or a string (in which case a method from the spider object with that name will be used) which will be called with every request extracted by this rule, and must return a request or None (to filter out the request).
Hence, implement that method and add cookies parameter to each request passed in and then return the modified request.
Resolved !
I use the folowing code :
def newsession(self, request):
session_id = random.randint(0,900000)
tagged = request
tagged.meta.update(cookiejar=session_id)
return tagged
In Rule I call newsession function through process_request (thanks Tomáš) :
Rule(LinkExtractor(allow=('/item/[a-z\+]+\-[0-9]+') ,deny=('sendMessage')), process_request='newsession', callback='parse_item', follow=False),

How to find the location URL in a Django response object?

Let's say I have a Django response object.
I want to find the URL (location).
However, the response header does not actually contain a Location or Content-Location field.
How do I determine, from this response object, the URL it is showing?
This is old, but I ran into a similar issue when doing unit tests. Here is how I solved the problem.
You can use the response.redirect_chain and/or the response.request['PATH_INFO'] to grab redirect urls.
Check out the documentation as well! Django Testing Tools: assertRedirects
from django.core.urlresolvers import reverse
from django.test import TestCase
class MyTest(TestCase)
def test_foo(self):
foo_path = reverse('foo')
bar_path = reverse('bar')
data = {'bar': 'baz'}
response = self.client.post(foo_path, data, follow=True)
# Get last redirect
self.assertGreater(len(response.redirect_chain), 0)
# last_url will be something like 'http://testserver/.../'
last_url, status_code = response.redirect_chain[-1]
self.assertIn(bar_path, last_url)
self.assertEqual(status_code, 302)
# Get the exact final path from the response,
# excluding server and get params.
last_path = response.request['PATH_INFO']
self.assertEqual(bar_path, last_path)
# Note that you can also assert for redirects directly.
self.assertRedirects(response, bar_path)
The response does not decide the url, the request does.
The response gives you the content of the response, not the url of it.

How to use session in TestCase in Django?

I would like to read some session variables from a test (Django TestCase)
How to do that in a clean way ?
def test_add_docs(self):
"""
Test add docs
"""
# I would access to the session here:
self.request.session['documents_to_share_ids'] = [1]
response = self.client.get(reverse(self.document_add_view_id, args=[1]), follow=True)
self.assertEquals(response.status_code, 200)
As of Django 1.7+ this is much easier. Make sure you set the session as a variable instead of referencing it directly.
def test_something(self):
session = self.client.session
session['somekey'] = 'test'
session.save()
andreaspelme's workaround is only needed in older versions of django. See docs
Unfortunately, this is not a easy as you would hope for at the moment. As you might have noticed, just using self.client.session directly will not work if you have not called other views that has set up the sessions with appropriate session cookies for you. The session store/cookie must then be set up manually, or via other views.
There is an open ticket to make it easier to mock sessions with the test client: https://code.djangoproject.com/ticket/10899
In addition to the workaround in the ticket, there is a trick that can be used if you are using django.contrib.auth. The test clients login() method sets up a session store/cookie that can be used later in the test.
If you have any other views that sets sessions, requesting them will do the trick too (you probably have another view that sets sessions, otherwise your view that reads the sessions will be pretty unusable).
from django.test import TestCase
from django.contrib.auth.models import User
class YourTest(TestCase):
def test_add_docs(self):
# If you already have another user, you might want to use it instead
User.objects.create_superuser('admin', 'foo#foo.com', 'admin')
# self.client.login sets up self.client.session to be usable
self.client.login(username='admin', password='admin')
session = self.client.session
session['documents_to_share_ids'] = [1]
session.save()
response = self.client.get('/') # request.session['documents_to_share_ids'] will be available
If you need to initialize a session for the request in tests to manipulate it directly:
from django.contrib.sessions.middleware import SessionMiddleware
from django.http import HttpRequest
request = HttpRequest()
middleware = SessionMiddleware()
middleware.process_request(request)
request.session.save()
If testing my django project with pytest, I can not see any modifications to the session that are made in the view. (That is because the Sessions middleware doesn't get called.)
I found the following approach to be useful:
from unittest.mock import patch
from django.test import Client
from django.contrib.sessions.backends import db
def test_client_with_session():
client = Client()
session = {} # the session object that will persist throughout the test
with patch.object(db, "SessionStore", return_value=session):
client.post('/url-that-sets-session-key/')
assert session['some_key_set_by_the_view']
client.get('/url-that-reads-session-key/')
This approach has the benefit of not requiring database access.
You should be able to access a Client's session variales through its session property, so I guess self.client.session['documents_to_share_ids'] = [1] should be what you are looking for!

Django Testing: determine which view was executed

In the Django testing documentation they promise that you can "Test that the correct view is executed for a given URL."
However I didn't find any possibility how to test which view was executed. I would expect that in the Response class but there's nothing about the executed view.
Thanks in advance.
You can extract the view function name thusly
from django.test.client import Client
c = Client()
response = c.get('/')
from django.core.urlresolvers import resolve
resolve(response.request["PATH_INFO"])[0].func_name
Dave's answer involves a HTTP request each time you're testing a url, which can be slow. If you just want to know what view a url resolves to, you can do that without using Client:
>>> from django.core.urlresolvers import get_resolver
>>> from myapp.views import func_to_test
>>> resolver = get_resolver(None)
>>> match = resolver.resolve('/some/path/')
>>> if match.func is func_to_test:
>>> print "correct function for resolution!"
Ryan Wilcox's post on route testing goes into more detail and provides techniques for making it even easier to test them.