I have a Django app that has two view requests. The first request ("add_item") is the add request, which does some validation and then ultimately adds an item to a user's queue. The second request hits a user's facebook feed, iterates through the posts, and adds some of them to their queue. Ideally, I don't want to replicate the logic for the add in that second request. I'd rather just call the first request within the second request's loop.
I can't figure out how to do this and still pass in the user's context in the request. Does anyone have an example of how to do this? I tried passing in the POST data and the user context that the first request requires:
request = { "POST": { "queue_id": 1, "data": "some_data_to_add" }, "user": request.user }
add_item(request)
However, request.user doesn't get serialized properly. I'm also pretty sure I can't just pass the POST data that way, I'm sure it also needs to be serialized somehow. In general, I think this is not the standard way to do something like this.
Create a separate a view which perform the task of adding items into the queue and call that function from both first and second view.
for example:
def adding_items(<necessary params>):
item add logic here.
def first_view(request):
# call adding items to update queue.
adding_items( <params> )
def second_view(request):
# call adding items to update queue.
adding_items( <params> )
Thats it :)
Related
In views, I have a function defined which is executed when the user submits the form online. After the form submission there are some database transactions that I perform and then based on the existing data in the database API's are triggered:
triggerapi():
execute API to send Email to the user and the administrator about
the submitted form
def databasetransactions():
check the data in the submitted form with the data in DB
if the last data submitted by the user is before 10 mins or more:
triggerapi()
def formsubmitted(request):
save the user input in variables
Databasetransactions()
save the data from the submitted form in the DB
In the above case, the user clicks on submit button 2 times in less than 5 milliseond duration. So 2 parallel data starts to process and both trigger Email which is not the desired behavior.
Is there a way to avoid this ? So that for a user session, the application should only accept the data once all the older data processing is completed ?
Since we are talking in pseudo-code, one way could be to use a singleton pattern for triggerapi() and return Not Allowed in case it is already istantiated.
There are multiple ways to solve this issue.
One of them would be to create a new session variable
request.session['activetransaction'] = True
This would however require you to pass request, unless it is already passed and we got a changed code portion. You can also add an instance/ class flag for it in the same way and check with it.
Another way, which might work if you need those submissions handled after the previous one, you can always add a while request.session['activetransaction']: and do the handling afterwards.
def formsubmitted(request):
if 'activetransaction' not in request.session or not request.session['activetransaction']:
request.session['activetransaction'] = True
# save the user input in variables
Databasetransactions()
# save the data from the submitted form in the DB
request.session['activetransaction'] = False
...
I have a set of URLs:
/home/
/register/
/login/
/puzzle/<pk>
All of the first 3 urls can make a request to the last url.
Is it possible to know which urls are calling the /puzzle/<pk>, from the view function attached to it?
Edit
So the problem is it's a puzzle game and every user has a level. If he completes a certain level only then he can proceed to the next level. Therefore, if a user has completed level 3, I'll always show him the html page with the url /puzzle/4.
The flow is :
A user registers. His level is 0. As soon as he registers he'll be redirected to puzzle/1/
A user logs in. If he has completed level x, as soon as he logs in he'll be redirected to /puzzle/x+1.
I've handled these 2 types.
But the problem is, say the user is in the page /puzzle/1/. Now if he manually changes the url to /puzzle/2/, from the view function attached to /puzzle/<pk>/, how can I handle the above case?
I believe you're looking for request.resolver_match. You could access the view name with request.resolver_match.view_name
Edit:
If I understand your model correctly, you need some logic like the following. You'll need to adjust the view name and params in reverse.
def view(request, puzzle_level):
if puzzle_level != request.user.level_completed + 1:
return redirect(
reverse(
'puzzle_view_name',
kwargs={'puzzle_level': request.user.level_completed + 1},
)
)
...
I need to scrap the data of each item from a website using Scrapy(http://example.com/itemview). I have a list of itemID and I need to pass it in a form in example.com.
There is no url change for each item. So for each request in my spider the url will always be the same. But the content will be different.
I don't wan't a for loop for handling each request. So i followed the below mentioned steps.
started spider with the above url
added item_scraped and spider_closed signals
passed through several functions
passed the scraped data to pipeline
trigerred the item_scraped signal
After this it automatically calls the spider_closed signal. But I want the above steps to be continued till the total itemID are finished.
class ExampleSpider(scrapy.Spider):
name = "example"
allowed_domains = ["example.com"]
itemIDs = [11111,22222,33333]
current_item_num = 0
def __init__(self, itemids=None, *args, **kwargs):
super(ExampleSpider, self).__init__(*args, **kwargs)
dispatcher.connect(self.item_scraped, signals.item_scraped)
dispatcher.connect(self.spider_closed, signals.spider_closed)
def spider_closed(self, spider):
self.driver.quit()
def start_requests(self):
request = self.make_requests_from_url('http://example.com/itemview')
yield request
def parse(self,response):
self.driver = webdriver.PhantomJS()
self.driver.get(response.url)
first_data = self.driver.find_element_by_xpath('//div[#id="itemview"]').text.strip()
yield Request(response.url,meta={'first_data':first_data},callback=self.processDetails,dont_filter=True)
def processDetails(self,response):
itemID = self.itemIDs[self.current_item_num]
..form submission with the current itemID goes here...
...the content of the page is updated with the given itemID...
yield Request(response.url,meta={'first_data':response.meta['first_data']},callback=self.processData,dont_filter=True)
def processData(self,response):
...some more scraping goes here...
item = ExamplecrawlerItem()
item['first_data'] = response.meta['first_data']
yield item
def item_scraped(self,item,response,spider):
self.current_item_num += 1
#i need to call the processDetails function here for the next itemID
#and the process needs to contine till the itemID finishes
self.parse(response)
My piepline:
class ExampleDBPipeline(object):
def process_item(self, item, spider):
MYCOLLECTION.insert(dict(item))
return
I wish I had an elegant solution to this. But instead it's a hackish way of calling the underlying classes.
self.crawler.engine.slot.scheduler.enqueue_request(scrapy.Request(url,self.yourCallBack))
However, you can yield a request after you yield the item and have it callback to self.processDetails. Simply add this to your processData function:
yield item
self.counter += 1
yield scrapy.Request(response.url,callback=self.processDetails,dont_filter=True, meta = {"your":"Dictionary"}
Also, PhantomJS can be nice and make your life easy, but it is slower than regular connections. If possible, find the request for json data or whatever makes the page unparseable without JS. To do so, open up chrome, right click, click inspect, go to the network tab, then enter the ID into the form, then look at the XHR or JS tabs for a JSON that has the data or next url you want. Most of the time, there will be some url made by adding the ID, if you can find it, you can just concatenate your urls and call that directly without having the cost of JS rendering. Sometimes it is randomized, or not there, but I've had fair success with it. You can then also use that to yield many requests at the same time without having to worry about phantomJS trying to do two things at once or having to initialize many instances of it. You could use tabs, but that is a pain.
Also, I would use a Queue of your IDs to ensure thread safety. Otherwise, you could have processDetails called twice on the same ID, though in the logic of your program everything seems to go linearly, which means you aren't using the concurrency capabilities of Scrapy and your program will go more slowly. To use Queue add:
import Queue
#go inside class definition and add
itemIDQueue = Queue.Queue()
#within __init__ add
[self.itemIDQueue.put(ID) for ID in self.itemID]
#within processDetails replace itemID = self.itemIDs[self.current_item_num] with
itemID = self.itemIDQueue.get()
And then there is no need to increment the counter and your program is thread safe.
I'm trying to call a view directly from another (if this is at all possible). I have a view:
def product_add(request, order_id=None):
# Works. Handles a normal POST check and form submission and redirects
# to another page if the form is properly validated.
Then I have a 2nd view, that queries the DB for the product data and should call the first one.
def product_copy_from_history(request, order_id=None, product_id=None):
product = Product.objects.get(owner=request.user, pk=product_id)
# I need to somehow setup a form with the product data so that the first
# view thinks it gets a post request.
2nd_response = product_add(request, order_id)
return 2nd_response
Since the second one needs to add the product as the first view does it I was wondering if I could just call the first view from the second one.
What I'm aiming for is just passing through the request object to the second view and return the obtained response object in turn back to the client.
Any help greatly appreciated, critism as well if this is a bad way to do it. But then some pointers .. to avoid DRY-ing.
Thanx!
Gerard.
My god, what was I thinking. This would be the cleanest solution ofcourse:
def product_add_from_history(request, order_id=None, product_id=None):
""" Add existing product to current order
"""
order = get_object_or_404(Order, pk=order_id, owner=request.user)
product = Product.objects.get(owner=request.user, pk=product_id)
newproduct = Product(
owner=request.user,
order = order,
name = product.name,
amount = product.amount,
unit_price = product.unit_price,
)
newproduct.save()
return HttpResponseRedirect(reverse('order-detail', args=[order_id]) )
A view is a regular python method, you can of course call one from another giving you pass proper arguments and handle the result correctly (like 404...). Now if it is a good practice I don't know. I would myself to an utiliy method and call it from both views.
If you are fine with the overhead of calling your API through HTTP you can use urllib to post a request to your product_add request handler.
As far as I know this could add some troubles if you develop with the dev server that comes with django, as it only handles one request at a time and will block indefinitely (see trac, google groups).
I have written a function having the following signature:
def action_handler(request, model):
This action_handler is used from different views and handles the actions for this views. One example is deleting objects. In this example the user selectes some objects, selects the delete action and then the user is presented a page to check whether he/she wants to really delete the selected objects. This is done by the following code:
context = {
'action_name' : selected_action,
'object_list' : object_list,
}
return render_to_response("crm/object_delete_check.html", context,
context_instance=RequestContext(request))
For the case that something goes wrong I want to redirect the user to the view from where the user called the action.
Thus I want to ask here whether it is possible to get the calling view from the request object or somewhere else from.
If the def "def action_handler(request, model):" is called from the view "contacts(request):" then i want to redirect the user to the view "contacts(request):" .
But the clue is I do not want to hard-code it since the def action_handler is called from different views. Using a simple "return" is also not possible, since I want to recall the view completely.
if goback: #goback being whatever criteria means "something went wrong"
default_back_url = "someurl_in_case_the_meta_is_messed_up"
back = request.META.get('HTTP_REFERER',default_back_url) #yeah they spelled referrer wrong
if back:
return HttpResponseRedirect(back)
else:
return HttpResponseRedirect(default_back_url)
while META can be faked, it's harder to fake than GET query strings.
You can pass previous page url through GET parameter:
/object_delete_check/?previous=/contacts/
(see contrib.auth.decorators.login_required for example)