Django view getting called twice (double GET request) - django

I'm creating a classifieds website in Django. A single view function handles global listings, city-wise listings, barter-only global listings and barter-only city-wise listings. This view is called ads.
The url patterns are written in the following order (note that each has a unique name although it's tied to the same ads view):
urlpatterns = patterns('',
url(r'^buy_and_sell/$', ads,name='classified_listing'),
url(r'^buy_and_sell/barter/$', ads,name='barter_classified_listing'),
url(r'^buy_and_sell/barter/(?P<city>[\w.#+-]+)/$', ads,name='city_barter_classified_listing'),
url(r'^buy_and_sell/(?P<city>[\w.#+-]+)/$', ads,name='city_classified_listing'),
)
The problem is that when I hit the url named classified_listing in the list above, the function ads gets called twice. I.e. here's what I see in my terminal:
[14/Jul/2017 14:31:08] "GET /buy_and_sell/ HTTP/1.1" 200 53758
[14/Jul/2017 14:31:08] "GET /buy_and_sell/None/ HTTP/1.1" 200 32882
This means double the processing. I thought urls.py returns the first url pattern matched. What am I doing wrong and what's the best way to fix this? All other calls work as expected btw (i.e. only once).
Note: Ask for more information in case I've missed something.
Great explanation to understand these type of occurences: https://groups.google.com/d/msg/django-users/CRMMYWix_60/KEIkguUcqxYJ

This issue has nothing to do with how url patterns are ordered in urls.py.
Like pointed out in the comments under the question, this has to do with problematic asset references in the HTML template.
What does that mean?
For instance, try curl -i http://localhost:8000/example/ >> output.txt in your terminal. Then open up output.txt in your editor of choice. Now search for href or src attributes where values are None (or otherwise malformed). That's one reason a double call is being created. That was the reason for me. I removed these, and the double call disappeared.
There's this old - but relevant - writeup about how to comprehensively diagnose this problem on your machine here: https://groups.google.com/forum/#!msg/django-users/CRMMYWix_60/KEIkguUcqxYJ
Happy testing.

As I can't comment on other answers, just to add for future wanderers that for me the "problem" was in a correctly formed but yet for the browser instructing <iframe src="#"..> tag. On django server the view was rendering twice, once with original request and then again by the hidden iframe element that I used for some of the modal popups later in the page usage.
After emptying the src attribute like <iframe src=""..> a second request is no longer initiated and my modals work fine.
The solution actually is from the link posted already in answers before [https://groups.google.com/forum/#!msg/django-users/CRMMYWix_60/KEIkguUcqxYJ][1]
where it is explained:
Note that it's a URI. That means something that is retrieved. Since
you've used the value "#fff", that will be interpreted by the browser as
a reference to the current page (#fff being an anchor, and not passed to
the server). Ergo, a second request is made.
that the iframe src # (anchor) is instructing the browser to load again the same URL, for the iframe element in my case.
I indeed had several style elements with #fff colors inside and whatnot, but this wasn't it, as browsers are smart enough to recognize this is not an anchor.
With available tools (browser only) I found to be easy to debug and find these initiation href/src attributes over the Network tab of your browser developer tools - in Chrome is just by clicking the Initiator link of the corresponding row - giving you the exact line from the page source that initiated the request to the same URL.

I struggled with the same problem and just wanted to share my experience with it. I had double requests all over my application but everything seemed to work as expected apart form it.
What Daniel Rossman pointet out in the comments was actually also true for my problem. I had a <link rel="shortcut icon" href="#"> in my base template which caused the double request, because of the #, which is a reference to the page itself. Once i removed it, i had no double requests anymore.
Hope this answer can save someone some debugging time.

I got double request in view function, in my scenario, this went wrong:
<img id="profile-img" src="#" alt="" class="profile-cover">
by setting src="" dismiss double request. it was a silly thing, I just thought it apply to a then must apply to img, but img actually send another request.

Related

Django "Change Page" loading in strange, single-letter scripts, all with 404 error

This issue is really strange. I am in the Django Admin, updating some data.
The "Change page" loads everything ok at first, but then certain Javascript behaviors aren't behaving properly, so I go to check the Javascript console, and I see 22 errors:
First notice the url (in yellow) of my Admin "Change Page":
and now see all the errors generated from that page (again, notice how the yellow is highlighting the same url as the "Change Page" form above:
Now the strangest part of all:
When I click on the last error, which is:
localhost/:1 Uncaught SyntaxError: Unexpected token <
Instead of showing me source code from the Django Change Page, it shows me markup from one of my Custom Templates instead! That should not be happening!
Any idea what could be going on here?
I figured out the problem.
I had been pulling in jQuery into my Admin Page with the following snippet:
class PageAdmin(admin.ModelAdmin):
# force jquery onto the page
class Media:
js = ('templates/js/vendor/jquery-1.12.2.min.js')
But because this js = () wasn't technically a tuple (it was missing a comma), it was trying to load scripts onto my admin page like this (notice the last letter in the example url is trying to spell out my string in the code snippet above):
example/t
example/e
example/m
example/p
example/l
example/a
example/t
example/e
example/s
example/
...
This is why it was pulling in my custom template, because it was hitting the url example/, which would have loaded in my custom index page.
Mystery solved! I hope this can help someone in the future.

content empty when using scrapy

Thanks for everyone in advance.
I encountered a problem when using Scrapy on Python 2.7.
The webpage I tried to crawl is a discussion board for Chinese stock market.
When I tried to get the first number "42177" just under the banner of this page (the number you see on that webpage may not be the number you see in the picture shown here, because it represents the number of times this article has been read and is updated realtime...), I always get an empty content. I am aware that this might be the dynamic content issue, but yet don't have a clue how to crawl it properly.
The code I used is:
item["read"] = info.xpath("div[#id='zwmbti']/div[#id='zwmbtilr']/span[#class='tc1']/text()").extract()
I think the xpath is set correctly and I have checked the return value of this response and it indeed told me that there is nothing under this directory. Results shown here:'read': [u'<div id="zwmbtilr"></div>']
If it has something, there should be something between <div id="zwmbtilr"> and </div>.
Really appreciated if you guys share any thoughts on this!
I just opened your link in Firefox with NoScript enabled. There nothing inside the <div #id='zwmbtilr'></div>. If I enable the javascripts, I can see the content you want. So, as you already new, it is a dynamic content issue.
Your first option is try to identify the request generated by javascript. If you can do that, you can send the same request from scrapy. If you can't do it, the next option is usually to use some package with javascript/browser emulation or someting like that. Something like ScrapyJS or Scrapy + Selenium.

django - ckeditor bug renders text/string in raw html format

am using django ckeditor. Any text/content entered into its editor renders raw html output on the webpage.
for ex: this is rendered output of ckeditor field (RichTextField) on a webpage;
<p><span style="color:rgb(0, 0, 0)">this is a test file ’s forces durin</span><span style="color:rgb(0, 0, 0)">galla’s good test is one that fails Thereafter, never to fail in real environment. </span></p>
I have been looking for a solution for a long time now but unable to find one :( There are some questions which are similar but none of those have been able to help. It will be helpful if any changes suggested are provided with the exact location where it needs to be changed. Needless to say I am a newbie.
Thanks
You need to mark the relevant variable that contains the html snippet in your template as safe
Obviously you should be sure, that the text comes from trusted users and is safe, because with the safe filter you are disabling a security feature (autoescaping) that Django applies per default.
If your ckeditor is part of a comment form and your mark the entered text as safe, anybody with access to the form could inject Javascipt and other (potentially nasty) stuff in your page.
The whole story is explained pretty well in the official docs: https://docs.djangoproject.com/en/dev/topics/templates/#automatic-html-escaping

How do you acquire the rewritten CFWheels URL of an given page?

CFWheels has the URLFor() function for getting the internal URL based on supplied arguments. Is there a way to get the internal URL without supplying any arguments?
For example:
Given a user navigates to "http://somedomain.com" or "http://somedomain.com/about/" or "http://somedomain.com/contact/" is there a method like ReWrittenURL() that returns something like "/" or "/about/" or "/contact/"?
Using URLFor() with no arguments returns "/home/index" or "/about/index" or "/contact/index".
CGI.SCRIPT_NAME returns "/rewrite.cfm"
Obviously with Javascript using document.location.href I can get what I'm after.
Does CGI.path_info have the value you're looking for?
edit
At first, I deleted this post, being utterly confounded. Now I've done a little test - I downloaded the latest wheels core files (1.1.6), extracted to an IIS 7.5 (with URL Rewrite module installed) + CF9 webserver, and edited the "web.config" file in the core root, setting "enabled='true'" for the rewrite rule. Also, since I was running this example from a subfolder, I changed the path from "/rewrite.cfm" to just "rewrite.cfm". This got me to the point where I was able to successfully requests urls like this:
http://server/wheelstest/wheels/wheels
From here, I edited the layout.cfm under views/wheels, adding:
<cfdump var="#cgi#">
When I then request the above URL (/wheelstest/wheels/wheels), I see the dump for the cgi scope. Under path_info, this is the value: /wheels/wheels.
Next, I added a blank "index.cfm" file under views/wheels.
When I request /wheelstest/wheels, I get this for path_info: "/wheels".
When I request /wheelstest/wheels/, I get this for path_info: "/wheels/".
When I request /wheelstest/wheels/index, I get this for path_info: "/wheels/index".
When I request /wheelstest/wheels/index/, I get this for path_info: "/wheels/index/".
So basically - cgi.path_info is doing for me exactly what you describe you want. What is different about your setup than mine, such that it isn't returning that value for you?
there might be a better way to do this... but here I go anyway
every page gets sent the #params#
<cfdump var="#params#">
<cfoutput>#params.action#/#params.controller#/#params.key#</cfoutput>
<cfabort>
try putting that in a controller and see the results
the problem is that if the objects inside the params object don't exist you get an error. So the path that gets generated needs to check if the struct key exists and edit accordingly.
CGI.Path_Info will give you the desired results. I've been trying different options however they all failed and went into the redirect loop. As soon as I switched CGI.path_info it all started well.

Problem reading a cookie from within a tracking pixel in IE

I wrote a simple pixel tracking program that works something like this
Step 1) tracker.com sets a cookie
Step 2) mysite.com displays <img src="tracker.com/tracking.php">. That image reads the cookie from Step 1 & does some processing.
Works great in Chrome, Firefox and Safari. But when tested in IE, the cookie can't be read in Step 2. It's as if the cookie doesn't exist -- but I know it does.
Any idea why IE pretends the cookie doesn't exist? I've tried messing with P3P headers, no luck.
Does your domain have a privacy policy? I forget what it's called, maybe p3p? Some random list of headers that you have to add.
Try adding the domain in the src attribute to trusted sites in IE. My guess is this is security, and you've got a rather arcane security measure you're coming up against.
If the cookie setting domain is 2 letters, I believe there is a bug within IE that prevents IE from doing cookies properly with 2 letter domains. If it isn't 2 letters, then nevermind.
It may be that IE is blocking 3rd-party cookies.
Its tricky without knowing more specifics of its use, but I'm trying at this late hour to figure out how to clone the cookie for the current domain using REMOTE_ADDR
So, the first answer was more about testing... try using JS to handle this -
From site-reference.com forums..
<script type='text/javascript'>
var track = new Image();
track.src="http://www.my-site.com/tracker.php?self=" + this.location;
</script>
*NOTE: Capital "I" in image, not lowercase!
Let us know! :D
Fred