I have two search inputs, first on home page another on search results itself. what i m try to do is receive query form home and redirect it to search results page according to this :
Like I search - html-5
redirect page should be - 127.0.0.1/html-5/find/?q=html-5
I have tried but unfortunately not getting the right way to it, please suggest me the correct way to do it.
I use these url patterns
url(r'^(?P<key>.*)/find/', FacetedSearchView.as_view(), name='haystack_search'),
url(r'^search/',category_query_view,name='category_query'),
then in category_query
def category_query_view(request):
category = request.GET.get('q')
print('hihi',category)
return HttpResponseRedirect(reverse('haystack_search', kwargs={'key':category},))
It is redirecting me to
127.0.0.1/html-5/find/
but i don't know how to add
/?q=html-5
in after this?
Oh, I get the right way, its pretty simple
def category_query_view(request):
category = request.GET.get('q')
print('hihi',category)
url = '{category}/find/?q={category}'.format(category=category)
return HttpResponseRedirect('/'+url)
Related
I have a set of URLs:
/home/
/register/
/login/
/puzzle/<pk>
All of the first 3 urls can make a request to the last url.
Is it possible to know which urls are calling the /puzzle/<pk>, from the view function attached to it?
Edit
So the problem is it's a puzzle game and every user has a level. If he completes a certain level only then he can proceed to the next level. Therefore, if a user has completed level 3, I'll always show him the html page with the url /puzzle/4.
The flow is :
A user registers. His level is 0. As soon as he registers he'll be redirected to puzzle/1/
A user logs in. If he has completed level x, as soon as he logs in he'll be redirected to /puzzle/x+1.
I've handled these 2 types.
But the problem is, say the user is in the page /puzzle/1/. Now if he manually changes the url to /puzzle/2/, from the view function attached to /puzzle/<pk>/, how can I handle the above case?
I believe you're looking for request.resolver_match. You could access the view name with request.resolver_match.view_name
Edit:
If I understand your model correctly, you need some logic like the following. You'll need to adjust the view name and params in reverse.
def view(request, puzzle_level):
if puzzle_level != request.user.level_completed + 1:
return redirect(
reverse(
'puzzle_view_name',
kwargs={'puzzle_level': request.user.level_completed + 1},
)
)
...
Alright, so I'm working on a scrapy based webcrawler, with some simple functionalities. The bot is supposed to go from page to page, parsing and then downloading. I've gotten the parser to work, I've gotten the downloading to work. I can't get the crawling to work. I've read the documentation on the Spider class, I've read the documentation on how parse is supposed to work. I've tried returning vs yielding, and I'm still nowhere. I have no idea where my code is going wrong. What seems to happen, from a debug script I wrote is the following. The code will run, it will grab page 1 just fine, it'll get the link to page two, it'll go to page two, and then it will happily stay on page two, not grabbing page three at all. I don't know where the mistake in my code is, or how to alter it to fix it. So any help would be appreciated. I'm sure the mistake is basic, but I can't figure out what's going on.
import scrapy
class ParadiseSpider(scrapy.Spider):
name = "testcrawl2"
start_urls = [
"http://forums.somethingawful.com/showthread.php?threadid=3755369&pagenumber=1",
]
def __init__(self):
self.found = 0
self.goto = "no"
def parse(self, response):
urlthing = response.xpath("//a[#title='Next page']").extract()
urlthing = urlthing.pop()
newurl = urlthing.split()
print newurl
url = newurl[1]
url = url.replace("href=", "")
url = url.replace('"', "")
url = "http://forums.somethingawful.com/" + url
print url
self.goto = url
return scrapy.Request(self.goto, callback=self.parse_save, dont_filter = True)
def parse_save(self, response):
nfound = str(self.found)
print "Testing" + nfound
self.found = self.found + 1
return scrapy.Request(self.goto, callback=self.parse, dont_filter = True)
Use Scrapy rule engine,So that don't need to write the next page crawling code in parse function.Just pass the xpath for the next page in the restrict_xpaths and parse function will get the response of the crawled page
rules=(Rule(LinkExtractor(restrict_xpaths= ['//a[contains(text(),"Next")]']),follow=True'),)
def parse(self,response):
response.url
I'm using lxml to scrape through a site. I want to scrape through a search result, that contains 194 items. My scraper is able to scrape only the first page of search results. How can I scrape the rest of the search results?
url = 'http://www.alotofcars.com/new_car_search.php?pg=1&byshowroomprice=0.5-500&bycity=Gotham'
response_object = requests.get(url)
# Build DOM tree
dom_tree = html.fromstring(response_object.text)
After this there are scraping functions
def enter_mmv_in_database(dom_tree,engine):
# Getting make, model, variant
name_selector = CSSSelector('[class="secondary-cell"] p a')
name_results = name_selector(dom_tree)
for n in name_results:
mmv = str(`n.text_content()`).split('\\xa0')
make,model,variant = mmv[0][2:], mmv[1], mmv[2][:-2]
# Now push make, model, variant in Database
print make,model,variant
By looking at the list I receive I can see that only the first page of search results is parsed. How can I parse the whole of search result.
I've tried to navigate through that website but it seems to be offline. Yet, I would like to help with the logic.
What I usually do is:
Make a request to the search URL (with parameters filled)
With lxml, extract the last page available number in a pagination div.
Loop from first page to the last one, making requests and scraping desired data:
for page_number in range(1, last+1):
## make requests replacing 'page_number' in 'pg' GET variable
url = "http://www.alotofcars.com/new_car_search.php?pg={}&byshowroomprice=0.5-500&bycity=Gotham'".format(page_number)
response_object = requests.get(url)
dom_tree = html.fromstring(response_object.text)
...
...
I hope this helps. Let me know if you have any further questions.
I have a django project that needs to search 2 different models and one of the models has 3 types that I need to filter based on. I have haystack installed and working in a basic sense (using the default url conf and SearchView for my model and the template from the getting started documentation is returning results fine).
The problem is that I'm only able to get results by using the search form in the basic search.html template and I'm trying to make a global search bar work with haystack but I can't seem to get it right and I'm not having a lot of luck with the haystack documentation. I found another question on here that led me to the following method in my search app.
my urls.py directs "/search" to this view in my search.views:
def search_posts(request):
post_type = str(request.GET.get('type')).lower()
sqs = SearchQuerySet().all().filter(type=post_type)
view = search_view_factory(
view_class=SearchView,
template='search/search.html',
searchqueryset=sqs,
form_class=HighlightedSearchForm
)
return view(request)
The url string that comes in looks something like:
http://example.com/search/?q=test&type=blog
This will get the query string from my global search bar but returns no results, however if I remove the .filter(type=post_type) part from the sqs line I will get search results again (albeit not filtered by post type). Any ideas? I think I'm missing something fairly obvious but I can't seem to figure this out.
Thanks,
-Sean
EDIT:
It turns out that I am just an idiot. The reason why my filtering on the SQS by type was returning no results was because I didn't have the type field included in my PostIndex class. I changed my PostIndex to:
class PostIndex(indexes.SearchIndex, indexes.Indexable):
...
type = indexes.CharField(model_attr='type')
and rebuilt and it all works now.
Thanks for the response though!
def search_posts(request):
post_type = str(request.GET.get('type')).lower()
sqs = SearchQuerySet().filter(type=post_type)
clean_query = sqs.query.clean(post_type)
result = sqs.filter(content=clean_query)
view = search_view_factory(
view_class=SearchView,
template='search/search.html',
searchqueryset=result,
form_class=HighlightedSearchForm
)
return view(request)
I'm getting ready to move an old Classic ASP site to a new Django system. As part of the move we have to setup some of the old URLs to point to the new ones.
For example,
http://www.domainname.com/category.asp?categoryid=105 should 301 to http://www.domainname.com/some-category/
Perhaps I'm regex stupid or something, but for this example, I've included in my URLconf this:
(r'^asp/category\.asp\?categoryid=105$', redirect_to, {'url': '/some-category/'}),
My thinking is that I have to escape the . and the ? but for some reason when I go to test this, it does not redirect to /some-category/, it just 404s the URL as entered.
Am I doing it wrong? Is there a better way?
To elaborate on Daniel Roseman's answer, the query string is not part of the URL, so you'll probably want to write a view function that will grab the category from the query string and redirect appropriately. You can have a URL like:
(r'^category\.asp', category_redirect),
And a view function like:
def category_redirect(request):
if 'categoryid' not in request.GET:
raise Http404
cat_id = request.GET['category']
try:
cat = Category.objects.get(old_id=cat_id)
except Category.DoesNotExist:
raise Http404
else:
return HttpResponsePermanentRedirect('/%s/' % cat.slug)
(Altered to your own tastes and needs, of course.)
Everything after the ? is not part of the URL. It's part of the GET parameters.