We are using axios to pass GET request to our django instance, that splits it into search terms and runs the search. This has been working fine until we ran into an edge case. We use urlencode to ensure that strings do not have empty spaces or others
So generalize issue, we have TextField called "name" and we want to search for term "A & B Company". However, issue is that when the request reaches django.
What we expected was that name=A%20&%20B%20Company&field=value would be parsed as name='A & B Company' and field='value'.
Instead, it is parsed as name='A ' 'B Company' and field='value'. The & symbol is incorrectly treated as separator, despite being encoded.
Is there a way to indicate django GET parameter that certain & symbols are part of the value, instead of separators for fields?
You can use the lib urllib
class ModelExample(models.Model):
name = models.TextField()
# in view...
from urllib.parse import parse_qs
instance = ModelExample(name="name=A%20&%20B%20Company&field=value")
dict_qs = parse_qs(instance.name)
dict_qs contains a dict with decoded querystring
You can find more informations about urllib.parse here: https://docs.python.org/3/library/urllib.parse.html
Related
I created an application with Flask.
Locally, everything works.
But now that I have sent to the server, I have a routing problem.
The url that causes the problem is built like this:
#app.route('/analyze/<string:keyword>')
The problem is that the keyword variable can contain accents (éèàç, etc ...) and spaces (example : sol stratifié)
The problem is that instead of laminate floor, I have sol%20stratifi%c3%a9 which appears on the screen, but also when I call the database.
I tried:
encode and decode utf- 8 on the keyword variable : it doesn't work
import codecs : it doesn't work
test to use the function : Flask URL route encoding problems
try to add .encode('utf-8') directly in the url route
would you have a solution ?
sol%20stratifi%c3%a9 is a percent-encoded string (URL quoted or %xx escaped in Python terminology).
The urllib.parse module defines functions that fall into two broad categories: URL parsing and URL quoting.
from urllib.parse import unquote
print( unquote( 'sol%20stratifi%c3%a9'))
sol stratifié
I'm trying to write my own Jekyll plugin to construct an api query from a custom tag. I've gotten as far as creating the basic plugin and tag, but I've run into the limits of my programming skills so looking to you for help.
Here's my custom tag for reference:
{% card "Arbor Elf | M13" %}
Here's the progress on my plugin:
module Jekyll
class Scryfall < Liquid::Tag
def initialize(tag_name, text, tokens)
super
#text = text
end
def render(context)
# Store the name of the card, ie "Arbor Elf"
#card_name =
# Store the name of the set, ie "M13"
#card_set =
# Build the query
#query = "https://api.scryfall.com/cards/named?exact=#{#card_name}&set=#{#card_set}"
# Store a specific JSON property
#card_art =
# Finally we render out the result
"<img src='#{#card_art}' title='#{#card_name}' />"
end
end
end
Liquid::Template.register_tag('cards', Jekyll::Scryfall)
For reference, here's an example query using the above details (paste it into your browser to see the response you get back)
https://api.scryfall.com/cards/named?exact=arbor+elf&set=m13
My initial attempts after Googling around was to use regex to split the #text at the |, like so:
#card_name = "#{#text}".split(/| */)
This didn't quite work, instead it output this:
[“A”, “r”, “b”, “o”, “r”, “ “, “E”, “l”, “f”, “ “, “|”, “ “, “M”, “1”, “3”, “ “]
I'm also then not sure how to access and store specific properties within the JSON response. Ideally, I can do something like this:
#card_art = JSONRESPONSE.image_uri.large
I'm well aware I'm asking a lot here, but I'd love to try and get this working and learn from it.
Thanks for reading.
Actually, your split should work – you just need to give it the correct regex (and you can call that on #text directly). You also need to escape the pipe character in the regex, because pipes can have special meaning. You can use rubular.com to experiment with regexes.
parts = #text.split(/\|/)
# => => ["Arbor Elf ", " M13"]
Note that they also contain some extra whitespace, which you can remove with strip.
#card_name = parts.first.strip
#card_set = parts.last.strip
This might also be a good time to answer questions like: what happens if the user inserts multiple pipes? What if they insert none? Will your code give them a helpful error message for this?
You'll also need to escape these values in your URL. What if one of your users adds a card containing a & character? Your URL will break:
https://api.scryfall.com/cards/named?exact=Sword of Dungeons & Dragons&set=und
That looks like a URL with three parameters, exact, set and Dragons. You need to encode the user input to be included in a URL:
require 'cgi'
query = "https://api.scryfall.com/cards/named?exact=#{CGI.escape(#card_name)}&set=#{CGI.escape(#card_set)}"
# => "https://api.scryfall.com/cards/named?exact=Sword+of+Dungeons+%26+Dragons&set=und"
What comes after that is a little less clear, because you haven't written the code yet. Try making the call with the Net::HTTP module and then parsing the response with the JSON module. If you have trouble, come back here and ask a new question.
I have a django based app with haystack and whoosh search engine. I want to provide an accent and special character independent search so that I can find indexed data with special characters also by using words without special chars:
Indexed is:
'café'
Search term:
'cafe'
'café'
I've written a provided a specific FoldingWhooshSearchBackend which uses a StemmingAnalyzer and aCharsetFilter(accent_map) as described in the following document:
https://gist.github.com/gregplaysguitar/1727204
However the search still doesn't work like expected, i.e. I cannot search with 'cafe' and find 'café'. I've looked into the search index using:
from whoosh.index import open_dir
ix = open_dir('myservice/settings/whoosh_index')
searcher = ix.searcher()
for doc in searcher.documents():
print doc
The special characters are still in the index.
Do I have to do something additional? Is is about changing the index template?
You have to write Haystack SearchIndex classes for your models. That's how you can prepare models data for the search index.
Example of myapp/search_index.py:
from haystack import site
from haystack import indexes
class UserProfileIndex(indexes.SearchIndex):
text = indexes.CharField(document=True)
def prepare_text(self, obj):
data = [obj.get_full_name(), obj.user.email, obj.phone]
original = ' '.join(data)
slugified = slugify(original)
return ' '.join([original, slugified])
site.register(UserProfile, UserProfileIndex)
If a user has name café, you will find his profile with bouth search terms café and cafe.
I think the best approach is to let Haystack create the schema for maximum forwards compatibility, and then hack the CharsetFilter in.
This code is working for me with Haystack 2.4.0 and Whoosh 2.7.0:
from haystack.backends.whoosh_backend import WhooshEngine, WhooshSearchBackend
from whoosh.analysis import CharsetFilter, StemmingAnalyzer
from whoosh.support.charset import accent_map
from whoosh.fields import TEXT
class FoldingWhooshSearchBackend(WhooshSearchBackend):
def build_schema(self, fields):
schema = super(FoldingWhooshSearchBackend, self).build_schema(fields)
for name, field in schema[1].items():
if isinstance(field, TEXT):
field.analyzer = StemmingAnalyzer() | CharsetFilter(accent_map)
return schema
class FoldingWhooshEngine(WhooshEngine):
backend = FoldingWhooshSearchBackend
am getting the following post data in my django app
POST
Variable Value
csrfmiddlewaretoken u'LHM3nkrrrrrrrrrrrrrrrrrrrrrrrrrdd'
id u'{"docs":[],"dr":1, "id":4, "name":"Group", "proj":"/al/p1/proj/2/", "resource":"/al/p1/dgroup/4/","route":"group", "parent":null'
am trying to get the id value in variable id i.e "id":4 (the value 4). When I do request.POST.get('id')I get the whole json string. u'{"docs":[],"dr":1, "id":4, "name":"Group", "proj":"/al/p1/proj/2/", "resource":"/al/p1/dgroup/4/","route":"group", "parent":null' How can I get the "id" in the string?
The data you are sending is simply a json string.
You have to parse that string before you can access data within it. For this you can use Python's json module (it should be available if you're using Python 2.7).
import json
data = json.loads( request.POST.get('id') )
id = data["id"]
If you somehow don't have the json module, you can get the simplejson module.
For more details, refer this question : best way to deal with JSON in django
That's happening because id is string, not dict as it should be. Please provide your template and view code to find source of problem.
I'm having trouble logging the output of an internationalized django variable.
My application code used the 'message' vaiable OK and it displays
in English and German (the two languages I am working with)
My problem is when I try to make a log of the said 'message' variable output.
I want to log in English and/or German (depending on what the end user is seeing)
I'm using string_concat and it says:
the lazy translations in result will only be converted to strings when result
itself is used in a string (usually at template rendering time).
Here is a code sample:
from django.utils.translation import ugettext_lazy as _
from django.utils.translation import string_concat
message = _('Profile Updated OK')
# Build a log message
fmt_msg = _(message)
log_message = 'Message Sent: '
log_message = string_concat(log_message, fmt_msg)
logger.info(log_message)
The log entry is not translated:
Message Sent: <django.utils.functional.__proxy__ object at 0x9c3c4ac>
Can anyone suggest how I can have the proxy object to be translated
when used with logging?
It's translating OK in my regular Django code just the logging.info() issue.
Cheers!
From the docs:
The result of a ugettext_lazy() call can be used wherever you would use a unicode string (an object with type unicode) in Python. If you try to use it where a bytestring (a str object) is expected, things will not work as expected, since a ugettext_lazy() object doesn't know how to convert itself to a bytestring.
Also, don't call _() twice.
logger.info(u'Message Sent: %s' % message)
You have to force Unicode strings, as documented in http://www.djangobook.com/en/2.0/chapter19/
u"Hello %s" % ugettext_lazy("people")
And do not call the function twice ;)