I'm using the http listener of the C++ REST SDK 2.8 and noticed the following. If I send the following URL to this listener:
http://my_server/my%2fpath?key=xxx%26yyy%3Dzzz
and I do:
auto uri = request.relative_uri();
auto v_path_components = web::uri::split_path(web::uri::decode(uri.path()));
auto m_query_components = web::uri::split_query(web::uri::decode(uri.query()));
then I find that v_path_components contains 2 elements ["my", "path"], and m_query_components contains 2 pairs [("key","xxx"), ("yyy","zzz")].
What I want and would have expected is v_path_components to contain 1 element ["my/path"], and m_query_components to contain 1 pair [("key","xxx&yyy=zzz")].
In order for the latter to achieve, relative_uri shouldn't decode/encode the uri, as that looses information. In addition, web::uri::decode() should be executed on the split results rather than before splitting. But, as the REST SDK itself as well as many samples shipped with it uses this in the above way, it leads me to believe that I might be wrong.
Could anyone confirm my findings or explain why I'm on the wrong track?
Your findings make sense.
Since you are decoding first, then the encoded ampersand (%3D) becomes a key/value pair separator. Same for the path components. The slash (%2f) becomes a path separator, and is parsed as such.
Related
I want to pass into a variable, the language of the user.
But, my client can't/didn't pass this information trough datalayer. So, the unique solution I've is to use the URL Path.
Indeed - The structure is:
http://www.website.be/en/subcategory/subsubcategory
I want to extract "en" information
No idea to get this - I check on Stack, on google, some people talk about regex, other ones about CustomJS, but, no result on my specific setup.
Do you have an idea how to proceed on this point ?
Many thanks !!
Ludo
Make sure the built in {{Page Path}} variable is enabled. Create a custom Javascript variable.
function() {
var parts = {{Page Path}}.split("/");
return parts[1];
}
This splits the path by the path delimiter "/" and gives you an array with the parts. Since the page path has a leading slash (I think), the first part is empty, so you return the second one (since array indexing starts with 0 the second array element has the index 1).
This might need a bit of refinement (for pages that do not start with a language signifier, if any), but that's the basic idea.
Regex is an alternative (via the regex table variable), but the above solution is a little easier to implement.
I am building image links in Python 3. I am getting a basic starter link and I am being given one or more elements that need to be switched out based on a pattern that is tied to that original link to create my new image link. I am being given the links and the elements and I can tie each link type to a pattern that I can store in the database. What I am looking for is the best way to create the pattern to switch out the data.
For Example:
The first link I get is: https://testurl.com/images/320/new.jpg
I am given a value of 450 and the pattern dictates that the 450 should be swapped out with the 320 to make a new link of: https://testurl.com/images/450/new.jpg.
The second link I get is http://newurl.com/stuff/image.jpg?resize=100
I am given a value of 200 and the pattern dictates that the 200 needs to replace the 100 to make a new link of http://newurl.com/stuff/image.jpg.resize=200
The third link I get is http://anotherlink.com/01/02/2016/today/newfolder/image.jpg I am given a string of 02/15/2016 that needs to be swapped out with the 01/02/2016 and a string of oldfolder that needs to be swapped out with the newfolder giving me a link of http://anotherlink.com/02/15/2016/today/oldfolder/image.jpg
Is there a way to store patterns that could handle all 3 different scenarios? Again, I can have a unique pattern for each, I just would need the same code to be able to execute all 3 patterns successfully.
I was thinking I could do this with a regex and I have done validation with it in the past but I'm not sure if there is a way to actually change out parts of strings based on a pattern instead of just matching.
My other thought was to create a template and do a find and replace loop until all the replace elements have been swapped out but that seems messy. Any suggestions?
I've got this code which should do what you ask, I haven't tested it yet, but add a comment if it doesn't work, or if it doesn't do what you want:
url="" #URL to use
replace="" #string to replace original with
if "resize=" in url:
for i in range(len(url)):
try:
int(url[-1:])
except Exception:
break
else:
url=url[:-1]
url+=str(replace)
elif [e for e in range(len(url)) if url[e:][:(e+10)].replace("/","").isalpha()==False]:
for e in range(len(url)):
if url[:(e+10)][e:].replace("/","").isdigit()==True:
if url[e:][:1]!="/":
url=url[:e]+replace+url[(e+10):]
url=url.replace("newfolder","oldfolder")
break
else:
url1=url.split("/")
url1[len(url1)-2]=replace
url=""
for i in url1:
url+=i+"/"
print(url)
Hope this helps.
EDIT:
I have now tested the code, and fixed the bugs, and it seems to work fine, but still add a comment if it doesn't work.
I've recently upgraded a CloudSearch instance from the 2011 to the 2013 API. Both instances have a field called sid, which is a text field containing a two-letter code followed by some digits e.g. LC12345. With the 2011 API, if I run a search like this:
q=12345*&return-fields=sid,name,desc
...I get back 1 result, which is great. But the sid of the result is LC12345 and that's the way it was indexed. The number 12345 does not appear anywhere else in any of the resulting document fields. I don't understand why it works. I can only assume that this type of query is looking for any terms in any fields that even contain the number 12345.
The reason I'm asking is because this functionality is now broken when I query using the 2013 API. I need to use the structured query parser, but even a comparable wildcard query using the simple parser is not working e.g.
q.parser=simple&q=12345*&return=sid,name,desc
...returns nothing, although the document is definitely there i.e. if I query for LC12345* it finds the document.
If I could figure out how to get the simple query working like it was before, that would at least get me started on how to do the same with the structured syntax.
Why it's not working
CloudSearch v1 (2011) had a different way of tokenizing mixed alpha+numeric strings. Here's the logic as described in the archived docs (emphasis mine).
If a string contains both alphabetic and numeric characters and is at
least three and no more than nine characters long, the alphabetic and
numeric portions of the string are treated as separate tokens. For
example, the string DOC298 is tokenized into two terms: doc 298
CloudSearch v2 (2013) text processing follows Unicode Text Segmentation, which does not specify that behavior:
Do not break within sequences of digits, or digits adjacent to letters (“3a”, or “A3”).
Solution
You should just be able to search *12345 to get back results with any prefix. There may be some edge cases like getting back results you don't want (things with more preceding digits like AB99912345); I don't know enough about your data to say whether those are real concerns.
Another option would would be to index the numeric prefix separately from the alphabetical suffix but that's additional work that may be unnecessary.
I'm guessing you are using Cloudsearch in English, so maybe this isn't your specific problem, but also watch out for Stopwords in your search queries:
https://docs.aws.amazon.com/cloudsearch/latest/developerguide/configuring-analysis-schemes.html#stopwords
In your example, the word "jo" is a stop word in Danish and another languages, and of course, supported languages, have a dictionary of stop words that has very common ones. If you don't specify a language in your text field, it will be English. You can see them here: https://docs.aws.amazon.com/cloudsearch/latest/developerguide/text-processing.html#text-processing-settings
It's a bit late, but I'm disappointed in myself for not coming up with something more elegant. Anyone have a better way to do this...
When you pass an OAuth code to Facebook, it response with a query string containing access_token and expires values.
access_token=121843224510409|2.V_ei_d_rbJt5iS9Jfjk8_A__.3600.1273741200-569255561|TxQrqFKhiXm40VXVE1OBUtZc3Ks.&expires=4554
Although if you request permission for offline access, there's no expires and the string looks like this:
access_token=121843224510409|2.V_ei_d_rbJt5iS9Jfjk8_A__.3600.1273741200-569255561|TxQrqFKhiXm40VXVE1OBUtZc3Ks.
I attempted to write a regex that would suffice for either condition. No dice. So I ended up with some really ugly Ruby:
s = s.split("=")
#oauth = {}
if s.length == 3
#oauth[:access_token] = s[1][0, s[1].length - 8]
#oauth[:expires] = s[2]
else
#oauth[:access_token] = s[1]
end
I know there must be a better way!
Split on the & symbol first, and then split each of the results on =? That's the method that can cope with the order changing, since it parses each key-value pair individually.
Alternatively, a regex that should work would be...
/access_token=(.*?)(?:&expires=(.*))/
If the format is strict, then you use this regex:
access_token=([^&]+)(?:&expires=(.*))?
Then access_token value is in \1, and expires, if there's any, will be in \2.
Query string parsing usually involves these steps:
If it is a complete URL, take the part after the first ?
split the rest on &
for each of the resulting name=value pairs, split them on =
URL-decode to the value and the name (!)
stuff the result into a data structure of your liking
Using regex is possible, but makes invalid assumptions about the state of URL-encoding the string is in and fails if the order of the query string changes, which is perfectly allowed and therefore cannot be ruled out. Better to encapsulate the above in a little parsing function or use one of the existing URL-handling libraries of your platform.
I am writing a program which will tokenize the input text depending upon some specific rules. I am using C++ for this.
Rules
Letter 'a' should be converted to token 'V-A'
Letter 'p' should be converted to token 'C-PA'
Letter 'pp' should be converted to token 'C-PPA'
Letter 'u' should be converted to token 'V-U'
This is just a sample and in real time I have around 500+ rules like this. If I am providing input as 'appu', it should tokenize like 'V-A + C-PPA + V-U'. I have implemented an algorithm for doing this and wanted to make sure that I am doing the right thing.
Algorithm
All rules will be kept in a XML file with the corresponding mapping to the token. Something like
<rules>
<rule pattern="a" token="V-A" />
<rule pattern="p" token="C-PA" />
<rule pattern="pp" token="C-PPA" />
<rule pattern="u" token="V-U" />
</rules>
1 - When the application starts, read this xml file and keep the values in a 'std::map'. This will be available until the end of the application(singleton pattern implementation).
2 - Iterate the input text characters. For each character, look for a match. If found, become more greedy and look for more matches by taking the next characters from the input text. Do this until we are getting a no match. So for the input text 'appu', first look for a match for 'a'. If found, try to get more match by taking the next character from the input text. So it will try to match 'ap' and found no matches. So it just returns.
3 - Replace the letter 'a' from input text as we got a token for it.
4 - Repeat step 2 and 3 with the remaining characters in the input text.
Here is a more simple explanation of the steps
input-text = 'appu'
tokens-generated=''
// First iteration
character-to-match = 'a'
pattern-found = true
// since pattern found, going recursive and check for more matches
character-to-match = 'ap'
pattern-found = false
tokens-generated = 'V-A'
// since no match found for 'ap', taking the first success and replacing it from input text
input-text = 'ppu'
// second iteration
character-to-match = 'p'
pattern-found = true
// since pattern found, going recursive and check for more matches
character-to-match = 'pp'
pattern-found = true
// since pattern found, going recursive and check for more matches
character-to-match = 'ppu'
pattern-found = false
tokens-generated = 'V-A + C-PPA'
// since no match found for 'ppu', taking the first success and replacing it from input text
input-text = 'u'
// third iteration
character-to-match = 'u'
pattern-found = true
tokens-generated = 'V-A + C-PPA + V-U' // we'r done!
Questions
1 - Is this algorithm looks fine for this problem or is there a better way to address this problem?
2 - If this is the right method, std::map is a good choice here? Or do I need to create my own key/value container?
3 - Is there a library available which can tokenize string like the above?
Any help would be appreciated
:)
So you're going through all of the tokens in your map looking for matches? You might as well use a list or array, there; it's going to be an inefficient search regardless.
A much more efficient way of finding just the tokens suitable for starting or continuing a match would be to store them as a trie. A lookup of a letter there would give you a sub-trie which contains only the tokens which have that letter as the first letter, and then you just continue searching downward as far as you can go.
Edit: let me explain this a little further.
First, I should explain that I'm not familiar with these the C++ std::map, beyond the name, which makes this a perfect example of why one learns the theory of this stuff as well as than details of particular libraries in particular programming languages: unless that library is badly misusing the name "map" (which is rather unlikely), the name itself tells me a lot about the characteristics of the data structure. I know, for example, that there's going to be a function that, given a single key and the map, will very efficiently search for and return the value associated with that key, and that there's also likely a function that will give you a list/array/whatever of all of the keys, which you could search yourself using your own code.
My interpretation of your data structure is that you have a map where the keys are what you call a pattern, those being a list (or array, or something of that nature) of characters, and the values are tokens. Thus, you can, given a full pattern, quickly find the token associated with it.
Unfortunately, while such a map is a good match to converting your XML input format to a internal data structure, it's not a good match to the searches you need to do. Note that you're not looking up entire patterns, but the first character of a pattern, producing a set of possible tokens, followed by a lookup of the second character of a pattern from within the set of patterns produced by that first lookup, and so on.
So what you really need is not a single map, but maps of maps of maps, each keyed by a single character. A lookup of "p" on the top level should give you a new map, with two keys: p, producing the C-PPA token, and "anything else", producing the C-PA token. This is effectively a trie data structure.
Does this make sense?
It may help if you start out by writing the parsing code first, in this manner: imagine someone else will write the functions to do the lookups you need, and he's a really good programmer and can do pretty much any magic that you want. Writing the parsing code, concentrate on making that as simple and clean as possible, creating whatever interface using these arbitrary functions you need (while not getting trivial and replacing the whole thing with one function!). Now you can look at the lookup functions you ended up with, and that tells you how you need to access your data structure, which will lead you to the type of data structure you need. Once you've figured that out, you can then work out how to load it up.
This method will work - I'm not sure that it is efficient, but it should work.
I would use the standard std::map rather than your own system.
There are tools like lex (or flex) that can be used for this. The issue would be whether you can regenerate the lexical analyzer that it would construct when the XML specification changes. If the XML specification does not change often, you may be able to use tools such as lex to do the scanning and mapping more easily. If the XML specification can change at the whim of those using the program, then lex is probably less appropriate.
There are some caveats - notably that both lex and flex generate C code, rather than C++.
I would also consider looking at pattern matching technology - the sort of stuff that egrep in particular uses. This has the merit of being something that can be handled at runtime (because egrep does it all the time). Or you could go for a scripting language - Perl, Python, ... Or you could consider something like PCRE (Perl Compatible Regular Expressions) library.
Better yet, if you're going to use the boost library, there's always the Boost tokenizer library -> http://www.boost.org/doc/libs/1_39_0/libs/tokenizer/index.html
You could use a regex (perhaps the boost::regex library). If all of the patterns are just strings of letters, a regex like "(a|p|pp|u)" would find a greedy match. So:
Run a regex_search using the above pattern to locate the next match
Plug the match-text into your std::map to get the replace-text.
Print the non-matched consumed input and replace-text to your output, then repeat 1 on the remaining input.
And done.
It may seem a bit complicated, but the most efficient way to do that is to use a graph to represent a state-chart. At first, i thought boost.statechart would help, but i figured it wasn't really appropriate. This method can be more efficient that using a simple std::map IF there are many rules, the number of possible characters is limited and the length of the text to read is quite high.
So anyway, using a simple graph :
0) create graph with "start" vertex
1) read xml configuration file and create vertices when needed (transition from one "set of characters" (eg "pp") to an additional one (eg "ppa")). Inside each vertex, store a transition table to the next vertices. If "key text" is complete, mark vertex as final and store the resulting text
2) now read text and interpret it using the graph. Start at the "start" vertex. ( * ) Use table to interpret one character and to jump to new vertex. If no new vertex has been selected, an error can be issued. Otherwise, if new vertex is final, print the resulting text and jump back to start vertex. Go back to (*) until there is no more text to interpret.
You could use boost.graph to represent the graph, but i think it is overly complex for what you need. Make your own custom representation.