How to use Regex to Genericise URLs in Azure App Insights - regex

I'm trying to get average performance for API Calls in AppInsights, but don't want each separate call together with the parameter.
For example,
GET /api/questions/identity/00434514 I want GET /api/questions/identity
I've used https://regexr.com/ and come up with .*\a\p\i((\/[a-zA-Z]*)*), which works on the website, but AppInsights returns the IDs instead
The request is:
requests
| extend newUrl = replace_regex(url, #".*\a\p\i((\/[a-zA-Z]*)*)", #"*")
| summarize RequestsCount=sum(itemCount), AverageDuration=avg(duration) by newUrl
| order by AverageDuration desc
I'm expecting:
It starts with anything, following by 'api'
It then must have any number of '/' followed by alpha characters (repeated any number of times)
What part of my regex does it not like.

Related

HTTP Body Regex

I want to automate a process of sending HTTP request using the regex to remove any character or number located between the = and the & with burpsuite ( Autorize Extension )
The body:
id=169413&token=2y10l02e7J2mBD6lTnSv9uHSZD5QylM2JsM21Hyi4J&sub_profile_id=14317
i want something similar at
id=&token=&sub_profile=
I tried with \=[A-Za-z1-9]+ but without results
it could be done in three steps example capture just id=Anything and remove 'Anything' then do it with the rest
How about
\=([^&]+)\&?

How can I use regex to construct an API call in my Jekyll plugin?

I'm trying to write my own Jekyll plugin to construct an api query from a custom tag. I've gotten as far as creating the basic plugin and tag, but I've run into the limits of my programming skills so looking to you for help.
Here's my custom tag for reference:
{% card "Arbor Elf | M13" %}
Here's the progress on my plugin:
module Jekyll
class Scryfall < Liquid::Tag
def initialize(tag_name, text, tokens)
super
#text = text
end
def render(context)
# Store the name of the card, ie "Arbor Elf"
#card_name =
# Store the name of the set, ie "M13"
#card_set =
# Build the query
#query = "https://api.scryfall.com/cards/named?exact=#{#card_name}&set=#{#card_set}"
# Store a specific JSON property
#card_art =
# Finally we render out the result
"<img src='#{#card_art}' title='#{#card_name}' />"
end
end
end
Liquid::Template.register_tag('cards', Jekyll::Scryfall)
For reference, here's an example query using the above details (paste it into your browser to see the response you get back)
https://api.scryfall.com/cards/named?exact=arbor+elf&set=m13
My initial attempts after Googling around was to use regex to split the #text at the |, like so:
#card_name = "#{#text}".split(/| */)
This didn't quite work, instead it output this:
[“A”, “r”, “b”, “o”, “r”, “ “, “E”, “l”, “f”, “ “, “|”, “ “, “M”, “1”, “3”, “ “]
I'm also then not sure how to access and store specific properties within the JSON response. Ideally, I can do something like this:
#card_art = JSONRESPONSE.image_uri.large
I'm well aware I'm asking a lot here, but I'd love to try and get this working and learn from it.
Thanks for reading.
Actually, your split should work – you just need to give it the correct regex (and you can call that on #text directly). You also need to escape the pipe character in the regex, because pipes can have special meaning. You can use rubular.com to experiment with regexes.
parts = #text.split(/\|/)
# => => ["Arbor Elf ", " M13"]
Note that they also contain some extra whitespace, which you can remove with strip.
#card_name = parts.first.strip
#card_set = parts.last.strip
This might also be a good time to answer questions like: what happens if the user inserts multiple pipes? What if they insert none? Will your code give them a helpful error message for this?
You'll also need to escape these values in your URL. What if one of your users adds a card containing a & character? Your URL will break:
https://api.scryfall.com/cards/named?exact=Sword of Dungeons & Dragons&set=und
That looks like a URL with three parameters, exact, set and Dragons. You need to encode the user input to be included in a URL:
require 'cgi'
query = "https://api.scryfall.com/cards/named?exact=#{CGI.escape(#card_name)}&set=#{CGI.escape(#card_set)}"
# => "https://api.scryfall.com/cards/named?exact=Sword+of+Dungeons+%26+Dragons&set=und"
What comes after that is a little less clear, because you haven't written the code yet. Try making the call with the Net::HTTP module and then parsing the response with the JSON module. If you have trouble, come back here and ask a new question.

Prevent URL encoding that is removing equals signs from URL

Working on a Django/React app. I have some verification emails links that look like the following:
https://test.example.com/auth/security_questions/f=ru&i=101083&k=7014c315f3056243534741610545c8067d64d747a981de22fe75b78a03d16c92
In dev env this works fine, but now that I am getting it ready for production, it isn't working. When I click on it, it converts it to:
https://test.example.com/auth/security_questions/f%3Dru&i%3D101083&k%3D7014c315f3056243534741610545c8067d64d747a981de22fe75b78a03d16c92/
This prevents react-router-dom from matching the correct URL, so a portion of the web application does not load properly.
The link is constructed using the following.
link = '%s/auth/security_questions/f=%s&i=%s&k=%s' % \
('https://test.example.com', 'ru', user.id, user.key)
Also, here is the url() that is catching the route:
url(r'^(?:.*)/$', TemplateView.as_view(template_name='index.html')),
These variables are supposed to be query parameters in a GET request. When you construct the link, you'll need to have a question mark in there somewhere separating the URL from the query string:
https://test.example.com/auth/security_questions/?f=ru&i=101083&k=7014c315...
^
|___ here
The conversion of = to url-encoded %3D etc is correct, and equivalent. Sometimes variables are part of the URL directly, but webapps don't use &-separated key/value pairs in that case.

Regex capture group in Varnish VCL

I have a URL in the form of:
http://some-site.com/api/v2/portal-name/some/webservice/call
The data I want to fetch needs
http://portal-name.com/webservices/v2/some/webservice/call
(Yes I can rewrite the application so it uses other URL's but we are testing varnish at the moment so for now it cannot be intrusive.)
But I'm having trouble getting the URL correctly in varnish VCL. The api part is replaced by an empty string, no worries but now the portal-name.
Things I've tried:
if (req.url ~ ".*/(.*)/") {
set req.http.portalhostname = re.group.0;
set req.http.portalhostname = $1;
}
From https://docs.fastly.com/guides/vcl/vcl-regular-expression-cheat-sheet and Extracting capturing group contents in Varnish regex
And yes, std is imported.
But this gives me either a
Syntax error at
('/etc/varnish/default.vcl' Line 36 Pos 35)
set req.http.portalhostname = $1;
or a
Symbol not found: 're.group.0' (expected type STRING_LIST):
So: how can I do this? When I have extracted the portalhostname I should be able to simply do a regsub to replace that value with an empty string and then prepend "webservices" and my URL is complete.
The varnish version i'm using: varnish-4.1.8 revision d266ac5c6
Sadly re.group seems to have been removed at some version. Similar functionality appears to be accessible via one of several vmods. See https://varnish-cache.org/vmods/

Why is my route not matching any action

Using play framework, I'm trying to match a route using a regular expression.
What I wanted is to use one action that maps all this urls:
mydomain.com/my-post-title-123
mydomain.com/another-post-title-124
mydomain.com/a-third-post-title-125
get this "123, 124 and 125" from the end of the url so the controller can use it. Basically ignore whatever post tile comes in and only use the number at the end.
I have the following on my routes.conf
GET /$postId<\d$> controllers.Posts.viewPost(postId: Int)
But I get the error page "Action not found"
You are missing the url prefix and "+" in the regex in the routes definition. Here is my route configuration and it works fine
#Regex test
GET /$prefix<.*>$postId<\d+$> controllers.Application.viewPost(prefix:String,postId: Int)
Controllers.Application.viewPost
def viewPost(prefix:String,postId:Int) = Action{
Ok("the post id is: "+postId+" the prefix is:"+prefix)
}
and the output will be
the post id is: 123 the prefix is "whatever/prefix/you/give"
** tested, it works.