I'm setting up an api endpoint which after parsing the url I would get a path such as /profile/<username> e.g. /profile/markzuck
The username is optional though as this endpoint returns the the authenticated users profile if username is blank
Rule set:
I'm not the best at regex but I've created an expression that requires /profile after that if there is a following / e.g. /profile/ then you need to have a <username> that matches (\w){1,15}. Also I want it to be allowed to match any number of combinations if there is another following / e.g. /profile/<username>/<if preceding "/" then anything else>
Although I'm not 100% sure my expression is correct this seems to work in JavaScript
/^\/(profile)(\/(?=(\w){1,15}))?/
Gorilla Mux though is different and it requires the route matching string to always start with a slash and some other things I don't understand like it can only use non-capturing groups
( found this out by getting this error: panic: route /{_dummy:profile/([a-zA-Z_])?} contains capture groups in its regexp. Only non-capturing groups are accepted: e.g. (?:pattern) instead of (pattern) )
I tried using the same expression I used for JavaScript which didn't work here. I created a more forgiving expresion handlerFunc("/{_dummy:profile\/[a-zA-Z_].*}") which does work however this doesn't really follow the same rule set I'm using in my JavaScript expresion.
I was able to come up with my working expresion from this SO post here
And Gorilla Mux's docs talks a little bit about how their regex works when explaining how to use the package in the intro section here
My question is what is a similar or equivalent expression to the rule set I described that will work in Gorilla Mux HandlerFunc()?
If you're doing this with mux, then I believe what you need is not regex, but multiple paths.
For the first case, use a path "/profile". For the one containing a user name, use another path "/profile/{userName}". If you really want to use a regex, you can do "/profile/{username:}" to validate the user name. If you need to process anything that comes after username, either register separate paths (/profile/{username}/otherstuff), or register a pathPrefix "/profile/{username}/" and process the remaining part of the URL manually.
Related
I'm trying to write a regular expression to target a URL but not any of its children. My regex is definitely pretty weak and could use some help.
Page I want to target (may include trailing slash and or UTM parameters): https://test.com/deals/
Example of a page I do not want to target: https://test.com/deals/Best-Sellers/c/901
My attempt:
.*Deals\/((?!Best).)*
You can use \/deals\/?(?:[?#]\S*)?$
Check on Regex101
This is a bit more permissive than what your question suggests but it might come in handy.
The main thing is that it tries to match /deals at the end of the line. This ensures that you won't match, say https://test.com/best-deals or similar but only the URL that ends with /deals. Also, the final / is optional - you might get https://test.com/deals.
In addition to that, the regex allows for the URL to end with # anchors or ? followed by parameters. The page might allow this right now or in the future - for example, if a link is used that leads to the same page (e.g. to a specific section), you'd get a # added to the URL. Or there might be something like a filter configuration embedded in the URL https://test.com/deals/?sort=price&productsPerPage=15&page=2&minPrice=100.
Finally, you should make your regex case insensitive to account for the fact the URL might also be https://test.com/Deals/. How you set this flag will depend on where you are using the regex, so I am just adding this as a reminder.
Unfortunately in the last versions of Chrome the negative network filter doesn't work anymore. I used this filter in order to exclude each http call containing a particular string. I asked a solution in Chrome dev tool forum but at the moment nobody answered.
So I would like to know if there is a way to resolve this problem (and exclude for example each call containing the string 'loadMess') with regex syntax.
Update (2018):
This is an update to my old answer to clarify that both bugs have been fixed for some time now.
Negate or exclude filtering is working as expected now. That means you can filter request paths with my.com/path (show requests matching this), or -my.com/path (show requests not matching this).
The regex solution also works after my PR fix made it in production. That means you can also filter with /my.com.path/ and /^((?!my.com/path).)*$/, which will achieve the same result.
I have left the old answer here for reference, and it also explains the negative lookup solution.
The pre-defined negative filters do work, but it doesn't currently allow you to do NOT filters on the names in Chrome stable, only CONTAINS. This is a bug that has been fixed in Chrome Canary.
Once the change has been pushed to Chrome stable, you should be able to do loadMess to filter only for that name, and -loadMess to filter out that name and leave the rest, as it was previously.
Workaround: Regex for matching a string not containing a string
^((?!YOUR_STRING).)*$
Example:
^((?!loadMess).)*$
Explanation:
^ - Start of string
(?!loadMess) - Negative lookahead (at this cursor, do not match the next bit, without capturing)
. - Match any character (except line breaks)
()* - 0 or more of the preceeding group
$ - End of string
Update (2016):
I discovered that there is actually a bug with how DevTools deals with Regex in the Network panel. This means the workaround above doesn't work, despite it being valid.
The Network panel filters on Name and Path (as discovered from the source code), but it does two tests that are OR'ed. In the case above, if you have loadMess in the Name, but not in the Path (e.g. not the domain or directory), it's going to match on either. To clarify, true || false === true, which means it will only filter out loadMess if it's found in both the Name and Path.
I have created an issue in Chromium and have subsequently pushed a fix to be reviewed. This has subsequently been merged.
This is answered here - for latest Chrome 58.0.3029.110 (Official Build) (64-bit)
https://stackoverflow.com/a/27770139/4772631
E.g.: If I want to exclude all gifs then just type -gif
Negative lookahead is recommended everywhere, but it does not work.
Instead, "-myregex" does work for me. Like this: -/(Violation|HMR)/.
Chrome broswer dev tools support regrex filter not very well.
When I want to hide some requests, it does not work as showed above. But you can use -hide1 -hide2 to hide the request you want.
Just leave a space between the conditions, and this does not match the regrex, I guess it may use string match other than regrex in principle
Filtering multiple different urls
You can negate symbol for filtering the network call.
Eg: -lab.com would filter lab.com urls.
But for filtering multiple urls you can use the | symbol in the regex
Eg: -/lab.com|mini.com/ This will filter lab.com and mini.com as well you can use it to filter many different websites or urls.
You can use "Invert" option to exclude the APIs matching a string in the Filter text box.
On latest chrome version (62) you have to use :
-mime-type:image/gif
I am trying to write a regular expression to be used in a Google Analytics goal that will match URLs containing
?package=whatever
and also
/success
The user will first visit a page like
www.website.com/become-client/?package=greatpackage
and if they purchase they will be lead to this page
www.website.com/become-client/?package=greatpackage/success
So based on this I could use the following regex
\?package\=greatpackage/success
This should match the correct destination and I would be able to use this in the goal settings in Analytics to create a goal for purchases of the greatpackage package.
But sometimes the website will use other parameters in addition to ?package. Like ?type, ?media and so on.
?type=business
Resulting in URLs like this
www.website.com/become-client/?package=greatpackage?type=business
and if they purchase they will be lead to this page
www.website.com/become-client/?package=greatpackage?type=business/success
Now the /success part is moved away from the ?package part. My questions is how do I write a regex that will still match this URL no matter what other parameters there may be in between the parts?
---update----
#jonarz proposed the following and it works like a charm.
\?package\=greatpackage(.*?)/success
But what if there are two products with nearly the same name. For example greatpackage and greatpackageULTRA. The code above will select both. If changing the product names is impossible, how can I then select only one of them?
The regex that would solve the problem introduced in the edit, would be:
\?package\=greatpackage((\?|\/)(.*?))?\/success(\/|\b)
Here is a test: https://regex101.com/r/jS4cH5/1 and it seems to suit your needs.
If you want to match an url like this one :
www.website.com/become-client/?package=greatpackage?type=business?other=nada/success
With a group to extract your package type :
.*\?package=([^\/?]+).*\/success
Without group (just matching the url if it's containing package=greatpackage and success)
.*\?package=greatpackage.*\/success
Without group and matching for any package type :
.*\?package=[^\/?]+.*\/success
You just need to add .* to match any char (except new lines). The [^/?]* part is there to be sure your package type isn't empty (ie : the first char isn't a / nor ?).
I am working on the ELK stack and as part of Logstash data transformation i am transforming data in Apache access logs.
One of the metric needed is to get a stat on different content types (aspx, php, gif, etc.).
From the log file I am trying to retrieve request url and then deduce the file type, for ex /c/dataservices/online.jsp?callBack is the request and I would get .aspx using the regular expression
\.\w{3,4}.
My regular expression wont work for request say /etc/designs/design/libs.min.1253.css this is returning me .min as the extension.
I am trying to get the last extension but it is not working. Please do suggest other approaches.
You need to anchor the match to the end of the string or the beginning of a query param ?. Try:
\.\w{3,4}($|\?)
Play with it here: https://regex101.com/r/iV3iM1/1
You're going to need a much fancier Regex.
Try this one.
([/.\w]+)([.][\w]+)([?][\w./=]+)?
This uses three capture groups. The first ([/.\w]+) matches your path up to the last .
The second ([.][\w]+) matches the final extension, and you can use the capture group to read it out.
The third ([?][\w./=]+)? matches the query string, which is optional.
I'm using CodeIgniter to write an app where a user will be allowed to register an account and is assigned a URL (URL slug) of their choosing (ex. domain.com/user-name). CodeIgniter has a URL routing feature that allows the utilization of regular expressions (link).
User's are only allowed to register URL's that contain alphanumeric characters, dashes (-), and under scores (_). This is the regex I'm using to verify the validity of the URL slug: ^[A-Za-z0-9][A-Za-z0-9_-]{2,254}$
I am using the url routing feature to route a few url's to features on my site (ex. /home -> /pages/index, /activity -> /user/activity) so those particular URL's obviously cannot be registered by a user.
I'm largely inexperienced with regular expressions but have attempted to write an expression that would match any URL slugs with alphanumerics/dash/underscore except if they are any of the following:
default_controller
404_override
home
activity
Here is the code I'm using to try to match the words with that specific criteria:
$route['(?!default_controller|404_override|home|activity)[A-Za-z0-9][A-Za-z0-9_-]{2,254}'] = 'view/slug/$1';
but it isn't routing properly. Can someone help? (side question: is it necessary to have ^ or $ in the regex when trying to match with URL's?)
Alright, let's pick this apart.
Ignore CodeIgniter's reserved routes.
The default_controller and 404_override portions of your route are unnecessary. Routes are compared to the requested URI to see if there's a match. It is highly unlikely that those two items will ever be in your URI, since they are special reserved routes for CodeIgniter. So let's forget about them.
$route['(?!home|activity)[A-Za-z0-9][A-Za-z0-9_-]{2,254}'] = 'view/slug/$1';
Capture everything!
With regular expressions, a group is created using parentheses (). This group can then be retrieved with a back reference - in our case, the $1, $2, etc. located in the second part of the route. You only had a group around the first set of items you were trying to exclude, so it would not properly capture the entire wild card. You found this out yourself already, and added a group around the entire item (good!).
$route['((?!home|activity)[A-Za-z0-9][A-Za-z0-9_-]{2,254})'] = 'view/slug/$1';
Look-ahead?!
On that subject, the first group around home|activity is not actually a traditional group, due to the use of ?! at the beginning. This is called a negative look-ahead, and it's a complicated regular expression feature. And it's being used incorrectly:
Negative lookahead is indispensable if you want to match something not followed by something else.
There's a LOT more I could go into with this, but basically we don't really want or need it in the first place, so I'll let you explore if you'd like.
In order to make your life easier, I'd suggest separating the home, activity, and other existing controllers in the routes. CodeIgniter will look through the list of routes from top to bottom, and once something matches, it stops checking. So if you specify your existing controllers before the wild card, they will match, and your wild card regular expression can be greatly simplified.
$route['home'] = 'pages';
$route['activity'] = 'user/activity';
$route['([A-Za-z0-9][A-Za-z0-9_-]{2,254})'] = 'view/slug/$1';
Remember to list your routes in order from most specific to least. Wild card matches are less specific than exact matches (like home and activity), so they should come after (below).
Now, that's all the complicated stuff. A little more FYI.
Remember that dashes - have a special meaning when in between [] brackets. You should escape them if you want to match a literal dash.
$route['([A-Za-z0-9][A-Za-z0-9_\-]{2,254})'] = 'view/slug/$1';
Note that your character repetition min/max {2,254} only applies to the second set of characters, so your user names must be 3 characters at minimum, and 255 at maximum. Just an FYI if you didn't realize that already.
I saw your own answer to this problem, and it's just ugly. Sorry. The ^ and $ symbols are used improperly throughout the lookahead (which still shouldn't be there in the first place). It may "work" for a few use cases that you're testing it with, but it will just give you problems and headaches in the future.
Hopefully now you know more about regular expressions and how they're matched in the routing process.
And to answer your question, no, you should not use ^ and $ at the beginning and end of your regex -- CodeIgniter will add that for you.
Use the 404, Luke...
At this point your routes are improved and should be functional. I will throw it out there, though, that you might want to consider using the controller/method defined as the 404_override to handle your wild cards. The main benefit of this is that you don't need ANY routes to direct a wild card, or to prevent your wild card from goofing up existing controllers. You only need:
$route['404_override'] = 'view/slug';
Then, your View::slug() method would check the URI, and see if it's a valid pattern, then check if it exists as a user (same as your slug method does now, no doubt). If it does, then you're good to go. If it doesn't, then you throw a 404 error.
It may not seem that graceful, but it works great. Give it a shot if it sounds better for you.
I'm not familiar with codeIgniter specifically, but most frameworks routing operate based on precedence. In other words, the default controller, 404, etc routes should be defined first. Then you can simplify your regex to only match the slugs.
Ok answering my own question
I've seem to come up with a different expression that works:
$route['(^(?!default_controller$|404_override$|home$|activity$)[A-Za-z0-9][A-Za-z0-9_-]{2,254}$)'] = 'view/slug/$1';
I added parenthesis around the whole expression (I think that's what CodeIgniter matches with $1 on the right) and added a start of line identifier: ^ and a bunch of end of line identifiers: $
Hope this helps someone who may run into this problem later.