Jmeter Regular Expression Extractor. How to save all returned values to a single variable? - regex

I'm quite new to Jmeter and already spent numerous hours to figure it out.
What i'm trying to achieve:
Using Post Processor Regex Extractor I wrote a regex that returns me several values (already tested it in www.regex101.com and it's working as expected). However, when I do this in Jmeter, I need to provide MatchNo. which in this case will only return to me one certain value. I sort of figured it out that negative digit in this field (Match No) suppose to return all values found. When I use Debug Sampler to find out how many values are returned and to what variables they are assigned, I see a lot of unfamiliar stuff. Please see examples below:
Text where regex to be parsed:
some data here...
"PlanDescription":"DF4-LIB 4224-NNJ"
"PlanDescription":"45U-LIP 2423-NNJ"
"PlanDescription":"PMH-LIB 131-NNJ"
some data here...
As I said earlier, at www.regex101.com I tested this with regex:
\"PlanDescription\":\"([^\"]*)\"
And all needed for me information are correct (with the group 1).
DF4-LIB 4224-NNJ
45U-LIP 2423-NNJ
PMH-LIB 131-NNJ
With the negative number (I tried -1, -2, -3 - same result) at MatchNo. field in Jmeter Regex Extractor field (which Reference Name is Plans) at the Debug Sampler I see the following:
Plans=
Plans_1=DF4-LIB 4224-NNJ
Plans_1_g=1
Plans_1_g0="PlanDescription":"DF4-LIB 4224-NNJ"
Plans_1_g1=DF4-LIB 4224-NNJ
Plans_2=45U-LIP 2423-NNJ
Plans_2_g=1
Plans_2_g0="PlanDescription":"45U-LIP 2423-NNJ"
Plans_2_g1=45U-LIP 2423-NNJ
Plans_3=PMH-LIB 131-NNJ
Plans_3_g=1
Plans_3_g0="PlanDescription":"PMH-LIB 131-NNJ"
Plans_3_g1=PMH-LIB 131-NNJ
I only need at this particular case - Jmeter regex to return 3 values that contain:
DF4-LIB 4224-NNJ
45U-LIP 2423-NNJ
PMH-LIB 131-NNJ
And nothing else. If anybody faced that problem before any help will be appreciated.

Based on output of the Debug Sampler, there's no problem, it's just how RegEx returns the response:
Plans_1,Plans_2,Plans_3 is the actual set of variables you wanted.
There should also be Plans_matchNr which should contain the number of matches (3 in your example). It's important if you loop through them (you will loop from 1 to the value of this variable)
_g sets of variables refer to matching groups per matching instance (3 in your case). Ignore them if you don't care about them. They are always publish, but there's no harm in that.
Once variables are published you can do a number of things:
Use them as ${Plans_1}, ${Plans_2}, ${Plans_3} etc. (as comment above noticed).
Use Plans_... variables in loop: refer to the next variable in the loop as ${__V(Plans_${i})}, where i is a counter with values between 1 and Plans_matchNr
You can also concatenate them into 1 variable using the following simple BeanShell Post-Processor or BeanShell Sampler script:
int count = 0;
String allPlans = "";
// Get number of variables
try {
count = Integer.parseInt(vars.get("Plans_matchNr"));
} catch(NumberFormatException e) {}
// Concatenate them (using space). This could be optimized using StringBuffer of course
for(int i = 1; i <= count; i++) {
allPlans += vars.get("Plans_" + i) + " ";
}
// Save concatenated string into new variable
vars.put("AllPlans", allPlans);
As a result you will have all old variables, plus:
AllPlans=DF4-LIB 4224-NNJ 45U-LIP 2423-NNJ PMH-LIB 131-NNJ

Related

Google Sheets: How can I extract partial text from a string based on a column of different options?

Goal: I have a bunch of keywords I'd like to categorise automatically based on topic parameters I set. Categories that match must be in the same column so the keyword data can be filtered.
e.g. If I have "Puppies" as a first topic, it shouldn't appear as a secondary or third topic otherwise the data cannot be filtered as needed.
Example Data: https://docs.google.com/spreadsheets/d/1TWYepApOtWDlwoTP8zkaflD7AoxD_LZ4PxssSpFlrWQ/edit?usp=sharing
Video: https://drive.google.com/file/d/11T5hhyestKRY4GpuwC7RF6tx-xQudNok/view?usp=sharing
Parameters Tab: I will add words in columns D-F that change based on the keyword data set and there will often be hundreds, if not thousands, of options for larger data sets.
Categories Tab: I'd like to have a formula or script that goes down the columns D-F in Parameters and fills in a corresponding value (in Categories! columns D-F respectively) based on partial match with column B or C (makes no difference to me if there's a delimiter like a space or not. Final data sheet should only have one of these columns though).
Things I've Tried:
I've tried a bunch of things. Nested IF formula with regexmatch works but seems clunky.
e.g. this formula in Categories! column D
=IF(REGEXMATCH($B2,LOWER(Parameters!$D$3)),Parameters!$D$3,IF(REGEXMATCH($B2,LOWER(Parameters!$D$4)),Parameters!$D$4,""))
I nested more statements changing out to the next cell in Parameters!D column (as in , manually adding $D$5, $D$6 etc) but this seems inefficient for a list thousands of words long. e.g. third topic will get very long once all dog breed types are added.
Any tips?
Functionality I haven't worked out:
if a string in Categories B or C contains more than one topic in the parameters I set out, is there a way I can have the first 2 to show instead of just the first one?
e.g. Cell A14 in Categories, how can I get a formula/automation to add both "Akita" & "German Shepherd" into the third topic? Concatenation with a CHAR(10) to add to new line is ideal format here. There will be other keywords that won't have both in there in which case these values will just show up individually.
Since this data set has a bunch of mixed breeds and all breeds are added as a third topic, it would be great to differentiate interest in mixes vs pure breeds without confusion.
Any ideas will be greatly appreciated! Also, I'm open to variations in layout and functionality of the spreadsheet in case you have a more creative solution. I just care about efficiently automating a tedious task!!
Try using custom function:
To create custom function:
1.Create or open a spreadsheet in Google Sheets.
2.Select the menu item Tools > Script editor.
3.Delete any code in the script editor and copy and paste the code below into the script editor.
4.At the top, click Save save.
To use custom function:
1.Click the cell where you want to use the function.
2.Type an equals sign (=) followed by the function name and any input value — for example, =DOUBLE(A1) — and press Enter.
3.The cell will momentarily display Loading..., then return the result.
Code:
function matchTopic(p, str) {
var params = p.flat(); //Convert 2d array into 1d
var buildRegex = params.map(i => '(' + i + ')').join('|'); //convert array into series of capturing groups. Example (Dog)|(Puppies)
var regex = new RegExp(buildRegex,"gi");
var results = str.match(regex);
if(results){
// The for loops below will convert the first character of each word to Uppercase
for(var i = 0 ; i < results.length ; i++){
var words = results[i].split(" ");
for (let j = 0; j < words.length; j++) {
words[j] = words[j][0].toUpperCase() + words[j].substr(1);
}
results[i] = words.join(" ");
}
return results.join(","); //return with comma separator
}else{
return ""; //return blank if result is null
}
}
Example Usage:
Parameters:
First Topic:
Second Topic:
Third Topic:
Reference:
Custom Functions
I've added a new sheet ("Erik Help") with separate formulas (highlighted in green currently) for each of your keyword columns. They are each essentially the same except for specific column references, so I'll include only the "First Topic" formula here:
=ArrayFormula({"First Topic";IF(A2:A="",,IFERROR(REGEXEXTRACT(LOWER(B2:B&C2:C),JOIN("|",LOWER(FILTER(Parameters!D3:D,Parameters!D3:D<>""))))) & IFERROR(CHAR(10)&REGEXEXTRACT(REGEXREPLACE(LOWER(B2:B&C2:C),IFERROR(REGEXEXTRACT(LOWER(B2:B&C2:C),JOIN("|",LOWER(FILTER(Parameters!D3:D,Parameters!D3:D<>""))))),""),JOIN("|",LOWER(FILTER(Parameters!D3:D,Parameters!D3:D<>""))))))})
This formula first creates the header (which can be changed within the formula itself as you like).
The opening IF condition leaves any row in the results column blank if the corresponding cell in Column A of that row is also blank.
JOIN is used to form a concatenated string of all keywords separated by the pipe symbol, which REGEXEXTRACT interprets as OR.
IFERROR(REGEXEXTRACT(LOWER(B2:B&C2:C),JOIN("|",LOWER(FILTER(Parameters!D3:D,Parameters!D3:D<>""))))) will attempt to extract any of the keywords from each concatenated string in Columns B and C. If none is found, IFERROR will return null.
Then a second-round attempt is made:
& IFERROR(CHAR(10)&REGEXEXTRACT(REGEXREPLACE(LOWER(B2:B&C2:C),IFERROR(REGEXEXTRACT(LOWER(B2:B&C2:C),JOIN("|",LOWER(FILTER(Parameters!D3:D,Parameters!D3:D<>""))))),""),JOIN("|",LOWER(FILTER(Parameters!D3:D,Parameters!D3:D<>"")))))
Only this time, REGEXREPLACE is used to replace the results of the first round with null, thus eliminating them from being found in round two. This will cause any second listing from the JOIN clause to be found, if one exists. Otherwise, IFERROR again returns null for round two.
CHAR(10) is the new-line character.
I've written each of the three formulas to return up to two results for each keyword column. If that is not your intention for "First Topic" and "Second Topic" (i.e., if you only wanted a maximum of one result for each of those columns), just select and delete the entire round-two portion of the formula shown above from the formula in each of those columns.

calculate range of values entered by user Custom function Google Appscript

I want to use arrayformula for my custom function if possible because I want to input a range of values
I also get this error: TypeError: Cannot read property "0" from null.
Also, this: Service invoked too many times in a short time: exec qps. Try Utilities.sleep(1000) between calls
var regExp = new RegExp("Item: ([^:]+)(?=\n)");
var matches=new regExp(input);
return matches[0];
}
Really appreciated some help
Edit:
Based on the second picture, I also try using this regex formula to find word start with "Billing address"
But for the first picture, I used regex formula to find word start with "Item"
The error appears the same for both custom function.
If you want to use a custom function which finds all the strings that start with Item or item and extracts the contents from after the finding, you can use the code provided below. The regular expression is checked by using the match() function and returns the desired result; otherwise, it will return null.
function ITEM(input) {
var regEx = /(?:I|i)tem\s*(.*)$/;
var matches = input.match(regEx);
if (matches && matches.length > 1) {
return matches[1];
} else {
return null;
}
}
If you want to use the RegExp like you did in the code you have shared, you should use \\ instead of \.
For checking and verifying the regular expressions you can use this site.
The Service invoked too many times in a short time: exec qps. Try Utilities.sleep(1000) between calls. error message you are getting is due to the fact that you are trying to call the custom function on too many cells - for example dragging the custom function on too many cells at once. You can check more about this error message here.

How to get the last value from jmeter Regular Expression?

I have used regular expression with the following configuration to extract id value:
Regex: `"id":"(.*?)"\,"batch":{"id":"(.*?)"}`
Template is `$1$`
match value is `N`
but I am not getting last value I am getting value on a random basis. please suggest how to get last id value.
The response is given below and wants to extract N1600-2016-11-18-1479503469487 value from the response.
14T21:00:00Z","version":1479157741885,"recordCount":92054,"statusIdentifier":"END"},{"id":"N1600-2016-11-15-1479244284441","batch":{"id":"N1600"},"asOf":"2016-11-15T21:00:00Z","runDate":"2016-11-15T21:00:00Z","version":1479244284441,"recordCount":91838,"statusIdentifier":"END"},{"id":"N1600-2016-11-16-1479330679449","batch":{"id":"N1600"},"asOf":"2016-11-16T21:00:00Z","runDate":"2016-11-16T21:00:00Z","version":1479330679449,"recordCount":91796,"statusIdentifier":"END"},{"id":"N1600-2016-11-17-1479417271814","batch":{"id":"N1600"},"asOf":"2016-11-17T21:00:00Z","runDate":"2016-11-17T21:00:00Z","version":1479417271814,"recordCount":91761,"statusIdentifier":"END"},{"id":"N1600-2016-11-18-1479503469487","batch":{"id":"N1600"},"asOf":"2016-11-18T21:00:00Z","runDate":"2016-11-18T21:00:00Z","version":1479503469487,"recordCount":91693,"statusIdentifier":"END"}]
After adding Beanshell
A. If you are sure that Nth element (say 4th) is the last value every time, then you can hard code the value in Match no. as 4.
B. If you are not sure it is always Nth (4) element, then do as follows:
Mention following values in RegExExtractor:
Regex: `"id":"(.*?)"\,"batch":{"id":"(.*?)"}`
Template is `$1$`
match value is -1 // to store all matched values
Add BeanShell PostProcessor as a child to the same sampler and add the following code:
int size = Integer.parseInt(vars.get("ID_matchNr")); // to get size of matched values
log.info("size:" + size ); // you can remove log stmts once it is working
String newId = "ID_" + size;
String lastId = vars.get(newId); // to retrieve the value
log.info("last id: " + lastId);
vars.put("lastID", lastId); // to store the value
you can refer the last value as follows:
${lastID}
Reference Image:
RegExExtractor:
BeanShell PostProcessor:
Why don't you parse the response as a JSON Object? It's going to be more maintainable than using regular expressions if your json object is modified in a future.

count the number of values returned by regex extractor in jmeter

How do I count the number of samples/values returned by jmeter extractors.
I am using regex to get the list of links to a variable using regex extractor.
By debug sampler shows that there many values returned. until like 9_g1
PlanLinksArray_9_g=1
PlanLinksArray_9_g0=/hix/admin/planmgmt/viewqhpdetail/gLe8eM5psNUTqo8aYXo20w
PlanLinksArray_9_g1=/hix/admin/planmgmt/viewqhpdetail/gLe8eM5psNUTqo8aYXo20w
How can i get the count of this to a variable in Jmeter??
AS per:
http://jmeter.apache.org/usermanual/component_reference.html#postprocessors
it will be called:
PlanLinksArray_9_matchNr : the number of matches found; could be 0
You can get extracted variables count as ${PlanLinksArray_matchNr} it will hold the number of variables like:
PlanLinksArray_1
PlanLinksArray_2
PlanLinksArray_3
etc.
If you need to include match groups as well, like
PlanLinksArray_1_g
PlanLinksArray_1_g0
PlanLinksArray_1_g1
etc.
you'll have to do some scripting.
Add a Beanshell PostProcessor after your Regular Expression extractor
Put the following code into the PostProcessor's "Script" area
JMeterVariables vars = new JMeterVariables();
Iterator iterator = vars.getIterator();
int counter = 0;
while (iterator.hasNext()) {
Map.Entry e = (Map.Entry) iterator.next();
if (e.getKey().startsWith("PlanLinksArray")) {
counter++;
}
}
vars.put("extractedValues", String.valueOf(counter));
Number of variables which names start with "PlanLinksArray" will be available as ${extractedValues} variable
See How to use BeanShell: JMeter's favorite built-in component guide for more Beanshell tips and tricks.

Validating a Salesforce Id

Is there a way to validate a Salesforce ID, maybe using RegEx? They are normally 15 chars or 18 chars but do they follow a pattern that we can use to check that it's a valid id.
There are two levels of validating salesforce id:
check format using regular expression [a-zA-Z0-9]{15}|[a-zA-Z0-9]{18}
for 18-characted ids you can check the the 3-character checksum:
Code examples provided in comments:
C#
Go
Javascript
Ruby
Something like this should work:
[a-zA-Z0-9]{15,18}
It was suggested that this may be more correct because it prevents Ids with lengths of 16 and 17 characters to be rejected, also we try to match against 18 char length first with 15 length as a fallback:
[a-zA-Z0-9]{18}|[a-zA-Z0-9]{15}
Just use instanceOf to check if the string is an instance of Id.
String s = '1234';
if (s instanceOf Id) System.debug('valid id');
else System.debug('invalid id');
The easiest way I've come across, is to create a new ID variable and assign a String to it.
ID MyTestID = null;
try {
MyTestID = MyTestString; }
catch(Exception ex) { }
If MyTestID is null after trying to assign it, the ID was invalid.
This regex has given me the optimal results so far.
\b[a-z0-9]\w{4}0\w{12}|[a-z0-9]\w{4}0\w{9}\b
You can also check for 15 chars, and then add an extra 3 chars optional, with an expression similar to:
^[a-z0-9]{15}(?:[a-z0-9]{3})?$
on i mode, or not:
^[A-Za-z0-9]{15}(?:[A-Za-z0-9]{3})?$
Demo
If you wish to simplify/modify/explore the expression, it's been explained on the top right panel of regex101.com. If you'd like, you can also watch in this link, how it would match against some sample inputs.
RegEx Circuit
jex.im visualizes regular expressions:
Javascript: /^(?=.*?\d)(?=.*?[a-z])[a-z\d]{18}$/i
These were the Salesforce Id validation requirements for me.
18 characters only
At least one digit
At least one alphabet
Case insensitive
Test cases
Should fail
1
a
1234
abgcde
1234aDcde
12345678901234567*
123456789012345678
abcDefghijabcdefgh
Should pass
1234567890abcDeFgh
1234abcd1234abcd12
abcd1234abcd1234ab
1abcDefhijabcdefgf
abcDefghijabcdefg1
12345678901234567a
a12345678901234567
For understanding the regex, please refer this thread
The regex provided by Daniel Sokolowski works perfectly to verify if the id is in the correct format.
If you want to verify if an id corresponds to an actual record in the database, you'll need to first find the object type from the first three characters (commonly known as prefix) and then query the object type:
boolean isValidAndExists(String key) {
Map<String, Schema.SObjectType> objTypes = Schema.getGlobalDescribe();
for (Schema.SObjectType objType : objTypes.values()) {
Schema.DescribeSObjectResult objDesc = objType.getDescribe();
if (objDesc.getKeyPrefix() == key.substring(0,3)) {
String objName = objDesc.getName();
String query = 'SELECT Id FROM ' + objName + ' WHERE Id = \'' + key + '\'';
SObject[] objs = Database.query(query);
return !objs.isEmpty();
}
}
return false;
}
Be aware that Schema.getGlobalDescribe can be an expensive operation and degrade the performance of your application if you use that often.
If you need to check that often, I recommend creating a Custom Setting or Custom Metadata to store the relation between prefixes and object types.
Assuming you want to validate Ids in Apex, there are a few approaches discussed in the other answers. Here is an alternative, with notes on the various approaches.
The try-catch method (credit to #matt_k) certainly works, but some folks worry about overhead, especially if testing many Ids.
I used instanceof Id for a long time (credit to #melani_s), until I discovered that it sometimes gives the wrong answer (e.g., '481D0B74-41CF-47E9').
Multiple answers suggest regexen. As the accepted answer correctly points out (credit to #zacheusz), 18 character Ids are only valid if their checksums are correct, which means the regex solutions can be wrong. That answer also helpfully provides code in several languages to test Id checksums. But not in Apex.
I was going to implement the checksum code in Apex, but then I realized the Salesforce had already done the work, so instead I just convert 18 digit Ids to 15 digit Ids (via .to15() which uses the checksum to fix capitalization, as opposed to truncating the string) and then back to 18 digits to let SF do the checksum calc, then I compare the original checksum and the new one. This is my method:
static Pattern ID_REGEX = Pattern.compile('[a-zA-Z0-9]{15}(?:[A-Z0-5]{3})?');
/**
* #description Determines if a string is a valid SalesforceId. Confirms checksum of 18 digit Ids.
* Works for cases where `x instanceof id` returns the wrong answer, like '481D0B74-41CF-47E9'.
* Does NOT check for the existence of a record with the given Id.
* #param s a string to validate
*
* #return true if the string `s` is a valid Salesforce Id.
*/
public static Boolean isValidId(String s) {
Matcher m = ID_REGEX.matcher(s);
if (m.matches() == false) return false; // if it doesn't match the regex it cannot be valid
if (s.length() == 15) return true; // if 15 char string matches the regex, assume it must be valid
String check = (Id)((Id)s).to15(); // Convert to 15 char Id, then to Id and back to string, giving correct 18-char Id
return s.right(3) == check.right(3); // if 18 char string matches the regex, valid if checksum correct
}
Additionally checking getSObjectType() != null would be perfect if we are dealing with Salesforce records
public static boolean isRecordId(string recordId){
try{
return string.isNotBlank(recordId) && ((Id)recordId.trim()).getSObjectType() != null;
}catch(Exception ex){
return false;
}
}