I'm retrieving a list of Work Items using the VSTS API and would like to display them on my web app. I can successfully return a list of the work items in the format below:
{"count":1,"value":[{"id":246,"rev":4,"fields":{"System.Id":246,"System.State":"New","System.Title":"test1"},"url":"https://example.visualstudio.com/_apis/wit/workItems/246"}]}
I have tried a regular expression to get the values from this HTTP response with the following code:
HttpResponseMessage getWorkItemsHttpResponse = client.GetAsync("_apis/wit/workitems?ids=" + ids + "&fields=System.Id,System.Title,System.State&asOf=" + workItemQueryResult.asOf + "&api-version=2.2").Result;
if (getWorkItemsHttpResponse.IsSuccessStatusCode)
{
result = getWorkItemsHttpResponse.Content.ReadAsStringAsync().Result;
// Regular expression to extract work item values to display
string parseWI = result.ToString();
var match = Regex.Match(parseWI, "\"System.ID\": (.*)");
workItemsToDisplay = (match.Groups[1].Value);
}
}
}
}
return workItemsToDisplay;
}
This is refusing to return anything though and leaves the textbox I display the workItemsToDisplay in empty. I'm not familiar with regular expressions and i'm sure this is where the issue stems from. Not sure if Microsoft already has sample code to construct a display of Work Items from the response.
Don't use a regex. That's JSON, use a JSON parsing library (JSON.Net is the de facto standard in the .NET world) and then you can easily retrieve specific fields.
Related
Well i'm a student , and i'm still learning the dart language and the flutter framework, I was trying to make an application that makes you able to login into a site with a http post request and get data by manipulating the response of the html source code with some regular expressions to get what you need from the website,
(something like data scraping)
I tried to do that but nothing worked as planned.
I did this project! years ago and it was for desktop, with vb.net, I used a library called xNet which helped me to do that.
For this case I used the http dart package.
Is this kind of work can be done with dart?
Is there any specific packages for this?
Is there any docs available ?
I know html is not a regular language, i asked if it is possible to use http requests to login into a site!?
if i can do that i can manipulate the response and get what i need with some regular expressions.
I wanna do something like
C#
using (HttpRequest req = new HttpRequest())
{
req.UserAgent = Http.ChromeUserAgent;
req.Cookies = new CookieDictionary(false);
req.Proxy = null;
req.IgnoreProtocolErrors = true;
req.AddParam("login", cin.Text);
req.AddParam("no_anti_inject_password", pass.Text);
try {
string Respo = req.Post("http://www.example.com/login.php").ToString;
// to with that 'Respo'
if (Respo.Contains("disconnect"))
{
//Logged
//example
Match NAME = Regex.Match(Respo, "(.*?)");
name.Text = "Name: " + NAME.Groups(1).Value;
}else{
//not logged
//some code...
}
catch{
//some exception
}
}
HTML is not a regular language and so a regular expression is not a good way to scrape data from html. You may be interested in package:html which implements an HTML parser.
I have integrated oAuth2 (Facebook, LinkedIn, etc) with my XPages app to allow for authentication to easily add comments (response docs). When a user authenticates, it has to redirect to the facebook/linkedin page, then return to complete the document creation. I use the state variable to do this, and pass it in the querystring of the url. When the page reloads and sees the state variable, it calls a "beforePageLoad" event and creates the response document if the user authenticated and has the correct state document.
My problem is when there is already a state parameter in the querystring. I want to replace the value, not add it to the end. I use a solution here from stackOverflow by ellemayo called updateQueryStringParameter. When I call it from my beforePageLoads it runs, but never replaces the parameter, it only appends it to the end. I end up with ...&state=E5A&state=E5F
I have a feeling that it is in the line,
return uri.replace(re, '$1' + key + "=" + value + '$2');
I can write the code using #ReplaceSubstring(), etc, but want to know if there are problems running regex in XPages SSJS. I read on Lotus.com that
A Regular Expression can be specified as Server-side, which uses the
Java (java.util.regex) API or Client-side, which uses the browser
JavaScript Regular Expression Engine. Client-side and Server-side
Regular Expression syntax is similar, but there are differences that a
user must be aware of.
Should I avoid regex in XPages SSJS ? I have it working extensively in client and in some field validations on the XPage itself.
Here is the call to the function:
if(#Contains( qString,"state=")){
qString=updateQueryStringParameter(qString, "state", linkDoc.getNoteID() );
}else{
qString="?"+qString+"&state=" + linkDoc.getNoteID()
}
the function:
function updateQueryStringParameter(uri, key, value) {
var re = new RegExp("([?&])" + key + "=.*?(&|$)", "i");
var separator = uri.indexOf('?') !== -1 ? "&" : "?";
if (uri.match(re)) {
// I also tried --> if (re.test(uri)) {
return uri.replace(re, '$1' + key + "=" + value + '$2');
}
else {
return uri + separator + key + "=" + value;
}
}
It was not an XPage or Regex problem. I was using the querystring provided by Domino the excludes the "?" as part of the querystring. when I send "?" + qString to the function, it works. Regex needed to know where to start looking, thus it never found the start of the query string.
Im trying to execute/consume a webservice and wondering if I am using the correct data type to return the results. String seems to work, but I receive an empty string. The service should be returning a simple string value without XML. There is a working version written in JS below, I have been asked to recreate it in Apex.
JS version (Working) - executed when a button is clicked
{!REQUIRESCRIPT("/soap/ajax/24.0/connection.js")}
{!REQUIRESCRIPT("/soap/ajax/24.0/apex.js")}
var xfolder = "TestFolder"
var parentid = "22K22"
var myvar = sforce.apex.execute("myWS","invokeWs", {folderName:xfolder,ObjectID:parentid});
window.alert('LiveLink folder created: ' + myvar);
APEX version (not working)
public with sharing class myTest {
public String getWSXMLResult() {
String tmpFolderName2 = 'TestFolder';
String tmpObjectID2 = '22K22';
String myWSXMLResult = myWS.invokeWs(tmpFolderName2,tmpObjectID2);
System.debug('XIXWS|' + myWSXMLResult);
return myWSXMLResult;
}
}
One thing I just noted while typing this out. I didn't specify the argument names for invokeWs, just the values..do I need to specify those values in the call to the WS? Such as..
myWS.invokeWs(folderName=tmpFolderName2,ObjectID=tmpObjectID2); -- this errors out btw
Thanks again everyone.
I'm testing a web service that returns JSON responses and I'd like to pull multiple values from the response. A typical response would contain multiple values in a list. For example:
{
"name":"#favorites",
"description":"Collection of my favorite places",
"list_id":4894636,
}
A response would contain many sections like the above example.
What I'd like to do in Jmeter is go through the JSON response and pull each section outlined above in a manner that I can tie the returned name and description as one entry to iterate over.
What I've been able to do thus far is return the name value with regular expression extractor ("name":"(.+?)") using the template $1$. I'd like to pull both name and description but can't seem to get it to work. I've tried using a regex "name":"(.+?)","description":"(.+?)" with a template of $1$$2$ without any success.
Does anyone know how I might pull multiple values using regex in this example?
You can just add (?s) to the regex to avoid line breaks.
E.g: (?s)"name":"(.+?)","description":"(.+?)"
It works for me on assertions.
It may be worth to use BeanShell scripting to process JSON response.
So if you need to get ALL the "name/description" pairs from response (for each section) you can do the following:
1. extract all the "name/description" pairs from response in loop;
2. save extracted pairs in csv-file in handy format;
3. read saved pairs from csv-file later in code - using CSV Data Set Config in loop, e.g.
JSON response processing can be implemented using BeanShell scripting (~ java) + any json-processing library (e.g. json-rpc-1.0):
- either in BeanShell Sampler or in BeanShell PostProcessor;
- all the required beanshell libs are currently provided in default
jmeter delivery;
- to use json-processing library place jar into JMETER_HOME/lib folder.
Schematically it will look like:
in case of BeanShell PostProcessor:
Thread Group
. . .
YOUR HTTP Request
BeanShell PostProcessor // added as child
. . .
in case of BeanShell Sampler:
Thread Group
. . .
YOUR HTTP Request
BeanShell Sampler // added separate sampler - after your
. . .
In this case there is no difference which one use.
You can either put the code itself into the sampler body ("Script" field) or store in external file, as shown below.
Sampler code:
import java.io.*;
import java.util.*;
import org.json.*;
import org.apache.jmeter.samplers.SampleResult;
ArrayList nodeRefs = new ArrayList();
ArrayList fileNames = new ArrayList();
String extractedList = "extracted.csv";
StringBuilder contents = new StringBuilder();
try
{
if (ctx.getPreviousResult().getResponseDataAsString().equals("")) {
Failure = true;
FailureMessage = "ERROR: Response is EMPTY.";
throw new Exception("ERROR: Response is EMPTY.");
} else {
if ((ResponseCode != null) && (ResponseCode.equals("200") == true)) {
SampleResult result = ctx.getPreviousResult();
JSONObject response = new JSONObject(result.getResponseDataAsString());
FileOutputStream fos = new FileOutputStream(System.getProperty("user.dir") + File.separator + extractedList);
if (response.has("items")) {
JSONArray items = response.getJSONArray("items");
if (items.length() != 0) {
for (int i = 0; i < items.length(); i++) {
String name = items.getJSONObject(i).getString("name");
String description = items.getJSONObject(i).getString("description");
int list_id = items.getJSONObject(i).getInt("list_id");
if (i != 0) {
contents.append("\n");
}
contents.append(name).append(",").append(description).append(",").append(list_id);
System.out.println("\t " + name + "\t\t" + description + "\t\t" + list_id);
}
}
}
byte [] buffer = contents.toString().getBytes();
fos.write(buffer);
fos.close();
} else {
Failure = true;
FailureMessage = "Failed to extract from JSON response.";
}
}
}
catch (Exception ex) {
IsSuccess = false;
log.error(ex.getMessage());
System.err.println(ex.getMessage());
}
catch (Throwable thex) {
System.err.println(thex.getMessage());
}
As well a set of links on this:
JSON in JMeter
Processing JSON Responses with JMeter and the BSF Post Processor
Upd. on 08.2017:
At the moment JMeter has set of built-in components (merged from 3rd party projects) to handle JSON without scripting:
JSON Path Extractor (contributed from ATLANTBH jmeter-components project);
JSON Extractor (contributed from UBIK Load Pack since JMeter 3.0) - see answer below.
I am assuming that JMeter uses Java-based regular expressions... This could mean no named capturing groups. Apparently, Java7 now supports them, but that doesn't necessarily mean JMeter would. For JSON that looks like this:
{
"name":"#favorites",
"description":"Collection of my favorite places",
"list_id":4894636,
}
{
"name":"#AnotherThing",
"description":"Something to fill space",
"list_id":0048265,
}
{
"name":"#SomethingElse",
"description":"Something else as an example",
"list_id":9283641,
}
...this expression:
\{\s*"name":"((?:\\"|[^"])*)",\s*"description":"((?:\\"|[^"])*)",(?:\\}|[^}])*}
...should match 3 times, capturing the "name" value into the first capturing group, and the "description" into the second capturing group, similar to the following:
1 2
--------------- ---------------------------------------
#favorites Collection of my favorite places
#AnotherThing Something to fill space
#SomethingElse Something else as an example
Importantly, this expression supports quote escaping in the value portion (and really even in the identifier name portion as well, so that the Javascript string I said, "What is your name?"! will be stored in JSON as AND parsed correctly as I said, \"What is your name?\"!
Using Ubik Load Pack plugin for JMeter which has been donated to JMeter core and is since version 3.0 available as JSON Extractor you can do it this way with following Test Plan:
namesExtractor_ULP_JSON_PostProcessor config:
descriptionExtractor_ULP_JSON_PostProcessor config:
Loop Controller to loop over results:
Counter config:
Debug Sampler showing how to use name and description in one iteration:
And here is what you get for the following JSON:
[{ "name":"#favorites", "description":"Collection of my favorite places", "list_id": 4894636 }, { "name":"#AnotherThing", "description":"Something to fill space", "list_id": 48265 }, { "name":"#SomethingElse", "description":"Something else as an example", "list_id":9283641 }]
Compared to Beanshell solution:
It is more "standard approach"
It performs much better than Beanshell code
It is more readable
I'm using selenium RC and I would like, for example, to get all the links elements with attribute href that match:
http://[^/]*\d+com
I would like to use:
sel.get_attribute( '//a[regx:match(#href, "http://[^/]*\d+.com")]/#name' )
which would return a list of the name attribute of all the links that match the regex.
(or something like it)
thanks
The answer above is probably the right way to find ALL of the links that match a regex, but I thought it'd also be helpful to answer the other part of the question, how to use regex in Xpath locators. You need to use the regex matches() function, like this:
xpath=//div[matches(#id,'che.*boxes')]
(this, of course, would click the div with 'id=checkboxes', or 'id=cheANYTHINGHEREboxes')
Be aware, though, that the matches function is not supported by all native browser implementations of Xpath (most conspicuously, using this in FF3 will throw an error: invalid xpath[2]).
If you have trouble with your particular browser (as I did with FF3), try using Selenium's allowNativeXpath("false") to switch over to the JavaScript Xpath interpreter. It'll be slower, but it does seem to work with more Xpath functions, including 'matches' and 'ends-with'. :)
You can use the Selenium command getAllLinks to get an array of the ids of links on the page, which you could then loop through and check the href using the getAttribute, which takes the locator followed by an # and the attribute name. For example in Java this might be:
String[] allLinks = session().getAllLinks();
List<String> matchingLinks = new ArrayList<String>();
for (String linkId : allLinks) {
String linkHref = selenium.getAttribute("id=" + linkId + "#href");
if (linkHref.matches("http://[^/]*\\d+.com")) {
matchingLinks.add(link);
}
}
A possible solution is to use sel.get_eval() and write a JS script that returns a list of the links. something like the following answer:
selenium: Is it possible to use the regexp in selenium locators
Here's some alternate methods as well for Selenium RC. These aren't pure Selenium solutions, they allow interaction with your programming language data structures and Selenium.
You can also get get HTML page source, then regular expression the source to return a match set of links. Use regex grouping to separate out URLs, link text/ID, etc. and you can then pass them back to selenium to click on or navigate to.
Another method is get HTML page source or innerHTML (via DOM locators) of a parent/root element then convert the HTML to XML as DOM object in your programming language. You can then traverse the DOM with desired XPath (with regular expression or not), and obtain a nodeset of only the links of interest. From their parse out the link text/ID or URL and you can pass back to selenium to click on or navigate to.
Upon request, I'm providing examples below. It's mixed languages since the post didn't appear to be language specific anyways. I'm just using what I had available to hack together for examples. They aren't fully tested or tested at all, but I've worked with bits of the code before in other projects, so these are proof of concept code examples of how you'd implement the solutions I just mentioned.
//Example of element attribute processing by page source and regex (in PHP)
$pgSrc = $sel->getPageSource();
//simple hyperlink extraction via regex below, replace with better regex pattern as desired
preg_match_all("/<a.+href=\"(.+)\"/",$pgSrc,$matches,PREG_PATTERN_ORDER);
//$matches is a 2D array, $matches[0] is array of whole string matched, $matches[1] is array of what's in parenthesis
//you either get an array of all matched link URL values in parenthesis capture group or an empty array
$links = count($matches) >= 2 ? $matches[1] : array();
//now do as you wish, iterating over all link URLs
//NOTE: these are URLs only, not actual hyperlink elements
//Example of XML DOM parsing with Selenium RC (in Java)
String locator = "id=someElement";
String htmlSrcSubset = sel.getEval("this.browserbot.findElement(\""+locator+"\").innerHTML");
//using JSoup XML parser library for Java, see jsoup.org
Document doc = Jsoup.parse(htmlSrcSubset);
/* once you have this document object, can then manipulate & traverse
it as an XML/HTML node tree. I'm not going to go into details on this
as you'd need to know XML DOM traversal and XPath (not just for finding locators).
But this tutorial URL will give you some ideas:
http://jsoup.org/cookbook/extracting-data/dom-navigation
the example there seems to indicate first getting the element/node defined
by content tag within the "document" or source, then from there get all
hyperlink elements/nodes and then traverse that as a list/array, doing
whatever you want with an object oriented approach for each element in
the array. Each element is an XML node with properties. If you study it,
you'd find this approach gives you the power/access that WebDriver/Selenium 2
now gives you with WebElements but the example here is what you can do in
Selenium RC to get similar WebElement kind of capability
*/
Selenium's By.Id and By.CssSelector methods do not support Regex and By.XPath only does where XPath 2.0 is enabled. If you want to use Regex, you can do something like this:
void MyCallingMethod(IWebDriver driver)
{
//Search by ID:
string attrName = "id";
//Regex = 'a number that is 1-10 digits long'
string attrRegex= "[0-9]{1,10}";
SearchByAttribute(driver, attrName, attrRegex);
}
IEnumerable<IWebElement> SearchByAttribute(IWebDriver driver, string attrName, string attrRegex)
{
List<IWebElement> elements = new List<IWebElement>();
//Allows spaces around equal sign. Ex: id = 55
string searchString = attrName +"\\s*=\\s*\"" + attrRegex +"\"";
//Search page source
MatchCollection matches = Regex.Matches(driver.PageSource, searchString, RegexOptions.IgnoreCase);
//iterate over matches
foreach (Match match in matches)
{
//Get exact attribute value
Match innerMatch = Regex.Match(match.Value, attrRegex);
cssSelector = "[" + attrName + "=" + attrRegex + "]";
//Find element by exact attribute value
elements.Add(driver.FindElement(By.CssSelector(cssSelector)));
}
return elements;
}
Note: this code is untested. Also, you can optimize this method by figuring out a way to eliminate the second search.