I am using JMeter and the HTTP Request sampler to load test an web API
Does anyone know if there is a way to configure JMeter so it logs a request as a failure if it takes longer than a certain threshold (e.g. 1000 ms)
Right now if only logs an error if the response is a 500 or something.
The easiest option would be adding a Duration Assertion
However if it is not flexible enough you can always go deeper and use Beanshell. Following updated code sample from previous response by Nithin CV poyyil will mark request as failed if duration exceeds some reasonable value:
long expectedDuration = 1000L;
long actualDuration = prev.getTime();
if (actualDuration > expectedDuration) {
prev.setSuccessful(false);
prev.setResponseMessage("Request took longer than " + expectedDuration);
}
See How to use BeanShell guide for reference and small cookbook.
My suggestion is you need to add Beanshell Postprocessor to request sampler for logging certain requests within particular response time threshold,
long duration = prev.getTime();
if(duration>1000){
log.error("Time Out..... !!!!");
}
Hope this may help you.
Related
I'm getting a browser error when using SustainSys.Saml2 library with my app:
400 Bad Request
Request Header Or Cookie Too Large
nginx/1.14.0
I think that reducing my cookie size might help and I only really need the email from the claim data, so I thought that if I could just save the email claim and remove the other claims, that it might reduce my cookie size and fix this error.
I read the response to a similar question (SustainSys.Saml2 Request length header too long) and looked for some information on how to implement AcsCommandResultCreated to remove unused claims (and hopefully reduce cookie size). I didn't find a lot of documentation, but did piece together some ideas and code to try and take a stab at it.
I've tried this code in my global.asax as well as in a controller action (that I made the "returnUrl" after Saml2/Acs). It doesn't look like my FedAuth cookie (set by Saml2/Acs) is any smaller. Any comments or suggestions? Thank you.
// Check if email claim exists
var principal = ClaimsPrincipal.Current;
var userEmail = principal.Claims.FirstOrDefault(claim => claim.Type == ClaimTypes.Email)?.Value;
// Create new command result that only contains the email claim
if (userEmail != null)
{
var emailClaim = principal.Claims.FirstOrDefault(claim => claim.Type == ClaimTypes.Email);
Sustainsys.Saml2.Configuration.Options.FromConfiguration.Notifications.AcsCommandResultCreated =
(commandResult, response) =>
{
var newCommandResult = new Sustainsys.Saml2.WebSso.CommandResult();
newCommandResult.Principal.Claims.Append(emailClaim);
commandResult = newCommandResult;
};
}
UPDATE:
It turned out that the test environment that I was using (which used nginx) needed to increase the request header buffer size. Adding these cookies increased the size to around 9500 bytes and nginx by default has a request header buffer size that is lower than that (I think 8000). Contacting the code owners of the test server running nginx, and increasing this solved my problem, without me having to reduce my cookie size.
Do you have a lot of failed authentication attempts? That can leave a lot of Saml2.XYZ correlation cookies around on the domain. Try checking the browser dev tools and clean those up.
The "headers too large" is usually something that happens when a user has tried signing in several times with a failure and those cookies get stuck. The real issue is usually something else - causing the authentication to fail and those correlation cookies to be accumulating.
If have multiple databases defined on a particular ArangoDB server, how do I specify the database I'd like an AQL query to run against?
Running the query through the REST endpoint that includes the db name (substituted into [DBNAME] below) ie:
/_db/[DBNAME]/_api/cursor
doesn't seem to work. The error message says 'unknown path /_db/[DBNAME]/_api/cursor'
Is this something I have to specify in the query itself?
Also: The query I'm trying to run is:
FOR col in COLLECTIONS() RETURN col.name
Fwiw, I haven't found a way to set the "current" database through the REST API. Also, I'm accessing the REST API from C++ using fuerte.
Tom Regner deserves primary credit here for prompting the enquiry that produced this answer. I am posting my findings here as an answer to help others who might run into this.
I don't know if this is a fuerte bug, shortcoming or just an api caveat that wasn't clear to me... BUT...
In order for the '/_db/[DBNAME/' prefix in an endpoint (eg full endpoint '/_db/[DBNAME/_api/cursor') to be registered and used in the header of a ::arangodb::fuerte::Request, it is NOT sufficient (as of arangodb 3.5.3 and the fuerte version available at the time of this answer) to simply call:
std::unique_ptr<fuerte::Request> request;
const char *endpoint = "/_db/[DBNAME/_api/cursor";
request = fuerte::createRequest(fuerte::RestVerb::Post,endpoint);
// and adding any arguments to the request using a VPackBuilder...
// in this case the query (omitted)
To have the database name included as part of such a request, you must additionally call the following:
request->header.parseArangoPath(endpoint);
Failure to do so seems to result in an error about an 'unknown path'.
Note 1: Simply setting the database member variable, ie
request->header.database = "[DBNAME]";
does not work.
Note 2: that operations without the leading '/_db/[DBNAME]/' prefix, seem to work fine using the 'current' database. (which at least for me, seems to be stuck at '_system' since as far as I can tell, there doesn't seem to be an endpoint to change this via the HTTP REST Api.)
The docs aren't very helpful right now, so just incase someone is looking for a more complete example, then please consider the following code.
EventLoopService eventLoopService;
// adjust the connection for your environment!
std::shared_ptr<Connection> conn = ConnectionBuilder().endpoint("http://localhost:8529")
.authenticationType(AuthenticationType::Basic)
.user(?) // enter a user with access
.password(?) // enter the password
.connect(eventLoopService);
// create the request
std::unique_ptr<Request> request = createRequest(RestVerb::Post, ContentType::VPack);
// enter the database name (ensure the user has access)
request->header.database = ?;
// API endpoint to submit AQL queries
request->header.path = "/_api/cursor";
// Create a payload to be submitted to the API endpoint
VPackBuilder builder;
builder.openObject();
// here is your query
builder.add("query", VPackValue("for col in collections() return col.name"));
builder.close();
// add the payload to the request
request->addVPack(builder.slice());
// send the request (blocking)
std::unique_ptr<Response> response = conn->sendRequest(std::move(request));
// check the response code - it should be 201
unsigned int statusCode = response->statusCode();
// slice has the response data
VPackSlice slice = response->slices().front();
std::cout << slice.get("result").toJson() << std::endl;
I have a backend listener that posts JMeter results to an external server. It works for most parts. However, I'd like to get cookie info for failed requests. Backend listener only gets SampleResult and I don't see an API to access cookie info from SampleResult. Does anyone have an idea?
View Result Tree listener is able to print out request cookies, so there must be a way. However, it's not obvious what's the class name for that particular listener. If anyone can point me to that source code, it'll be a starting point.
With current JMeter implementation it is not possible unless your create your own AbstractBackendListenerClient implementation which will be cookies-aware.
Also depending on how do you parse result you can substitute a "not interesting" SampleResult field (for example Response Message) with cookie values for failed requests. It can be done using i.e. JSR223 PostProcessor and the following Groovy code
if (!prev.isSuccessful()) {
def manager = sampler.getCookieManager()
def cookies = new StringBuilder()
for (int i = 0; i < manager.getCookieCount(); i++) {
cookies.append(manager.get(i).getName()).append('=').append(manager.get(i).getValue())
}
prev.setResponseMessage(cookies.toString())
}
Hard to help without seeing some code, but one general direction could be this: you can check if your SampleResult is of type HTTPSampleResult, and if yes, get its cookies, using getCookies() method. Whole process could be wrapped as the following function:
public String getCookiesForHttpResult(SampleResult sampleResult) {
if (sampleResult instanceof HTTPSampleResult) {
HTTPSampleResult httpSampleResult = (HTTPSampleResult) sampleResult;
return httpSampleResult.getCookies();
}
// not HTTP result
return null;
}
Something like that is done in JMeter's RequestViewHTTP.java
Getting CookieManager in backend listener can be problematic, since it's mutable: by the time backend listener reads from CookieManager, list of cookies may be different from the time of request execution.
I'm testing API with Postman and I have a problem:
My request goes to sort of middleware, so either I receive a full 1000+ line JSON, or I receive PENDING status and empty array of results:
{
"meta": {
"status": "PENDING",
"missing_connectors_count": 0,
"xxx_type": "INTERNATIONAL"
},
"results": []
}
The question is, how to loop this request in Postman until I will get status SUCCESS and results array > 0?
When I'm sending those requests manually one-by-one it's ok, but when I'm running them through Collection Runner, "PENDING" messes up everything.
I found an awesome post about retrying a failed request by Christian Baumann which allowed me to find a suitable approach to the exact same problem of first polling the status of some operation and only when it's complete run the actual tests.
The code I'd end up if I were you is:
const maxNumberOfTries = 3; // your max number of tries
const sleepBetweenTries = 5000; // your interval between attempts
if (!pm.environment.get("tries")) {
pm.environment.set("tries", 1);
}
const jsonData = pm.response.json();
if ((jsonData.meta.status !== "SUCCESS" && jsonData.results.length === 0) && (pm.environment.get("tries") < maxNumberOfTries)) {
const tries = parseInt(pm.environment.get("tries"), 10);
pm.environment.set("tries", tries + 1);
setTimeout(function() {}, sleepBetweenTries);
postman.setNextRequest(request.name);
} else {
pm.environment.unset("tries");
// your actual tests go here...
}
What I liked about this approach is that the call postman.setNextRequest(request.name) doesn't have any hardcoded request names. The downside I see with this approach is that if you run such request as a part of the collection, it will be repeated a number of times, which might bloat your logs with unnecessary noise.
The alternative I was considering is writhing a Pre-request Script which will do polling (by sending a request) and spinning until the status is some kind of completion. The downside of this approach is the need for much more code for the same logic.
When waiting for services to be ready, or when polling for long-running job results, I see 4 basic options:
Use Postman collection runner or newman and set a per-step delay. This delay is inserted between every step in the collection. Two challenges here: it can be fragile unless you set the delay to a value the request duration will never exceed, AND, frequently, only a small number of steps need that delay and you are increasing total test run time, creating excessive build times for a common build server delaying other pending builds.
Use https://postman-echo.com/delay/10 where the last URI element is number of seconds to wait. This is simple and concise and can be inserted as a single step after the long running request. The challenge is if the request duration varies widely, you may get false failures because you didn't wait long enough.
Retry the same step until success with postman.setNextRequest(request.name);. The challenge here is that Postman will execute the request as fast as it can which can DDoS your service, get you black-listed (and cause false failures), and chew up a lot of CPU if run on a common build server - slowing other builds.
Use setTimeout() in a Pre-request Script. The only downside I see in this approach is that if you have several steps needing this logic, you end up with some cut & paste code that you need to keep in sync
Note: there are minor variations on these - like setting them on a collection, a collection folder, a step, etc.
I like option 4 because it provides the right level of granularity for most of my cases. Note that this appears to be the only way to "sleep" in a Postman script. Now standard javascript sleep methods like a Promise with async and await are not supported and using the sandbox's lodash _.delay(function() {}, delay, args[...]) does not keep script execution on the Pre-request script.
In Postman standalone app v6.0.10, set your step Pre-request script to:
console.log('Waiting for job completion in step "' + request.name + '"');
// Construct our request URL from environment variables
var url = request['url'].replace('{{host}}', postman.getEnvironmentVariable('host'));
var retryDelay = 1000;
var retryLimit = 3;
function isProcessingComplete(retryCount) {
pm.sendRequest(url, function (err, response) {
if(err) {
// hmmm. Should I keep trying or fail this run? Just log it for now.
console.log(err);
} else {
// I could also check for response.json().results.length > 0, but that
// would omit SUCCESS with empty results which may be valid
if(response.json().meta.status !== 'SUCCESS') {
if (retryCount < retryLimit) {
console.log('Job is still PENDING. Retrying in ' + retryDelay + 'ms');
setTimeout(function() {
isProcessingComplete(++retryCount);
}, retryDelay);
} else {
console.log('Retry limit reached, giving up.');
postman.setNextRequest(null);
}
}
}
});
}
isProcessingComplete(1);
And you can do your standard tests in the same step.
Note: Standard caveats apply to making retryLimit large.
Try this:
var body = JSON.parse(responseBody);
if (body.meta.status !== "SUCCESS" && body.results.length === 0){
postman.setNextRequest("This_same_request_title");
} else {
postman.setNextRequest("Next_request_title");
/* you can also try postman.setNextRequest(null); */
}
I was searching for an answer to the same question and thought of a possible solution as I was reading your question.
Use postman workflow to rerun your request every time you don't get the response you're looking for. Anyway, that's what I'm gonna try.
postman.setNextRequest("request_name");
https://www.getpostman.com/docs/workflows
I didn't succeed to find the complete guidelines for this issue that's why I decided to invest some time and to describe all steps of the process from A to Z.
I will be observing an example where we will need to pass through transaction ids and in each iteration to change query param for next transaction id from the list.
Step 1. Prepare your request
https://some url/{{queryParam}}
Add {{queryParam}} variable for changing it from pre-request script.
If you need a token for request you should add it here, in Authorization tab.
Save request to collection (Save button in the right corner). For demonstration purpose I will use "Transactions Request" name. We will need to use this name later on.
Step 2. Prepare pre-request script
In postman use tab Pre-request Script to change transactionId variable from query param to actual transaction id.
let ids = pm.collectionVariables.get("TransactionIds");
ids = JSON.parse(ids);
const id = ids.shift();
console.log('id', id)
postman.setEnvironmentVariable("transactionId", id);
pm.collectionVariables.set("TransactionIds", JSON.stringify(ids));
pm.collectionVariables.get - gets array of transaction ids from collection variables. We will set it up in Step 4.
ids.shift() - we use it to remove id that we will use from our ids list (to prevent running twice on the same id)
postman.setEnvironmentVariable("transactionId", id) - change transaction id from query param to actual transaction id
pm.collectionVariables.set("TransactionIds", JSON.stringify(ids)) - we are setting up a new collection of variables that now does not include the id that was handled.
Step 3. Prepare Tests
In postman use tab Tests to create a loop logic. Tests will be executed after the request execution, so we can use it to make next request.
let ids = pm.collectionVariables.get("TransactionIds");
ids = JSON.parse(ids);
if (ids && ids.length > 0){
console.log('length', ids.length);
postman.setNextRequest("Transactions Request");
} else {
postman.setNextRequest(null);
}
postman.setNextRequest("Transactions Request") - calls a new request, in this case it will call the "Transactions Request" request
Step 4. Run Collections
In Postman from the left side bar you should choose Collections (click on it) and then choose a tab Variables.
This is the collection variables. In our example we used TransactionIds as a variable, so put in Current Value the array of transaction ids on which you want to loop.
Now you can click on Run (the button from right corner, near Save button) to run our loop requests.
You will be proposed to choose on which request you want to perform an action. Choose the request that we’ve created "Transactions Request".
It will run our request with pre-request script and with logic that we’ve set in Tests. In the end postman will open a new window with summary of our run.
So in my Test Plan I have a Cookie Manager setup inside my Thread Group which sets a specific Cookie value for 1 Cookie. Let's call it, MYID. I'm trying to figure out a way to verify that this specific Cookie's value was used to complete this one HTTP Request, because if I set my MYID to a specific value *(which actually tells which web server to go to), say to "Server1", but Server1 is down, unavailable, etc... HAProxy should change this and send you to Server2.
So basically I want to try and make sure that Cookie MYID was equal to "Server1" all the way through the HTTP Request.
I am trying to use a BeanShell PostProcessor to verify the Cookie's value after the request is ran, but when I tried using some code I have inside a PreProcessor that sets a cookie in a different Test Plan of mine I get an error saying:
Error Message:
Typed variable declaration : Attempt to resolve method: getCookieManager() on undefined variable or class name: sampler
And below here is the Code slightly modified from a BeanShell PreProcessor in another Test Plan I have...
CODE:
import org.apache.jmeter.protocol.http.control.Cookie;
import org.apache.jmeter.protocol.http.control.CookieManager;
CookieManager manager = sampler.getCookieManager();
for (int i = 0; i < manager.getCookieCount(); i++) {
Cookie cookie = manager.get(i);
if (cookie.getName().equals("MYID")) {
if (cookie.getValue().equals("Server1")) {
log.info("OK: The Cookie contained the Correct Server Number...");
} else {
log.info("ERROR: The Cookie did NOT contain the Correct Server Number...");
}
break;
}
}
For the error, I was thinking the "sampler" object was no longer available since the Request was already run, or something along those lines, but I'm not sure...
Or, is there another JMeter object I should be using instead of the "BeanShell PostProcessor" in order to verify the Cookie's value was correct..?
Any thoughts or suggestion would be greatly appreciated!
Thanks in Advance,
Matt
If you trying to get cookie manager from the parent sampler in the Beanshell PostProcessor - you need to use ctx.getCurrentSampler(), not "sampler" as it is not exposed in script variables.
So just change this line:
CookieManager manager = sampler.getCookieManager();
to
CookieManager manager = ctx.getCurrentSampler().getCookieManager();
And your script should start working as you expect.
ctx is a shorthand to JMeterContext instance and getCurrentSampler() method name is self-explanatory.
For more information on Beanshell scripting check out How to use BeanShell: JMeter's favorite built-in component guide.