I am using postman to test my service. Each test case contains 4 requests: A, B, C, D. Right now, I can test them slightly automatically by writing script that put the value from the previous request to the next one (A to B, B to C).
It's kind of fast but I still need to switch between tabs and press Send 4 times. Then I found out that I can use runner to help me run all of them in a single click.
It's sound great until I realize that I need to send the D request right after the C request sent without waiting for C to response, otherwise my test case will always fail.
Is there anyway to accomplish this? Does postman support a function to trigger another request?
Related
Using Siddhi CEP 3.1.2 as standalone library, I am using try figure out how to correctly trigger the callback handler under a number of conditions.
Two events joined by logical AND
a AND b
I find with the above that if i provide both events, then i can trigger the callback handler, however I've also seen that if i subsequently provide either a or b then the handler is also triggered. I did not expect the latter to happen as I assumed there would be no match due to the previous execution of the handler... Is there a way to clear the streams following a successful match?
Two events joined by logical AND including a within.
a AND b within 5 sec
I've also found in the second case, that the "within" option is seemingly ignored. The callback is triggered regardless of the time gap between the events a and b.
Have I misunderstood the Siddhi documentation? I'd appreciate any guidance on these. Thanks
These are two bugs based on our testing and fixed with the PR #436.
The following test cases are added to ensure the correct behavior:
Test A and B
Test every (A and B)
Test A -> B and C within 1 sec
You will get these fixes from the next release onwards or else you can build Siddhi from source and test them right now. Here is a blog post on how to build and use Siddhi from source: Siddhi 4.0.0 Early Access.
Hard to write a good title for this question. I am developing a performance test in Gatling for a SOAP Webservice. I'm not very experienced with Gatling so I'm learning things as I go, but this conundrum has me entirely stumped.
One of the scenarios I am implementing a test for is an order-process consisting of several unique consecutive calls to the webservice, one of which is a polling call that returns the current status of the ordering process. Simplified, this call gets a SOAP Response with a status that can be of three types:
PROCESSING - Signifying the order is still processing.
ORDER_OK - Order completed without errors.
EVERYTHING_ELSE - A group of varying error-statuses and other results.
What I want to do, is have Gatling continuously poll the webservice until the processing-status changes - and then check that the status says it completed successfully. Polling continuously is easily implemented, but performing the check after it completes is turning out to be a far greater challenge than it has any business being.
So far, this is what I've done to solve the polling:
exec { session => session.set("status", "PROCESSING") }
.asLongAs(session => session("status").as[String].equals("PROCESSING")) {
exec(http("Poll order")
.post("/MyWebService")
.body(ELFileBody("bodies/ws/pollOrder.xml"))
.check(
status.is(200),
regex("soapFault").notExists,
regex("pollResponse").exists,
xpath("//*[local-name(.)='result']").exists.saveAs("status")
)
).exitHereIfFailed.pause(5 seconds)
}
This snip appears to be performing the polling correctly, it continues to poll until the orderStatus changes from processing to something else. I need to check the status to see if it changed to the response I am interested in however, because I don't know what it is, and only one of the many results it can be should cause the scenario to continue for that user.
A potential fix would be to add more checks in that call that go something like this:
.check(regex("EVERYTHING_ELSE_XYZ")).notExists
The service can return a LOT of different "not a happy day" messages however and I'm only really interested in the two other ones, so it would be preferable for me to be able to do a check only for the two valid happy-day responses. Checking if one exact thing exists seems far more sensible than checking that dozens of things don't.
What I thought I would be able to do was performing a check on the status variable in the users session when the step exits the asLongAs-loop, and continue/exit the scenario for that user. As it's a session-variable I could probably do this in the next step of the total scenario and break the run for that user there, but that would also mean the error is reported in the wrong place, and the next calls fault-% would be polluted by errors from the previous call.
Using pseudocode, being able to do something like this immediately after it exits the asLongAs loop would have been perfect:
if (session("status").as[String].equals("ORDER_OK")) ? continueTheScenario : failTheScenario
but I've not been able to do anything similar to that inside a gatling-chain. It's almost starting to appear impossible to do something like that, but can anyone see a solution to this that I'm not seeing?
Instead of "exists", use "in" to check that the result is one of the 2 valid values.
I forget, is there ever a situation where you may not get an http response back? Let's say you send a request to some API, and it bombs on their side. They're supposed to set a status code if that happens but I assume there have to be times where there could be other variables that could fail in which you might not get a response back.
I'm trying to setup some of my TDD. I think testing whether I get a non-null response back is a good first 'simplest as possible' test to start out with.
Well, I would suggest that having a test for checking only that response is not null is almost worthless. TDD is not about writing infinite little tests to develop something (like testing that constructor actually creates an object etc.), but that is another topic altogether.
Back on the topic, there could be a situation where the network fails, so you wouldn't get a response at all.
I have been assigned for automated testing of Web services to achieve the following in soapUI:
Basically before the soap message is send everytime it should be replaced with another test data. For example, if I have 10 test cases, then the next test case overwrites the previous one. Test Case 10 is what I finally see on the screen. The advantage is that you only need one soap request message and through a loop the soap message is refilled with the new test data(data driven) and sent.
The problem I consider now is that for each response message applies different assertions. It’s not possible to have global assertions for all response messages because every response message looks different and therefore it is necessary to have specific assertions for each response message. The problem is that I can’t hold the assertions for the specific response message because it’s then rewritten by the next response message. The response message by testcase 10 is what I finally see on the screen. You got the problem
A possible solution:
Save each response message separately.
Get each response message separately and enter the corresponding assertions for that specific respons.
Does anyone have an elegant solution or experience with the above and got it working well ?
You can solve this using a data source and data loop.
In the data source, have your input values AND your assertion.
Then, you just make the response assertion a variable pointing to the data source. Structure would then look like this:
- Test Steps
-- Data Source
-- SOAP Request
---- Assertion
-- Data Loop
We are building a REST service that will take about 5 minutes to execute. It will be only called a few times a day by an internal app. Is there an issue using a REST (ie: HTTP) request that takes 5 minutes to complete?
Do we have to worry about timeouts? Should we be starting the request in a separate thread on the server and have the client poll for the status?
This is one approach.
Create a new request to perform ProcessXYZ
POST /ProcessXYZRequests
201-Created
Location: /ProcessXYZRequest/987
If you want to see the current status of the request:
GET /ProcessXYZRequest/987
<ProcessXYZRequest Id="987">
<Status>In progress</Status>
<Cancel method="DELETE" href="/ProcessXYZRequest/987"/>
</ProcessXYZRequest>
when the request is finished you would see something like
GET /ProcessXYZRequest/987
<ProcessXYZRequest>
<Status>Completed</Status>
<Results href="/ProcessXYZRequest/Results"/>
</ProcessXYZRequest>
Using this approach you can easily imagine what the following requests would give
GET /ProcessXYZRequests/Pending
GET /ProcessXYZRequests/Completed
GET /ProcessXYZRequests/Failed
GET /ProcessXYZRequests/Today
Assuming that you can configure HTTP timeouts using whatever framework you choose, then you could request via a GET and just hang for 5 mins.
However it may be more flexible to initiate an execution via a POST, get a receipt (a number/id whatever), and then perform a GET using that 5 mins later (and perhaps retry given that your procedure won't take exactly 5 mins every time). If the request is still ongoing then return an appropriate HTTP error code (404 perhaps, but what would you return for a GET with a non-existant receipt?), or return the results if available.
As Brian Agnew points out, 5 minutes is entirely manageable, if somewhat wasteful of resources, if one can control timeout settings. Otherwise, at least two requests must be made: The first to get the result-producing process rolling, and the second (and third, fourth, etc., if the result takes longer than expected to compile) to poll for the result.
Brian Agnew and Darrel Miller both suggest similar approaches for the two(+)-step approach: POST a request to a factory endpoint, starting a job on the server, and later GET the result from the returned result endpoint.
While the above is a very common solution, and indeed adheres to the letter of the REST constraints, it smells very much of RPC. That is, rather than saying, "provide me a representation of this resource", it says "run this job" (RPC) and then "provide me a representation of the resource that is the result of running the job" (REST). EDIT: I'm speaking very loosely here. To be clear, none of this explicitly defies the REST constraints, but it does very much resemble dressing up a non-RESTful approach in REST's clothing, losing out on its benefits (e.g. caching, idempotency) in the process.
As such, I would rather suggest that when the client first attempts to GET the resource, the server should respond with 202 "Accepted" (http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.2.3), perhaps with "try back in 5 minutes" somewhere in the response entity. Thereafter, the client can poll the same endpoint to GET the result, if available (otherwise return another 202, and try again later).
Some additional benefits of this approach are that single-use resources (such as jobs) are not unnecessarily created, two separate endpoints need not be queried (factory and result), and likewise the second endpoint need not be determined from parsing the response from the first, thus simpler. Moreover, results can be cached, "for free" (code-wise). Set the cache expiration time in the result header according to how long the results are "valid", in some sense, for your problem domain.
I wish I could call this a textbook example of a "resource-oriented" approach, but, perhaps ironically, Chapter 8 of "RESTful Web Services" suggests the two-endpoint, factory approach. Go figure.
If you control both ends, then you can do whatever you want. E.g. browsers tend to launch HTTP requests with "connection close" headers so you are left with fewer options ;-)
Bear in mind that if you've got some NAT/Firewalls in between you might have some drop connections if they are inactive for some time.
Could I suggest registering a "callback" procedure? The client issues the request with a "callback end-point" to the server, gets a "ticket". Once the server finishes, it "callbacks" the client... or the client can check the request's status through the ticket identifier.