Does any web site test framework facilitate fine-grained concurrent testing - unit-testing

I have a legacy .NET application that is implemented using Application variables and makes heavy use of Session data as well. There are some anecdotal reports of bugs that seem to point toward concurrency errors, i.e. multiple sessions clobbering shared application-level data.
I want to develop some automated tests that let me control concurrent access in a fine-grained fashion, i.e.
Create two HTTP clients with fresh sessions
Request /my/page/1 with client 1
Request /my/page/2 with client 2
POST data with client 2
POST data with client 1
Issue parallel request for /my/page/results from both clients
etc.
Are there any libraries that make this sort of testing easier or will I have to roll my own to some extent?
I'm aware of Selenium and WatiN, but have not personally used either project. From reading the docs, neither appears to be a good match.
Perhaps the best option is just plain NUnit and making good use of the .NET WebClient class?

Clarification: What you want is a well-defined series of steps to be executed synchronously. This means your test code does not need to be multithreaded, more precisely, it must not be multithreaded, or you lose control over the order the steps are executed.
The only thing you need is two browser instances and the ability to dispatch the test steps to the two browsers.
To do so, I presume you can use either of Selenium or WatiN as follows (not verified, it's only in my mind)
Selenium (with WebDriver):
using (var firefox1 = new FirefoxDriver(profile))
using (var firefox2 = new FirefoxDriver(profile))
{
requestPageOne(firefox1);
requestPageTwo(firefox2);
postPageTwo(firefox2);
postPageOne(firefox1);
//// ...
}
WatiN
using (var firefox1 = new FireFox(url1))
using (var firefox2 = new FireFox(url2))
//// ...

Related

Only aysnc methods lsited when call wcf

I am trying to call a WCFService from Windows phone 8 which connects and returns data fine when i use WCFTestClient but when I refreence the service using add reference and then try to access in code only the async methods are showing in intelesence. I have not delciared my methods as aync how can i ensure I can access my other methods as I calling a webservice does it need be aysnc.
// Constructor
public MainPage()
{
InitializeComponent();
// Sample code to localize the ApplicationBar
//BuildLocalizedApplicationBar();
IcuroServiceClient _db = new IcuroServiceClient();
var json =_db.GetPersonByIdAsync(1);
}
And if so How would I convert a method that is as simple as below to Aysnc ?. I am used to asmx services and new to WCF.
public string GetListByUserId(string userId)
{
List<curoList> myList = _db.GetAllListsByUserId(userId);
var json = JsonConvert.SerializeObject(myList, Newtonsoft.Json.Formatting.None);
return json;
}
Its grayed out here for me mate In my normal signature im returning a string but the asyncs dont look like their returning anything just void .
Windows phone is based literally and figuratively on the Silverlight motif of keeping all service calls async only. It is a two step process where one has to think backwards. Here are the steps
Provide a callback method whose job it is to handle the resulting data or error in an appropriate fashion.
Then make the call as one normally would (but calling the async version) to start the process off on a separate thread.
Be cognizant of not directly loading to the GUI in the callback method for the result is on a different thread; note that loading to VM properties is fine for the most part whereas the update to any GUI subscribed bindings will be done on the thread.
Hot to Resolve
In the example given before the call to to db.GetPersonByIdAsync(1) provide (find using intellisense) the async subscription method that has the call back.
It may be prefixed with either a Load method or a Begin call, for the PersonById.
In the editor, the providing of a method is usually done by intellisense which where one can tab an example into code. That is so one doesn't have to root out the parameters for the callback method.
Possible example, shows the how to do it:
client.GetPersonByIdAsyncCompleted += MyMethodToHandleResultforGetPerson; // var personId = e.Result;
client.GetPersonByIdAsync(1);
Why
When Silverlight was designed, having a program wait (even if its busy waiting) on any thread would slow down the browser experience. By requiring all database (service) calls to be asynchronous ensures that the developer can't slow down the browser, in particular by bad code. Loosely speaking that motif carried over to the windows phone where Silverlight was brought over as the main operations for the phone.
It took me awhile to get used to the process, but now that I have worked on Silverlight and Windows phone, I actually use the async motif in WPF even though I don't necessarily have too, it makes sense in terms of data processing and thread management.
It is not easier for the developer per-se for everyone learns in a synchronous hello world, but it is adapted for the needs of the target platform and once learned it becomes second nature.

When to use httptest.Server and httptest.ResponseRecorder

As title, when to use httptest.Server and httptest.ResponseRecorder?
It seems to me that I can also test my handlers to return correct response using httptest.Server. I can simply start a httptest.Server given with my implementation of handlers, then do validations on the response's body.
Please correct if I'm wrong, I am learning Go + TDD
When you just want to check, if your http.Handler does what it should, you don't need to use httptest.Server. Just call your handler with an httptest.ResponseRecorder instance and check the output as in the example.
The possible uses of httptest.Server are numerous, so here are just a couple that come to my mind:
If your code depends on some external services and APIs, you can use a test server to emulate them. (Although I personally would isolate all code dealing with external data sources and then use them through interfaces, so that I could easily create fake objects for my tests.)
If you work on a client-server application, you can use a test server to emulate the server-side when testing the client-side.

Web-app with clojure using hot-swapping of code

I'm thinking of writing a web-app in clojure that can update itself without restarting or loosing state.
I've seen some articles where Clojure apps can perform so-called hot-swapping of code. Meaning that they can update their own functions at runtime. Would this be safe to perform on a web-server?
To get hot-swap for code is tricky to get right, if possible at all.
It depends on the changeset and the running application too.
Issues:
old vars may litter namespaces and cause subtle conflicts, bugs
redefinition of multiple vars is not atomic
There may be old vars in a namespace that will not be there if you restart the application, however will interfere if you just redefine some of the functions and keep the app running without restart.
The other issue is atomicity: redefining multiple functions i.e. changing multiple vars is not atomic. If you change functions in one or more namespace that code in some other namespace depends on, reloading the namespaces with the new code is not atomic.
Generally, you are better off either
having a proxy hold the requests until your app restarts
spinning up a new app instance parallel to the "old version" and use a proxy to switch from the new version after the new version is ready to process requests
OTP applications in Erlang support this. Basically, it will spin the new version of your application up and start sending requests to the new version of your application. It will keep the old version alive until it has completed processing requests and then shut it down.

distributed unit testing/scenario based unit testing with boost.test

I am developing unit test cases for an application using Boost.test libraries. There are certain APIs which can directly be tested.
But, there are APIs which require interaction between test machines. So for example, execution of a certain API in machine 1 should trigger an API in test machine 2 and its response needs to be used again in machine 1 for successful completion.
How can I synchronize this ? Does Boost provide other libraries for this interaction? If there are any other approaches, kindly suggest them.
Thanks in advance for your time and help.
There are two kinds of tests you can write for this interaction:
Unit test - using mocks/faks you can fake the calls from the first component and fake the calls from the 2nd component back. This way you can test the internal logic of the first component - for example make sure that if no response were returned a time-out exception is raised.
Integration/acceptance test - create both components as part of the test and configure them and raise the call from component one.
In both kinds of tests you might be required to use events and WaitForSingleObject to make sure that the test won't end before the response has returned.

Should my unit tests be touching an API directly when testing a wrapper for that API?

I have some written a number of unit tests that test a wrapper around a FTP server API.
Both the unit tests and the FTP server are on the same machine.
The wrapper API gets deployed to our platform and are used in both remoting and web service scenarios. The wrapper API essentially takes XML messages to perform tasks such as adding/deleting/updating users, changing passwords, modifying permissions...that kinda thing.
In a unit test, say to add a user to a virtual domain, I create the XML message to send to the API. The API does it's work and returns a response with status information about whether the operation was successful or failed (error codes, validation failures etc).
To verify whether the API wrapper code really did do the right thing (if the response indicated success), I invoke the FTP server's COM API and query its store directly to see if, for example when creating a user account, the user account really did get created.
Does this smell bad?
Update 1: #Jeremy/Nick: The wrapper is the focus of the testing, the FTP server and its COM API are 3rd party products, presumably well tested and stable. The wrapper API has to parse the XML message and then invoke the FTP server's API. How would I verify, and this may be a silly case, that a particular property of the user account is set correctly by the wrapper. For example setting the wrong property or attribute of an FTP account due to a typo in the wrapper code. A good example being setting the upload and download speed limits, these may get transposed in the wrapper code.
Update 2: thanks all for the answers. To the folks who suggested using mocks, it had crossed my mind, but the light hasn't switched on there yet and I'm still struggling to get my head round how I would get my wrapper to work with a mock of the FTP server. Where would the mocks reside and do I pass an instance of said mocks to the wrapper API to use instead of calling the COM API? I'm aware of mocking but struggling to get my head round it, mostly because I find most of the examples and tutorials are so abstract and (I'm ashamed to say) verging on the incomprehensible.
You seem to be mixing unit & component testing concerns.
If you're unit-testing your wrapper, you should use a mock FTP server and don't involve the actual server. The plus side is, you can usually achieve 100% automation like this.
If you're component-testing the whole thing (the wrapper + FTP server working together), try to verify your results at the same level as your tests i.e. by means of your wrapper API. For example, if you issue a command to upload a file, next, issue a command to delete/download that file to make sure that the file was uploaded correctly. For more complex operations where it's not trivial to test the outcome, then consider resorting to the COM API "backdoor" you mentioned or perhaps involve some manual verification (do all of your tests need to be automated?).
To verify whether the API wrapper code really did do the right thing (if the response indicated success), I invoke the FTP server's COM API
Stop right there. You should be mocking the FTP server and the wrapper should operate against the mock.
If your test runs both the wrapper and the FTP server, you are not Unit Testing.
To test your wrapper with a mock object, you can do the following:
Write a COM object that has the same interface as the FTP server's COM API. This will be your mock object. You should be able to interchange the real FTP server and your mock object by passing the interface pointer of either to your wrapper by means of dependency injection.
Your mock object should implement hard-coded behaviour based on the methods called on its interface (which mimics FTP server API) and also based on the argument values used:
For example, if you have an UploadFile method you can blindly return a success result and perhaps store the file name that was passed in in an array of strings.
You could simulate an upload error when you encounter a file name with "error" in it.
You could simulate latency/timeout when you encounter a file name with "slow" in it.
Later on, the DownloadFile method could check the internal string array to see if a file with that name was already "uploaded".
The pseudo-code for some test cases would be:
//RealServer theRealServer;
//FtpServerIntf ftpServerIntf = theRealServer.getInterface();
// Let's test with our mock instead
MockServer myMockServer;
FtpServerIntf ftpServerIntf = myMockServer.getInterface();
FtpWrapper myWrapper(ftpServerIntf);
FtpResponse resp = myWrapper.uploadFile("Testing123");
assertEquals(FtpResponse::OK, resp);
resp = myWrapper.downloadFile("Testing123");
assertEquals(FtpResponse::OK, resp);
resp = myWrapper.downloadFile("Testing456");
assertEquals(FtpResponse::NOT_FOUND, resp);
resp = myWrapper.downloadFile("SimulateError");
assertEquals(FtpResponse::ERROR, resp);
I hope this helps...
I agree with Nick and Jeremy about not touching the API. I would look at mocking the API.
http://en.wikipedia.org/wiki/Mock_object
If it's .NET you can use:
Moq: http://code.google.com/p/moq/
And a bunch of other mocking libraries.
What are you testing the wrapper or the API. The API should work as is, so you don't need to test it I would think. Focus your testing efforts on the wrapper and pretend like the API doesn't exist, when I write a class that does file access I don't unit test the build in streamreader...I focus on my code.
I would say your API should be treated just like a database or a network connection when testing. Don't test it, it isn't under your control.
It doesn't sound like you're asking "Should I test the API?" — you're asking "Should I use the API to verify whether my wrapper is doing the right thing?"
I say yes. Your unit tests should assert that your wrapper passes along the information reported by the API. In the example you give, for instance, I don't know how you would avoid touching the API. So I don't think it smells bad.
The only time I can think of when it might make sense to dip into the lower level API to verify results if if the higher-level API is write-only. For example, if you can create a user using the high-level API, then there should be a high-level API to get the user accounts, too. Use that.
Other folks have suggested mocking the lower-level API. That's good, if you can do it. If the lower-level component is mocked, checking the mocks to make sure the right state is set should be okay.