Ordered Selenium unit tests n - unit-testing

I have a small problem, I have created some Selenium tests. The problem is I can't order the testcases I have created. I know unit testing should not be ordered but this is what I need in my situation. I have to follow these steps: login first, create a new customer, change some details about the customer and finally log out.
Since there is no option to order unit tests in NUnit I can't execute this.
I already tried another option, to create a unittest project in Visual Studio, because Visual Studio 2012 has the ability to create a ordered unit test. But this is not working because I can't run a unit test while I am running my ASP.NET project. Another solution file is also not a good option because I want to verify my data after it has been submitted by a Selenium test.
Does someone of you have another solution to solve my problem?

If you want to test all of those steps in a specific order (and by the sounds of it, as a single session) then really it's more like an acceptance test you are talking about; and in that case it's not a sin to write more complex test methods and Assert your conditions after each step.
If you want to test each step in true isolation (a pure unit test) then each unit test must be capable of running by itself without any reference to any other tests; but when you're testing the actual site UI itself this isn't really an option for you.
Of course if you really you want to have every single test somehow setup every single dependency without reference to any other actions (e.g in the last test you would need to fake the login token, your data layer will have to pretend that you added a new customer, etc. A lot of work for dubious benefit...)
I say this based on the assumption that you already have unit tests written for the server-side controllers, layers, models, etc, that you run without any reference to the actual site running in a browser and are therefore confident that the various back-end part of your site do what they are supposed to do
In your case I'd recommend more of a hybrid integration/acceptance test
void Login(IWebDriver driver)
{
//use driver to open browser, navigate to login page, type user/password into box and press enter
}
void CreateNewCustomer(IWebDriver driver)
{
Login(driver);
//and then use driver to click "Create Customer" link, etc, etc
}
void EditNewlyCreatedCustomer(IWebDriver driver)
{
Login(driver);
CreateNewCustomer(driver);
//do your selenium stuff..
}
and then your test methods:
[Test]
void Login_DoesWhatIExpect()
{
var driver = new InternetExplorerDriver("your Login URL here");
Login(driver);
Assert(Something);
}
[Test]
void CreateNewCustomer_WorksProperly()
{
var driver = new InternetExplorerDriver("your Login URL here");
CreateNewCustomer(driver);
Assert(Something);
}
[Test]
void EditNewlyCreatedCustomer_DoesntExplodeTheServer()
{
var driver = new InternetExplorerDriver("your Login URL here");
EditNewlyCreatedCustomer(driver);
Assert(Something);
}
In this way the order of the specific tests do not matter; certainly if the Login test fails then the CreateNewCustomer and EditNewlyCreatedCustomer tests will also fail but that's actually irrelevant in this case as you are testing an entire "thread" of operation

Related

Visual Test With Applitools Of a simple CSS class change

would like to know if there is a tutorial to learn how to test with this tool this type of change?. I meant... I needed to introduce a CSS class change in a component but I don't know how to test it... i've checked youtube but I cant quite understand how would be the best approach...
Edit: Applitools customer support is very helpful - consider contacting them for a more personal assistance.
Applitools is a cloud service that smartly detects changes to your UI that are noticeable to the human eye.
It does so by comparing screenshots taken during your automatic end-to-end tests to baseline images taken during the last test that you approved.
I mention this because you tagged your question as unit-testing. This is not unit-testing as it requires launching a browser (automatically - using Selenium for example) to run the automated tests.
Per your question - you should make sure that you have a baseline image that contains the component in discussion before your class change (do it by running an automated test that calls Applitools Eyes API to create a checkpoint (eyes.checkWindow) when your test reaches a page that contains this component. Then, approve the new test as a baseline, apply your class change and run the test again.
Applitools will let you know if a visual difference occurred following your change. So, for example, if this is just an under-the-hood change - you will not see a visual difference, but if this actually affects the UI then you'll see a visual difference. If you're happy with this difference you simply approve this run as your new baseline.
To show an example I'll use the Applitools tutorial for Selenium + Java. See the comments - you run this once, make your class change and run it again:
import com.applitools.eyes.Eyes;
import com.applitools.eyes.RectangleSize;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import java.net.URISyntaxException;
public class TestApplitoolsWebsite {
public static void main(String[] args) throws URISyntaxException, InterruptedException {
WebDriver driver = new FirefoxDriver();
Eyes eyes = new Eyes();
// This is your api key, make sure you use it in all your tests.
eyes.setApiKey("YOUR_API_KEY");
try {
// Start visual testing with browser viewport set to 1024x768.
// Make sure to use the returned driver from this point on.
driver = eyes.open(driver, "Applitools", "Test Web Page", new RectangleSize(1024, 768));
driver.get("http://applitools.com");
// Visual validation point #1
eyes.checkWindow("Main Page");
driver.findElement(By.cssSelector(".features>a")).click();
// Visual validation point #2
// Let's assume this page contains the class you'll change
eyes.checkWindow("Features page");
// End visual testing. Validate visual correctness.
eyes.close();
} finally {
// Abort test in case of an unexpected error.
eyes.abortIfNotClosed();
driver.close();
}
}
}
Of course - if your class change impacts several places in the application / website then it's best to make sure your test covers all these places and that you take screenshots of all these places.

Grails. mocked data from a unit test is available in an integration test

Im having a failed integration test because of test pollution (tests pass or fail depending on which order they are run in).
What baffles me a bit however is that it seems that a unit test where i have mocked some data with mockDomain(Media.class,[new Movie(...)]) is still present and available in other tests, even integration tests.
Is this the expected behaviour? why doesn't the test framework clean up after itself for each test?
EDIT
Really strange, the documentation states that:
Integration tests differ from unit tests in that you have full access to the Grails environment within the test. Grails will use an in-memory HSQLDB database for integration tests and clear out all the data from the database in between each test.
However in my integration test i have the following code
protected void setUp() {
super.setUp()
assertEquals("TEST POLLUTION!",0,Movie.count())
...
}
Which gives me the output:
TEST POLLUTION! expected:<0> but was:<1>
Meaning that there is data present when there shouldn't be!
Looking at the data that is present int he Movie.list() i find that the data corresponds to data set in a previous test (unit test)
protected void setUp() {
super.setUp()
//mock the superclass and subclasses as instances
mockDomain(Media.class,[
new Movie(id:1,name:'testMovie')
])
...
}
Any idea's of why im experiencing these issues?
It's also possible that the pollution is in the test database. Check the DataSources.groovy to see what is being used for the test environment. If you have it set to use a database where
dbCreate is set to something other than "create-drop", any previous contents of the database could also be showing up.
If this is the case, the pollution has come from an entirely difference source. Instead of coming from the unit tests, it has actually come from the database, but when switching to run the integration tests you get connected to a real database with all the data it contains.
We experienced this problem, in that our test enviroment was set to have dbCreate as "update". Quite why this was set for integration tests puzzled me, so I switched to use dbCreate as "create-drop" and make sure that when running test suites we started with a clean database.

How do I use an API of a windows service?

I've got a big windows service application. It performs actions on a time bound basis. Sometimes I need to be able to use some of it's functionality in isolation from the rest of the application. Currently I've got a battery of 'unit tests' which call into various sources and perform the desired functionality. My problem is these are not unit tests, they are the way we're exposing the API. If we run all the unit tests in the project, we'll be damaging some of our production data.
My question is how do I go about accessing some of the functionality of the application without unit testing? I was thinking of perhaps something like an interpreter over the top of it where you can call various parts of the functionality, but am not really that sure where to start.
An example of a unit test in our code will be:
[TestMethod]
public void TransferFunds()
{
int accountNumberTo = 123456;
int accountNumberFrom = 654321;
var accountFrom = Store.GetAccount(accountNumberFrom);
var accountTo = Store.GetAccount(accountNumberTo);
double amountToTransfer = 1000;
DateTime transactionDate = new DateTime(2010,01,01);
Store.TransferFunds(accountFrom, AccountTo, amountToTransfer, transactionDate);
var client = BankAccountService.Client();
client.Contribute(accountNumberTo, amountToTransfer, transactionDate);
client.Contribute(accountNumberFrom, amountToTransfer, transactionDate);
}
How can we move this out of unit tests, but still have the ability to run code like this?
Your setup sounds very dangerous. I would create separate console applications for your different needs. I would also remove recommend that you remove all unittests that endangers your production data. Having that sort of unittests is just down-right bad!

How do I Unit Test the Output of a View in MonoRail?

I've been trying to write some initial NUnit unit tests for MonoRail, having got some basics working already. However, while I've managed to check whether a Flash["message"] value has been set by a controller action, the BaseControllerTest class doesn't seem to store the output for a view at all, so whether I call RenderView or the action itself, nothing gets added to the Response.OutputContent data.
I've also tried calling InPlaceRenderView to try to get it to write to a StringWriter, and the StringWriter also seems to get nothing back - the StringBuilder that returns is also empty.
I'm creating a new controller instance, then calling
PrepareController(controller,"","home","index");
So far it just seems like the BaseControllerTest is causing any output to get abandoned. Am I missing something? Should this work? I'm not 100% sure, because while I'm also running these unit tests in MonoDevelop on Linux, although MonoRails is working OK there.
While I haven't got an ideal method for testing Views, this is possibly less important when ViewComponents can be tested adequately. To test views within the site itself, I can use Selenium. While in theory that can be made part of an NUnit test suite, that didn't run successfully under MonoDevelop in my tests (failing to start the connection to Selenium RC consistently, despite the RC interactive session working fine). However, the Selenium tests can be run as a set from Firefox, which is not too bad - unit testing with NUnit, then Integration/System testing scripting using a Selenium suite, and that setup will work in a Linux/MonoDevelop setup.
As for testing the underlying elements, you can check for redirections and check the flash value set or the like, so that's all fine, and for testing ViewComponents the part-mocked rendering does return the rendered output in an accessible form, so they've proved much easier to test in NUnit (with a base test class of BaseViewComponentTest) as follows:
[Test]
public void TestMenuComponentRendersOK()
{
var mc = new MenuComponent();
PrepareViewComponent(mc);
var dict = new System.Collections.Specialized.ListDictionary();
dict.Add("data",getSampleMenuData());
dict.Add("Name","testmenu");
// other additional parameters
mc.RenderComponent(mc,dict);
Assert.IsTrue(this.Output.Contains(""),"List items should have been added");
}

How to use "Pex and Moles" library with Entity Framework?

This is a tough one because not too many people use Pex & Moles or so I think (even though Pex is a really great product - much better than any other unit testing tool)
I have a Data project that has a very simple model with just one entity (DBItem). I've also written a DBRepository within this project, that manipulates this EF model. Repository has a method called GetItems() that returns a list of business layer items (BLItem) and looks similar to this (simplified example):
public IList<BLItem> GetItems()
{
using (var ctx = new EFContext("name=MyWebConfigConnectionName"))
{
DateTime limit = DateTime.Today.AddDays(-10);
IList<DBItem> result = ctx.Items.Where(i => i.Changed > limit).ToList();
return result.ConvertAll(i => i.ToBusinessObject());
}
}
So now I'd like to create some unit tests for this particular method. I'm using Pex & Moles. I created my moles and stubs for my EF object context.
I would like to write parametrised unit test (I know I've first written my production code, but I had to, since I'm testing Pex & Moles) that tests that this method returns valid list of items.
This is my test class:
[PexClass]
public class RepoTest
{
[PexMethod]
public void GetItemsTest(ObjectSet<DBItem> items)
{
MEFContext.ConstructorString = (#this, name) => {
var mole = new SEFContext();
};
DBRepository repo = new DBRepository();
IList<BLItem> result = repo.GetItems();
IList<DBItem> manual = items.Where(i => i.Changed > DateTime.Today.AddDays(-10));
if (result.Count != manual.Count)
{
throw new Exception();
}
}
}
Then I run Pex Explorations for this particular parametrised unit test, but I get an error path bounds exceeded. Pex starts this test by providing null to this test method (so items = null). This is the code, that Pex is running:
[Test]
[PexGeneratedBy(typeof(RepoTest))]
[Ignore("the test state was: path bounds exceeded")]
public void DBRepository_GetTasks22301()
{
this.GetItemsTest((ObjectSet<DBItem>)null);
}
This was additional comment provided by Pex:
The test case ran too long for these inputs, and Pex stopped the analysis. Please notice: The method Oblivious.Data.Test.Repositories.TaskRepositoryTest.b__0 was called 50 times; please check that the code is not stuck in an infinite loop or recursion. Otherwise, click on 'Set MaxStack=200', and run Pex again.
Update attribute [PexMethod(MaxStack = 200)]
Question
Am I doing this the correct way or not? Should I use EFContext stub instead? Do I have to add additional attributes to test method so Moles host will be running (I'm not sure it does now). I'm running just Pex & Moles. No VS test or nUnit or anything else.
I guess I should probably set some limit to Pex how many items should it provide for this particular test method.
Moles is not designed to test the parts of your application that have external dependencies (e.g. file access, network access, database access, etc). Instead, Moles allows you to mock these parts of your app so that way you can do true unit testing on the parts that don't have external dependencies.
So I think you should just mock your EF objects and queries, e.g., by creating in-memory lists and having query methods return fake data from those lists based on whatever criteria is relevant.
I am just getting to grips with pex also ... my issues surrounded me wanting to use it with moq ;)
anyway ...
I have some methods similar to your that have the same problem. When i increased the max they went away. Presumably pex was satisfied that it had sufficiently explored the branches. I have methods where i have had to increase the timeout on the code contract validation also.
One thing that you should probably be doign though is passing in all the dependant objects as parameters ... ie dont instantiate the repo in the method but pass it in.
A general problem you have is that you are instantiating big objects in your method. I do the same in my DAL classes, but then i am not tryign to unit test them in isolation. I build up datasets and use this to test my data access code against.
I use pex on my business logic and objects.
If i were to try and test my DAL code id have to use IOC to pass the datacontext into the methods - which would then make testing possible as you can mock the data context.
You should use Entity Framework Repository Pattern: http://www.codeproject.com/KB/database/ImplRepositoryPatternEF.aspx