Im new Q language and i want to unit test the Analytics code (q code). I'm using the Delta Control ide for development. In KX documents i came across the K4unit and i would like to use this. i have few questions here
How do i load the k4unit code inside Analytics Library in the Delta Control Ide.
I would like to create a framework for unit test , so that i would be used across.
Any lights or help would be greatly appreciated.
Related
My Intention is to test the functional flows of the application. So i think best way is testing UI for the steps of the functional flow. (E.g : Add item, after search it and add pricing etc.). My concern is to find best automation tool which should support for desktop application. does anyone have handheld experience of doing this. As a coding guy i don't have experience in Testing.
Additionally: the application is huge application which have connected with oracle database. So it need to clear the database as startup of the testing. So tool should support this matter also.
Programming language : C#
I found Selenium (but this is only for web application), Renorex and IBM rational Functional Tester.
is my approach(test UI steps for Functional steps) correct? have any one work on above tools or any other best tools whcih support to C# windows application.
I think NUnit is best for text unit test in this project. am i correct? according to my research google i found this as result.
I'll try to keep it short. A good fit for you case
automation tool which should support for desktop application.
and
Programming language : C#
will be the VisualStudio's build in Coded UI test. All actions performed on the desktop will be recorded. Pause the recording if you are performing actions that may lead to sensitive data being included in the recording. When coded UI tests are automated, they have to interact with the desktop when you run them, unlike other automated tests.
As far as you try to keep the tests close to the Test pyramid
the QA effort should be acceptable and your
approach(test UI steps for Functional steps)
will be correct and feasible. In general try to avoid long and fragile tests, except for some E2E tests that will exercise the (most business valuable) end-user journeys.
With C# should be relatively easy to drop oracle database for your
need to clear the database as startup of the testing.
Such DB cleaning is often used Fixture strategy. Take a look at this Persistent Fixture. Usage of such DB fixtures can be put to work for all conditions that your UI tests expect/rely on (avoid UI setup for tests), example - create available and applicable bonuses.
Another way for your UI layer tests could be some of the .Net based Sikuli implementations:
SikuliSharp
SikuliIntegrator
sikuli4net
We are working on billing system (java based module)for that we would like to have a testing framework. That testing framework should be able to adoptable for any type of billing scenarios(eg: utility bill payments, water/electricity/or any other type billing) Normally the billing entity have common attributes like customer name/usage/ etc..I would like to pick a suitable testing platform to test our billing module.
It can be opensource/licensed software.
Can anybody suggest such a framework/engine?
If you wanna go for open source tool, then I'd recommend Selenium Webdriver with TestNG Framework. You can get lot of documentation and help on web.
You can go for Cucumber for describing various test scenarios (and their dependencies).
These scenarios will be backed by jUnit for gluing the description to executable code. jUnit will typically be used to write low level tests as well (for use by developers).
Cucumber has the benefit of giving you reports and can serve well for discussion with users and their representatives.
I would suggest following based on your given little description about your need :
1 - For functionality testing and to make all your scenarios automate use Selenium WebDriver
2 -Then if you want to priorities your testes , want to run tests through XML , want to run multiple tests then you can use TestNG Framework
Above are totally open source tool and you will get real benefit of those by scripting. They allow you to do scripting using programming languages like Java , Python , ruby and a little more. You will get all details once you visit my given links in above 2 points.
According to your given requirements I think above 2 tools are enough to make everything automate for testing.
I am working with Google Maps on Android 4.0 and I would like to know:
Is there a testing framework (or anything..) that can interact with the Google Maps API? What are the possibilities when interacting/testing with Google Maps on Android? Is it not possible to find all pins on the map and perform a click on them or perhaps determine zoom level?
There seem to be a couple questions on this website dealing with similar issues/questions with no answer.
I realize "interact with" is a broad term. I am really looking for any kind of help whether it is a suggestion or just to tell me something is not possible.
I have been using JUnit but it seems limited by itself. Just today I started looking at Robotium since the majority of tests I need to do are UI based. I am new to Robotium so maybe it is possible with this and I have not discovered it yet?
When our team created an Android application that used Google maps, I was able to create tests using the R.id for the map fragment. It is important to note that I had access to the Application's code, so I knew what variables to look for.
Prior to creating this test, I wanted to make sure that I was targeting the correct R.id, so I went into the R file, copied the value and placed it in
assertEquals(id.satellite,copiedRValue);
which returned true, and then build the test around the changeable variable
initialID = solo.waitForFragmentById(id.satellite); //where id.satellite is defined in the R file, eventually would time out and throw error if fragment was not present
//save the map type, leave
//and do other awsome stuff
//before coming back to the map
finalID = solo.waitForFragmentById(id.satellite);//capture the map fragment that is now displayed, again would time out if this specific fragment was not visible
//assert that the fragments are equal
assertTrue(initialID.equals(finalID);
The biggest problem that I had with this test and with other Robotium tests is that I had click events happening to bring in menus and sometimes Robotium would not perform the click, and the test would fail on the click.
This was my first go at testing with Robotium, so there might be other ways to manipulate the R.id values to create a tighter test.
In case anyone wants to know after much searching I finally found something that can test Google Maps. Things such as zoom level and I believe tap pin (method is called tapMapMarkerItem()) are supported. I have not tested the pin tap yet tho.
Apparently the awesome Robotium does not support map testing by itself. Nicholas Albion was nice enough to create an extension to provides testing support for maps on Android. Thank you so much Nicholas!
So here it is:
1. Download the Robotium jars from robotium.org (I found this helpful http://www.vogella.com/articles/AndroidTesting/article.html - by Lars Vogel)
2. Download the extension from https://github.com/nalbion/robotium-maps
How can I perform unit testing in Simulink, or preferably, Stateflow?
I'm a fan of agile software methods, including test driven development. I'm responsible for the development of safety critical control software and we're using Matlab/Simulink/Stateflow for the development of it. This toolset is selected because of the link with plant (hardware) models. (model-in-the-loop, hardware-in-the-loop)
I have found some links on Stackoverflow: Unit-testing framework for MATLAB: xunit, slunit and doctest.
Does anyone have experience in using those or different unit test frameworks?
How to link this to continuous integration systems (i.e. Hudson)?
EDIT: This is now much easier and getting easier all the time with the Jenkins plugin for MATLAB
ORIGINAL ANSWER:
As Craig mentioned there is indeed a framework in MATLAB introduced in R2013a. Furthermore, this framework added a TAPPlugin in R2014a which outputs the Test Anything Protocal. Using that protocol you can set up your CI build with a TAPPlugin (eg. Jenkins, TeamCity) so that the CI system can fail the build if the tests fail.
Your CI build may look like a shell command to start MATLAB and run all your tests:
/your/path/to/matlab/bin/matlab -nosplash -nodisplay -nodesktop -r "runAllMyTests"
Then the runAllMyTests creates the suite to run and runs it with the tap output being redirected to a file. You'll need to tweak specifics here, but perhaps this can help you get started:
function runAllMyTests
import matlab.unittest.TestSuite;
import matlab.unittest.TestRunner;
import matlab.unittest.plugins.TAPPlugin;
import matlab.unittest.plugins.ToFile;
try
% Create the suite and runner
suite = TestSuite.fromPackage('packageThatContainsTests', 'IncludingSubpackages', true);
runner = TestRunner.withTextOutput;
% Add the TAPPlugin directed to a file in the Jenkins workspace
tapFile = fullfile(getenv('WORKSPACE'), 'testResults.tap');
runner.addPlugin(TAPPlugin.producingOriginalFormat(ToFile(tapFile)));
runner.run(suite);
catch e;
disp(e.getReport);
exit(1);
end;
exit force;
EDIT: I used this topic as the first two posts of a new developer oriented blog launched this year
Unit testing Simulink is not straightforward, unfortunately. Mathworks have the SystemTest. Alternatively, you can roll-your-own Simulink testing framework, which is the approach that we've followed and is not too difficult, but you may need to built test-harnesses programmatically.
In order to integrate with CI, you need to create a function/script that executes all the tests, then you can use the command-line parameters for MATLAB.exe to run a script on start-up. I'm not sure anyone has a good way of integrating the test reports with the CI software, though. Just look at the number of comments in Unit-testing framework for MATLAB.
With 2015a Matlab introduces a new product name "Simulink Test". Perhaps that'll simplify this mess.
http://www.mathworks.com/products/simulink-test/features.html#manage-test-plans-and-test-execution
R2016b introduces integration between Simulink Test and MATLAB Unit Testing framework. Tests created with Simulink Test using Test Manager (*.mldatx) are recognized by and can be run natively using the MATLAB Unit Test Runner and thus you can generate JUnit style XML test results or TAP test results facilitating Continuous integration workflows.
See this reference for more information: https://www.mathworks.com/help/sltest/ug/run-test-files-using-matlab-unit-test.html?s_tid=gn_loc_drop
The documentation shows an example of producing TAP results using matlab.unittest.plugins.TAPPlugin but you can use XMLPlugin (https://www.mathworks.com/help/matlab/ref/matlab.unittest.plugins.xmlplugin-class.html) instead just as easily.
This should open up a better integration within just MATLAB environment even without CI in the picture with the ability to have MATLAB and Simulink Tests together in the same test suite and have them run together seamlessly. For example if you have a directory MYDIR with both native MATLAB unit tests and Simulink Tests, you can do something as simple as the follows to execute both kinds of tests in one shot:
results = runtests(MYDIR)
If your system is complex, you should decompose it using Model Reference and test each of these independently.
An other solution (more "old school") is to put your main blocks in a library and to create small models.
To test these submodels and especially those with a State Machine (Stateflow), the best is to create temporal test cases
with the Signal builder block. You have a powerful function signalbuilder to interact with this block and load test cases. My method is to get for each case of each submodel an input file and an output file. Your outputs of the model are the "correct" output and the one from the blocks. The model is run with sim (no external inputs) and the 2 outputs are compared with a script the indicated which signal is different (and when).
You could use an existent system but I prefer to use my own to run each case (or some of them).
I don't have any public code for that but that's the way I use. I don't use a CIS so I can't answer the second part of your question.
Matlab (since 2013b) has built-in support for xUnit, in the form of the Unit Testing Framework.
I haven't used it but since it's possible to run simulink from Matlab with sim() then this framework can be used to test your simulink models. You libraries and possibly models will need a wrapper to execute it as the other answerers have noted.
There are plenty of examples on the Mathworks site, unfortunately non of them run simulink models. I'd code an example for you, but I don't have ML2013b :-(
In order to initiate your tests from a CI (I use Jenkins) then you can call matlab to run a .m file that runs your test suite, this example cmd script will call Run_Tests.m from Matlab:
IF EXIST "C:\Program Files (x86)\MATLAB\R2013b\bin\win32\matlab.exe" (
REM WinXP
"C:\Program Files (x86)\MATLAB\R2013b\bin\win32\matlab.exe" -r "Run_Tests;exit" -logfile matlab.log
) ELSE (
REM Win7
"C:\Program Files\MATLAB\R2013b\bin\win32\matlab.exe" -r "Run_Tests;exit" -logfile matlab.log
)
Note that if a startup.m exists in the directory that you call Matlab from, then it'll be executed automatically beforeRun_Tests.m`.
I think you are searching for something like EZTEST. It is intended for your special purpose: Test driven development for Simulink and Stateflow on unit level. For safety critical software, there is also a Safety Manual included, which describes what is covered regarding ISO 26262.
Edit: I am a developer of this software, so my opinion may be biased. But I am not involved in the marketing or sale of the product. I am just posting this, because I know that there are little to none unit test frameworks out there, meeting the questioner's needs (as the answers might suppose).
Testing units using SIL and PIL is also supported. Unfortunately I am not familiar with Hudson, so I cannot address this part of the question.
I've seen different solutions to the problem of unit testing Simulink models. Simulink Verification & Validation which did not support xUnit concepts of test runners and test suites at the time of examination, TPT being overloaded with functionality, not easy to use and very hard in terms of changeability and maintainability.
Furthermore I've seen custom solutions with Matlab scripts and Excel tables which were lightweight but also difficult in terms of understandability and maintainability. I'd still not recommend using any of these approaches, at least not for unit testing.
In the end, we ended up using a C unit testing framework (CUnit) testing the generated code. While this definitely has the disadvantage that you have to generate code before testing, it also has a lot of advantages, like easy integration into CI systems, high flexibility of writing unit tests, fast execution of unit tests and last but not least refactoring capabilities in terms of switching from Simulink to another model-based environment or to hand-written code. Especially the last point should not be underestimated since I have seen many Simulink models that should have been hand-written modules in the first place. Nowadays, I'd recommend using GoogleTest instead of CUnit.
Does anyone know of a way to do continuous integration with R programming? I'm aware of tools like the svUnit package to do the unit tests, but has anyone tried to run these with Hudson/Jenkins?
I do not see any particular problem. These things tend to be scripted so could just
point to the top of your repository
N minutes after each checkin, loop over source directories
invoke R CMD check on each
your package has to be set to use unit test, for which you can use
RUnit which is the initial unit testing for R ; it is widely used
testthat which is a newer package by Hadley, and used by many of his packages
svUnit by Philippe which AFAIK never caught on quite as much as the other two.
That is really not any different from continuous integration with compiled languages. Your question is really about how to do unit testing within R, and that question has been covered before.
at office we have been using Hudson/Jenkins for quite a while. I have contributed integration of svUnit to Jenkins so I would strongly advise you to use svUnit before trying anything else.
have a look at the two libraries I maintain: logging and delftfews or at myself trying to follow zoo and redistribute it on github.
I have not been doing housekeeping recently so the three scripts (in zoo, logging, delftfews) are all slightly different. the one in my zoo version will stop if any test fails. this is practical when you are running R CMD check, but probably less of a good idea when doing continuous integration.
Hudson/Jenkins supports running bash scripts. I think that you can use this as an entrance to your R world. In R, a simple way to keep result is to use sink("toYourFile.txt") and then use CI's result display function to show the toYourFile.txt.