I'm using the Postman Jetpacks tests collections, they work very well, but I must run the test inside the Postman application.
Actually I need to run the tests I designed in Postman Jetpack using a command line to make it scriptable. My approach is to use Newman commands line to be able to execute the test collections. However looks like I can only execute requests using Newman instead of tests collections or even isolated tests.
Is it posible to run tests collection created on Jetpack with Newman?
After some research I found out that is possible to execute Postman Jetpack collections with Newman, Newman will also pass the tests and display the information properly.
This is an interesting way to do Integration tests in for APIs
Related
I am revising a kotlin-js web browser application.
Currently tests are run through selenide and are limited as they interact entirely through the DOM and cannot call the code or inspect data.
More tests are needed, and I am thinking that an actual js framework is needed such as Qunit, mocha or jasmine etc.
The project is configured through gradle, but I have not found how to have tests run by a gradle project that simulate being run in the browser.
The overall question is, how to best deliver unit tests?
questions:
Is there a better approach than the selenid approach?
Is there a kotlin-js based test alternative that can be run from a gradle task?
What combination has been found to work, ideally without resorting to node.js in order to run a browser app. e.g instructions on using Qunit, jasmine, mocha or an other test running as a gradle task?
Alternatively (not preferred) Is there some way to call javascript code (functions etc) and access page global variables from the selenide test code?
Any answers to either question appreciated.
Straight up been looking for answers to this question for months, still no idea how to actually do it - how does one automate tests that run in the browser? Selenium tests run on the backend and of course can be automated, and we are doing this now. Recently I have been tasked with automating browser based unit tests and I have a significant knowledge gap -
I don't know how to actually do it -
For example - how do does an automated test runner collect test results and exit codes of (unit) tests that run in the browser? Can anyone explain how this actually done and the steps to accomplish this?
Is Karma the best tool to accomplish this?
You can use http://phantomjs.org/. phantomjs is a headless webbrowser, which you can see as fullstack webbrowser without gui, usable as library. Together with karma you can execute your unit tests without relying on any GUI implementations.
Here a blogpost, which explains the different components in such a secenario http://orizens.com/wp/topics/my-setup-for-testing-js-with-jasmine-karma-phantomjs-angularjs/
Means you can execute your karma unit tests on a headless linux server.
Clarification:
The need for phantomjs doesn't come from unit tests. The need for phantomjs comes from the fact that your js unit tests depend onto the browser api.
Its a good design principle to structure the code, that the coupling to the browser api is not cluttered over all the code. Try to introduce a thin layer which encapsulates the browser api dependencies. Like this you can test your js mostly without the need for phantomjs.
To execute your unit tests with phantomjs can take its time. If you have a lot of unit tests, its better to factor out the dependencies to the browser api. So you can execute more tests without the need for phantomjs and only a minority of the unit tests need to be executed with phantomjs.
You can probably use cucumber. If you have 20 test cases, that you can need to execute.
You can create a feature file which will contain all the scenarios.
The Runner classes and Method on what needs to be done can be defined in a different package. Let's say you have a scenario to
1. Open browser.
2. Enter google link.
3. Login using credentials.
Create a feature file with the above information.
Use a Cucumber runner class. And create package methods such as
#When("^Open Browser$")
public void open_Browser() throws Throwable {
WebDriver driver = new FirefoxDriver();
driver.get("www.google.com");
}
similarly you can create different methods to run. To run the jar, you can use Command Line Interface technique.
Great piece of article LINK
Basics:
This is for Python automation, you'll need to have some previous knowledge/experience.
pip install selenium
pip install nose
Above should be executed in cmd or shell...
For this test we will open the AWeber website at http://www.aweber.com
using Firefox, and make sure that the title of the page is "AWeber Email Marketing Services & Software Solutions for Small Business".
import unittest
from selenium import webdriver
class AweberTest(unittest.TestCase):
#classmethod
def setUpClass(cls):
cls.driver = webdriver.Firefox()
def test_title(self):
self.driver.get('https://www.aweber.com')
self.assertEqual(
self.driver.title,
'AWeber Email Marketing Services & Software Solutions for Small Business')
#classmethod
def tearDownClass(cls):
cls.driver.quit()
Running the test using nose:
nose aweber.py
Next test, clicking on elements:
self.driver.get('https://www.aweber.com')
order_tab = self.driver.find_element_by_css_selector('#ordertab>a')
order_tab.click()
There are many selectors in that we can use find_element_by_(css/xpath/name/id) - Locating elements
And in this case we used the method click but we can also .send_keys("asdf") ,scroll, execute java script using
browser.execute_script("alert('I canNNNNN not do javascript')")
Full code example: LINK-Pastebin
Is there a way to run just a given acceptance test? We have a bunch of integration tests and I just want to see how my acceptance test runs. From the command line, I'm using ember test --serve, but I'd like to run just this specific test. All my tests are in one file and I only want to run those.
Thanks
Nevermind, I found it.
ember test --server --filter="my_test_name"
I have integrated ember-cli-blanket in my Jenkins job successfully. And the ember test will generate a output file in JSON format(eg: coverage-report.json).
Now for making it more user friendly or making it more understandable, I need to show a chart/graph which describes the result of test(content of coverage-report.json), after the build finishes successfully.
I have tried some Jenkins plugin like, cucumber plugin, cucumber reports plugin, cucumber JSON test reporting etc. But nothing works.
Is there any way to achieve this? Any help is appreciated.
We are using Codeigniter and have 2 options to call our API controllers:
we can use a client that calls the controller's url through Curl,
we can use a client that calls the controller from the command line.
This is perfectly fine for the functionality of our site. However, when I run PHPUnit, the coverage reports for the Controllers are blank while the coverage reports for all Models are correct.
In tracing how xdebug creates the reports, it appears that using the Curl-based client or the CLI client are called outside of the scope of the test function, so xdebug_get_code_coverage() does not track the controller code that is executed.
Is it possible to configure xdebug to recognize code coverage in this scenario? Is it possible to call Codeigniter controllers within the scope of the PHPUnit test function? Any other possible solutions?
Yes, that's easily possible. See http://www.phpunit.de/manual/current/en/selenium.html for more information about it
Basically you put some special files in your web root:
PHPUnit_Extensions_SeleniumTestCase can collect code coverage information for tests run through Selenium:
Copy PHPUnit/Extensions/SeleniumTestCase/phpunit_coverage.php into your webserver's document root directory.
In your webserver's php.ini configuration file, configure PHPUnit/Extensions/SeleniumTestCase/prepend.php and PHPUnit/Extensions/SeleniumTestCase/append.php as the auto_prepend_file and auto_append_file, respectively.
In your test case class that extends PHPUnit_Extensions_SeleniumTestCase, use protected
$coverageScriptUrl = 'http://host/phpunit_coverage.php';
to configure the URL for the phpunit_coverage.php script.
When running a URL with the GET parameter PHPUNIT_SELENIUM_TEST_ID, the coverage information gets tracked and PHPUnit can collect it by requesting the coverageScriptUrl.
An alternative: see our SD PHP Test Coverage tool.
It doesn't use xdebug to collect coverage data, so it won't have xdebug's specific problems. It instruments a script to collect test coverage data; once instrumented, no matter how the script is executed, you will get test coverage data.
(The instrumentation is temporary; you throw the instrumented code away once you have
the test coverage data collected, so it doesn't affect your production code base).
This approach does require you to explicitly list all PHP scripts for which you want coverage data; you can ignore some if you want. Usually it isn't worth the bother; most of the users simply list all PHP scripts.