I want to classified tests to run them for different purposes. As far as, I search I could not an option to tag some test and run test on demand. I also looked at Chaijs if it has such a feature, but could not be able to find a solution.
What I want to do is add some tag like the following:
#health_check
pm.test("Status code is 200", function () {
pm.response.to.have.status(200);
});
#smoke
pm.test("Product should be correct", function () {
var jsonData = pm.response.json();
var product = pm.variables.get("PRODUCT");
pm.expect(jsonData.meta.appId).to.eql(product);
});
and run it like as following or anyway achieve this:
$ newman run mycollection.json -tag smoke
That tag type option is not something that's ever been available in the app or in Newman.
You could use the --folder option on the CLI and organise your collections to include requests within those specific folders, that cover those scenarios.
For example:
$ newman run mycollection.json --folder healthCheck --folder smoke
Alternatively, you could use a global variable to act as a switch in the control if specific tests/groups of tests are run.
Wrapping the test in an if statement like this:
if(pm.globals.get('smoke_check') === "runCheck") {
pm.test("Status code is 200", () => pm.response.to.have.status(200));
}
Then passing in a global variable in the cli using the --global-var flag.
$ newman run mycollection.json --global-var "smoke_check=runCheck"
Related
I have a collection of simple API requests, and I'm running into an issue with one of the tests I've created.
I have a request which creates some content, and I use the built-in '$randomMACAddress' variable to generate a random name for the list I'm creating. This is stored in an env var called 'listName'. In a test further down in the flow, I assert that the list I'm retrieving has a name which matches '$randomMACAddress':
pm.test("List details are correct"), function () {
pm.expect(jsonData.name).equal($randomMACAddress);
}
This test PASSES.
I have the same check in another test, but this time it fails, and Postman tells me the following:
ReferenceError: $randomMACAddress is not defined
The test in that request is as follows:
pm.test("List details are correct", function () {
pm.expect(jsonData.id).equal(pm.globals.get('listID'));
pm.expect(jsonData.name).equal($randomMACAddress);
pm.expect(jsonData.products[0].skuId).equal('xyz');
});
The requests/tests are run at the same time (collection runner) and I'm baffled as to why that assertion fails on the latter test.
Tried to initialise things differently, but that hasn't worked.
The way to access the stored data would be via the pm.variables.get("variable_name") function and not using $randomMACAddress.
More info here:
https://learning.getpostman.com/docs/postman/environments_and_globals/variables
Also, the chai syntax would be to.equal()
When running tests using a command such as ember test --module="Acceptance | example", how could I capture the module option within the environment.js config file?
The aim here is to pass a flag indicating the acceptance test module into the application instance when testing. The flag will simply be used in the application to check whether to execute a block of code.
Update 23/09/18:
One idea I've had is that perhaps I can pass a flag into the application instance from the beforeEach hook in acceptance test modules. This would be better because I wouldn't have to include the --module option in the test command, and I'd be able to run all my tests at once. I'm not sure this is possible or how to do this but it's currently my best start.
I've had success implementing my idea. In the acceptance tests, I've got:
import config from '../../config/environment';
module('Acceptance | example', function(hooks) {
setupApplicationTest(hooks);
hooks.beforeEach(function() {
config.APP.testModule = 'Acceptance | example';
...
});
...
I can then use this by importing the config, then accessing the flag with:
config.APP.testModule
For now at least, I'm going to use this. I will hold off from accepting this answer for a while because it would still be great to hear if anyone has any other ideas or suggestions on how to improve this!
all. I'm trying to implement an app that using parse server as backend.
And I'm trying to use mocha/chai to do the unit test for the cloud code function.
Like the code below.
const { expect } = require('chai');
const { server } = require('../index.js');
const Parse = require('parse/node');
let loggedUser;
let loggedUserSessionToken;
describe('SMS APIs', function() {
before('Initialize parse server.', function(done) {
Parse.initialize("appId");
Parse.serverURL = 'http://localhost:1337/parse';
done();
});
after('Close server', function(done) {
done();
server.close();
});
it('Pass', function(done) {
expect(1).to.equal(1);
done();
})
)};
After I run yarn mocha. The command line shows lots of log message. It is hard to read the mocha test result. like the picture below. Is there any method to turn off parse logger?
command line logger image
Take a look how the parse-server repo does it: helper.js
the key is to set 'silent: true' in the parse-server configuration.
I do this by using the wonderful config package, creating a test.js config that sets silent to true and then setting NODE_ENV=test when running my unit tests. Sounds like a lot to do, but this pattern is commonly reused for many things. Good luck!
I would like to run "ember test" without JShint. My ultimate goal is to set run jshint in development environment and not run it in production build.
I first started to turn off the option in Brocfile.js
http://discuss.emberjs.com/t/disable-ember-cli-hinting/6731/2
var app = new EmberApp({
hinting: false
});
It worked, so I decided to try
var EmberApp = require('ember-cli/lib/broccoli/ember-app'),
isProduction = ( process.env.EMBER_ENV || 'development' ) === 'production';
if( isProduction ){
app.hinting = false;
}
Then I realize the process.env.EMBER_ENV, doesn't seem to work. But little did I know I was probably running the wrong command.
ember test
The command didn't specify any environment, so I tried
ember test --environment=production
ember test --environment production
which result in an exception:
Build failed.
Path or pattern "test-loader.js" did not match any files
Error: Path or pattern "test-loader.js" did not match any files
at Object.multiGlob (.../node_modules/ember-cli/node_modules/broccoli-kitchen-sink-helpers/index.js:202:13)
Next, I try to read node_modules/ember-cli/lib/broccoli/ember-app.js, I see:
var isProduction = this.env === 'production';
this.tests = options.hasOwnProperty('tests') ? options.tests : !isProduction;
this.hinting = options.hasOwnProperty('hinting') ? options.hinting : !isProduction;
But I don't really know how this.env get pass in the ember test and I can't get the environment within the Brocfil.js. I am assuming by default, hinting would respect the isProduction value if it isn't defined.
And searching further more, got me to https://github.com/rwjblue/ember-cli-test-loader, which seems to be related.
My questions are:
1. Is there a way to run ember test without jshint via CLI?
2. Can this be set using config/environment.js?
3. Can I set a breakpoint to debug the Brocfile.js file? I tried with chrome:localhost:4200, I don't any node_modules file being loaded.
Thanks in advance. I am an extreme newbie to javascript and ember..
try
ember test --query=nolint
the latest ember-cli-qunit uses this query param to disable lint checks in tests
here is the commit where it was added if you are interested
I have multiple packages under a subdirectory under src/,
running the tests for each package with go test is working fine.
When trying to run all tests with go test ./... the tests are running but it fails..
the tests are running against local database servers, each test file has global variables with db pointers.
I tried to run the tests with -parallel 1 to prevent contention in the db, but the tests still fail.
what can be the issue here?
EDIT: some tests are failing on missing DB entries, I completely clear the DB before and after each test. the only reason I can think of why this is happening is because of some contention between tests.
EDIT 2:
each one of my test files has 2 global variables (using mgo):
var session *mgo.Session
var db *mgo.Database
also it has the following setup and teardown functions:
func setUp() {
s, err := cfg.GetDBSession()
if err != nil {
panic(err)
}
session = s
db = cfg.GetDB(session)
db.DropDatabase()
}
func tearDown() {
db.DropDatabase()
session.Close()
}
each tests startup with setUp() and defer tearDown()
also cfg is:
package cfg
import (
"labix.org/v2/mgo"
)
func GetDBSession() (*mgo.Session, error) {
session, err := mgo.Dial("localhost")
return session, err
}
func GetDB(session *mgo.Session) *mgo.Database {
return session.DB("test_db")
}
EDIT 3:
I changed cfg to use a random database, the tests passed.
it seems that the tests from multiple packages are running somewhat in parallel.
is it possible to force go test to run everything sequentially across packages ?
Update: As pointed out by #Gal Ben-Haim, adding the (undocumented) go test -p 1 flag builds and tests all packages in serial. As put by the testflag usage message in the Go source code:
-p=n: build and test up to n packages in parallel
Old answer:
When running go test ./..., the tests of the different packages are in fact run in parallel, even if you set parallel=1 (only tests within a specific package are guaranteed to be run one at a time). If it is important that the packages be tested in sequence, like when there is database setup/teardown involved, it seems like the only way right now is to use the shell to emulate the behavior of go test ./..., and forcing the packages to be tested one by one.
Something like this, for example, works in Bash:
find . -name '*.go' -printf '%h\n' | sort -u | xargs -n1 -P1 go test
The command first lists all the subdirectories containing *.go files. Then it uses sort -u to list each subdirectory only once (removing duplicates). Finally all the subdirectories containing go files get fed to go test via xargs. The -P1 indicates that at most one command is to be run at a time.
Unfortunately, this is a lot uglier than just running go test ./..., but it might be acceptable if it is put into a shell script or aliased into a function that's more memorable:
function gotest(){ find $1 -name '*.go' -printf '%h\n' | sort -u | xargs -n1 -P1 go test; }
Now all tests can be run in the current directory by calling:
gotest .
apparently running go test -p 1 runs everything sequentially (including build), I haven't see this argument in go help test or go help testflag
I am assuming that because the packages individually pass that in this situation you are also dropping the DB before that test as well.
Therefore it sounds like the state of the DB for each package test is expected to be empty.
So between each set of the package tests the DB must be emptied. There are two ways around this, not knowing your entire situation I will briefly explain both options:
Option 1. Test Setup
Add an init() function to the start of each package _test file which you then put processing to remove the DB. This will be run before the init() method of the actual package:
func init() {
fmt.Println("INIT TEST")
// My test state initialization
// Remove database contents
}
Assuming that the package also had a similar print line you would see in the output (note the stdout output is only displayed when the a test fails or you supply the -v option)
INIT TEST
INIT PACKAGE
Option 2. Mock the database
Create a mock for the database (unless that is specifically what you are testing). The mock db can always act like the DB is blank for the starting state of each test.
Please try out the following github repository.
https://github.com/appleboy/golang-testing
Copy coverage.sh to /usr/local/bin/coverage and change permission.
$ curl -fsSL https://raw.githubusercontent.com/appleboy/golang-testing/master/coverage.sh /usr/local/bin/coverage
$ chmod +x /usr/local/bin/coverage