I know it is best practice to run all unit test cases after any change to make sure not breaking anything. However, some times, e.g. debugging, I really want to run only one single test case. It seems Xcode doesn't provide any such feature in UI, while other testing framework such as JUnit has such features.
Is there any workaround to have only one testcase run in Xcode?
P.S. most of my test cases are logic tests. So, they are not run in iPhone device.
Xcode 4 now have this feature. Simply create a "run scheme" that has the test cases that you want to run.
Open menu "Product|Edit Scheme..."
Click on "Edit..."
In the left pane, expand the "Test" section.
In the right pane, expand the test bundle and uncheck the test cases you don't need to run.
⌃+⌥+⌘+U
You can also use the keyboard short cut of Control-Option-Command-U
Expert taken from Apple Documentation
Product > Perform Action > Test . This dynamic menu item senses the current test method in which the editing insertion point is positioned when you’re editing a test method and allows you to run that test with a keyboard shortcut. The command’s name adapts to show the test it will run, for instance, Product > Perform Action > Test testAddition. The keyboard shortcut is Control-Option-Command-U.
I'm sure that no one missed the release of Xcode 5 and its ability to easily run a single test case, but adding this answer just for completeness.
In Xcode 5 you can run a single test case by clicking the little play button that you can find next to the test in the test navigator or next to the test method in the editor. In both places it shows up when you hover over it.
You can also use xctool from the commandline with the --only argument, which will only run the specified testcase(s).
As you have noted, the OCUnit test framework marks methods whose name start with 'test' as test cases. This is done at runtime
In practice, your test cases should run so fast that it should not matter how many are enabled; your debugger should be able to stop inside your test case very quickly after you press "Debug".
That being said, the quickest way to disable some tests is probably to use an #if 0 / #endif block. The feature to disable test cases dynamically does not exist in Xcode / OCUnit, since there is no GUI component.
In theory it should be doable, because at runtime (and before all tests are run) there are ways to access the test list in OCUnit, but this requires modifications to the OCUnit source code, which is not desirable (it will be wiped out in the next Xcode update, for one).
Finally, if that feature is important to you, you can easily write your own test harness that mostly replicates what OCUnit does.
Then you can tweak it to your heart's content, add UI, etc.
It is not difficult, and a little educational. Here's a good link to get you started:
http://gusmueller.com/blog/archives/2009/10/how_to_write_your_own_automated_testing_framework.html
Go to View -> Navigators -> Tests (⌘-6)
Find your test case or test and secondary click on it
Run your case/test
Related
Am using testify's test suite support to write unit tests. This results in my test file having a single TestFooBar(t *testing.T) that kicks of the suite.Run while all my individual tests become a part of my test suite struct with method signatures like - func (suite *myTestSuite) TestMyStuff().
I have observed that GoLand can identify all methods with a signature similar to TestFooBar(t *testing.T) and put a green play icon next to it. It will allow me to run/debug these methods individually. However, all the test methods that are a part of the test suite as described above, won't be identified and cannot be ran or debugged individually in the IDE.
Any way to tell GoLand that myTestSuite structure has many tests that will allow me to execute them individually inside the IDE?
(Similar question here but that is just talking about command line, while my question is specifically for the IDE.)
At the moment, the IDE does not support recognizing tests from testify. There's an issue for that, https://youtrack.jetbrains.com/issue/GO-3066, and we hope we'll get it done soon.
As a workaround, you can edit manually a Run Configuration via Run | Edit Configurations... | + | Go Test and pass the arguments to the Go Tool in order to pick up the test that you need to debug.
Workaround: Remove the suite receiver, do the test and then put the suite receiver back.
I'm a new NUnit user, using NUnit 3.9 under Visual Studio Community 2017. I'm using it on a pet open source library project, and it's going well once I got the hang of it.
The library accesses a publicly available government website via a documented API. Most of my tests use local data, so that I have a stable bed to compare against, and so that I can test without going out to the website every time.
I would like to set it up so that normally, the tests that hit the server do not run. I run the tests over and over as I tweak the code, and just as a matter of courtesy, don't want to bang on the server. Also, I'd like to be able to test even when the remote system is down or when I don't have Internet access.
Is there any way to group or tag my tests so that normally only the ones using local data run, but that I can still, when necessary, run the ones that exercise the server access? Either specifying "run these" or "exclude these" would be fine.
I've grouped the tests into two different classes, UnitTestOffline.cs and UnitTestOnline.cs, and was hoping I could somehow run the tests on a class-by-class basis, but haven't found a way to do that.
You'll get better answers if you say specifically how you run your tests, since there are a number of ways to do it. Since you mention VS2017, I'm going to assume that you are using the NUnit 3 VS Adapter, but let us know if you are using some other approach.
In the VS adapter, use the dropdown to display your tests by class. Right click on the class for which you want to run tests and run them.
If you decide to categorize tests using the CategoryAttribute, you can display tests by "trait" in Visual Studio. As before, right click on the group you want to run tests for and run them.
If you get a lot of tests, you might want to put your unit tests in one assembly and your integration tests in another. In that case, display the tests by project, right click on the project you want and run them.
All of this can also be done using the nunit3-console command-line runner as well. To select by class or category, you use the --where option. To select by assembly, you merely enter the name of the assembly you want on the command-line.
Seems like you want to categorize your tests (unit test, integration tests...) and run only the unit tests... you could use [Category] for that.
In the nunit GUI you could /include /exclude category after that and run only the one you want.
And probably that the filtering of Visual Studio could work.
Try to see one of the solution suggested here as well
I understand that I should use the sure-fire plugin for unit tests, and failsafe for integration. I can run unit tests with mvn test and integration tests with mvn verify but this annoys me for 2 reasons:
I'd prefer to be able to select any test class (or method in that class) and run it individually by a simple click, rather than typing it into terminal every time.
The terminal returns the test results in ugly black/white paragraphs, requiring me to sift through them. I'd much prefer to have the results returned in a visually organized manner, similar to if I right-clicked on the test class in IntelliJ and click 'RunDemoTest`. This produces:
I find the error results much easier to sift through, for example it shows red/green #Test results on the left, and on the right it cleanly organizes the error into
Expected : 3
Actual :1
I'm sure there are advantages to using terminal for automated test runs later into production, but during development I don't find the terminal conducive to my tinkering.
How do I benefit from IntelliJ's visual feedback of test results, while simultaneously ensuring unit & integration tests are run separately, and preserving my freedom to pick and choose which test classes and test methods I can run at any time?
I'm assuming I can't have my cake and eat it too. Please explain.
If you are using the IntelliJ view "Maven Projects" you can very easy toggle on/off the exection of maven integrated tests.
Via "Run/Debug Configurations" you can create test executions that match your reqirement for a comfortable UI.
After these steps, there is a new entry in the drop down list "Run/Debug configurations". When you start the new JUnit Test configuration, the defined tests are executed and the results are presented exactly in the same manner as the screenshot in your question.
The options in my second screenshot allow a very flexible definition of the scope. You don't have go to every java file and click on the green arrows in the editor view.
This configuration isn't related to any maven configuration, and you can use them at any time in your coding process.
When using WebStorms as a test runner every unit test is run. Is there a way to specify running only one test? Even only running one test file would be better than the current solution of running all of them at once. Is there a way to do this?
I'm using Mocha.
not currently possible, please vote for WEB-10067
You can double up the i on it of d on describe and the runner will run only that test/suite. If you prefix it with x it will exclude it.
There is a plugin called ddescribe that gives you a gui for this.
You can use the --grep <pattern> command-line option in the Extra Mocha options box on the Mocha "Run/Debug Configurations" screen. For example, my Extra Mocha options line says:
--timeout 5000 --grep findRow
All of your test *.js files, and the files they require, still get loaded, but the only tests that get run are the ones that match that pattern. So if the parts you don't want to execute are tests, this helps you a lot. If the slow parts of your process automatically get executed when your other modules get loaded with require, this won't solve that problem. You also need to go into the configuration options to change the every time you want to run tests matching a different pattern, but this is quick enough that it definitely saves me time vs. letting all my passing tests run every time I want to debug one failing test.
You can run the tests within a scope when you have a Mocha config setting by using .only either on the describe or on the it clauses
I had some problems getting it to work all the time, but when it went crazy and kept running all my tests and ignoring the .only or .skip I added to the extra mocha options the path to one of the files containing unit tests just like in the example for node setup and suddenly the .only feature started to work again regardless of the file the tests were situated in.
I am using MSTest and running unit tests on Team Foundation Server. I am doing wildcard test execution as here; also see image.
I want to use an arbitrary test settings file for one specific dll / test assembly (let's call it foo.dll) and not use any test settings file/use the default settings for every other dll that is grabbed by the wildcard. What is the best way to do that?
I have an image that might help illustrate:
(The above is an image of the screen at Process tab-->Basic-->Automated Tests-->Test Assembly, within the Team Foundation Build template.)
With the standard, out of the box process templates you can't do what you're after. You can't wildcard assemblies and exclude some at the same time, and you can't attach a test settings file for just some assemblies. The reason being that under the hood it's doing a single test run and the settings apply for the whole test run.
As an idea, you could copy and customize the build process template, and specifically alter the section relating to running tests to do what you need. Add a second test run to it for example so you can do one set of tests without the run settings and another run with them. For both test runs use the wildcard specification and then use the criteria/arguments tab to decide which tests to run. In the DLL you want to exclude from the first run, tag the tests with an attribute and ignore those in your first test run. For the second include just tests with that attribute.