I am using VisualPHPUnit and I am trying to organize my tests into suites (due to the fact that Selenium IDE does not export PHPUnit test suites).
I am currently implementing the option of the configuration XML file.
Yet,it is very limiting cause I want to run test suites on demand and not uploading each time on the server a new XML file (aka test suite).
I know that I can create an XML file with many test-suites in it but I would like to run them individually.
As you understand i am struggling towards DontRepeatYourself principle and code reuse.You know..just choose login.php , then the testcase and them logout.php and run them.
Is something like this possible???
Moreover...would it be difficult VisualPHPUnit to parse only one XML file and create a dropdown box of testsuites to choose from and run?
This is my XML file
<phpunit>
<!-- This is required for VPU to work correctly -->
<listeners>
<listener class="PHPUnit_Util_Log_JSON"></listener>
</listeners>
<testsuites>
<testsuite name="TestSuite1">
<file>/var/www/VisualPHPUnit/app/unitTests/Login/Login.php</file>
<file>/var/www/VisualPHPUnit/app/unitTests/CreateCourse/CreateCourse1.php</file>
<file>/var/www/VisualPHPUnit/app/unitTests/Logout.php</file>
</testsuite>
</testsuites>
</phpunit>
actually the PHP formaters are once again available for Selenium IDE. You may want to check this out:
https://addons.mozilla.org/en-us/firefox/addon/selenium-ide-php-formatters/
there are 2 php formaters: PHP_unit and PHP_Selenium
Related
Can I somehow use the same Eclipse plug-in project for both a generated EMF model and the corresponding generated EMF Edit code?
Normally these two components reside in two different projects, the EMF Edit one with the suffix .edit to its name. I find this superfluous, since there is so little code in the Edit project, and it is so closely related to the model code.
I have tried setting both the modelDirectory and the editDirectory Gen Model attributes to (different) directories in the same project, but that seems to lead to endless confusion and build problems. I think maybe the two generation steps overwrite each others project setting files.
After some more experimentation it seems like it works fine to have EMF and EMF Edit generated code in the same project.
The things I had to do to make it work are the following:
Setting the genmodel property modelDirectory and editDirectory to the same directory. Otherwise I got a build error saying "The type ... is already defined in ...".
Setting the genmodel property bundleManifest="false". Otherwise the plug-in ID is overwritten by the generation process.
Apart from this I also set updateClasspath="false" to avoid that the generation process messes around with that.
The automatic updates to the manifest and plugin.xml files seem to be the following:
Set plug-in ID
Add exported packages
Add EMF extensions to plugin.xml
2 and 3 needs to be performed manually if they are desired. That would involve adding entries to plugin.xml similar to these:
<extension point="org.eclipse.emf.ecore.generated_package">
<!-- #generated model -->
<package
uri="somePackage"
class="somePackage.SomePackage"
genModel="model/model.xcore"/>
</extension>
<extension point="org.eclipse.emf.edit.itemProviderAdapterFactories">
<!-- #generated model -->
<factory
uri="somePackage"
class="somePackage.someClass"
supportedTypes=
"org.eclipse.emf.edit.provider.IEditingDomainItemProvider
org.eclipse.emf.edit.provider.IStructuredItemContentProvider
org.eclipse.emf.edit.provider.ITreeItemContentProvider
org.eclipse.emf.edit.provider.IItemLabelProvider
org.eclipse.emf.edit.provider.IItemPropertySource"/>
</extension>
I need to assign my test results to use cases.
Currently I have TestNG tests for my classes (unit tests).
Obviously those tests are classes exist because of use cases, but there is no obvious 1-1 mapping.
Is it possible to configure TestNG reports to include custom groups in reporting?
Like
F02UC01 parse input
for this use case I have test classes:
com.company.product.input.ParseTest
F03UC02 produce output
for this use case I have test
com.company.product.input.OutputTest
com.company.product.input.AnotherOutputTest
Ideally, I do not want to rerun or rewrite existing tests. I just want another test report, with diffrent grouping criterias.
Usually you do this by creating a suite xml file. E.g.
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="Use Case Tests">
<test name="F02UC01 parse input">
<classes>
<class name="com.company.product.input.ParseTest"/>
</classes>
</test>
<test name="F03UC02 produce output">
<classes>
<class name="com.company.product.input.OutputTest"/>
<class name="com.company.product.input.AnotherOutputTest"/>
</classes>
</test>
</suite>
Maybe you can select them by packages instead of classes.
<packages>
<package name="com.company.product.input.*"/>
</packages>
You can also mix classes and packages selection. See the testng documentation.
EDIT
I am running all tests anyway. I just want another report, where some of the tests are use cases.
I guess in this case you have to implement your own IReporter.
I would create an annotation that I can add to test methods to logically group them. E.g.
#Test
#TestTag("F02UC01 parse input")
public void someTest(){
}
and then use the IAnnotationFinder in my custom reporter to report tests grouped by the annotation's value.
I'm trying to use a custom appender in a clojure application using logback & clojure tools.logging.
My config looks like this:
<appender name="Sentry" class="net.kencochrane.raven.logback.SentryAppender">
<dsn>some-dsn</dsn>
</appender>
This results in the following runtime error:
The following loggers will not work because they were created during the default configuration phase of the underlying logging system.....
A quick google search turns up a document to use a JoranConfigurator programatically to perform a multi-step configuration, but I actually don't see an exposed method in tools.logging to perform multi-step configuration. Any advise other than modifying tools.logging? Am I missing something obvious? Thanks for your time.
tools.logging doesn't do any configuration of the underlying logging system. In the case of logback it just talks to the logging system via slf4j. The configuration of the actual logging is totally on the logback side.
An easy way might be to use a logback.xml config file and Logback dependencies with clojure/tools.logging. Based on Sentry docs it looks like you need the following dependencies:
[org.clojure/tools.logging "0.3.1" :exclusions [org.clojure/clojure]]
[org.slf4j/slf4j-api "1.7.25"]
[org.codehaus.janino/janino "3.0.7"] ; for conditional config processing
[ch.qos.logback/logback-classic "1.2.3"]
[ch.qos.logback/logback-core "1.2.3"]
[com.getsentry.raven/raven-logback "8.0.2"]
The Sentry docs page also has snippets of Logback XML config that you can use. For MDC attributes, you may want to take a look at Cambium.
here's a simple setup using sentry-java-logback,
with deps:
[org.clojure/tools.logging "0.4.1"]
[ch.qos.logback/logback-classic "1.2.3"]
[io.sentry/sentry-logback "3.2.0"]
and logback.xml:
<!-- Sentry -->
<appender name="SENTRY" class="io.sentry.logback.SentryAppender">
<encoder>
<pattern>${defaultPattern}</pattern>
</encoder>
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>ERROR</level>
</filter>
<options>
<dsn>https://........</dsn>
<release>1.0.0</release>
<serverName>ABCDE</serverName>
<environment>production</environment>
</options>
<minimumBreadcrumbLevel>DEBUG</minimumBreadcrumbLevel>
</appender>
<!-- ....THIS MIGHT CHANGE.... -->
<root level="DEBUG">
<appender-ref ref="STDOUT"/>
<appender-ref ref="SENTRY"/>
</root>
you can read more on Sentry docs
My SQL files aren't being executed when I install or update a new component. Initially I found it weird that my workmate needed to execute his SQL files directly into PHPMyAdmin, but isn't Joomla! supposed to automatically run the files?
Here's how I set up my XML:
ReforcoDigital.xml
<?xml version="1.0" encoding="utf-8"?>
<extension type="component" version="2.5.0" client="site" method="upgrade">
<name>Reforco Digital</name>
<author>Rodrigo Pereira</author>
<creationDate>14/05/2013</creationDate>
<copyright>Copyright</copyright>
<license>Licença</license>
<authorEmail>rodrigo-c-pereira#hotmail.com</authorEmail>
<authorUrl>http://www.site.com</authorUrl>
<version>0.1</version>
<description>Sistema de Reforço Digital</description>
<install>
<sql>
<file driver="mysql" charset="utf8">sql/install.mysql.utf8.sql</file>
</sql>
</install>
<update>
<schemas>
<schemapath type="mysql">sql/updates/mysql</schemapath>
</schemas>
</update>
<files folder="site">
<folder>language</folder>
<folder>models</folder>
<folder>views</folder>
<filename>reforcodigital.php</filename>
<filename>controller.php</filename>
<filename>index.html</filename>
</files>
<administration>
<menu>Reforco Digital</menu>
<files folder="admin">
<folder>sql</folder>
<folder>views</folder>
<filename>controller.php</filename>
<filename>reforcodigital.php</filename>
<filename>index.html</filename>
</files>
</administration>
</extension>
Folder admin/sql has the install.mysql.utf8.sql file, and admin/sql/updates/mysql has the file 0.1.sql; both have the very same SQL code. What am I possibly missing here?
The fact that I was updating my component with a 0.1.sql over and over was the problem. Changed the filename to 0.2.sql and it did the job.
My understanding is that Joomla! will look for the update files in alphabetical order, and judge if that version has been used or not; correct me if that's wrong. Since there was a higher version than 0.1, it did the job.
I commented earlier that I was having the same problem as you, but I'm not. Mine is a completely different problem that I'll describe after I explain what's happening with your file.
So, your sql folder resides within your admin component folder (this is normal), but your install/uninstall sql XML entries don't describe this. So, I believe that if you make this change, you'll find it working as expected:
<install>
<sql folder="admin">
<file driver="mysql" charset="utf8">sql/install.mysql.utf8.sql</file>
</sql>
</install>
My problem is that the update section will only execute a schema update when a schema exists in the #__schema table - and install.sql isn't executed on update. So if you're upgrading your extension and adding a table to it, you can't really rely on the XML installer to create your new table unless your extension already had tables before the update.
I'm going with an install.php file to search for and create the table if necessary.
I have following, very simple, XML config for PHPUnit:
<phpunit bootstrap="/_tests/TestAutoload.php">
<testsuites>
<testsuite name="Unit Tests">
<directory suffix=".php">_tests</directory>
</testsuite>
</testsuites>
</phpunit>
How to exclude certain file in this directory from test suite? I tried <exclude> and <blacklist>, but it doesn't seem to work in this context. Also couldn't find any other documentation than phpunit.de one, which doesn't mention anything about it. Else than that, this config works perfectly.
To exclude the file name TestCase.php.
add this to your phpunit.xml
<testsuites>
<testsuite name="BLABLA">
<directory suffix=".php">./tests</directory>
<exclude>./tests/TestCase.php</exclude>
</testsuite>
</testsuites>
Here is an additional excerpt from a real-live test-suite I can confirm it working with:
...
<testsuites>
<testsuite name="n98-magerun-tests">
<directory>./tests</directory>
<exclude>tests/N98/Magento/Command/Installer/UninstallCommandTest.php</exclude>
</testsuite>
...
There are a number of ways to not run a particular test - putting it into a blacklist so it's never run may not be the way - as changing it means editing the blacklist, and you'll often endup bouncing it in and out of version control.
There are several other ways that may be more appropriate:
If a test is not yet ready to run:
$this->markTestIncomplete('This test has not been implemented yet.');
If there's an outside reason it should not be run, skip it:
if (!extension_loaded('mysqli')) {
$this->markTestSkipped('The MySQLi extension is not available.');
}
You can also put that into the setUp() function, so it will skip all the tests in a test-class.
You can make a test dependant on a previous one succeeding:
public function testEmpty()
{
$stack = array();
$this->assertTrue(empty($stack));
return $stack; // also sends this variable to any following tests - if this worked
}
/**
* only runs if testEmpty() passed
*
* #depends testEmpty
*/
public function testPush(array $stack)
{
}
The #group -name- annotation is one of the best ways to specifically stop, or run one group of tests
/**
* #group database
* #group remoteTasks
*/
public function testSomething()
{
}
testSomething() is now in two groups, and if either is added on the command line (or in the config.xml) --exclude-group parameter. it won't be run. Likewise, you could run only tests that belong to a particular group - say named after a feature, or bug report.
2021 update
On top of some good single answers above here is the entire approach that allows you to apply a consistent, clean, more architecture-driven tests organization and a convinient fast testing workflow. With it you manage your tests, directories and testsuites from phpunit.xml and run tests in groups or by one as needed. So do the following:
Group your tests in the directories (e.g. tests/Unit, tests/Feature, tests/Integration);
Make testsuites grouping your directories with <testsuite> element (note you have to wrap multiple <testsuite> tags in <testsuites> tag in later versions of PHPUnit);
Make an all testsuite that combines all default tests you would run as a full test suite and assign it via defaultTestSuite="all" key within <phpunit> element like this:
<phpunit .. some other keys
defaultTestSuite="all">
<testsuite name="all">
<directory suffix="Test.php">./tests/Feature/</directory>
<directory suffix="Test.php">./tests/Unit/</directory>
</testsuite>
</phpunit>
If you need make a dedicated Tinker test suite with tests that you could use for tinkering, keeping example tests etc. you would exclude from normal testing workflow. Do not inlcude it in the all test suite.
So you will be able to:
use phpunit CLI command to always run the default all tests.
use CLI to filter on testsuite, test file or single test level for any of your test suites e.g.:
phpunit --testsuite SuiteOne,
phpunit --filter SomeTest or
phpunit --filter SomeTest::test_some_test_method
combine --testsuite with --filter arguments
Couple this workflow with the capability to run the current test or test file from within your IDE (for Sublime Text there are Mac and Linux/Windows plugins) and you will be completelly equipped to instantly chose what test to execute.
Whit this PHPUnit configuration-file I have made very good experiences.
<?xml version="1.0" encoding="UTF-8"?>
<phpunit
convertErrorsToExceptions="true"
convertNoticesToExceptions="true"
convertWarningsToExceptions="true"
colors="true"
processIsolation="true"
stopOnFailure="true"
syntaxCheck="false"
backupGlobals="false"
bootstrap="test-bootstrap.php">
<testsuites>
<testsuite name="php-dba-cache">
<directory suffix="Test.php">tests</directory>
</testsuite>
</testsuites>
<logging>
<log type="coverage-html"
target="build/coverage"
charset="UTF-8"
yui="true"
highlight="true"
lowUpperBound="35"
highLowerBound="70"/>
</logging>
<filter>
<whitelist addUncoveredFilesFromWhitelist="true">
<directory suffix=".php">src</directory>
<exclude>
<file>test-bootstrap.php</file>
</exclude>
</whitelist>
</filter>
</phpunit>
https://github.com/gjerokrsteski/php-dba-cache
The phpunit documentation is a bit minimalistic when it comes to exclusion in a testsuite. Apparently, only entire directories can be excluded but not individual files. I would be very happy to be proven wrong. The workaround seems to be using the #group feature as posted above by Alister Bulman.
It's kind of a pain needing to tag every single test in those test suites I'd like to keep.
For Phpunit 6.5, exclude is under whitelist
<filter>
<whitelist>
<directory suffix=".php">src</directory>
<exclude>
<directory>src/Migrations</directory>
<file>src/kernel.php</file>
</exclude>
</whitelist>
</filter>
I came across this with a test class that I wanted to extended for other tests. PHPUnit would issue warnings about the test file containing no tests. Simply declaring the class inside the file abstract caused PHPunit to quiet down about that.
abstract class SetupSomeCommonThingsTestCase {
protected function setUp(): void {
parent:setUp();
...
}
}
Hey there, Make sure that you put your exclusions in the Whitelist. Example:
<phpunit>
<filter>
<blacklist>
<directory suffix=".php">/not/even/looked/at/</directory>
</blacklist>
<whitelist>
<directory suffix=".php">/path/to/test/dir/</directory>
<exclude>
<file suffix=".php">/path/to/fileToExclude.php</file>
</exclude>
</whitelist>
</filter>
</phpunit>
http://www.phpunit.de/manual/current/en/appendixes.configuration.html#appendixes.configuration.blacklist-whitelist