Log4net cannot find configuration file when run from Visual Studio/Microsoft Test Framework - unit-testing

We are writing unit tests for our business layer running under .NET 4.0. The business layer is a straightforward C# class library that usually runs within SOAP and REST web services. Our application uses log4net within a separate wrapper assembly for logging. The C# code in the logging assembly has an assembly info directive that tells log4net the name of the configuration file, a la-
[assembly: log4net.Config.XmlConfigurator(ConfigFile="AcmeLogging.config", Watch=true)]
Initializing the log4net through the wrapper works fine in the web services. When we initialize it from with our unit test assembly it does not appear to see the configuration file. The configuratino file is configured through properties to be copied to the execution directory, and we do see it in the bin\debug directory. A quick console test program using the logging assembly running from within that same folder works fine. The curious thing is that the behavior problems are intermittent, and pop up on different developers' machines at different times and cannot be cured in any deterministic way.
Stepping through the wrapper assembly code, the log4netLogManager.GetLogger() call appears to return correctly, but the list of appenders returned by log.Logger.Repository.GetAppenders() is empty. Since this incorrect behavior is the same whether the file is in the Bin\Debug folder or not, we believe that it is not seeing the file.
Any clues as to what we're missing about running log4net in the Microsoft Test Framework would be greatly appreciated.

Unit tests are unique in the fact that if you need configuration files in your unit tests you need to include them as deployment items. Here is an example of how I do this within my test class:
[TestClass]
[DeploymentItem("hibernate.cfg.xml")]
public class AsyncForwardingAppenderTest
{
}
In addition to the Deployment attribute you need to Enable Deployment in your test settings. To do this go to Test->Edit Test Settings->. Then click on Deployment area on right. Click the checkbox Enable Deployment.
After doing this and running your test your config file should be located in your test results folder. TestResults\username_machine date stamp\Out. If your config file is not located in this folder it will not work. This is what the DeploymentItem attribute does. It sticks the file in this Out folder.
If you don't want to include the DeploymentItem attribute on every test class what I did was to create a base test class that all of the tests that use log4net inherit from and mark it with the DeploymentItem attribute.

Related

How to set path at test discovery in visual studio?

How can I set the path to my external binaries during test discovery in visual studio's Test Explorer? After that how to make sure, it uses the correct paths?
I use windows 10 and VS 2019. I have a solution that builds some binaries and some tests into different folders. Also, I have some 3rd party dependencies, each in its own folder.
Something like:
solutionDir/
-ownBinaries/
-testBinaries/
-externalBinaries/
I'd like to use the Test Explorer to run my tests. For this purpose, I use a .runsettings file. I installed Google Test adapter via NuGet (later it will run on CI, so this is the only option). The automatic runsetting discovery is disabled, and this file is selected as the runsettings file. It overrides the workingDir to my ownBinaries folder, and extend the PATH enviroment variable with the externalBinaries. The relevant parts are:
<SolutionSettings>
<Settings>
<AdditionalTestExecutionParam>-testdirectory=$(SolutionDir)</AdditionalTestExecutionParam>
<WorkingDir>$(SolutionDir)ownBinaries</WorkingDir>
<PathExtension>$(SolutionDir)externalBinaries</PathExtension>
</Settings>
</SolutionSettings>
This is works fine, after my tests are discovered, but I have problems when it tries to discover my tests.
I use google test and c++, so the test discovery tries to run those tests with the --gtest-list-tests argument, then populate the view with the test name, case, etc. The binaries are just fine, builds without error, I can run them from the debugger, and they produce the output I want.
But the test explorer won't show them, because it doesn't set the externalBinaries path.
This is what lead me to this situation.
First I copied every binaries next to my test exe, namely into the testBinaries folder. Then, I could run it in the cmd with the --gtest-list-tests argument. Everything was fine, all my test names showed up. Started VS, and Test Explorer discovered all my tests, it was able to run them.
Then I done a clean build, so the external stuff deleted from the testBin folder. The Test Explorer cached the test names, so it was able to run them.
Restart VS. Test Explorer tries to discover my tests. but it fails whit this helping message: (removed date+time)
Google Test Adapter: Test discovery starting...
Failed to run test executable 'D:\MySolution\testBinaries\SBCUnitTest.exe': One or more errors occurred.
Check out Google Test Adapter's trouble shooting section at https://github.com/csoltenborn/GoogleTestAdapter#trouble_shooting
In particular: launch command prompt, change into directory '..\ownBinaries', and execute the following command to make sure your tests can be run in general.
D:\MySolution\testBinaries\SBCUnitTest.exe --gtest_list_tests -testdirectory=
Found 0 tests in executable D:\MySolution\testBinaries\SBCUnitTest.exe
Test discovery completed, overall duration: 00:00:00.3022924
Have you noticed that -testDirectory= is empty despite it is set in the runsettings file?
I'm completely lost how I can proceed with it. This workaround is quite heavy to copy all files, then delete all but the test binaries each time when I start VS.
Here is the link for the Troubleshooting section mentioned in the error message.
I've read through the readme file on github, also the runsetting docs on Microsoft's website.
Edit
I made progress with the VsTest.console.exe, I can successfully run all my tests with the proper arguments as below:
& "VSTest.console.exe" *_uTest.exe /Settings:..\MySolution.gta.runsettings /TestAdapterPath:"..\packages\GoogleTestAdapter.0.18.0\build\_common\"
I use the same *.runsettings and *.gta_settings_helper files. Those files are used to get absolute paths for the dependencies. I could run this from different folders, but then I had to adjust the arguments (test discovery pattern, relative path to runsettings, and relative path to GTA).
Great news, that it successfully runs on Azure (it uses vstest.console).
Edit 2
Tried to merge the workingDir and pathExtension nodes, so only one needed (the pathExtension). No success.
Tried to install Test adapter for google test in the VS installer, delete the runsetting file, and set the properties in VS->Tools->Options then Test adapter for google test. Even the example pathExtension didn't worked for me.
Found the extended logs under %AppData%/Local/Temp/TestAdapter/someNumber/*.txt and in that log I've found one line as the runsettings file. I paste here the formatted version of the log
<RunSettings>
<GoogleTestAdapterSettings>
<SolutionSettings>
<Settings>
<WorkingDir>$(SolutionDir)</WorkingDir>
<PathExtension>$(SolutionDir)externalBinaries</PathExtension>
</Settings>
</SolutionSettings>
<ProjectSettings>
</ProjectSettings>
<GoogleTestAdapterSettings>
<SolutionSettings>
<Settings>
</Settings>
</SolutionSettings>
<ProjectSettings>
</ProjectSettings>
</GoogleTestAdapterSettings>
</GoogleTestAdapterSettings>
</RunSettings>
Does anybody know why is there an empty google test adapter setting? Where does it comes from? I think this is overwrites my settings.
It turned out, before first run the relative paths are not known.
Trivial solution
Add the full path to the PATH Extension under Visual Studio -> Options -> Test Adapter for Google Test settings. Meanwhile the custom *.runsetting file is not selected.
Using this method all my tests are discoverable, but it is a manual setting for each repo cloned.

sonar coverage reports for multiple java unit test reports

i have a java project called Customer, under this i have another 8 modules, among these 5 modules have junit test classes and have separate ant build files for each modules. I have created(generated unit test reports) jacoco.exec for each project, now, i like to combine these 5 modules' unit test reports into one report and display (or) display unit test reports for each module wise in sona coverage section. Can you please provide any suggestion for this.
Thanks,
Joseph
What you want to do is read each module's test report. You can do that with multi-module project configuration. It will probably be most straight-forward if you handle it all from your main build.xml. I've not tested this from Ant, but something like this should work
<property name="sonar.projectKey" ...
<!-- all normal properties here -->
<property name="sonar.modules" value="module1,module2..." />
<property name="module1.sonar.jacoco.reportPath" value="...
Note that if the Jacoco reports are all in a standard location, you may not even need to specify them because child modules inherit their parents' properties. The docs for multi-module project configuration aren't written with Ant syntax in mind, but you should be able to work through that as long as you keep this in mind from the SonarQube Scanner for Ant docs:
The SonarQube Scanner for Ant is an Ant Task that is wrapper of SonarQube Scanner, which works by invoking SonarQube Scanner and passing to it all properties named following a sonar.* convention. This has the downside of not being very Ant-y, but the upside of providing instant availability of any new analysis parameter introduced by a new version of a plugin or of SonarQube itself.
Thanks a lot. again to Ann.. It works to me, my issues resolved(coverage and multiple languages for each module) after added separate module for each sub project in sonar-project.properties as mentioned below.. getting 'coverage' results for modules in Sonarqube..
# Modules
sonar.modules=common,help,mobile,partners
common.sonar.projectBaseDir=C:/dev/workspaces/hg/customer
common.sonar.sources=common/src/main/java,common/web
common.sonar.tests=common/src/test/java
common.sonar.binaries=common/bin
common.sonar.junit.reportsPath=common/test/reports/junitreport
common.sonar.surefire.reportsPath=common/test/reports/junitreport
common.sonar.jacoco.reportPath=common/coverage/common.exec
help.sonar.projectBaseDir=C:/dev/workspaces/hg/customer
help.sonar.sources=help/src/main/java,help/web
help.sonar.tests=help/src/test/java
help.sonar.binaries=help/bin
help.sonar.junit.reportsPath=help/test/reports/junitreport
help.sonar.surefire.reportsPath=help/test/reports/junitreport
help.sonar.jacoco.reportPath=help/coverage/help.exec
This worked for me:
sonar.modules=common,help
sonar.sources=./src/main/java
sonar.binaries=./target/classes
sonar.tests=./src/test/java
sonar.junit.reportsPath=./target/surefire-reports
The path should be relative because sonar automatically navigates to the basedir/module_name/target/surefire-reports

Grails test-app classpath

I'm trying to use test support classes within my tests. I want these classes to be available for all different test types.
My directory structure is as follows;
/test/functional
/test/integration
/test/unit
/test/support
I have test helper classes within the /test/support folder that I would like to be available to each of the different test types.
I'm using GGTS and I've added the support folder to the classpath. But whenever I run my integration tests running 'test-app' I get a compiler 'unable to resolve class mypackage.support.MyClass
When I run my unit tests from within GGTS the support classes are found and used. I presume this is because the integration tests run my app in its own JVM.
Is there any way of telling grails to include my support package when running any of my tests?
I don't want my test support classes to be in my application source folders.
The reason that it works for your unit tests inside the IDE is that all source folders get compiled into one directory, and that is added to your classpath along with the jars GGTS picks up from the project dependencies. This is convenient but misleading, because it doesn't take into account that Grails uses different classpaths for run-app and each of the test phases, which you see when you run the integration tests. GGTS doesn't really run the tests; it runs the same grails test-app process that you do from the commandline, and captures its output and listens for build events so it can update its JUnit view.
It's possible to add extra jar files to the classpath for tests because you can hook into an Ant event and add it to the classpath before the tests start. But the compilation process is a lot more involved and it looks like it would be rather ugly/hackish to get it working, and would likely be brittle and stop working in the future when the Grails implementation changes.
Here are some specifics about why it'd be non-trivial. I was hoping that you could call GrailsProjectTestCompiler.compileTests() for your extra directory, but you need to compile it along with the test/unit directory for unit tests and the test/integration directory for integration tests, and the compiler (GrailsProjectTestCompiler) presumes that each test phase only needs to compile that one directory. That compiler uses Gant, and each test phase has its own Grailsc subclass (org.grails.test.compiler.GrailsTestCompiler and org.grails.test.compiler.GrailsIntegrationTestCompiler) registered as taskdefs. So it should be possible to subclass them and add logic to compile both the standard directory and the shared directory, and register those as replacements, but that requires also subclassing and reworking GrailsProjectTestRunner (which instantiates GrailsProjectTestCompiler), and hooking into an event to replace the projectTestRunner field in _GrailsTest.groovy with your custom one, and at this point my brain hurts and I don't want to think about this anymore :)
So instead of all this, I'd put the code in src/groovy and src/java, but in test-specific packages that make it easy to exclude the compiled classes from your WAR files. You can do that with a grails.war.resources closure in BuildConfig.groovy, e.g.
grails.war.resources = { stagingDir ->
println '\nDeleting test classes\n'
delete(verbose: true) {
// adjust as needed to only delete test-specific classes
fileset dir: stagingDir, includes: '**/test/**/*.class'
}
println '\nFinished deleting test classes\n'
}

Team Build: Cannot find generated private accessor

We have been using TeamBuild and test for our continuous integration build for about 4 months and this issue just popped up the other day in one of our test assemblies when running tests on one of our test assemblies on the build server only.
{NameSpace}.Order_Accessor, Version=0.0.0.0, Culture=neutral, PublicKeyToken='{xxx}' or one of its dependencies. The system cannot find the file specified.
This is not being caused by problems with Publicize.exe noted elsewhere. The unit tests run fine locally and the generated assembly does get created.
The part that is confusing is that we use generated private accessor assemblies in numerous projects including the one with this issue above without any problems. Investigating this issue further, the generated assembly never gets copied to the TestResults//Out directory which appears to be the problem. I determined this was the issue by adding the assembly from SolutionRoot using the TestRunConfig deployment settings which results in an assembly manifest does not match error.
I am wondering if anyone has had issues with these assemblies being copied properly. We are running TeamSystem 2008 on Windows Server 2008 if that helps.
Below is the test run deployment error for the file in question as well
Run has the following issue(s):
TESTTOOLSTASK : warning : Test Run deployment issue: Failed to get the file for deployment item 'E-mail Templates\OrderConfirmation.txt' (output directory 'E-mail Templates') specified by the test 'EmailServiceTest.EnqueueTest': System.IO.DirectoryNotFoundException: Could not find a part of the path 'Continuous Integration Build\Binaries\Continuous Integration Test\E-mail Templates\OrderConfirmation.txt'.
TESTTOOLSTASK : warning : Test Run deployment issue: Failed to get the file for deployment item 'E-mail Templates\OrderConfirmation.htm' (output directory 'E-mail Templates') specified by the test 'Email.Tests.EmailServiceTest.EnqueueTest': System.IO.DirectoryNotFoundException: Could not find a part of the path 'Continuous Integration Build\Binaries\Continuous Integration Test\E-mail Templates\OrderConfirmation.htm'
TESTTOOLSTASK : warning : Test Run deployment issue: The assembly or module 'Services.Order_Accessor' directly or indirectly referenced by the test container '\continuous integration build\binaries\continuous integration test\services.order.supplierintegration.tests.dll' was not found.
Looks like your TestRunConfig deployment expects that the files are in "Continuous Integration Build\ Binaries\"
I've highlighted the "Binaries" part for a reason. This is a folder the $BinariesRoot property points to (unless overridden), and it is used by the compiler. Therefore, I'd suggest you double-check that:
The Publicize tool (or whatever tool you use to generate the assemblies) is configured to copy the output into ($BinariesRoot)Continuous Integration Test
There are provisions in the team build type to copy the .htm and the .txt files to ($BinariesRoot)Continuous Integration Test

TFS UnitTesting not deploying local copy assembly to test dir when on build server

I have an assembly that needs to be in the test output dir for my tests to run.
I have the assembly referenced as a local copy in the project but on the build server this gets ignored.
The two ways I have found to fix this are
Add a special attribute to test method that will make sure the file is there for each test.
[DeploymentItem("my assembly")]
This is not very practical as this assembly is required for almost every test in the assembly.
Add test run config file with special deployment section. I am using a TestContainer in my build scripts to run the tests I think that this may be the reason my included test run config does not get picked up and the assembly not copied. I would prefer to not have a vsmdi test list file as I am trying to run all tests and I feel this would be a violation of DRY.
Any suggestions on how I can get my tests running?
As my assembly was being dynamicly loaded the unit test framework was not copying it.
I added a explict refrence to it by calling typeof on one of the types in the assembly and all is fine.
Thanks Jerome Laban for your help with this one.