Is there a way to log all assertions, not just failing ones?
I wonder whether AssertJ provides an interface to access the details of the assertion (WritableAssertionInfo) and the result of the assertion? Is there any way to hook into the assertion process and add a simple log output?
I checked the documentation, the Java API and some related questions (like this one) but couldn't find a solution to my use-case.
Nope, only failed assertions lead to (error) messages, there is recording of what is being asked.
I'm curious to know what problem you are trying to solve if you can share it.
Related
How to control output from Twisted-trial tests?
I have looked up for different solutions, but I'm quite new to testing, so I can't find a fitting solution or can't use it correctly.
In general, I try to make autotesting system for my project like BuildBot. But BuildBot doesn't fit me because it reacts only to "On change sources" hook from Mercurial and I want to use other hooks too.
On THIS page from BuildBot documentation I found this information:
One advantage of trial is that the Buildbot happens to know how to
parse trial output, letting it identify which tests passed and which
ones failed. The Buildbot can then provide fine-grained reports about
how many tests have failed, when individual tests fail when they had
been passing previously, etc.
Does it mean that there is no way but to parse information from test-output?
Other possible solutions?
Besides, I looked up in Twisted documentation and found this class IReporter.
Is it a solution and if it is, how can I use it?
If it isn't, are there any other solutions?
P.S. Please, note, that tests have already been written, so I can only start them and can't modify source code.
You can format output from trial arbitrarily by writing a reporter plugin. You found the interface for that plugin already - IReporter.
Once you write such a plugin, you'll be able to use it by adding --reporter=yourplugin to your trial command line arguments.
You can see the list of reporter plugins that are already available on your system using trial --help-reporters. If you have python-subunit installed then you'll see subunit which is a machine-parseable format that might already satisfy your requirements. Unfortunately it's still a subunit v1 reporter and subunit v2 is better in a number of ways. Still, it might suffice.
I am building an application that will send, read, and respond to emails. The problem is that if an email is part of a thread, it is likely to contain history information. Unfortunately, there is no consistency in how history information is displayed (sometimes it's marked with arrows >>>> sometimes with a rule, etc.) so it's not easily found by regular expression.
I am currently using several regexs that solve many cases, and adding new ones as they arise. Is this the best way to handle the problem?
Does anyone have a better solution?
In all honesty, there is no good way to handle this problem. Most email systems have a default way of doing this. Setting using regular expressions for the most common ones will get you pretty far because most email systems run on default settings for this. But you have to understand that this can be customized very easily from one email server to another meaning that there is no one-size-fits-all solution.
But...if good enough is good enough, just keep doing what you're doing or don't use forwards/replies.
I frequently write programs consisting of nothing but assertions to test my code. While debugging, OSX tries to be helpful by asking whether I would like to "Click Report to see more detailed information and send a report to Apple" whenever an assertion fails.
Is there a way to disable this feature for my programs? I am using make and g++.
Give this a shot, however it'll disable them system wide.
defaults write com.apple.CrashReporter DialogType none
I searched for unit testing tool and I found the suitable one is NUnit and I think it good but my problem that this tool only show test method result only (pass or fail) and I need to show not only pass or fail also the output .How can I show the output using NUnit or if there another unit testing tool its also good ?If its not supported please suggest me how can I solve it.
All ideas are welcomed
Piping the output of System.Console will work for NUnit, but it's not your best option.
For passing tests, you shouldn't need to review the console output to verify that the tests have passed. If you are, you're doing it wrong. Tests should be automated and repeatable without human intervention. Verifying by hand doesn't scale and creates false positives.
On the other hand, having console output for failing tests is helpful, but it will only provide information that could otherwise be inferred from attaching a debugger. That's a lot of extra effort to add console logging to your application for little benefit.
Instead, make sure that your error messages are meaningful. When writing your tests make sure your assertions are explicit. Always try to use the assertion that closely fits the object you are asserting and provide a failure message that explains why the test is important.
For example:
// very bad
Assert.IsTrue( collection.Count == 23 );
The above assertion doesn't really provide much help when the test fails. As NUnit formats the output of the assertions, this assertion won't help you as it will state something like "expecting <True> but was <False>".
A more appropriate assert will provide more meaningful test failures.
// much better
Assert.AreEqual(23, collection.Count,
"There should be a minimum of 23 items by default.");
This provides a much more meaningful failure message: "Expecting <23> but was <0>: There should be a minimum of 23 items by default."
On the bottom bar of NUnit you can click Text Output and that shows all debug and console output.
It depends where are you want to output the data from a test.
I believe you mentioned something another from File, Log, Console, Debug output.
As an alternative NUnit allow to output any message in the regular tests output stream, just use following utility methods:
For successful test
Assert.Pass( string message, object[] parms );
For failed test
Assert.Fail( string message, object[] parms );
More details see here
This post is loooooong after the question was asked, but wanted to chime in. Yes, you can accomplish a lot in unit/integration tests and probably do most of what you need. So, I agree, do as much as you can in your test methods.
But sometimes, providing some output is useful. Especially if you need to further verify the results and that verification cannot be accomplished via your unit test. Think of an external system that your dev/test environment has no or limited access to.
As an example, let's say you are hitting a webapi to CREATE a claim and the response is the new claim number. But the api does not expose methods to GET a claim, and you need to verify some other data that was created when you made the webapi call. In this case, you could use the outputted claim numbers to manually check the remote system.
FWIW
I'm looking for a very easy and quick way to generate some SAML assertions. This is only going to be used for testing (using SOAP UI). So I just need something that can generate a valid assertion, signed or unsigned, that I can then drop into SOAPUI and send off to my Web Service. I know how to add the assertion to the SOAP message and all that other good stuff, I just need some valid test assertions.
Any ideas?
Thanks.
I don't think you will find one. SAML has so many profiles/bindings. It's almost impossible to generate a single assertion that meets requirement of every relying party.
Your best bet is to capture a real assertion and use it as a template. Just replace the field on the fly in SOAPUI.
This is the approach used by Google SSO client library (now deprecated). You can find the examples here,
http://code.google.com/p/google-apps-sso-sample/downloads/list