Related
Supposed I have a function void Myclass::func(x), and various other code make thousands of calls to it. Now I want to know some stats of the argument x, for example, the average, max, min, or even a distribution graph.
void Myclass::func(int x) {
while(foo.doFancyStuff(x)) {
// ...
}
}
Here's some ad-hoc methods that come to my mind:
Print each value of x to log. Then use external tools/scripts to analyze them.
Caveats: Mixed with other info in logs. Writing each value of x into external log or file on filesystem is slow.
Define global variable to store them and analyze at the end of execution of interest.
Caveats: Global variables are bad. They will get confusing in the future.
Store them in class Myclass.
Caveats: Not reusable code. What about next time I want to analyze Otherclass::doOtherStuff(y) ? And bad integration, because that stats code should not be coupled with the Myclass itself.
Is there any tool/library to do this? I'm using Visual Studio on Windows, so would like an answer usable for this platform. Cross-platform tools are welcome, too.
Here is an example using the scripting API of lldb (which works on Windows, too). Take this trivial program,
void func(int x) {}
int main(int, char **)
{
for (int i = 0; i < 1000; ++i)
func(i);
}
which you can analyze with such a script
import lldb
import os
fArgs = []
def analyzeFrame(frame, bpLocation, dict):
variables = frame.GetVariables(True, False, False, False)
x = variables.GetValueAtIndex(0).GetValueAsSigned()
fArgs.append(x)
return False
debugger = lldb.SBDebugger.Create()
debugger.SetAsync(False)
target = debugger.CreateTargetWithFileAndArch("pathToYourExecutable", "")
bp = target.BreakpointCreateByName("func", 4, lldb.SBFileSpecList(), lldb.SBFileSpecList())
bp.SetScriptCallbackFunction("analyzeFrame")
process = target.Launch(target.GetDebugger().GetListener(), [], [],
None, None, None, os.getcwd(), 0, False, lldb.SBError())
print("max: {}".format(max(fArgs)))
print("min: {}".format(min(fArgs)))
You need to make sure the python interpreter finds the lldb module. The path can be seen by executing lldb -P on the command line.
Unfortunately, there is no single simple answer. What you want is a variation of instrumentation debugging, which means that someone needs to inject additional code in your class to handle this case.
For a portable way, the only option you have is to add a cache of the previous values, then in your class destructor, output the statistics that you want. The way you cache data is up to you, you can design a simple Stats<> class, member of the class that you want to monitor and make calls to it to store new values. That would be what I would try first, as it's portable, almost clean and reusable.
When I raise my owns exceptions in my Python libraries, the exception stack shows the raise-line itself as the last item of the stack. This is obviously not an error, is conceptually right, but points the focus on something that is not useful for debugging when you're are using code externally, for example as a module.
Is there a way to avoid this and force Python to show the previous-to-last stack item as the last one, like the standard Python libraries.
Due warning: modifying the behaviour of the interpreter is generally frowned upon. And in any case, seeing exactly where an error was raised may be helpful in debugging, especially if a function can raise an error for several different reasons.
If you use the traceback module, and replace sys.excepthook with a custom function, it's probably possible to do this. But making the change will affect error display for the entire program, not just your module, so is probably not recommended.
You could also look at putting code in try/except blocks, then modifying the error and re-raising it. But your time is probably better spent making unexpected errors unlikely, and writing informative error messages for those that could arise.
you can create your own exception hook in python. below is the example of code that i am using.
import sys
import traceback
def exceptionHandler(got_exception_type, got_exception, got_traceback):
listing = traceback.format_exception(got_exception_type, got_exception, got_traceback)
# Removing the listing of statement raise (raise line).
del listing[-2]
filelist = ["org.python.pydev"] # avoiding the debuger modules.
listing = [ item for item in listing if len([f for f in filelist if f in item]) == 0 ]
files = [line for line in listing if line.startswith(" File")]
if len(files) == 1:
# only one file, remove the header.
del listing[0]
print>>sys.stderr, "".join(listing)
And below are some lines that I have used in my custom exception code.
sys.excepthook = exceptionHandler
raise Exception("My Custom error message.")
In the method exception you can add file names or module names in list "filenames" if you want to ignore any unwanted files. As I have ignored the python pydev module since I am using pydev debugger in eclipse.
The above is used in my own module for a specific purpose. you can modify and use it for your modules.
I'd suggest to not use the Exception mechanism to validate arguments, as tempting as that is. Coding with exceptions as conditionals is like saying, "crash my app if, as a developer, I don't think of all the bad conditions my provided arguments can cause. Perhaps using exceptions for things not only out of your control but also which is under control of something else like the OS or hardware or the Python language would be more logical, I don't know. In practice however I use exceptions as you request a solution for.
To answer your question, in part, it is just as simple to code thusly:
class MyObject(object):
def saveas(self, filename):
if not validate_filename(filename):
return False
...
caller
if not myobject.saveas(filename): report_and_retry()
Perhaps not a great answer, just something to think about.
I am used to test drive my code. Now that I am new to Go I am trying to get it right as fast as possible. I am using the testing package in the standard library which seem to be good enough. (I also like that it is not yet another external dependency. We are currently at 2 dependencies overall - compared to any Java- or Ruby project.....) Anyways - it looks like an assert in golang looks like this:
func TestSomething(t *testing.T) {
something := false
if something {
t.Log("Oh noes - something is false")
t.Fail()
}
}
I find this verbose and would like to do it on one line instead:
Assert( something, "Oh noes - something is false" )
or something like that. I hope that I have missed something obvious here. What is the best/idiomatic way to do it in go?
UPDATE: just to clarify. If I was to do something like this:
func AssertTrue(t *testing.T, value bool, message string) {
if value {
t.Log(message)
t.Fail()
}
}
and then write my test like this
func TestSomething(t *testing.T) {
something := false
AssertTrue(t, something, "Oh noes - something is false")
}
then it would not be the go way to do it?
There are external packages that can be integrated with the stock testing framework.
One of them I wrote long ago, gocheck, was intended to sort that kind of use case.
With it, the test case looks like this, for example:
func (s *Suite) TestFoo(c *gocheck.C) {
// If this succeeds the world is doomed.
c.Assert("line 1\nline 2", gocheck.Equals, "line 3")
}
You'd run that as usual, with go test, and the failure in that check would be reported as:
----------------------------------------------------------------------
FAIL: foo_test.go:34: Suite.TestFoo
all_test.go:34:
// If this succeeds the world is doomed.
c.Assert("line 1\nline 2", gocheck.Equals, "line 3")
... obtained string = "" +
... "line 1\n" +
... "line 2"
... expected string = "line 3"
Note how the comment right above the code was included in the reported failure.
There are also a number of other usual features, such as suite and test-specific set up and tear down routines, and so on. Please check out the web page for more details.
It's well maintained as I and other people use it in a number of active projects, so feel free to join on board, or follow up and check out the other similar projects that suit your taste more appropriately.
For examples of gocheck use, please have a look at packages such as mgo, goyaml, goamz, pipe, vclock, juju (massive code base), lpad, gozk, goetveld, tomb, etc. Also gocheck, manages to test itself. It was quite fun to bootstrap that.
But when You try write test like Uncle Martin, with one assert in test and long function names, then simple assert library, like http://github.com/stretchr/testify/assert can make it much faster and easier
I discourage writing test in the way you seem to have desire for. It's not by chance that the whole stdlib uses the, as you call it, "verbose" way.
It is undeniably more lines, but there are several advantages to this approach.
If you read Why does Go not have assertions? and s/error handling/test failure reporting/g you can get a picture of why the several "assert" packages for Go testing are not a good idea to use,
Once again, the proof is the huge code base of the stdlib.
The idiomatic way is the way you have above. Also, you don't have to log any message if you don't desire.
As defined by the GO FAQ:
Why does Go not have assertions?
Go doesn't provide assertions. They are undeniably convenient, but our
experience has been that programmers use them as a crutch to avoid
thinking about proper error handling and reporting. Proper error
handling means that servers continue operation after non-fatal errors
instead of crashing. Proper error reporting means that errors are
direct and to the point, saving the programmer from interpreting a
large crash trace. Precise errors are particularly important when the
programmer seeing the errors is not familiar with the code.
We understand that this is a point of contention. There are many
things in the Go language and libraries that differ from modern
practices, simply because we feel it's sometimes worth trying a
different approach.
UPDATE
Based on your update, that is not idiomatic Go. What you are doing is in essence designing a test extension framework to mirror what you get in the XUnit frameworks. While there is nothing fundamentally wrong, from an engineering perspective, it does raise questions as to the value + cost of maintaining this extension library. Additionally, you are creating an in-house standard that will potentially ruffle feathers. The biggest thing about Go is it is not C or Java or C++ or Python and things should be done the way the language is constructed.
Our unit tests fire off child processes, and sometimes these child processes crash. When this happens, a Windows Error Reporting dialog pops up, and the process stays alive until this is manually dismissed. This of course prevents the unit tests from ever terminating.
How can this be avoided?
Here's an example dialog in Win7 with the usual settings:
If I disable the AeDebug registry key, the JIT debugging option goes away:
If I disable checking for solutions (the only thing I seem to have control over via the control panel), it looks like this, but still appears and still stops the program from dying until the user presses something. WerAddExcludedApplication is documented to also have this effect.
A summary from the answers by jdehaan and Eric Brown, as well as this question (see also this question):
N.B. These solutions may affect other error reporting as well, e.g. failure to load a DLL or open a file.
Option 1: Disable globally
Works globally on the entire user account or machine, which can be both a benefit and a drawback.
Set [HKLM|HKCU]\Software\Microsoft\Windows\Windows Error Reporting\DontShowUI to 1.
More info: WER settings.
Option 2: Disable for the application
Requires modification to the crashing program, described in documentation as best practice, unsuitable for a library function.
Call SetErrorMode: SetErrorMode(SetErrorMode(0) | SEM_NOGPFAULTERRORBOX); (or with SEM_FAILCRITICALERRORS). More info: Disabling the program crash dialog (explains the odd arrangement of calls).
Option 2a: Disable for a function:
Requires modification to the crashing program, requires Windows 7/2008 R2 (desktop apps only) or higher, described in documenation as preferred to SetErrorMode, suitable for a thread-safe library function.
Call and reset SetThreadErrorMode:
DWORD OldThreadErrorMode = 0;
SetThreadErrorMode(SEM_FAILCRITICALERRORS,& OldThreadErrorMode);
…
SetThreadErrorMode (z_OldThreadErrorMode, NULL);
More info: not much available?
Option 3: Specify a handler
Requires modification to the crashing program.
Use SetUnhandledExceptionFilter to set your own structured exception handler that simply exits, probably with reporting and possibly an attempt at clean-up.
Option 4: Catch as an exception
Requires modification to the crashing program. For .NET applications only.
Wrap all code into a global try/catch block. Specify the HandleProcessCorruptedStateExceptionsAttribute and possibly also the SecurityCriticalAttribute on the method catching the exceptions. More info: Handling corrupted state exceptions
Note: this might not catch crashes caused by the Managed Debugging Assistants; if so, these also need to be disabled in the application.
Option 5: Stop the reporting process
Works globally on the entire user account, but only for a controlled duration.
Kill the Windows Error Reporting process whenever it shows up:
var werKiller = new Thread(() =>
{
while (true)
{
foreach (var proc in Process.GetProcessesByName("WerFault"))
proc.Kill();
Thread.Sleep(3000);
}
});
werKiller.IsBackground = true;
werKiller.Start();
This is still not completely bullet-proof though, because a console application may crash via a different error message, apparently displayed by an internal function called NtRaiseHardError:
The only solution is to catch all exceptions at a very high level (for each thread) and terminate the application properly (or perform another action).
This is the only way to prevent the exception from escaping your app and activating WER.
Addition:
If the exception is something you do not except to happen you can use an AssertNoThrow(NUnit) or alike in another Unit Test framework to enclose the code firing the child processes. This way you would also get it into your Unit test report. This is in my opinion the cleanest possible solution I can think of.
Addition2:
As the comments below show, I was mistaken: you cannot always catch the asynchronous exceptions, it depends on what the environment allows. In .NET some exceptions are prevented from being caught, what makes my idea worthless in this case...
For .NET: There are complicated workarounds involving the use of AppDomains, leading to an unload of an AppDomain instead of a crash of the whole application. Too bad...
http://www.bluebytesoftware.com/blog/PermaLink,guid,223970c3-e1cc-4b09-9d61-99e8c5fae470.aspx
http://www.develop.com/media/pdfs/developments_archive/AppDomains.pdf
EDIT:
I finally got it. With .NET 4.0 You can add the HandleProcessCorruptedStateExceptions attribute from System.Runtime.ExceptionServices to the method containing the try/catch block. This really worked! Maybe not recommended but works.
using System;
using System.Reflection;
using System.Runtime.InteropServices;
using System.Runtime.ExceptionServices;
namespace ExceptionCatching
{
public class Test
{
public void StackOverflow()
{
StackOverflow();
}
public void CustomException()
{
throw new Exception();
}
public unsafe void AccessViolation()
{
byte b = *(byte*)(8762765876);
}
}
class Program
{
[HandleProcessCorruptedStateExceptions]
static void Main(string[] args)
{
Test test = new Test();
try {
//test.StackOverflow();
test.AccessViolation();
//test.CustomException();
}
catch
{
Console.WriteLine("Caught.");
}
Console.WriteLine("End of program");
}
}
}
Try setting
HKCU\Software\Microsoft\Windows\Windows Error Reporting\DontShowUI
to 1. (You can also set the same key in HKLM, but you need admin privs to do that.)
This should prevent WER from showing any UI.
Edit:
For personn interested in a cleaner way to implemenent that, have a look to that answer.
In my job I often need to use third-made API to access remote system.
For instance to create a request and send it to the remote system:
#include "external_lib.h"
void SendRequest(UserRequest user_request)
{
try
{
external_lib::Request my_request;
my_request.SetPrice(user_request.price);
my_request.SetVolume(user_request.quantity);
my_request.SetVisibleVolume(user_request.quantity);
my_request.SetReference(user_request.instrument);
my_request.SetUserID(user_request.user_name);
my_request.SetUserPassword(user_request.user_name);
// Meny other member affectations ...
}
catch(external_lib::out_of_range_error& e)
{
// Price , volume ????
}
catch(external_lib::error_t& e)
{
// Here I need to tell the user what was going wrong
}
}
Each lib's setter do checks the values that the end user has provided, and may thow an exception when the user does not comply with remote system needs. For instance a specific user may be disallowed to send a too big volume. That's an example, and actually many times users tries does not comply: no long valid instrument, the prices is out of the limit, etc, etc.
Conseqently, our end user need an explicit error message to tell him what to modify in its request to get a second chance to compose a valid request. I have to provide hiim such hints
Whatever , external lib's exceptions (mostly) never specifies which field is the source
of aborting the request.
What is the best way, according to you, to handle those exceptions?
My first try at handling those exceptions was to "wrap" the Request class with mine. Each setters are then wrapped in a method which does only one thing : a try/catch block. The catch block then throws a new exceptions of mine : my_out_of_range_volume, or my_out_of_range_price depending on the setter. For instance SetVolume() will be wrapped this way:
My_Request::SetVolume(const int volume)
{
try
{
m_Request.SetVolume(volume);
}
catch(external_lib::out_range_error& e)
{
throw my_out_of_range_volume(volume, e);
}
}
What do you think of it? What do you think about the exception handling overhead it implies? ... :/
Well the question is open, I need new idea to get rid of that lib constraints!
If there really are a lot of methods you need to call, you could cut down on the code using a reflection library, by creating just one method to do the calling and exception handling, and passing in the name of the method/property to call/set as an argument. You'd still have the same amount of try/catch calls, but the code would be simpler and you'd already know the name of the method that failed.
Alternatively, depending on the type of exception object that they throw back, it may contain stack information or you could use another library to walk the stack trace to get the name of the last method that it failed on. This depends on the platform you're using.
I always prefer a wrapper whenever I'm using third party library.
It allows me to define my own exception handling mechanism avoiding users of my class to know about external library.
Also, if later the third party changes the exception handling to return codes then my users need not be affected.
But rather than throwing the exception back to my users I would implement the error codes. Something like this:
class MyRequest
{
enum RequestErrorCode
{
PRICE_OUT_OF_LIMIT,
VOLUME_OUT_OF_LIMIT,
...
...
...
};
bool SetPrice(const int price , RequestErrorCode& ErrorCode_out);
...
private:
external_lib::Request mRequest;
};
bool MyRequest::SetPrice(const int price , RequestErrorCode& ErrorCode_out)
{
bool bReturn = true;
try
{
bReturn = mRequest.SetPrice(price);
}
catch(external_lib::out_of_range_error& e)
{
ErrorCode_out = PRICE_OUT_OF_LIMIT;
bReturn = false;
}
return bReturn;
}
bool SendRequest(UserRequest user_request)
{
MyRequest my_request;
MyRequest::RequestErrorCode anErrorCode;
bool bReturn = my_request.SetPrice(user_request.price, anErrorCode);
if( false == bReturn)
{
//Get the error code and process
//ex:PRICE_OUT_OF_LIMIT
}
}
I think in this case I might dare a macro. Something like (not tested, backslashes omitted):
#define SET( ins, setfun, value, msg )
try {
ins.setfun( value );
}
catch( external::error & ) {
throw my_explanation( msg, value );
}
and in use:
Instrument i;
SET( i, SetExpiry, "01-01-2010", "Invalid expiry date" );
SET( i, SetPeriod, 6, "Period out of range" );
You get the idea.
Although this is not really the answer you are looking for, but i think that your external lib, or you usage of it, somehow abuses exceptions. An exception should not be used to alter the general process flow. If it is the general case, that the input does not match the specification, than it is up to your app to valid the parameter before passing it to the external lib. Exceptions should only be thrown if an "exceptional" case occurrs, and i think whenever it comes to doing something with user input, you usually have to deal with everything and not rely on 'the user has to provide the correct data, otherwise we handle it with exceptions'.
nevertheless, an alternative to Neil's suggestions could be using boost::lambda, if you want to avoid macros.
In your first version, you could report the number of operations that succeeded provided the SetXXX functions return some value. You could also keep a counter (which increases after every SetXXX call in that try block) to note what all calls succeeded and based on that counter value, return an appropriate error message.
The major problem with validating each and every step is, in a real-time system -- you are probably introducing too much latency.
Otherwise, your second option looks like the only way. Now, if you have to write a wrapper for every library function and why not add the validation logic, if you can, instead of making the actual call to the said library? This IMO, is more efficient.