Not all tests of Value-Parameterized tests are run in GoogleTest - c++

I'm writing tests with Googletest in Eclipse Oxygen 4.7.0 and I'm using Value-Parameterization. There is some weird behavior going on that I cannot explain so I hoped someone here might know what is going on.
The fixture class for value-parameterization should create 256 tests for each TEST_P. However, upon running the test for the first time after booting my computer, at least one TEST_P creates less tests than it should. Sometimes it is only 40, sometimes it is 180 or any number just not 256. The tests that do run pass and I don't get an error message. When I hit run again, all tests run as they should.
Here is what the fixture, one TEST_P and the instantiation look like:
typedef struct mystruct
{
mystruct()
{
myparam = new FooStruct;
bool1 = true;
bool2 = true;
bool3 = false;
bool4 = true;
bool5 = true;
bool6 = true;
uint8_t1 = 0;
uint8_t2 = 2;
uint8_t3 = 2;
int32_t1 = 0;
int32_t2 = 0;
int32_t3 = 0;
int32_t4 = 0;
}
~mystruct()
{
delete myparam;
myparam = NULL;
}
void setInt32_t(int32_t newint1, int32_t newint2, int32_t newint3, int32_t newint4)
{
int32_t1 = newint1;
int32_t2 = newint2;
int32_t3 = newint3;
int32_t4 = newint4;
}
FooStruct *myparam;
bool bool1, bool2, bool3, bool4, bool5, bool6;
uint8_t uint8_t1, uint8_t2, uint8_t3;
int32_t int32_t1, int32_t2, int32_t3, int32_t4;
}MYSTRUCT_T;
class MyTest : public testing::TestWithParam
< ::std::tr1::tuple<uint8_t, uint8_t, uint8_t, bool, bool, bool, bool> >
{
public:
MYSTRUCT_T *testparams;
protected:
virtual void SetUp()
{
_Initializer();
testparams = new mystruct();
testparams->uint8_t1 = ::std::tr1::get<0>(GetParam());
testparams->uint8_t2 = ::std::tr1::get<1>(GetParam());
testparams->uint8_t3 = ::std::tr1::get<2>(GetParam());
testparams->bool1 = ::std::tr1::get<3>(GetParam());
testparams->bool2 = ::std::tr1::get<4>(GetParam());
testparams->bool3 = ::std::tr1::get<5>(GetParam());
testparams->bool4 = ::std::tr1::get<6>(GetParam());
//bool5 and bool6 are unused here
}
virtual void TearDown()
{
delete testparams;
}
};
TEST_P(MyTest, NameOfTheTest)
{
testparams->bool1
? (testparams->uint8_t3 == 2)
? testparams->setInt32_t(0, 0, 0, 0)
: testparams->limits(1, 1, 1, 1)
: (testparams->uint8_t3 == 2)
? testparams->setInt32_t(2, 2, 2, 2)
: testparams->setInt32_t(3, 3, 3, 3);
int bar = 0;
_SetParam(testparams);
if(!_CheckAndSetParam(testparams, bar))
{
FAIL();
return;
}
_DoSomething(bar);
EXPECT_EQ(0, Global_Var[bar].status());
int32_t baz = 1;
_DoMore(bar, baz, testparams->bool2);
EXPECT_EQ(1, Global_Var[bar].status());
}
using testing::Bool;
using testing::Values;
using testing::Range;
INSTANTIATE_TEST_CASE_P(InstantiationName, MyTest,
::testing::Combine(Range((uint8_t)0, (uint8_t) 4), Range((uint8_t)0, (uint8_t)2), Range((uint8_t)2, (uint8_t)4), Bool(), Bool(), Bool(), Bool()));
I am utterly clueless as to why this keeps happening. A colleague just cloned the repository and on his machine the tests ran without a problem first try.
If any further detail or more information about my computer is needed, I am happy to provide it.
EDIT: I now noticed that there is an error that is being displayed when this behaviour occurs. Above the Console there is a small status bar (I don't know what it is called or how to best describe it) that continuously shows which test is currently running. When the tests get interrupted it says there: "I/O error: Pipe closed". Unfortunately, I couldn't find anything so far regarding that output. So the question is still unanswered
Another Edit : Coming back because the issue has arisen again when I thought it was already solved. I haven't read too much about pipe usage in the GoogleTest Documentation but will do now as I remember seeing it being mentioned at some point. Maybe that will lead me to a clue. The reason, however, I am editing this question again is to describe what I see when the error occurs.
Only one thing is constant among the crashes: the error type which is
I/O Error: : Pipe closed
Here is how I can invoke it: I edit something in the .cpp where the tests sit and start a run/debug. If there were changes and a build is made, there will likely be a crash.
The crash will after any amount of tests run which seems random. Usually, it happens after around 1800-2600 tests (out of ~2800). With an increasing amount of tests, there seems to be an increased chance in the run crashing. So I assume there will be a point where I can't run the tests anymore because this error will definitely happen.
In the C/C++ Unit view I see all the tests that ran. All of them have the green check even the last one that seems to make the run crash- at least most of the time. Sometimes it shows the blue "play"-button as if the test were still running, yet in the debug view it states "terminated".
That is about everything I know so far and have observed so any help is greatly appreciated.

Related

Record with List of Records Unit Test Failing

I have the following records:
namespace FunctionalInnovate.Domain
module Voting=
open System
type Vote = {
VoteId : int
SuggestionId : int
VotePreference : int
}
type Suggestion = {
SuggetionId : int
SuggestionText : string
}
type User = {
UserId : int
Votes : Vote list
}
From these records, I have created the following Unit Test:
module ``Voting Tests``
open NUnit.Framework
open FsUnit
open FunctionalInnovate.Domain.Voting
[<Test>]
let``Get Vote from given SuggestionId``()=
let votePreference = 1
let vote1 = { VoteId = 1; SuggestionId = 34; VotePreference = votePreference }
let vote2 = { VoteId = 2; SuggestionId = 654; VotePreference = votePreference}
let votes = [ vote1; vote2; ]
let user = {UserId = 321; Votes = votes}
Assert.IsTrue(true) |> ignore
I've just changed the Assert statement to something simple at this point as it keeps failing at present. Using ReSharper, when I run the test I get an error. Upon investigation, it appears to be a problem with the List of Votes being assigned to the Votes type within my user declaration. What I find strange though is that if I load all of the types into FSI and then load each line of the test in turn, it works without any errors.
Anyone got any ideas as to why the unit test is failing?
The error ReSharper Test Runner is producing is:
System.MissingMethodException : Method not found: 'Void User..ctor(Int32, Microsoft.FSharp.Collections.FSharpList`1<Vote>)'.
at Voting Tests.Get Vote from given SuggestionId()
As per #Tomas Petricek and #John Palmer's advice, it was an issue with my ''FSharp.Core.dll'' version.
The version of this DLL in my UnitTest project was much older than what was in my Domain project.
All I did to fix it was going into NuGet Package Manager and update that specific package. Upon running my tests again, everything was green and healthy.

LLVM API: correct way to create/dispose

I'm attempting to implement a simple JIT compiler using the LLVM C API. So far, I have no problems generating IR code and executing it, that is: until I start disposing objects and recreating them.
What I basically would like to do is to clean up the JIT'ted resources the moment they're no longer used by the engine. What I'm basically attempting to do is something like this:
while (true)
{
// Initialize module & builder
InitializeCore(GetGlobalPassRegistry());
module = ModuleCreateWithName(some_unique_name);
builder = CreateBuilder();
// Initialize target & execution engine
InitializeNativeTarget();
engine = CreateExecutionEngineForModule(...);
passmgr = CreateFunctionPassManagerForModule(module);
AddTargetData(GetExecutionEngineTargetData(engine), passmgr);
InitializeFunctionPassManager(passmgr);
// [... my fancy JIT code ...] --** Will give a serious error the second iteration
// Destroy
DisposePassManager(passmgr);
DisposeExecutionEngine(engine);
DisposeBuilder(builder);
// DisposeModule(module); //--> Commented out: Deleted by execution engine
Shutdown();
}
However, this doesn't seem to be working correctly: the second iteration of the loop I get a pretty bad error...
So to summarize: what's the correct way to destroy and re-create the LLVM API?
Posting this as Answer because the code's too long. If possible and no other constraints, try to use LLVM like this. I am pretty sure the Shutdown() inside the loop is the culprit here. And I dont think it would hurt to keep the Builder outside, too. This reflects well the way I use LLVM in my JIT.
InitializeCore(GetGlobalPassRegistry());
InitializeNativeTarget();
builder = CreateBuilder();
while (true)
{
// Initialize module & builder
module = ModuleCreateWithName(some_unique_name);
// Initialize target & execution engine
engine = CreateExecutionEngineForModule(...);
passmgr = CreateFunctionPassManagerForModule(module);
AddTargetData(GetExecutionEngineTargetData(engine), passmgr);
InitializeFunctionPassManager(passmgr);
// [... my fancy JIT code ...] --** Will give a serious error the second iteration
// Destroy
DisposePassManager(passmgr);
DisposeExecutionEngine(engine);
}
DisposeBuilder(builder);
Shutdown();
/* program init */
LLVMInitializeNativeTarget();
LLVMInitializeNativeAsmPrinter();
LLVMInitializeNativeAsmParser();
LLVMLinkInMCJIT();
ctx->context = LLVMContextCreate();
ctx->builder = LLVMCreateBuilderInContext(ctx->context);
LLVMParseBitcodeInContext2(ctx->context, module_template_buf, &module) // create module
do IR code creation
{
function = LLVMAddFunction(ctx->module, "my_func")
LLVMAppendBasicBlockInContext(ctx->context, ...
LLVMBuild...
...
}
optional optimization
{
LLVMPassManagerBuilderRef pass_builder = LLVMPassManagerBuilderCreate();
LLVMPassManagerBuilderSetOptLevel(pass_builder, 3);
LLVMPassManagerBuilderSetSizeLevel(pass_builder, 0);
LLVMPassManagerBuilderUseInlinerWithThreshold(pass_builder, 1000);
LLVMPassManagerRef function_passes = LLVMCreateFunctionPassManagerForModule(ctx->module);
LLVMPassManagerRef module_passes = LLVMCreatePassManager();
LLVMPassManagerBuilderPopulateFunctionPassManager(pass_builder, function_passes);
LLVMPassManagerBuilderPopulateModulePassManager(pass_builder, module_passes);
LLVMPassManagerBuilderDispose(pass_builder);
LLVMInitializeFunctionPassManager(function_passes);
for (LLVMValueRef value = LLVMGetFirstFunction(ctx->module); value;
value = LLVMGetNextFunction(value))
{
LLVMRunFunctionPassManager(function_passes, value);
}
LLVMFinalizeFunctionPassManager(function_passes);
LLVMRunPassManager(module_passes, ctx->module);
LLVMDisposePassManager(function_passes);
LLVMDisposePassManager(module_passes);
}
optional for debug
{
LLVMVerifyModule(ctx->module, LLVMAbortProcessAction, &error);
LLVMPrintModule
}
if (LLVMCreateJITCompilerForModule(&ctx->engine, ctx->module, 0, &error) != 0)
my_func = (exec_func_t)(uintptr_t)LLVMGetFunctionAddress(ctx->engine, "my_func");
LLVMRemoveModule(ctx->engine, ctx->module, &ctx->module, &error);
LLVMDisposeModule(ctx->module);
LLVMDisposeBuilder(ctx->builder);
do
{
my_func(...);
}
LLVMDisposeExecutionEngine(ctx->engine);
LLVMContextDispose(ctx->context);
/* program finit */
LLVMShutdown();

TaskDialogIndirect is returning an unusual error code

I'm using TaskDialogIndirect to display prompts to the user. Normally this works just fine, but sometimes, after the program has been running for a while, it begins returning an error code that the MSDN entry does not list as being one of the error codes this function can return.
0x80040001 OLE_E_ADVF "Invalid advise flags"
I have checked all the inputs to the function against previous successful calls in the same run. Aside from differences in the string to be displayed, they are identical. (the strings are even the same length.)
// create task dialog struct
TASKDIALOGCONFIG tdc;
ZeroMemory(&tdc, sizeof(TASKDIALOGCONFIG));
tdc.cbSize = sizeof(tdc);
tdc.dwFlags = (((dwMessageBoxFlags & MB_OKCANCEL) == MB_OKCANCEL) ? TDF_ALLOW_DIALOG_CANCELLATION : 0) | TDF_POSITION_RELATIVE_TO_WINDOW;
tdc.hwndParent = hwndOwner;
tdc.hInstance = LGetHInstance();
tdc.pszContent = usrText.wsz;
tdc.pButtons = _pButtons;
tdc.cButtons = nButtons;
tdc.pszMainIcon = pszTaskDialogIcon;
tdc.pszWindowTitle = usrCaption.wsz;
tdc.nDefaultButton = nDefaultButton;
// display it now
int iButton = 0;
BOOL b = 0;
HRESULT hResult = TaskDialogIndirect(&tdc, &iButton, NULL, &b);
NEW INFORMATION
At the same time that TaskDialogIndirect stops behaving correctly, ShellExecute also stops working, as does CreateFile.
This was actually caused by an event handle leak elsewhere. When the available handles ran out, no more API calls which needed to create a handle could succeed. They did return a rather odd set of error codes though, none of which was "out of handles".

Odd ColdFusion Behavior--Abort Not Honored

Using ColdFusion 9.01, occasionally, we have observed an issue where an error may be occurring within a CFC function and when we attempt to add writeDump(foo); and abort; calls to debug the error ColdFusion does not honor those calls.
Example:
private void function index(Event)
{
var rc = Event.getCollection();
var prc = Event.getCollection(private=true);
/** NOT HONORED! **/
writeDump(var=rc);
abort;
prc.JSON = {};
prc.JSON.show = variables.APIProxy.call(
handler = 'shows'
,action = 'read'
,event = arguments.Event
/** THE ERROR IS OCCURING HERE **/
,params = { language=lcase(rc.language.getLanguage_Medium()), show=rc.show_name }
);
prc.JSON.showEpisodes = variables.APIProxy.call(
handler = 'episodes'
,action = 'index'
,event = arguments.Event
,params = { language=lcase(rc.language.getLanguage_Medium()), show=rc.show_name, detail=true }
);
prc.JSON.products = variables.APIProxy.call(
handler = 'products'
,action = 'index'
,event = arguments.Event
,params = { language=lcase(rc.language.getLanguage_Medium()), detail=true }
);
Event.addAssets(
'model/product.js
,model/show.js
,collection/product_mobile.js
,collection/show_mobile.js
,view/product_mobile.js
,view/productList.js
,view/show_mobile.js
,view/showList.js
,model/episode.js
,view/episode_mobile.js
,view/episodeList.js
,collection/episode_mobile.js
,collection/product_mobile.js
,mobile/episodeObject.css
,mobile/show.js
,mobile/show.css
,mobile/category.css
');
Event.setLayout('layout.mobile');
Event.setView("show/index_mobile");
return;
}
I believe we have successfully eliminated caching. I am curious if anyone else has encountered this.
Thank you.
Aaron
I'm guessing that the error is a parse error, not a true runtime error, so it gets thrown before the function actually executes. It's not actually skipping over your abort, it just fails to parse (or execute) the entire thing.
I'm not sure why you're getting a parse error there, but I do know the CF code that handles struct literals is somewhat flaky.
The issue was with the struct literals declared within the argument calls to a function.
i'm going to go out on a limb here and say that your issue might have something to do with this bug:
http://cfbugs.adobe.com/cfbugreport/flexbugui/cfbugtracker/main.html#bugId=86960
is there anything in your app that executes in the onRequestEnd() method?
it would be helpful to tell us what exactly is happening and/or the output you're getting when the issue happens.

What are the best practices for unit testing properties with code in the setter?

I'm fairly new to unit testing and we are actually attempting to use it on a project. There is a property like this.
public TimeSpan CountDown
{
get
{
return _countDown;
}
set
{
long fraction = value.Ticks % 10000000;
value -= TimeSpan.FromTicks(fraction);
if(fraction > 5000000)
value += TimeSpan.FromSeconds(1);
if(_countDown != value)
{
_countDown = value;
NotifyChanged("CountDown");
}
}
}
My test looks like this.
[TestMethod]
public void CountDownTest_GetSet_PropChangedShouldFire()
{
ManualRafflePresenter target = new ManualRafflePresenter();
bool fired = false;
string name = null;
target.PropertyChanged += new PropertyChangedEventHandler((o, a) =>
{
fired = true;
name = a.PropertyName;
});
TimeSpan expected = new TimeSpan(0, 1, 25);
TimeSpan actual;
target.CountDown = expected;
actual = target.CountDown;
Assert.AreEqual(expected, actual);
Assert.IsTrue(fired);
Assert.AreEqual("CountDown", name);
}
The question is how do I test the code in the setter? Do I break it out into a method? If I do it would probably be private since no one else needs to use this. But they say not to test private methods. Do make a class if this is the only case? would two uses of this code make a class worthwhile? What is wrong with this code from a design standpoint. What is correct?
The way you've got is fine (call the setter and then check the get returns the expected value).
Make sure you choose a selection of test values that exercise all the paths in that setter. A single set/get test isn't sufficient coverage.