During testing and debugging of an app, i noticed there was an Exception that mostly happens during debug testing only, inside a for-loop that iterates over a list:
[ERROR:flutter/lib/ui/ui_dart_state.cc(177)] Unhandled Exception: Concurrent modification during iteration: Instance(length:0) of '_GrowableList'.
I have searched around and found that it mostly happens if you change the list itself during the iteration, but i cannot see where it happens in the code:
Main function:
static Future<void> save(EntryModel entry) async {
...
List<TagModel> tagsList = entry.tags;
List<int> tagIdsInserted = [];
if (tagsList != null && tagsList.isNotEmpty) {
for (TagModel tag in tagsList) {
//Error happens inside this loop
int tagIdInserted = await TagContract.save(tag); //this function does not alter the tag in any way.
if (tagIdInserted == null || tagIdInserted <= 0) {
throw Exception('Invalid TagID!');
}
tagIdsInserted.add(tagIdInserted);
}
}
What happen is during the first iteration it runs fine, but the second or third one the List<TagModel> tagsList suddenly becomes empty, including from the initial object (the entry passed to the function).
Also i noticed that during runs without debugging it runs mostly fine, but i am not sure if that is because i am not catching the error.
Thanks in advance.
Try to avoid using await inside a loop, it is just too dangerous.
You have to understand how asynchronous code execute. If an await is encountered and the Future is unable to return synchronously, the runtime will suspend the execution of this function and jump to whatever other jobs that are on the top of the queue.
So when the await is encountered, the runtime will start executing some god-knows-where code and those code touched your tagsList.
Try to understand the following example. This will directly triggers the exception.
void main() {
List<int> ids = [1,2,3];
test(ids);
ids.add(1); // If the async function get suspended, this becomes the top of the queue.
}
void test(List<int> ids) async {
for (final id in ids) {
await Future.delayed(Duration(milliseconds: 10));
}
}
In async programming, avoid writing an await who depends on exposed shared states.
For a list of async tasks, always prepare them in an Iterable<Future>, then use Future.wait to synchronize them and get the result in a single await.
For your code
final results = await Future.wait(tagsList.map((tag)=>TagContract.save(tag)))
Related
I'm trying to write my own torrent program based on libtorrent rasterbar and I'm having problems getting the alert mechanism working correctly. Libtorrent offers function
void set_alert_notify (boost::function<void()> const& fun);
which is supposed to
The intention of of the function is that the client wakes up its main thread, to poll for more alerts using pop_alerts(). If the notify function fails to do so, it won't be called again, until pop_alerts is called for some other reason.
so far so good, I think I understand the intention behind this function. However, my actual implementation doesn't work so good. My code so far is like this:
std::unique_lock<std::mutex> ul(_alert_m);
session.set_alert_notify([&]() { _alert_cv.notify_one(); });
while (!_alert_loop_should_stop) {
if (!session.wait_for_alert(std::chrono::seconds(0))) {
_alert_cv.wait(ul);
}
std::vector<libtorrent::alert*> alerts;
session.pop_alerts(&alerts);
for (auto alert : alerts) {
LTi_ << alert->message();
}
}
however there is a race condition. If wait_for_alert returns NULL (since no alerts yet) but the function passed to set_alert_notify is called before _alert_cw.wait(ul);, the whole loop waits forever (because of second sentence from the quote).
For the moment my solution is just changing _alert_cv.wait(ul); to _alert_cv.wait_for(ul, std::chrono::milliseconds(250)); which reduces number of loops per second enough while keeping latency low enough.
But it's really more workaround then solution and I keep thinking there must be proper way to handle this.
You need a variable to record the notification. It should be protected by the same mutex that owns the condition variable.
bool _alert_pending;
session.set_alert_notify([&]() {
std::lock_guard<std::mutex> lg(_alert_m);
_alert_pending = true;
_alert_cv.notify_one();
});
std::unique_lock<std::mutex> ul(_alert_m);
while(!_alert_loop_should_stop) {
_alert_cv.wait(ul, [&]() {
return _alert_pending || _alert_loop_should_stop;
})
if(_alert_pending) {
_alert_pending = false;
ul.unlock();
session.pop_alerts(...);
...
ul.lock();
}
}
I'm running the process asynchronously using QEvent, after my process completes I'm just setting promise and trying to get it using future get function.
However it is not working asynchronously, it's working synchronously.
May I know what wrong I am doing?
code:
Win::Win(QObject *parent) :
MyCustomWidget(parent)
{
qDebug()<<execute().get();
}
std::future<int> Win::execute()
{
auto promise = std::make_shared<std::promise<int>>();
auto temp = [=]
{
///process
qDebug()<<"done";
promise->set_value(100);
};
QApplication::postEvent(this,new MyCustomEvent(temp));
////QApplication::processEvents() without this line its not working may know the reason
//// But without this line and without future get also its working
return promise->get_future();
}
I'm using NSubstitute to mock a class that my method under test uses. This mocked class may throw a particular exception under certain conditions.
The method that I'm testing has some "retry" logic that it executes when it catches this exception. I'm trying to test this retry logic. So, I need a particular method of this mocked class to throw the exception sometimes, but not other times. Unfortunately, the method that throws this exception has no parameters, so I can't base the throw logic on parameters.
How can I make the mocked object's method throw the exception either:
A) ...the first N times it's called
or
B) ...based on the parameters some other method that's called before it
or
C) ...under any other condition other than the parameters passed in
To give you a clearer picture of what I'm trying to do, my code is something like:
IDataSender myDataSender = GetDataSender();
int ID = GetNextAvailableID();
myDataSender.ClearData();
myDataSender.Add(ID,"DataToSend");
bool sendSuccess = false;
while (!sendSuccess)
{
try
{
myDataSender.SendData();
sendSuccess = true;
}
catch (IDCollisionException)
{
ID++;
MyDataSender.ClearData();
myDataSender.Add(ID,"DataToSend");
}
}
So, I need to test my retry logic, and I need to simulate that IDCollisionException. However, I can't have the SendData() throwing the exception every single time, or the retry loop will never succeed.
What can I do here?
If I understand the question correctly, you can use When..Do and close over a local variable to get this behaviour.
const int throwUntil = 3;
var callsToSendData = 0;
var dataSender = Substitute.For<IDataSender>();
dataSender
.When(x => x.SendData())
.Do(x =>
{
callsToSendData++;
if (callsToSendData < throwUntil)
{
throw new DbCollisionException();
}
});
Similarly, you can also use callbacks to locally capture parameters passed to other methods, and access them within the Do block (rather than just using a counter).
I am trying to unit test a method that uses the Wait() method on an IObservable however my test never completes - the Wait never finishes. My test contains the following:
var scheduler = new TestScheduler();
var input1 = scheduler.CreateColdObservable<List<string>>(
new Recorded<Notification<List<string>>>(100, Notification.CreateOnNext(new List<string> { "John", "Harry" })),
new Recorded<Notification<List<string>>>(200, Notification.CreateOnCompleted<List<string>>())
);
I am using Moq to setup the response on my method by returning input1. For example
myObj.Setup(f => f.GetStrings()).Returns(input1);
It doesn't actually matter about the details of myObj. I start the scheduler and call my method which contains a Wait(e.g somewhere in my method I call
var results = myObj.GetStrings().Wait();
But this never returns. I suspect I am using the scheduler wrong but I am not sure.
Regards
Alan
Summary
The problem is that you are creating a cold observable and advancing the scheduler before you have subscribed to it.
Detail
If you call the blocking Wait() operation on a single threaded test, you are dead in the water at that point. This is because the TestScheduler's internal clock only advances when you call Start() or one of the AdvanceXXX() methods and, since you have a cold observable, the event times you specify are relative the point of subscription. There are also some nuances to calling Start() which I will explain below.
So, as Wait will block, you might try to call it on another thread, but it's still tricky. Consider the following code, which is similar to yours:
void Main()
{
var scheduler = new TestScheduler();
var source = scheduler.CreateColdObservable(
new Recorded<Notification<int>>(100, Notification.CreateOnNext(1)),
new Recorded<Notification<int>>(200, Notification.CreateOnCompleted<int>()));
// (A)
int result = 0;
var resultTask = Task.Run(() => { result = source.Wait(); });
// (B)
resultTask.Wait();
Console.WriteLine(result);
}
This code tries to wait on a background thread. If we insert a call to scheduler.Start() at point (A), then source.Wait() will block forever.
This is because Start() will ONLY advance the internal clock of the TestScheduler until all currently scheduled events are executed. With a cold observable, events are scheduled relative to the virtual time of subscription. Since there are no subscribers at point (A), you will find that TestScheduler.Now.Ticks will report 0 even after the call to Start().
Hmmm. Things get even worse if we move the call to scheduler.Start() to point B. Now we have a race condition! It's a race condition that will almost always result in the test hanging at the call to resultTask.Wait(). This is because the chances are that the resultTask will not have had time to execute it's action and subscribe to source before the scheduler.Start() call executes - and so time once again will not advance.
A deterministic execution is therefore very hard to achieve - there is no nice way to announce that the Wait() call has been issued before advancing time, since the Wait() call itself will block. Inserting a long enough delay before calling Start() will work, but kind of defeats the object of using the TestScheduler:
// (B)
Task.Delay(2000).Wait();
scheduler.AdvanceBy(200);
What this question really demonstrates to me (IMHO) is that calling Wait() and blocking a thread is almost always a bad idea. Look for using methods like LastAsync() instead, and/or using continuations to get hold of results to asynchronous methods.
I can't recommend the approach due to the complexity, but here is a deterministic solution that makes use of an extension method to signal when a subscription has been made.
void Main()
{
var scheduler = new TestScheduler();
var source = scheduler.CreateColdObservable(
new Recorded<Notification<int>>(100, Notification.CreateOnNext(1)),
new Recorded<Notification<int>>(200, Notification.CreateOnCompleted<int>()));
// (A)
var waitHandle = new AutoResetEvent(false);
int result = 0;
var resultTask = Task.Run(() =>
{
result = source.AnnounceSubscription(waitHandle).Wait();
});
// (B)
waitHandle.WaitOne();
scheduler.Start();
resultTask.Wait();
Console.WriteLine(result);
}
public static class ObservableExtensions
{
public static IObservable<T> AnnounceSubscription<T>(
this IObservable<T> source, AutoResetEvent are)
{
return Observable.Create<T>(o =>
{
var sub = source.Subscribe(o);
are.Set();
return sub;
});
}
}
Recommended approach for testing Rx
A more idiomatic use of the TestScheduler is to create an observer to collect results, and then assert they meet expectations. Something like:
void Main()
{
var scheduler = new TestScheduler();
var source = scheduler.CreateColdObservable(
new Recorded<Notification<int>>(100, Notification.CreateOnNext(1)),
new Recorded<Notification<int>>(200, Notification.CreateOnCompleted<int>()));
var results = scheduler.CreateObserver<int>();
// here you would append to source the Rx calls that do something interesting
source.Subscribe(results);
scheduler.Start();
results.Messages.AssertEqual(
new Recorded<Notification<int>>(100, Notification.CreateOnNext(1)),
new Recorded<Notification<int>>(200, Notification.CreateOnCompleted<int>()));
}
Finally, if you derive a unit test class from ReactiveTest you can take advantage of OnNext, OnCompleted and OnError helper methods to create Recorded<Notification<T>> instances in a more readable fashion.
I have a simulation program. In the main class of the simulation I am "creating + adding" and "removing + destroying" Agents.
The problem is that once in a while (once every 3-4 times I run the program) the program crashes because I am apparently calling a function of an invalid agent in the main loop. The program works just fine most of the time. There are normally thousands of agents in the list.
I don't know how is it possible that I have invalid Agents in my Loop.
It is very difficult to debug the code because I receive the memory exception inside the "Agent::Step function" (which is too late because I cannot understand how was the invalid Agent in the list and got called).
When I look into the Agent reference inside the Agent::Step function (exception point) no data in the agent makes sense, not even the initialized data. So it is definitely invalid.
void World::step()
{
AddDemand();
// run over all the agents and check whether they have remaining actions
// Call their step function if they have, otherwise remove them from space and memory
list<Agent*>::iterator it = agents_.begin();
while (it != agents_.end())
{
if (!(*it)->AllIntentionsFinished())
{
(*it)->step();
it++;
}
else
{
(*it)->removeYourselfFromSpace(); //removes its reference from the space
delete (*it);
agents_.erase(it++);
}
}
}
void World::AddDemand()
{
int demand = demandIdentifier_.getDemandLevel(stepCounter_);
for (int i = 0; i < demand; i++)
{
Agent* tmp = new Agent(*this);
agents_.push_back(tmp);
}
}
Agent:
bool Agent::AllIntentionsFinished()
{
return this->allIntentionsFinished_; //bool flag will be true if all work is done
}
1- Is it possible that VStudio 2012 optimization of Loops (i.e. running in multi-thread if possible) creates the problem?
2- Any suggestions on debugging the code?
If you're running the code multi-threaded, then you'll need to add code to protect things like adding items to and removing items from the list. You can create a wrapper that adds thread safety for a container fairly easily -- have a mutex that you lock any time you do a potentially modifying operation on the underlying container.
template <class Container>
thread_safe {
Container c;
std::mutex m;
public:
void push_back(typename Container::value_type const &t) {
std::lock_guard l(m);
c.push_back(t);
}
// ...
};
A few other points:
You can almost certainly clean your code up quite a bit by having the list hold Agents directly, instead of a pointer to an Agent that you have to allocate dynamically.
Your Agent::RemoveYourselfFromSpace looks/sounds a lot like something that should be handled by Agent's destructor.
You can almost certainly do quite a bit more to clean up the code by using some standard algorithms.
For example, it looks to me like your step could be written something like this:
agents.remove_if([](Agent const &a) { return a.AllIntentionsFinished(); });
std::for_each(agents.begin(), agents.end(),
[](Agent &a) { a.step(); });
...or, you might prefer to continue using an explicit loop, but use something like:
for (Agent & a : agents)
a.step();
The problem is this:
agents_.erase(it++);
See Add and remove from a list in runtime
I don't see any thread-safe components in the code you showed, so if you are running multiple threads and sharing data between them, then absolutely you could have a threading issue. For instance, you do this:
(*it)->removeYourselfFromSpace(); //removes its reference from the space
delete (*it);
agents_.erase(it++);
This is the worst possible order for an unlocked list. You should: remove from the list, destruct object, delete object, in that order.
But if you are not specifically creating threads which share lists/agents, then threading is probably not your problem.