My async call is returning before list is populated in forEach loop - list

I have a routine which gets a list of filenames from the device, then reads the file(s) to build a list. However, the calling routine always returns with zero items. I print the filenames, so I know they exist, however, it appears that the async is returning before I read the files. I used similar code when making an HTTP call. But, something here is causing the routine to return the list even though it hasn't completed. Perhaps, it is possible that I am calling it at the wrong time? I am calling retrieveItems here:
#override
void initState() {
super.initState();
retrieveItems();
}
Eventually I will have a refresh button, but for now I'd simply like the list to populate with the data from the files...
--------------------
Callee
Future<List<String>> readHeaderData() async {
List<String> l = new List();
List<String> files = await readHeaders(); // Gets filenames
files.forEach((filename) async {
final file = await File(filename);
String contents = await file.readAsString();
User usr = User.fromJson(json.decode(contents));
String name = usr.NameLast + ", " + usr.NameFirst;
print(name);
l.add(name);
}
return l;
Caller
void retrieveItems() async {
LocalStorage storage = new LocalStorage();
await storage.readHeaderData().then((item) {
try {
if ((item != null ) &&(item.length >= 1)) {
setState(() {
users.clear();
_users.addAll(item);
});
} else {
setState(() {
_users.clear();
final snackbar = new SnackBar(
content: new Text('No users found.'),
);
scaffoldKey.currentState.showSnackBar(snackbar);
});
}
} on FileNotFoundException catch (e) {
print(e.toString()); //For debug only
setState(() {
_users.clear();
});
});
}
});

This code
Future<List<String>> readHeaderData() async {
List<String> l = new List();
List<String> files = await readHeaders(); // Gets filenames
files.forEach((filename) async {
final file = await File(filename);
String contents = await file.readAsString();
User user = User.fromJson(json.decode(contents));
String name = user.NameLast + ", " + user.NameFirst;
print(name);
l.add(name);
}
return l;
}
returns the list l and then processes the asyc forEach(...) callbacks
If you change it to
Future<List<String>> readHeaderData() async {
List<String> l = new List();
List<String> files = await readHeaders(); // Gets filenames
for(var filename in files) { /// <<<<==== changed line
final file = await File(filename);
String contents = await file.readAsString();
User user = User.fromJson(json.decode(contents));
String name = user.NameLast + ", " + user.NameFirst;
print(name);
l.add(name);
}
return l;
}
the function will not return before all filenames are processed.
files.forEach((filename) async {
means that you can use await inside the callback, but forEach doesn't care about what (filename) async {...} returns.

Also possible
await Future.forEach(yourList, (T elem) async { ...async staff });

To expand on Günter's comment regarding using list.map(f), here's an example of converting a forEach call so that it works correctly.
Broken example
Incorrectly assumes forEach will wait on futures:
Future<void> brokenExample(List<String> someInput) async {
List<String> results;
someInput.forEach((input) async {
String result = await doSomethingAsync(input);
results.add(result);
});
return results;
}
Corrected example
Waits on the async functions to complete, using Future.wait and .map():
Future<void> correctedExample(List<String> someInput) async {
List<String> results;
await Future.wait(someInput.map((input) async {
String result = await doSomethingAsync(input);
results.add(result);
}));
return results;
}

I encountered the similar issue. The problem is that dart will NOT wait for "forEach" contrary to public believe. There are two solutions:
1) Convert forEach to for loop as indicated by others. Another is use Future:
2) await Future.forEach(list, (item) async {
// your code
final result = await getMyResult();
});

Another option
Future.wait(someList.map((item) => something_returns_future(item)));

Related

Using Linq (Select, Where...) On asynchronous list

I have an asynchronous list (toListAsync) and I need to perform a linq select statement on it but it doesn't work because Task does not contain a definition for select.
What can I do?
public async Task<List<FilesTable>> List()
{
using (var ctx = Configuration.OpenContext(false))
{
return await ctx.FilesTables.Include(e => e.Subsection).ToListAsync();
}
}
// The above is in the first file(FilesTable.cs),
// the below in a different file(file_apis.cs)
public async Task<List<FilesModel>> GetFiles()
{
return await new FilesTable().List().Select(e => (FilesModel)e).ToList();
}
Add using System.Linq; namespace.
Then you can do this:
(await yourMethod.ToListAsync()).Select(a => new { });
You should add parenthesis before await keyword and after ToListAsync() method
From performance PoV it is better to use filters first, then at the end use ToListAsync, so you can use:
MyList.Where(x => ...).Select(x => ...).ToListAsync();
Otherwise you will be loading unnecessary amount of data to the memory, and this will reduce the performance depending on the amount of data. But if you do filter before ToList only the required data will be loaded.
[Update]
With reference to your code, you can a new method to the FilesTable.cs that will do filter and return the desired list:
public async Task<List<FilesModel>> ListFiles()
{
using(var cts = Configuration.OpenContext(false))
{
return await ctx.FilesTables.Include(e => e.Subsection).Select(e => (FilesModel)e).ToListAsync();
}
}
You can use or brekets for waiting the iteration end for doing the select, or you can use IAsyncEnumerable
public async IAsyncEnumerable<FilesTable> List()
{
using (var ctx = Configuration.OpenContext(false))
{
return ctx.FilesTables.Include(e => e.Subsection).AsAsyncEnumerable();
}
}
public async IAsyncEnumerable<FilesModel> GetFiles()
{
await foreach(var value in new FilesTable().List())
{
yield return value;
}
}
// or using that nuget package System.Linq.Async NuGet package
public async Task<List<FilesModel>> GetFiles()
{
return await new FilesTable().List().ToListAsync();
}

Is there a solution to asynchronously wait for C++ data on Flutter's invokeMethod?

Currently, this is how I read from C++ using Flutter:
final Uint8List result = await platform.invokeMethod(Common.MESSAGE_METHOD, {"message": buffer});
It is handled by Kotlin like this:
MethodChannel(flutterEngine.dartExecutor.binaryMessenger, CHANNEL).setMethodCallHandler { call, result ->
if (call.method == MESSAGE_METHOD) {
val message: ByteArray? = call.argument<ByteArray>("message")
//... //response = Read something FROM C++
result.success(response)
Since this happens in the main thread, if I take too much time to answer, I make Flutter's UI slow.
Is there a solution to get C++ data in an async way?
I know that Flutter has support for event channels to send data back from C++ to Flutter. But what about just requesting the data on the Flutter side and waiting for it to arrive in a Future, so I can have lots of widgets inside a FutureBuilder that resolves to something when ready?
If reading something from C++ is a heavy process, You can use AsysncTask to perform it in the background for android.
internal class HeavyMsgReader(var result: MethodChannel.Result) : AsyncTask<ByteArray?, Void?, String?>() {
override fun doInBackground(vararg message: ByteArray?): String {
//... //response = Read something FROM C++
return "response"
}
override fun onPostExecute(response: String?) {
result.success(response)
}
}
Calling async task:
MethodChannel(flutterEngine.dartExecutor.binaryMessenger, CHANNEL).setMethodCallHandler { call, result ->
if (call.method == MESSAGE_METHOD) {
val message: ByteArray? = call.argument<ByteArray>("message")
HeavyMsgReader(result).execute(message);
Hopefully this will work
import 'dart:async';
Future<Uint8List> fetchData(buffer) async {
final Uint8List result = await platform.invokeMethod(Common.MESSAGE_METHOD, {"message": buffer});
return result;
}
And just call it, like this
fetchData(buffer).then((result) => {
print(result)
}).catchError(print);
Proof that its working:
import 'dart:async';
Future<String> fetchUserOrder() async {
await Future.delayed(Duration(seconds: 5));
return 'Callback!';
}
Future<void> main() async {
fetchUserOrder().then((result) => {
print(result)
}).catchError(print);
while(true){
print('main_thread_running');
await Future.delayed(Duration(seconds: 1));
}
}
output:
main_thread_running
main_thread_running
main_thread_running
main_thread_running
main_thread_running
Callback!
main_thread_running
main_thread_running
...

How can I test / mock Hive (Flutter) open box logic in repo?

Sorry if this seems a dumb question. I'm learning clean architecture as dictated by Rob Martin, and I've having a tiny bit of trouble writing one of my tests.
I wrote a couple functions in a Hive repo. Here's the code
import 'package:hive/hive.dart';
import 'package:movie_browser/features/SearchMovie/domain/entities/movie_detailed_entity.dart';
abstract class HiveMovieSearchRepoAbstract {
Future<void> cacheMovieDetails(MovieDetailed movie);
Future<MovieDetailed> getCachedMovieDetails(String id);
}
// const vars to prevent misspellings
const String MOVIEDETAILSBOX = "MovieDetailedBox";
const String SEARCHBOX = "SearchBox";
class HiveMovieSearchRepo implements HiveMovieSearchRepoAbstract {
Box movieDetailsBox = Hive.box(MOVIEDETAILSBOX) ?? null;
// TODO implement searchbox
// final searchBox = Hive.box(SEARCHBOX);
Future<void> cacheMovieDetails(MovieDetailed movie) async {
/// expects a MovieDetailed to cache. Will cache that movie
movieDetailsBox ?? await _openBox(movieDetailsBox, MOVIEDETAILSBOX);
movieDetailsBox.put('${movie.id}', movie);
}
Future<MovieDetailed> getCachedMovieDetails(String id) async {
/// expects a string id as input
/// returns the MovieDetailed if cached previously
/// returns null otherwise
movieDetailsBox ?? await _openBox(movieDetailsBox, MOVIEDETAILSBOX);
return await movieDetailsBox.get('$id');
}
_openBox(Box box, String type) async {
await Hive.openBox(type);
return Hive.box(type);
}
}
I can't think of how to test this? I want two cases, one where the box is already opened, and one case where it isn't.
Specifically, it's these lines I want to test
movieDetailsBox ?? await _openBox(movieDetailsBox, MOVIEDETAILSBOX);
_openBox(Box box, String type) async {
await Hive.openBox(type);
return Hive.box(type);
}
I thought about mocking the Box object then doing something like....
when(mockHiveMovieSearchRepo.getCachedMovieDetails(some_id)).thenAnswer((_) async => object)
but wouldn't that bypass the code I want tested and always show as positive?
Thanks so much for the help
i don't know if i fully understand your question but you can try something like this
abstract class HiveMovieSearchRepoAbstract {
Future<void> cacheMovieDetails(MovieDetailed movie);
Future<MovieDetailed> getCachedMovieDetails(String id);
}
// const vars to prevent misspellings
const String MOVIEDETAILSBOX = "MovieDetailedBox";
const String SEARCHBOX = "SearchBox";
class HiveMovieSearchRepo implements HiveMovieSearchRepoAbstract {
final HiveInterface hive;
HiveMovieSearchRepo({#required this.hive});
#override
Future<void> cacheMovieDetails(MovieDetailed cacheMovieDetails) async {
/// expects a MovieDetailed to cache. Will cache that movie
try {
final moviedetailbox = await _openBox(MOVIEDETAILSBOX);
moviedetailbox.put('${movie.id}', movie);
} catch (e) {
throw CacheException();
}
}
Future<MovieDetailed> getCachedMovieDetails(String id) async {
/// expects a string id as input
/// returns the MovieDetailed if cached previously
/// returns null otherwise
try {
final moviedetailbox = await _openBox(MOVIEDETAILSBOX);
if (moviedetailbox.containsKey(boxkeyname)) {
return await movieDetailsBox.get('$id');
}
return null;
} catch (e) {
return CacheException();
}
}
Future<Box> _openBox(String type) async {
try {
final box = await hive.openBox(type);
return box;
} catch (e) {
throw CacheException();
}
}
}
And to test it you can do something like this
class MockHiveInterface extends Mock implements HiveInterface {}
class MockHiveBox extends Mock implements Box {}
void main() {
MockHiveInterface mockHiveInterface;
MockHiveBox mockHiveBox;
HiveMovieSearchRepo hiveMovieSearchRepo;
setUp(() {
mockHiveInterface = MockHiveInterface();
mockHiveBox = MockHiveBox();
hiveMovieSearchRepo = HiveMovieSearchRepo(hive: mockHiveInterface);
});
group('cacheMoviedetails', () {
test(
'should cache the movie details',
() async{
//arrange
when(mockHiveInterface.openBox(any)).thenAnswer((_) async => mockHiveBox);
//act
await hiveMovieSearchRepo.cacheMovieDetails(tcacheMovieDetails);
//assert
verify(mockHiveBox.put('${movie.id}', tmovie));
verify(mockHiveInterface.openBox("MovieDetailedBox"));
});
});
group('getLocalCitiesAndCountriesAtPage', () {
test('should when', () async {
//arrange
when(mockHiveInterface.openBox(any))
.thenAnswer((realInvocation) async => mockHiveBox);
when(mockHiveBox.get('$id'))
.thenAnswer((realInvocation) async => tmoviedetails);
//act
final result =
await hiveMovieSearchRepo.getCachedMovieDetails(tId);
//assert
verify(mockHiveInterface.openBox(any));
verify(mockHiveBox.get('page${tpage.toString()}'));
expect(result, tmoviedetails);
});
});
}
You should add some tests also for the CacheExeption().
Hope this help you.
So, I wrote this post 9 months. Stackoverflow just sent me a notification saying it's a popular question, so I'll answer it for anyone else wondering the same thing
Easy way to make this testable is change Box to an arg passed into the class, like so
abstract class ClassName {
final Box movieDetailsBox;
final Box searchBox;
ClassName({
this.moveDetailsBox,
this.searchBox,
});
}
this makes the boxes mockable and testable
You should mock the hive interface and box;

Flutter + AWS: It always gives broken pipe error

I'm trying to compare faces with AWS rekognition API. but somehow I'm getting "broken pipe" error all the time. There is no problem on aws keys and photos. I'm trying to get more info from http.post but It just says "broken pipe", it doesn't give any detail, unfortunately.
Scenario;
User takes 2 photos (working)
on second taken, I will parse images to bytes (working)
send bytes with standard request to aws API (doesn't work)
I changed the image quality to the lowest as well, but It didn't help.
Main.dart code
import 'dart:async';
import 'dart:io';
import 'package:flutter/material.dart';
import 'package:camera/camera.dart';
import 'testa.dart';
import 'package:path_provider/path_provider.dart';
List<CameraDescription> cameras;
Future<void> main() async {
cameras = await availableCameras();
runApp(MyApp());
}
class MyApp extends StatelessWidget {
// This widget is the root of your application.
#override
Widget build(BuildContext context) {
return MaterialApp(
title: 'Flutter Demo',
theme: ThemeData(
// This is the theme of your application.
//
// Try running your application with "flutter run". You'll see the
// application has a blue toolbar. Then, without quitting the app, try
// changing the primarySwatch below to Colors.green and then invoke
// "hot reload" (press "r" in the console where you ran "flutter run",
// or simply save your changes to "hot reload" in a Flutter IDE).
// Notice that the counter didn't reset back to zero; the application
// is not restarted.
primarySwatch: Colors.blue,
),
home: MyHomePage(title: 'Flutter Demo Home Page'),
);
}
}
class MyHomePage extends StatefulWidget {
MyHomePage({Key key, this.title}) : super(key: key);
// This widget is the home page of your application. It is stateful, meaning
// that it has a State object (defined below) that contains fields that affect
// how it looks.
// This class is the configuration for the state. It holds the values (in this
// case the title) provided by the parent (in this case the App widget) and
// used by the build method of the State. Fields in a Widget subclass are
// always marked "final".
final String title;
#override
_MyHomePageState createState() => _MyHomePageState();
}
class _MyHomePageState extends State<MyHomePage> {
int _counter = 0;
CameraController controller;
#override
void initState() {
super.initState();
controller = CameraController(cameras[0], ResolutionPreset.low);
controller.initialize().then((_) {
if (!mounted) {
return;
}
setState(() {});
});
}
void _incrementCounter() {
setState(() {
// This call to setState tells the Flutter framework that something has
// changed in this State, which causes it to rerun the build method below
// so that the display can reflect the updated values. If we changed
// _counter without calling setState(), then the build method would not be
// called again, and so nothing would appear to happen.
_counter++;
});
}
Future<String> _checkImage(String filePath, String secondPath) async {
File sourceImagefile, targetImagefile; //load source and target images in those File objects
String accessKey, secretKey, region ; //load your aws account info in those variables
print(filePath);
print(secondPath);
targetImagefile = File(filePath);
sourceImagefile = File(secondPath);
print(targetImagefile.existsSync());
print(sourceImagefile.existsSync());
accessKey = '';
secretKey = '';
region = 'eu-west-1';
RekognitionHandler rekognition = new RekognitionHandler(accessKey, secretKey, region);
String labelsArray = await rekognition.compareFaces(sourceImagefile, targetImagefile);
return labelsArray;
}
Widget cameraPart() {
if (!controller.value.isInitialized) {
return Container();
}
return AspectRatio(
aspectRatio:
controller.value.aspectRatio,
child: CameraPreview(controller));
}
#override
Widget build(BuildContext context) {
// This method is rerun every time setState is called, for instance as done
// by the _incrementCounter method above.
//
// The Flutter framework has been optimized to make rerunning build methods
// fast, so that you can just rebuild anything that needs updating rather
// than having to individually change instances of widgets.
return Scaffold(
appBar: AppBar(
// Here we take the value from the MyHomePage object that was created by
// the App.build method, and use it to set our appbar title.
title: Text(widget.title),
),
body: cameraPart(),
floatingActionButton: FloatingActionButton(
onPressed: takePhoto,
tooltip: 'Increment',
child: Icon(Icons.add),
), // This trailing comma makes auto-formatting nicer for build methods.
);
}
Future<String> get360PhotoFolder() async {
final Directory appFolder = await getAppFolder();
final String dirPath = '${appFolder.path}/photos360';
await Directory(dirPath).create(recursive: true);
return dirPath;
}
String firstPath = '';
String secondPath = '';
Future<bool> takePhoto() async {
final String dirPath = await get360PhotoFolder();
final String filePath = '$dirPath/${timestamp()}_test.jpg';
try {
debugPrint('photo taken - $filePath');
await controller.takePicture(filePath);
setState(() {
if (firstPath == '') {
print('a');
firstPath = filePath;
}else if (secondPath == '') {
print('b');
secondPath = filePath;
_checkImage(firstPath, secondPath).then((value) {
print(value);
}).catchError((error) {
print(error);
});
firstPath = '';
secondPath = '';
}
});
} on CameraException catch (e) {
print([e.code, e.description]);
return true;
}
return false;
}
String timestamp() => DateTime.now().millisecondsSinceEpoch.toString();
Future<Directory> getAppFolder() async =>
await getApplicationDocumentsDirectory();
}
AWS rekognition code
import 'dart:async';
import 'dart:convert';
import 'dart:io';
import 'package:intl/intl.dart';
import 'testb.dart';
class RekognitionHandler {
final String _accessKey, _secretKey, _region;
RekognitionHandler(this._accessKey, this._secretKey, this._region);
Future<String> _rekognitionHttp(String amzTarget, String body) async {
String endpoint = "https://rekognition.$_region.amazonaws.com/";
String host = "rekognition.$_region.amazonaws.com";
String httpMethod = "POST";
String service = "rekognition";
var now = new DateTime.now().toUtc();
var amzFormatter = new DateFormat("yyyyMMdd'T'HHmmss'Z'");
String amzDate =
amzFormatter.format(now); // format should be '20170104T233405Z"
var dateFormatter = new DateFormat('yyyyMMdd');
String dateStamp = dateFormatter.format(
now); // Date w/o time, used in credential scope. format should be "20170104"
int bodyLength = body.length;
String queryStringParamters = "";
Map<String, String> headerParamters = {
"content-length": bodyLength.toString(),
"content-type": "application/x-amz-json-1.1",
"host": host,
"x-amz-date": amzDate,
"x-amz-target": amzTarget
};
String signature = Signature.generateSignature(
endpoint,
service,
_region,
_secretKey,
httpMethod,
now,
queryStringParamters,
headerParamters,
body);
String authorization =
"AWS4-HMAC-SHA256 Credential=$_accessKey/$dateStamp/$_region/$service/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-date;x-amz-target, Signature=$signature";
headerParamters.putIfAbsent('Authorization', () => authorization);
//String labelsArray = "";
StringBuffer builder = new StringBuffer();
try {
HttpClient httpClient = new HttpClient();
httpClient.connectionTimeout = Duration(minutes: 10);
HttpClientRequest request = await httpClient.postUrl(Uri.parse(endpoint));
request.headers.set('content-length', headerParamters['content-length']);
request.headers.set('content-type', headerParamters['content-type']);
request.headers.set('host', headerParamters['host']);
request.headers.set('x-amz-date', headerParamters['x-amz-date']);
request.headers.set('x-amz-target', headerParamters['x-amz-target']);
request.headers.set('Authorization', headerParamters['Authorization']);
request.write(body);
HttpClientResponse response = await request.close();
await for (String a in response.transform(utf8.decoder)) {
builder.write(a);
}
} catch (e) {
print(e);
}
return Future.value(builder.toString());
}
Future<String> compareFaces(
File sourceImagefile, File targetImagefile) async {
try {
List<int> sourceImageBytes = sourceImagefile.readAsBytesSync();
String base64SourceImage = base64Encode(sourceImageBytes);
List<int> targetImageBytes = targetImagefile.readAsBytesSync();
String base64TargetImage = base64Encode(targetImageBytes);
String body =
'{"SourceImage":{"Bytes": "$base64SourceImage"},"TargetImage":{"Bytes": "$base64TargetImage"}}';
String amzTarget = "RekognitionService.CompareFaces";
String response = await _rekognitionHttp(amzTarget, body);
return response;
} catch (e) {
print(e);
return "{}";
}
}
}

How tp send big Data set from loopback.js remote method

I am reading oracle ref Cursor and storing in p_ref out param and using the streams from resultset.toquerystream() method I am reading the data and pushing the data in array variable and once reached end of the stream I am sending the data.
its working fine for small data set of 1000 records but failing with error ERR_EMPTY_RESPONSE as you can see I am storing the data in array and returning the data once we reach end
but its failing with "ERR_EMPTY_RESPONSE"
oracledb.getConnection(
ds.settings,
function (err, connection) {
if (err) {
cb(err.message, []);
return;
}
let count = 0;
let params = {
p_ref: {type: oracledb.CURSOR, dir: oracledb.BIND_OUT}
};
let cursor = null;
let queryStream = null;
let data = [];
connection.execute(
sql,
params,
{outFormat: oracledb.OBJECT},
function (err, result) {
if (err) {
cb(err.message, []);
doRelease(connection);
return;
}
cursor = result.outBinds.p_ref;
queryStream = cursor.toQueryStream();
queryStream.on('data', function (row) {
count++;
data.push(row);
});
queryStream.on('error', function (err) {
if (!cbCalled) {
cbCalled = true;
cb(err.message);
doRelease(connection);
}
});
queryStream.on('end', function () {
console.log('Rows selected: ' + count);
cb('', data);
doRelease(connection);
});
queryStream.on('close', function () {
if (!cbCalled) {
console.log(count);
doRelease(connection);
}
});
}
);
}
);
Reference from here https://github.com/oracle/node-oracledb/blob/master/doc/api.md
var stream = connection.queryStream('SELECT employees_name FROM employees');
stream.on('error', function (error) { // handle any error... });
stream.on('data', function (data) { // handle data row... });
stream.on('end', function () { // release connection... });
stream.on('metadata', function (metadata) { // access metadata of query });
Try removing the call to doRelease() from the 'end' event.