Ionic 2 / cordova-plugin-file File.writeFile() refuses to create binary file correctly (png image) - ionic2

In summary
File.writeFile() creates a PNG file of 0 bytes when trying to write a Blob made from base64 data.
In my application, I am trying to create a file that consists of base64 data stored in the db. The rendered equivalent of the data is a small anti-aliased graph curve in black on a transparent background (never more that 300 x 320 pixels) that has previously been created and stored from a canvas element. I have independently verified that the stored base64 data is indeed correct by rendering it at one of various base64 encoders/decoders available online.
Output from "Ionic Info"
--------------------------------
Your system information:
Cordova CLI: 6.3.1
Gulp version: CLI version 3.9.1
Gulp local:
Ionic Framework Version: 2.0.0-rc.2
Ionic CLI Version: 2.1.1
Ionic App Lib Version: 2.1.1
Ionic App Scripts Version: 0.0.39
OS:
Node Version: v6.7.0
--------------------------------
The development platform is Windows 10, and I've been testing directly on a Samsung Galaxy S7 and S4 so far.
I know that the base64 data has to be converted into binary data (as a Blob) first, as File does not yet support writing base64 directly in to an image file. I found various techniques with which to do this, and the code which seems to suit my needs the most (and reflects a similar way I would have done it in java is illustrated below):
Main code from constructor:
this.platform.ready().then(() => {
this.graphDataService.getDataItem(this.job.id).then((data) =>{
console.log("getpic:");
let imgWithMeta = data.split(",")
// base64 data
let imgData = imgWithMeta[1].trim();
// content type
let imgType = imgWithMeta[0].trim().split(";")[0].split(":")[1];
console.log("imgData:",imgData);
console.log("imgMeta:",imgType);
console.log("aftergetpic:");
// this.fs is correctly set to cordova.file.externalDataDirectory
let folderpath = this.fs;
let filename = "dotd_test.png";
File.resolveLocalFilesystemUrl(this.fs).then( (dirEntry) => {
console.log("resolved dir with:", dirEntry);
this.savebase64AsImageFile(dirEntry.nativeURL,filename,imgData,imgType);
});
});
});
Helper to convert base64 to Blob:
// convert base64 to Blob
b64toBlob(b64Data, contentType, sliceSize) {
//console.log("data packet:",b64Data);
//console.log("content type:",contentType);
//console.log("slice size:",sliceSize);
let byteCharacters = atob(b64Data);
let byteArrays = [];
for (let offset = 0; offset < byteCharacters.length; offset += sliceSize) {
let slice = byteCharacters.slice(offset, offset + sliceSize);
let byteNumbers = new Array(slice.length);
for (let i = 0; i < slice.length; i++) {
byteNumbers[i] = slice.charCodeAt(i);
}
let byteArray = new Uint8Array(byteNumbers);
byteArrays.push(byteArray);
}
console.log("size of bytearray before blobbing:", byteArrays.length);
console.log("blob content type:", contentType);
let blob = new Blob(byteArrays, {type: contentType});
// alternative way WITHOUT chunking the base64 data
// let blob = new Blob([atob(b64Data)], {type: contentType});
return blob;
}
save the image with File.writeFile()
// save the image with File.writeFile()
savebase64AsImageFile(folderpath,filename,content,contentType){
// Convert the base64 string in a Blob
let data:Blob = this.b64toBlob(content,contentType,512);
console.log("file location attempt is:",folderpath + filename);
File.writeFile(
folderpath,
filename,
data,
{replace: true}
).then(
_ => console.log("write complete")
).catch(
err => console.log("file create failed:",err);
);
}
I have tried dozens of different decoding techniques, but the effect is the same. However, if I hardcode simple text data into the writeFile() section, like so:
File.writeFile(
folderpath,
"test.txt",
"the quick brown fox jumps over the lazy dog",
{replace: true}
)
A text file IS created correctly in the expected location with the text string above in it.
However, I've noticed that whether the file is the 0 bytes PNG, or the working text file above, in both cases the ".then()" consequence clause of the File Promise never fires.
Additionally, I swapped the above method and used the Ionic 2 native Base64-To-Gallery library to create the images, which worked without a problem. However, having the images in the user's picture gallery or camera roll is not an option for me as I do not wish to risk a user's own pictures while marshalling / packing / transmitting / deleting the data-rendered images. The images should be created and managed as part of the app.
User marcus-robinson seems to have experienced a similar issue outlined here, but it was across all file types, and not just binary types as seems to be the case here. Also, the issue seems to have been closed:
https://github.com/driftyco/ionic/issues/5638
Anybody experiencing something similar, or possibly spot some error I might have caused? I've tried dozens of alternatives but none seem to work.

I had similar behaviour saving media files which worked perfectly on iOS. Nonetheless, I had the issue of 0 bytes file creation on some Android devices in release build (dev build works perfectly). After very long search, I followed the following solution
I moved the polyfills.js script tag to the top of the index.html in the ionic project before the cordova.js tag. This re-ordering somehow the issue is resolved.
So the order should look like:
<script src="build/polyfills.js"></script>
<script type="text/javascript" src="cordova.js"></script>
Works on ionic 3 and ionic 4.
The credits go to 1

I got that working with most of your code:
this.file.writeFile(this.file.cacheDirectory, "currentCached.jpeg", this.b64toBlob(src, "image/jpg", 512) ,{replace: true})
The only difference i had was:
let byteCharacters = atob(b64Data.replace(/^data:image\/(png|jpeg|jpg);base64,/, ''));
instead of your
let byteCharacters = atob(b64Data);
Note: I did not use other trimming etc. like those techniques you used in your constructor class.

Related

Download image using Dynamics 365 Web API

I am trying to download the image from Dataverse using Dynamics Web API.
I am able to succeed in that using {{webapiurl}}sample_imageattributedemos(d66ecb6c-4fd1-ec11-a7b5-6045bda5603f)/entityimage/$value
But when I try to download the full/actual size image - I am getting the file with the reduced size - {{webapiurl}}sample_imageattributedemos(d66ecb6c-4fd1-ec11-a7b5-6045bda5603f)/entityimage/$value?fullsize=true.
I tried to download the image using the sample code where additionally I have added CanStoreFullImage = true attribute.
Please find below code snippet for the reference:
CreateAttributeRequest createEntityImageRequest = new CreateAttributeRequest
{
EntityName = _customEntityName.ToLower(),
Attribute = new ImageAttributeMetadata
{
SchemaName = "EntityImage", //The name is always EntityImage
//Required level must be AttributeRequiredLevel.None
RequiredLevel = new AttributeRequiredLevelManagedProperty(AttributeRequiredLevel.None),
DisplayName = new Microsoft.Xrm.Sdk.Label("Image", 1033),
Description = new Microsoft.Xrm.Sdk.Label("An image to show with this demonstration.", 1033),
CanStoreFullImage = true,
IsPrimaryImage = false,
}
};
How can I achieve this - to download the full size image using Web API?
the correct syntax is size=full, not fullsize=true
to build such requests you can use my tool Dataverse REST Builder, you can find the operations to deal with Image fields under the Manage Image Data request type

Offline Maps HERE-SDK - offline search and navigation

I am using HERE-Android-SDK to build a simple navigation solution with offline maps.
While using the offline mode for the search addresses and calculation of a route, I can see that there are results returned from the address-search, which are not included in the installed offline map datasets. Is there anything additional which I need to do, in order to get only search results which are located inside the offline-map data installed on my device?
I am using the following code snippets.
download offline maps for a specific country:
mapsLoader.selectDataGroup(MapPackage.SelectableDataGroup.TruckAttributes)
mapsLoader.installMapPackages(listOf(mapPackageId))
search request for addresses:
val term = "New York"
val center = GeoCoordinate(lastWayPoint.latitude, lastWayPoint.longitude)
val request = SearchRequest(term)
request.connectivity = Request.Connectivity.OFFLINE
request.locale = Locale.GERMAN
request.setSearchCenter(center)
request.collectionSize = 5
request.execute { data, error ->
if (error != ErrorCode.NONE) return
// handle search results here
}
Thanks for all of your help in advance!

Postman accessing the stored results in the database leveldb

So I have a set of results in Postman from a runner on a collection using some data file for iterations - I have the stored data from the runner in the Postman app on Linux, but I want to know how I can get hold of the data. There seems to be a database hidden away in the ~/.config directory (/Desktop/file__0.indexeddb.leveldb) - that looks like it has the data from the results there.
Is there anyway that I can get hold of the raw data - I want to be able to save the results from the database and not faff around with running newman or hacking a server to post the results and then save, I already have 20000 results in a collection. I want to be able to get the responseData from each post and save it to a file - I will not execute the posts again, I need to just work out a way
I've tried KeyLord, FastNoSQL (this crashes), levelDBViewer(Jar), but not having any luck here.
Any suggestions?
inline 25024 of runner.js a simple yet hack for small numbers of results I can do the following
RunnerResultsRequestListItem = __WEBPACK_IMPORTED_MODULE_2_pure_render_decorator___default()(_class = class RunnerResultsRequestListItem extends __WEBPACK_IMPORTED_MODULE_0_react__["Component"] {
constructor(props) {
super(props);
var text = props.request.response.body,
blob = new Blob([text], { type: 'text/plain' }),
anchor = document.createElement('a');
anchor.download = props.request.ref + ".txt";
anchor.href = (window.webkitURL || window.URL).createObjectURL(blob);
anchor.dataset.downloadurl = ['text/plain', anchor.download, anchor.href].join(':');
anchor.click();
it allows me to save but obviously I have to click save for now, anyone know how to automate the saving part - please add something here!

Can I convert browser generated image blob to image file for upload?

I'm using fabric.js to dynamically create textures in Threes.js, and I need to save the textures to AWS. I'm using meteor-slingshot, which normally takes images passed in through a file selector input. Here's the uploader:
var uploader = new Slingshot.Upload("myFileUploads");
uploader.send(document.getElementById('input').files[0], function (error, downloadUrl) {
if (error) {
console.error('Error uploading', uploader.xhr.response);
alert (error);
}
else {
Meteor.users.update(Meteor.userId(), {$push: {"profile.files":downloadUrl}});
}
});
Uploading works fine from the drive ... but I'm generating my files in the browser, not getting them from the drive. Instead, they are generated from a canvas element with the following method:
generateTex: function(){
var canvTex = document.getElementById('texture-generator');
var canvImg = canvTex.toDataURL('image/jpeg');
var imageNew = document.createElement( 'img' );
imageNew.src = canvImg;
}
This works great as well. If I console.log the imageNew, I get my lovely image with base 64 encoding:
<img src=​"data:​image/​jpeg;​base64,/​9j/​
4AAQSkZJRgABAQAAAQABAAD/2wBDAAMCAgICAgMCAgID
//....carries on to 15k or so characters
If I console.log a file object added from the drive via filepicker ( not generated from a canvas ), I can see what the file object should look like:
file{
lastModified: 1384216556000
lastModifiedDate: Mon Nov 11 2013 16:35:56 GMT-0800 (PST)
name: "filename.png"
size: 3034
type: "image/png"
webkitRelativePath: ""
__proto__: File
}
But I can't create a file from the blob for upload, because there is no place in the file object to add the actual data.
To sum up I can:
Generate an image blob and display it in a dom element
Upload files from the drive using meteor-slingshot
inspect the existing file object
But I don't know how to convert the blob into a named file, so I can pass it to the uploader.
I don't want to download the image, (there are answers for that), I want to upload it. There is a "chrome only" way to do this with the filesystem API but I need something cross browser (and eventually cross platform). If someone could help me with this, I would have uncontainable joy.
Slingshot supports blobs just as well as files: https://github.com/CulturalMe/meteor-slingshot/issues/22
So when you have a canvas object called canvTex and a Slingshot.Upload instance called uploader, then uploading the canvas image is as easy as:
canvTex.toBlob(function (blob) {
uploader.send(blob, function (error, downloadUrl) {
//...
});
});
Because blobs have no names, you must take that into account when defining your directive. Do not attempt to generate a key based on the name of the file.

Parametrising encoded web service requests

I am trying to parametrise web service requests in a web performance test. Using Fiddler2 I have recorded a sequence of over 60 web service requests for a transaction performed by my desktop application and saved them as a .webtest file. This web test runs without any errors and the responses that I have checked look correct.
When the web service requests are viewed in Visual Studio 2012 they appear in plain text and so I should be able to edit them to parametrise the values in the SOAP requests. For example, most of the requests contain the text <Database>db1a</Database> (actually it has <Database>db1a</Database>) and I want to change them to get the database name from a context parameter. There are several other items to replace with parameters. For this one transaction there are over 60 web service requests and I have other transactions to record. The .webtest file contains XML and the requests looks like:
<Request Method="POST" Version="1.1" Url="http://example.com/somewhere.asmx" ThinkTime="83" Timeout="60" ParseDependentRequests="True" FollowRedirects="True" RecordResult="True" Cache="False" ResponseTimeGoal="0" Encoding="utf-8">
<Headers>
<Header Name="Content-Type" Value="text/xml; charset=utf-8" />
<Header Name="SOAPAction" Value=""http://example.com/webservices/VariousActionNamesHere"" />
</Headers>
<StringHttpBody ContentType="text/xml; charset=utf-8">PAA/AHgAbQBsACAAdg
... lots more characters not shown
+AA==</StringHttpBody>
</Request>
The StringHttpBody field contains an encoded version of the SOAP request. Visual Studio shows it as plain text. What is the encoding of this field and how can I decode and encode it?
I have installed Release 3.0 of the “Web and Load Test Plugins for Visual Studio Team Test” from http://teamtestplugins.codeplex.com/ . They provide a slightly better interface for editing the SOAP requests one at a time. But they do not allow mass changes.
Converting the web test to a coded web test (ie into C#) shows the SOAP requests as simple text and they could be edited there but I would prefer to keep the flexibility of a .webtest file.
Update: I have posted a partial answer to the question. Whilst it works, it feels the wrong way to do the work because it feels too complicated. So I am looking for a better overall approach.
StringHttpBody is base64 encoded. The raw body of the request is converted to a UTF-16 byte array and then base64 encoded, like so:
Convert.ToBase64String(Encoding.Unicode.GetBytes(oSession.GetRequestBodyAsString()));
For a quick view, you can copy/paste this string into Fiddler's Tools > TextWizard screen then use the From Base64 option to decode.
Here is part of an answer to working with the StringHttpBody fields. This is about decoding and encoding the fields to allow easier understanding and modification.
Read the input XML and find the contents of the StringHttpBody fields. Replace each field contents with the result of calling the following routine on the original contents. Write the all the lines to a new intermediate file. The byte array contains UTF-16 characters as high and low bytes. (All the characters I have seen so far have high byte zero.)
private string DecodeBody(string source) {
byte[] outBytes = Convert.FromBase64String(source);
StringBuilder sb = new StringBuilder();
Assert( (outBytes.Length % 2) != 0 );
for (int ix = 0; ix < outBytes.Length; ix += 2) {
Assert(outBytes[ix] != 0);
sb.Append((char)outBytes[ix + 1]);
}
return sb.ToString();
}
Now have a file containing a simple text version of the .webtest file. This file can easily be edited to parameterise fields of the requests. Have an routine used similar to the one above and writing to another intermediate file. The routine has statements such as:
source = source.Replace("<Database>db1a</Database>", "<Database>{{DatabaseName}}</Database>");
The final intermediate file is then reencoded to create a new .webtest file. Just as before the contents of the StringHttpBody fields are found and replaced with the result of calling a routine. The encoding routine is:
private string EncodeBody(string source) {
StringBuilder sb = new StringBuilder();
byte[] outBytes = new byte[2 * source.Length];
for (int ix = 0; ix < source.Length; ix++) {
char ch = source[ix];
outBytes[2 * ix] = (byte)(((int)ch) & 0xFF);
outBytes[2 * ix + 1] = (byte)((((int)ch) / 256) & 0xFF);
}
sb.Append(Convert.ToBase64String(outBytes));
return sb.ToString();
}
The flow of files is thus:
decode original.webtest > intermediate1
parameterise intermediate1 > intermediate2
encode intermediate2 > final.webtest
On the small number of .webtest files I have tried the encode operation is the inverse of the decode operation, the original file from before decoding is identical to the file after encoding. Having the two intermediate files allows easy checking and searching of the contents of the unencoded file and the effect of the parameterise step.
For
[StringHttpBody ContentType="application/json"]
To Decode the body:
var encodedString = childNode.InnerText;
var encodedStringBytes = Convert.FromBase64String(encodedString);
var decodedString = Encoding.Unicode.GetString(encodedStringBytes);
JObject jsonString =JsonConvert.DeserializeObject(decodedString);
To Encode the body:
childNode.InnerText =
Convert.ToBase64String(Encoding.Unicode.GetBytes(JsonConvert.SerializeObject(jsonString)));
I think this might help.