Serialize Json / Web Services to Observable Collection Model - web-services

I want to ask, so i've consume a web service api and than serialize it into a observable collection of Model.
My question is how i can use this observable collection everywhere, so i don't have to call/get/consume from web services everytime?
So just call the api one time and then can use the data everytime without callng API again?
Thanks

As #thang mentioned above there are many ways to store the data in the app to eliminate calling web service each time.
I will suggest you the way I am doing it:
1.When I retrieve the JSON data from the Web Api I am parsing it to Observable Collection:
ObservableCollection<User> usersList = JsonConvert.DeserializeObject<ObservableCollection<User>>(responseJson).Users;
2.Once I have my list I can also save serialized objects (in JSON format) to the text file (remember that JSON is nothing else like string):
private async void saveUsersToFile(string serializedUsersListAsJson)
{
StorageFolder storageFolder = ApplicationData.Current.LocalFolder;
StorageFile usersFile = await storageFolder.CreateFileAsync("users.txt", CreationCollisionOption.OpenIfExists);
await FileIO.WriteTextAsync(usersFile, serializedUsersListAsJson);
}
This step allows you to store the data even if the app is closed and relaunched.
3.When you launch the app you can invoke below method to read data from the file:
private async void retrieveNotes()
{
StorageFolder storageFolder = ApplicationData.Current.LocalFolder;
StorageFile usersFile = await storageFolder.CreateFileAsync("users.txt", CreationCollisionOption.OpenIfExists)
string serializedUsersList = await FileIO.ReadTextAsync(usersFile );
// Deserialize JSON list to the ObservableCollection:
if(serializedUsersList!=null)
{
var usersList= JsonConvert.DeserializeObject<ObservableCollection<User>>(serializedUsersList);
}
4.Last step is to declare Observable Collection field in Pages where you need to use it. For instance if you need to pass this list between Pages you can just use:
Frame.Navigate(typeof(MainPage), usersList);
Remember to read the data from the file once the app is launched. After that you can just use it while app is running. My suggestion is to cache data each time you connect Web Api to retrieve new data.
Hope this will help. If you want to read more about data storage please read below post on my blog:
https://mobileprogrammerblog.wordpress.com/2016/05/23/universal-windows-10-apps-data-storage/

To save the data for next time user open the app
Store the data in local Sqlite database, or serialized the collection to a local file to use later.
To use the data in same section
Store the data in a common object, and retrieve it every time you need to initialize the ViewModel
To use the data across Win 10 device
Store the file / database in OneDrive and sync when needed
If the data size is small and you dont have critical need to have 100% sync data, store it inside roaming folder

Related

Does Django have a method of storage like HTML5's localStorage or sessionStorage?

Does Django have a method of storage like HTML5's localStorage or sessionStorage?
I want to use the Django/Django-Rest-Framework as the backend of my project.
but whether the Django has a convenient storage method to server my project? if in the HTML5 there are localStorage and sessionStorage, which is very useful.
EDIT
I want to use a simple method to store my temporary data, such as, if there is a requirement to share the data.
such as I have 3 providers (a_provider, b_provider, c_provider), they can process a origin_data.
in a function,
def process_data():
a_provider(get_data()) # process a
b_provider(get_data()) # process b
c_provider(get_data()) # process c
the get_data() can get the shared data.
rather than every process to return the processed data as param to pass into other provider.
There are some 'Offline Solutions' you can check out here
However, If you are trying to completely run in local-storage Django probably isn't your choice. Some new development on this particular topic is being explored by the awesome team at BeeWare.
Hope this helps.

How to call on Web Service API and route data into Azure SQL Database?

Having configured an Azure SQL Database, I would like to feed some tables with data from an HTTP REST GET call.
I have tried Microsoft Flow (whose HTTP Request action is utterly botched) and I am now exploring Azure Data Factory, to no avail.
The only way I can currently think of is provisioning an Azure VM and install Postman with Newman. But then, I would still need to create a Web Service interface to the Azure SQL Database.
Does Microsoft offer no HTTP call service to hook up to an Azure SQL Database?
Had the same situation a couple of weeks ago and I ended up building the API call management using Azure Functions. No problem to use the Azure SDK's to upload the result to e.g BLOB store or Data Lake. And you can add whatever assembly you need to perform the HTTP post operation.
From their you can easily pull it with Data Factory to a Azure SQL db.
I would suggest you write yourself an Azure Data Factory custom activity to achieve this. I've done this for a recent project.
Add a C# class library to your ADF solution and create a class that inherits from IDotNetActivity. Then in the IDictionary method make the HTTP web request to get the data. Land the downloaded file in blob storage first, then have a downstream activity to load the data into SQL DB.
public class GetLogEntries : IDotNetActivity
{
public IDictionary<string, string> Execute(
IEnumerable<LinkedService> linkedServices,
IEnumerable<Dataset> datasets,
Activity activity,
IActivityLogger logger)
{
etc...
HttpWebResponse myHttpWebResponse = (HttpWebResponse)httpWebRequest.GetResponse();
You can use the ADF linked services to authenticate against the storage account and define where container and file name you want as the output etc.
This is an example I used for data lake. But there is an almost identical class for blob storage.
Dataset outputDataset = datasets.Single(dataset => dataset.Name == activity.Outputs.Single().Name);
AzureDataLakeStoreLinkedService outputLinkedService;
outputLinkedService = linkedServices.First(
linkedService =>
linkedService.Name ==
outputDataset.Properties.LinkedServiceName).Properties.TypeProperties
as AzureDataLakeStoreLinkedService;
Don't bother with an input for the activity.
You will need an Azure Batch Service as well to handle the compute for the compiled classes. Check out my blog post on doing this.
https://www.purplefrogsystems.com/paul/2016/11/creating-azure-data-factory-custom-activities/
Hope this helps.

How to store and load models from Ember data to and from local files?

I'm giving it a go at making an Ember Electron app and wanting to save some model data to a json or json-api formatted file instead of pushing it to a remote API, what would be the best way to do this?
I see that it is possible to call .toJSON() on model objects to get a basic JSON representation of the model, from which I can save it to disk using electron-settings, however I'm at a loss on how to load the data in.
From what I can see, using the "push" function in Ember data allows you to import data, however it seems it expects JSON-API format instead:
https://guides.emberjs.com/v2.9.0/models/pushing-records-into-the-store/
I've found it is easiest to store the data in JSON API spec format. To get this out of the system simply call var data = model.serialize({includeId: true}) this will give you the JSON API Specced version of your data.
Or to get a load of models:
var jsonModels = [];
allModels.forEach(function(model) {
jsonModels.pushObject(model.serialize({includeId: true}).data)
}, this);
return {data: JSON.stringify(jsonModels)};
You can then call this.get('store').pushPayload(data); to load that JSON into your store.

Upload Spark RDD to REST webservice POST method

Frankly i'm not sure if this feature exist?sorry for that
My requirement is to send spark analysed data to file server on daily basis, file server supports file transfer through SFTP and REST Webservice post call.
Initial thought was to save Spark RDD to HDFS and transfer to fileserver through SFTP.
I would like to know is it possible to upload the RDD directly by calling REST service from spark driver class without saving to HDFS.
Size of the data is less than 2MB
Sorry for my bad english!
There is no specific way to do that with Spark. With that kind of data size it will not be worth it to go through HDFS or another type of storage. You can collect that data in your driver's memory and send it directly. For a POST call you can just use plain old java.net.URL, which would look something like this:
import java.net.{URL, HttpURLConnection}
// The RDD you want to send
val rdd = ???
// Gather data and turn into string with newlines
val body = rdd.collect.mkString("\n")
// Open a connection
val url = new URL("http://www.example.com/resource")
val conn = url.openConnection.asInstanceOf[HttpURLConnection]
// Configure for POST request
conn.setDoOutput(true);
conn.setRequestMethod("POST");
val os = conn.getOutputStream;
os.write(input.getBytes);
os.flush;
A much more complete discussion of using java.net.URL can be found at this question. You could also use a Scala library to handle the ugly Java stuff for you, like akka-http or Dispatch.
Spark itself does not provide this functionality (it is not a general-purpose http client).
You might consider using some existing rest client library such as akka-http, spray or some other java/scala client library.
That said, you are by no means obliged to save your data to disk before operating on it. You could for example use collect() or foreach methods on your RDD in combination with your REST client library.

How to use Ember Adapter

Why the 'Todos.ApplicationAdapter = DS.FixtureAdapter.extend();' replace to '
Todos.ApplicationAdapter = DS.LSAdapter.extend({
namespace: "todos-emberjs"
});
' can be achieved local stores?
what's the meaning of 'namespace: "todos-emberjs"'?
There are how much kinds of adapters? And I should how to use them? How to define an adapter?
(Check out the picture here to see where ADAPTER component fits in)
I just went through EmberJS tutorial recently and from what I understood:
1)What are EmberJS adapters?
The adapters are objects that take care of communication between your application and a server. Whenever your application asks the store for a record that it doesn't have cached, it will ask the adapter for it. If you change a record and save it, the store will hand the record to the adapter to send the appropriate data to your server and confirm that the save was successful.
2)What types of EmberJS adapters are available?
Right now I am only aware of DS.RESTAdapter which is used by default by the store (it communicates with an HTTP server by transmitting JSON via XHR), DS.FixtureAdapter(something like in-memory storage which is not persistent) and DS.LSAdapter(something like local-storage which is persistent).
3)Why LSAdapter instead of FixtureAdapter in Todos tutorial?
FixtureAdapter stores data in-memory and thus whenever you refresh your page, the data gets reassigned to initial values. But LSAdapter is available on github which uses persistent storage to store and retrieve data, hence enabling you to retain all the changes even after you refresh your page.
4)Why namespace: "todos-emberjs"?
If your JSON API lives somewhere other than on the host root, you can set a prefix that will be added to all requests. For example, if your JSON APIs are available at /todo-emberjs/ you would want it to be used as a prefix to all the URLs that you are going to call. In that case, set namespace property to todo-emberjs.
(Hope it helps, loving EmberJS btw !)