I have two pieces of JSON that I've successfully decoded sequentially. I would like to take the new html_fragment and update my existing html_fragment. Generally this would be simple but my data structure is giving me difficulties:
type PostDataContainer
= PostDataContainer PostData
type alias PostData =
{ title : String
, comments : List Comment
}
type alias Comment =
{ comment_id : Int
, html_fragment : String
}
type alias CommentHtml =
{ id : Int
, html_fragment : String
}
I've just gotten CommentHtml and would like to update the existing html_fragment in Comment. This is what I have so far:
MergeCommentHtml commentHtmlData ->
case commentHtmlData of
Err err ->
Debug.log ("Error decoding CommentHtmlData" ++ toString err)
( mdl, Cmd.none )
Ok commentHtml ->
case mdl.maybePostDataContainer of
Just (PostDataContainer postData) ->
let
updatedCommentData = -- I dont know how to calculate this?
in
( { mdl | postData = { postData | comments = updatedCommentData } }, Cmd.none )
Note that commentHtml here is a List CommentHtml. Any thoughts on how to update my old comment.html_fragment with the new values in commentHtml?
Option 1:
just decode the data as it stands. When it's time to display it, arrange it appropriately via some function you write like rawJsonDataToNicerData.
Option 2:
Suppose you implement the following function:
-- given a new comment, and some PostData, return the new version of the PostData
updateData : CommentHtml -> PostData -> PostData
-- so now, assuming we can decode a CommentHtml with commentHtmlDeocder
-- we can do the following
dataUpdaterDecoder : Decoder (PostData -> PostData)
dataUpdaterDecoder
commentHtmlDecoder |> Decoder.andThen (\commentHtml -> updateData commentHtml)
Now wherever we were going to decode a commentHtmlDeocder we can decode a dataUpdaterDecoder instead, and use a bunch of these to update our data.
Here is an example of a relational data decoder in action using the idea above:
https://ellie-app.com/3KWmyJmMrDsa1
Given that commentHtmlData is a List according to a comment, I think the easiest approach is to convert it to a Dict keyed by id, then map over the existing comments looking for the comment_id in the dict. If it exists, replace html_fragment, if not then return the original unmodified:
let
commentHtmlDict =
commentHtmlData
|> List.map (\c -> (c.id, c))
|> Dict.fromList
updatedCommentData =
postData.comments
|> List.map (\comment ->
case Dict.get comment.comment_id commentHtmlDict of
Just commentHtml ->
{ comment | html_fragment = commentHtml.html_fragment }
Nothing ->
comment
)
Related
I need to generate a value with a different type from my passed type. This is the first time I write on ocaml-like, and for example, in a familiar me haskell I would use Data.Generics.
How I have understood I need to use decorator and ppx. I wrote simple example
let recordHandler = (loc: Location.t, _recFlag: rec_flag, _t: type_declaration, fields: list(label_declaration)) => {
let (module Builder) = Ast_builder.make(loc);
let test = [%str
let schema: Schema = { name: "", _type: String, properties: [] }
]
let moduleExpr = Builder.pmod_structure(test);
[%str
module S = [%m moduleExpr]
]
}
let str_gen = (~loc, ~path as _, (_rec: rec_flag, t: list(type_declaration))) => {
let t = List.hd(t)
switch t.ptype_kind {
| Ptype_record(fields) => recordHandler(loc, _rec, t, fields);
| _ => Location.raise_errorf(~loc, "schema is used only for records.");
};
};
let name = "my_schema";
let () = {
let str_type_decl = Deriving.Generator.make_noarg(str_gen);
Deriving.add(name, ~str_type_decl) |> Deriving.ignore;
};
And
open Ppxlib;
let _ = Driver.run_as_ppx_rewriter()
But in using in rescript code
module User = {
#deriving(my_schema)
type my_typ = {
foo: int,
};
};
I caught:
schema is not supported
. And I made myself sure me to connect it right when I had changed #deriving(my_schema) for #deriving(abcd) and #deriving(sschema).
I got different error
Ppxlib.Deriving: 'abcd' is not a supported type deriving generator.
And my last experiment was to copy past existing library deriving accessors .
ppx_accessor
I copied-pasted it and renamed for accessors_2. And I got same error such as experiment.
accessors_2 is not supported
Also I haven't found examples "ppx rescript". Can you please help me.
What am I doing wrong (ALL , I know)
I have found answer in the article
Dropping support for custom PPXes such as ppx_deriving (the deriving
attribute is now exclusively interpreted as bs.deriving)
I've got a working Apple Watch workout app. My metadata saves and all workout data flows to iPhone. I'm also able retrieve and display the data. But when i try to add arrays converted to ... json strings ... to metadata, the app crashes on save. Every time. I've tried numerous variations, always it's the same. Here's the latest code to crash... and it's perfectly fine.
GOOD CODE that works, but return string...
CRASHES with every HKWorkoutSession save.
func toJSON(array: [[String: Any]]) throws -> String {
let data = try JSONSerialization.data(withJSONObject: array, options: [])
return String(data: data, encoding: .utf8)!
}
NOW ... when my configuration class is converted with strings created using the function below, metadata saves just fine... and i'm back where i started. Wondering how to restore the [[String:Any]] arrays from String.
SAVES on WatchOS 3
This code creates a string from array of dictionaries.
What I'm needing help with is function to restore strings created using this function back into original form of [ [ String : Any ] ]
func joinedRepresentationOfArrayOfArrays(newArray: [[String : Any]]) -> String {
var newString = ""
for dictionary in newArray {
newString = newString.appending("[")
for (key, value) in dictionary {
newString = newString.appending("[\(key) : \(value), ")
}
newString = newString.appending("], ")
}
newString = newString.appending("], ")
return newString
}
I'm trying to save data in an image metadata in iOS/Swift3. It does not appear that CG will let you save out custom tags (is that true?) so I JSON encoded my dictionary and put the result as a string into the TIFF tag's ImageDescription. When I load the image and get the metadata back...
if let data = NSData(contentsOfFile:oneURL.path), let imgSrc = CGImageSourceCreateWithData(data, options as CFDictionary) {
let allmeta = CGImageSourceCopyPropertiesAtIndex(imgSrc, 0, options as CFDictionary) as? [String : AnyObject]
The allMeta contains (among other things):
▿ 0 : 2 elements
- key : ImageDescription
- value : {
"CameraOrientationW" : 0.1061191,
"CameraOrientationZ" : -0.01305595,
"CameraOrientationX" : 0.01319851,
"CameraOrientationY" : 0.9941801
}
Which has the JSON data, yay! So now I simply have to get the TIFF metadata, get the ImageDescription from that, and de-JSON it...
let tiffmeta = allmeta?["{TIFF}"]
if let tiffMeta = tiffmeta {
let descmeta = tiffMeta["ImageDescription"]
var descdata = descmeta?.data(usingEncoding: NSUTF8StringEncoding)!
let descdict = try? JSONSerialization.jsonObject(with: descdata, options: [])
But this will not compile. Xcode puts an error on the let descdata line:
Value of type 'MDLMaterialProperty??' has no member 'data'
I tried casting it to String on the line above, at which point it complains I didn't unwrap the optional MDLMaterialProperty.
Am I missing something obvious here?
So just to close this one, this appears to be a problem in the compiler. I made a number of minor changes to the syntax, nothing that had any actual effect on the code, and suddenly it decided the object was indeed a string.
I want to do a client-side js_of_ocaml application with a server in OCaml, with contraints described below, and I would like to know if the approach below is right or if there is a more efficient one. The server can sometimes send large quantities of data (> 30MB).
In order to make the communication between client and server safer and more efficient, I am sharing a type t in a .mli file like this :
type client_to_server =
| Say_Hello
| Do_something_with of int
type server_to_client =
| Ack
| Print of string * int
Then, this type is marshalled into a string and sent on the network. I am aware that on the client side, some types are missing (Int64.t).
Also, in a XMLHTTPRequest sent by the client, we want to receive more than one marshalled object from the server, and sometimes in a streaming mode (ie: process the marshal object received (if possible) during the loading state of the request, and not only during the done state).
These constraints force us to use the field responseText of the XMLHTTPRequest with the content-type application/octet-stream.
Moreover, when we get back the response from responseText, an encoding conversion is made because JavaScript's string are in UTF-16. But the marshalled object being binary data, we do what is necessary in order to retrieve our binary data (by overriding the charset with x-user-defined and by applying a mask on each character of the responseText string).
The server (HTTP server in OCaml) is doing something simple like this:
let process_request req =
let res = process_response req in
let s = Marshal.to_string res [] in
send s
However, on the client side, the actual JavaScript primitive of js_of_ocaml for caml_marshal_data_size needs an MlString. But in streaming mode, we don't want to convert the javascript's string in a MlString (which can iter on the full string), we prefer to do the size verification and unmarshalling (and the application of the mask for the encoding problem) only on the bytes read. Therefore, I have writen my own marshal primitives in javascript.
The client code for processing requests and responses is:
external marshal_total_size : Js.js_string Js.t -> int -> int = "my_marshal_total_size"
external marshal_from_string : Js.js_string Js.t -> int -> 'a = "my_marshal_from_string"
let apply (f:server_to_client -> unit) (str:Js.js_string Js.t) (ofs:int) : int =
let len = str##length in
let rec aux pos =
let tsize =
try Some (pos + My_primitives.marshal_total_size str pos)
with Failure _ -> None
in
match tsize with
| Some tsize when tsize <= len ->
let data = My_primitives.marshal_from_string str pos in
f data;
aux tsize
| _ -> pos
in
aux ofs
let reqcallback f req ofs =
match req##readyState, req##status with
| XmlHttpRequest.DONE, 200 ->
ofs := apply f req##responseText !ofs
| XmlHttpRequest.LOADING, 200 ->
ignore (apply f req##responseText !ofs)
| _, 200 -> ()
| _, i -> process_error i
let send (f:server_to_client -> unit) (order:client_to_server) =
let order = Marshal.to_string order [] in
let msg = Js.string (my_encode order) in (* Do some stuff *)
let req = XmlHttpRequest.create () in
req##_open(Js.string "POST", Js.string "/kernel", Js._true);
req##setRequestHeader(Js.string "Content-Type",
Js.string "application/octet-stream");
req##onreadystatechange <- Js.wrap_callback (reqcallback f req (ref 0));
req##overrideMimeType(Js.string "application/octet-stream; charset=x-user-defined");
req##send(Js.some msg)
And the primitives are:
//Provides: my_marshal_header_size
var my_marshal_header_size = 20;
//Provides: my_int_of_char
function my_int_of_char(s, i) {
return (s.charCodeAt(i) & 0xFF); // utf-16 char to 8 binary bit
}
//Provides: my_marshal_input_value_from_string
//Requires: my_int_of_char, caml_int64_float_of_bits, MlStringFromArray
//Requires: caml_int64_of_bytes, caml_marshal_constants, caml_failwith
var my_marshal_input_value_from_string = function () {
/* Quite the same thing but with a custom Reader which
will call my_int_of_char for each byte read */
}
//Provides: my_marshal_data_size
//Requires: caml_failwith, my_int_of_char
function my_marshal_data_size(s, ofs) {
function get32(s,i) {
return (my_int_of_char(s, i) << 24) | (my_int_of_char(s, i + 1) << 16) |
(my_int_of_char(s, i + 2) << 8) | (my_int_of_char(s, i + 3));
}
if (get32(s, ofs) != (0x8495A6BE|0))
caml_failwith("MyMarshal.data_size");
return (get32(s, ofs + 4));
}
//Provides: my_marshal_total_size
//Requires: my_marshal_data_size, my_marshal_header_size, caml_failwith
function my_marshal_total_size(s, ofs) {
if ( ofs < 0 || ofs > s.length - my_marshal_header_size )
caml_failwith("Invalid argument");
else return my_marshal_header_size + my_marshal_data_size(s, ofs);
}
Is this the most efficient way to transfer large OCaml values from server to client, or what would time- and space-efficient alternatives be?
Have you try to use EventSource https://developer.mozilla.org/en-US/docs/Web/API/EventSource
You could stream json data instead of marshaled data.
Json.unsafe_input should be faster than unmarshal.
class type eventSource =
object
method onmessage :
(eventSource Js.t, event Js.t -> unit) Js.meth_callback
Js.writeonly_prop
end
and event =
object
method data : Js.js_string Js.t Js.readonly_prop
method event : Js.js_string Js.t Js.readonly_prop
end
let eventSource : (Js.js_string Js.t -> eventSource Js.t) Js.constr =
Js.Unsafe.global##_EventSource
let send (f:server_to_client -> unit) (order:client_to_server) url_of_order =
let url = url_of_order order in
let es = jsnew eventSource(Js.string url) in
es##onmessage <- Js.wrap_callback (fun e ->
let d = Json.unsafe_input (e##data) in
f d);
()
On the server side, you then need to rely on deriving_json http://ocsigen.org/js_of_ocaml/2.3/api/Deriving_Json to serialize your data
type server_to_client =
| Ack
| Print of string * int
deriving (Json)
let process_request req =
let res = process_response req in
let data = Json_server_to_client.to_string res in
send data
note1: Deriving_json serialize ocaml value to json using the internal representation of values in js_of_ocaml. Json.unsafe_input is a fast deserializer for Deriving_json that rely on browser-native JSON support.
note2: Deriving_json and Json.unsafe_input take care of ocaml string encoding
I use the Exchange webservices to extract attachments from exchange mailserver.
When i call the code on linux with mono a certain text attachment contain some mixed-up strings.
like so
"sam winglin vz" becomes "sainglin vz" -> so it is missing "m w".
I see this about 3 times in a 150kb file. 3 bytes are missing in the linux output vs the windows output.
When i extract it from visual studio the text attachment is perfect.
It is like this example
Save attachments from exchange inbox
Any idea in what direction i should look to fix this?
Code:
#r "Microsoft.Exchange.WebServices.dll"
open Microsoft
open Microsoft.Exchange.WebServices.Data
open System
open System.Net
type PgzExchangeService(url,user,password) =
let service = new ExchangeService(ExchangeVersion.Exchange2007_SP1,
TimeZoneInfo.CreateCustomTimeZone("Central Standard Time",new TimeSpan(-6, 0, 0),"(GMT-06:00) Central Time (US & Canada)","Central Standard Time"))
do
ServicePointManager.ServerCertificateValidationCallback <- ( fun _ _ _ _ -> true )
service.Url <- new Uri(url)
service.Credentials <- new WebCredentials(user, password, "domain")
member this.Service with get() = service
member this.InboxItems = this.Service.FindItems(WellKnownFolderName.Inbox, new ItemView(10))
member this.GetFileAttachments ( item : Item ) =
let emailMessage =
EmailMessage.Bind( this.Service,
item.Id,
new PropertySet(BasePropertySet.IdOnly, ItemSchema.Attachments))
item, emailMessage.Attachments |> Seq.choose (fun attachment -> match box attachment with
| :? FileAttachment as x -> Some(x) | _ -> None)
let mailAtdomain = new PgzExchangeService("https://xx.xx.XX.XX/EWS/Exchange.asmx", "user", "passw")
let printsave (item : Item ,att : seq<FileAttachment>) =
if (Seq.length att) > 0 then
printfn "%A - saving %i attachments" item.Subject (Seq.length att)
att |> Seq.iter ( fun attachment -> printfn "%A" attachment.Name
attachment.Load(#"/tmp/test/" + attachment.Name ) )
// filter so we only have items with attachements and ...
let itemsWithAttachments = mailAtdomain.InboxItems
|> Seq.map mailAtdomain.GetFileAttachments
|> Seq.iter printsave
The code doesn't run on Windows with mono due to a bug in TimeZoneInfo
This sample code runs on linux but not on windows. because of the TimeZoneInfo bug.
But with this the code that works on linux to extract attachments.
Try csv attachments and see if the result is the same. i loose data ! about 3 bytes every somemany lines
mail me if you need the sample csv attachment that gives the problem
Here is a c# version that i used for testing. Running from VS2010 it works perfect but on linux with mono the attachment has wrong size , some bytes are missing?!.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Microsoft.Exchange.WebServices.Data;
using System.Net;
namespace Exchange_SDP_Attachment_Extracter
{
public class PgzExchangeService
{
public void Extract()
{
ExchangeService service = new ExchangeService (ExchangeVersion.Exchange2007_SP1,TimeZoneInfo.Local);
service.Credentials = new NetworkCredential("user", "pass", "domain");
service.Url = new Uri("https://xx.xx.xx.xx/EWS/Exchange.asmx");
ServicePointManager.ServerCertificateValidationCallback = delegate { return true; };
FindItemsResults<Item> findResults = service.FindItems(WellKnownFolderName.Inbox, new ItemView(10));
foreach (Item item in findResults.Items)
{
EmailMessage e = EmailMessage.Bind
(service,
item.Id,
new PropertySet(BasePropertySet.IdOnly, ItemSchema.Attachments));
foreach ( Attachment att in e.Attachments )
{
if (att is FileAttachment)
{
FileAttachment fileAttachment = (FileAttachment)att;
fileAttachment.Load(#"/tmp/testsdp/" + fileAttachment.Name);
}
}
}
}
}
class Program
{
static void Main(string[] args)
{
PgzExchangeService pgz = new PgzExchangeService();
pgz.Extract();
}
}
}
My suggestion would be to try examining the text attachment with a hex editor. There has be something unusual about those three occurrences. You need to find out what those three lines have in common before any of us can recommend a course of action to you.