Put in double quotes all between two patterns in every line - regex

I have ndjson file, with strings like this:
{"created": "2016-03-08 00:00:00 UTC", "changed": "2016-03-08 08:51:56 UTC", "rev": 28990, "status": 1, "user": [{"user_id": null, "name": null, "loyaltyCard": 123456789012}], "id": "26680533564", "tax": null, "products": [{"price": 289, "quantity": 1, "coupon": null, "id": "4122"}], "shipping": 0.0}
I need to take in double quotes values of "loyaltyCard", considering, that it could be digits, letters (cyrillic also) and anything else.
Expecting to see something like:
UTC", "rev": 280, "status": 1, "user": [{"user_id": null, "name": null, "loyaltyCard": "123456789012"}], "id": "26680533564", "tax": null, "products": [{"price": 289, "quantity": 1, "coupon": null, "id": "4122"}], "shipping": 0.0}
UTC", "rev": 56990, "status": 1, "user": [{"user_id": 543445, "name": null, "loyaltyCard": "1233bla456bla"}], "id": "5454580534", "tax": null, "products": [{"price": 869, "quantity": 2, "coupon": null, "id": "86854"}], "shipping": 0.0}

If you want to wrap up loyaltyCard value in double quotes, you can use a regex like this:
(loyaltyCard": )([^}]*)\}
With a replacement string:
\1"\2"}
Regex demo
Update for Vim: to find/replace in vim you can use:
:%s/\(loyaltyCard": \)\([^}]*\)\}/\1"\2"}/
^ ^----------------------------^ ^-----^
| + Pattern1 (w/capture groups) + Pattern 2 (w/ group refs)
+ substitute cmd

s/: (\d+)/: "$1"/ might do the trick

This is one of the way:
m/"loyaltyCard"\:\s*"([^"]*)"/g;
If you want to give a escape sequence for quotes your wish.

Related

how to put two data in a title?

I want to display a list of products that uses an API to retrieve the data.
the array I use to display data looks like this :
Array [
Object {
"amount": 2671.25,
"balance": 0,
"client_id": 1,
"created_at": "2020-05-06T17:42:26Z",
"discount": 0,
"discount_type": 0,
"id": 19,
"items": Array [
Object {
"cost": 2400,
"currency": "EUR",
"description": "",
"name": "Apple MacBook '' LED 500 Go SSD 32 Go",
"product_id": 5,
"quantity": 1,
"tax_rate_id": 1,
},
Object {
"cost": 54.25,
"currency": "EUR",
"description": "blablabla",
"product_id": 2,
"quantity": 5,
"tax_rate_id": 4,
},
],
"po_number": "",
"public_notes": "TEST 6 : Acomptes",
"quote_date": "2020-05-06",
"quote_number": "D0019",
"quote_status": 40,
"terms": "",
"updated_at": "2020-05-06T18:08:06Z",
},
It works, it's great. But I would like to improve the display.
For example, when I unroll my lilste to get the detail, I have the price.
<List.Item title={item.amount}/>
Ok, it works, cool, but I would like to add the currency (€ or other) How can I do it? I tried this but it doesn't work:
<List.Item title={item.amount}{item.currency}/>
<List.Item title={item.amount} + {item.currency}/>
<List.Item title={item.amount} + '€'/>
<List.Item title='{item.amount} + €'/>
Use Template Literals
Template literals are string literals allowing embedded expressions.
It takes that form:
`string text ${expression} string text`
So you could do something like
<List.Item title={`${item.amount} ${item.currency}`} />

How to extract the "ZTlhMDNhOGVkZV8xNTY3Mjk3MjAw" value from the respective response in JMeter? This value "ZTlhMDNhOGVkZV8xNTY3Mjk3MjAw" is dynamic

Response:
{
"service_name": "signup",
"message": "Sign Up has been done successfully",
"global_error": "",
"error": [],
"data": {
"session_key": "8f29d7c93e7089841208e94a7d98fc22",
"user_profile": {
"user_id": 65,
"user_unique_id": "e9a03a8ede",
"dob": "Dec 06, 1998",
"first_name": "FC7155313",
"last_name": "FC1791398",
"user_name": "FCwqim178",
"email": "fc_slekjbp#mailinator.com",
"phone_no": "3362239492",
"balance": "0",
"status": "2",
"image": "http://dummy.projects.com/app/assets/img/default_user.png",
"currency": "$",
"profile_status": 1,
"require_otp": false,
"existing_user": 0,
"master_country_id": null,
"master_state_id": "3919"
},
"verification_link": "http://dummy.projects.com/activation/ZTlhMDNhOGVkZV8xNTY3Mjk3MjAw",
"send_email_otp": false,
"send_phone_otp": false
},
"response_code": 200
}
I am using JMeter and want to pass "ZTlhMDNhOGVkZV8xNTY3Mjk3MjAw" value in next API.
This value "ZTlhMDNhOGVkZV8xNTY3Mjk3MjAw" is generating dynamic for each new user registration.
But my regex is not working.
The regular expression that I wrote:
,"verification_link":"http://dummy.projects.com/activation/(.+?)","send_email_otp":
Template: $1$
Match No.: 1
Your response seems to be a JSON entity therefore there is a high change that it looks like:
{
"service_name": "signup",
"message": "Sign Up has been done successfully",
"global_error": "",
"error": [
],
"data": {
"session_key": "8f29d7c93e7089841208e94a7d98fc22",
"user_profile": {
"user_id": 65,
"user_unique_id": "e9a03a8ede",
"dob": "Dec 06, 1998",
"first_name": "FC7155313",
"last_name": "FC1791398",
"user_name": "FCwqim178",
"email": "fc_slekjbp#mailinator.com",
"phone_no": "3362239492",
"balance": "0",
"status": "2",
"image": "http://dummy.projects.com/app/assets/img/default_user.png",
"currency": "$",
"profile_status": 1,
"require_otp": false,
"existing_user": 0,
"master_country_id": null,
"master_state_id": "3919"
},
"verification_link": "http://dummy.projects.com/activation/ZTlhMDNhOGVkZV8xNTY3Mjk3MjAw",
"send_email_otp": false,
"send_phone_otp": false
},
"response_code": 200
}
so this "send_email_otp" bit can easily go to the next line and your regular expression will not match anything in this situation.
I would recommend amending your regex to look something like:
"verification_link":\s?"http://dummy.projects.com/activation/(\w+)"
Demo:
References:
JMeter: Regular Expressions
Using RegEx (Regular Expression Extractor) with JMeter
Perl 5 Regex Cheat sheet
Your regex matches only the first part of your URL, and not the part that you actually want. Try this instead:
http:\/\/dummy\.projects\.com\/activation\/(.+)\",
Regex Demo
The (.+?) part matches between 1 and unlimited time, as few times as possible (called lazy match, indicated by +?). So you match the first character of whatever comes after activation/ and then stop. You do not get the entire value, as you want.

Egrep special expressions like \w in bracket expressions []

I am trying to use extended grep to extract data from a JSON. The regex I use is functional on my regexr instance, but for some reason it doesn't work in bash.
I tried many things, notably the bare double dash and various minor edits to the regex for escaping.
#!/bin/bash
networks='{ "networks": [ { "admin_state_up": true, "availability_zone_hints": [], "availability_zones": [], "created_at": "2019-03-12T23:45:13Z", "description": "", "id": "7188504a-72cb-4590-a9b0-414732017837", "ipv4_address_scope": null, "ipv6_address_scope": null, "is_default": false, "mtu": 1450, "name": "BLUE", "port_security_enabled": true, "project_id": "187d635aec4c43fe8e8918afb3a5c82e", "provider:network_type": "vxlan", "provider:physical_network": null, "provider:segmentation_id": 86, "revision_number": 2, "router:external": false, "shared": false, "status": "ACTIVE", "subnets": [], "tags": [], "tenant_id": "187d635aec4c43fe8e8918afb3a5c82e", "updated_at": "2019-03-12T23:45:13Z" }, { "admin_state_up": true, "availability_zone_hints": [], "availability_zones": [], "created_at": "2019-03-12T23:45:13Z", "description": "", "id": "ed82083f-0a7c-4322-a4fb-de8db23e2bae", "ipv4_address_scope": null, "ipv6_address_scope": null, "is_default": false, "mtu": 1450, "name": "RED", "port_security_enabled": true, "project_id": "187d635aec4c43fe8e8918afb3a5c82e", "provider:network_type": "vxlan", "provider:physical_network": null, "provider:segmentation_id": 108, "revision_number": 2, "router:external": false, "shared": false, "status": "ACTIVE", "subnets": [], "tags": [], "tenant_id": "187d635aec4c43fe8e8918afb3a5c82e", "updated_at": "2019-03-12T23:45:13Z" }, { "admin_state_up": true, "availability_zone_hints": [], "availability_zones": [], "created_at": "2019-03-12T23:45:13Z", "description": "", "id": "1eb6647e-869e-4e83-9468-43e2c320bccc", "ipv4_address_scope": null, "ipv6_address_scope": null, "is_default": false, "mtu": 1450, "name": "public", "port_security_enabled": true, "project_id": "187d635aec4c43fe8e8918afb3a5c82e", "provider:network_type": "vxlan", "provider:physical_network": null, "provider:segmentation_id": 32, "revision_number": 2, "router:external": false, "shared": false, "status": "ACTIVE", "subnets": [], "tags": [], "tenant_id": "187d635aec4c43fe8e8918afb3a5c82e", "updated_at": "2019-03-12T23:45:13Z" } ] }'
result=`echo $networks | grep -oE '"(id|name)": "([\w+-]+)"'`
echo $result
The aforementioned code doesn't work but if I switch to the following regex, it works. I just need to add extraction for id field too to be able to extract ids and names using \2 back reference (group 2)
grep -oE '"(id|name)": "(\w+)"'
Can you help me understand why the script doesn't work?
Full formatted JSON
{
"networks": [{
"admin_state_up": true,
"availability_zone_hints": [],
"availability_zones": [],
"created_at": "2019-03-12T23:45:13Z",
"description": "",
"id": "7188504a-72cb-4590-a9b0-414732017837",
"ipv4_address_scope": null,
"ipv6_address_scope": null,
"is_default": false,
"mtu": 1450,
"name": "BLUE",
"port_security_enabled": true,
"project_id": "187d635aec4c43fe8e8918afb3a5c82e",
"provider:network_type": "vxlan",
"provider:physical_network": null,
"provider:segmentation_id": 86,
"revision_number": 2,
"router:external": false,
"shared": false,
"status": "ACTIVE",
"subnets": [],
"tags": [],
"tenant_id": "187d635aec4c43fe8e8918afb3a5c82e",
"updated_at": "2019-03-12T23:45:13Z"
}, {
"admin_state_up": true,
"availability_zone_hints": [],
"availability_zones": [],
"created_at": "2019-03-12T23:45:13Z",
"description": "",
"id": "ed82083f-0a7c-4322-a4fb-de8db23e2bae",
"ipv4_address_scope": null,
"ipv6_address_scope": null,
"is_default": false,
"mtu": 1450,
"name": "RED",
"port_security_enabled": true,
"project_id": "187d635aec4c43fe8e8918afb3a5c82e",
"provider:network_type": "vxlan",
"provider:physical_network": null,
"provider:segmentation_id": 108,
"revision_number": 2,
"router:external": false,
"shared": false,
"status": "ACTIVE",
"subnets": [],
"tags": [],
"tenant_id": "187d635aec4c43fe8e8918afb3a5c82e",
"updated_at": "2019-03-12T23:45:13Z"
}, {
"admin_state_up": true,
"availability_zone_hints": [],
"availability_zones": [],
"created_at": "2019-03-12T23:45:13Z",
"description": "",
"id": "1eb6647e-869e-4e83-9468-43e2c320bccc",
"ipv4_address_scope": null,
"ipv6_address_scope": null,
"is_default": false,
"mtu": 1450,
"name": "public",
"port_security_enabled": true,
"project_id": "187d635aec4c43fe8e8918afb3a5c82e",
"provider:network_type": "vxlan",
"provider:physical_network": null,
"provider:segmentation_id": 32,
"revision_number": 2,
"router:external": false,
"shared": false,
"status": "ACTIVE",
"subnets": [],
"tags": [],
"tenant_id": "187d635aec4c43fe8e8918afb3a5c82e",
"updated_at": "2019-03-12T23:45:13Z"
}]
}
According to man grep:
The Backslash Character and Special Expressions
The symbol \w is a synonym for [[:alnum:]] and \W is a synonym for [^[:alnum:]]. ... A bracket expression is a list of characters enclosed by [ and ]. ... To include a literal ] place it first in the list. Similarly, to include a literal ^ place it anywhere but first. Finally, to include a literal - place it last.
Basically, \w is literally replaced by those characters when evaluated, giving you "([[[:alnum:]]+-]+)", which in a US standard locale gives you "([[a-zA-Z0-9]+-]+)".
Since a bracket expression is truncated by the first ] it sees (unless it is the first element of a bracket expression), the group is only [[[:alnum:]]+, or "1 or more of a digit, letter, and [. This expression is followed by -]+, meaning "exactly one hyphen and one or more ]". This is obviously pretty terrible.
If you try
echo $networks | grep -oE '"(id|name)": "([[:alnum:]+-]+)"'
I.e., \w without the outer bracket expression, the relevant part means "a group (surrounded by ") comprised of one or more digits, letters, hyphens, and plus signs", which outputs:
"id": "7188504a-72cb-4590-a9b0-414732017837"
"name": "BLUE"
"id": "ed82083f-0a7c-4322-a4fb-de8db23e2bae"
"name": "RED"
"id": "1eb6647e-869e-4e83-9468-43e2c320bccc"
"name": "public"
Using PERL (-P) instead of Extended (-E) regexp, looks like the \w is interpreted as expected, without escaping issue: note the -oP
result=$( echo $networks | grep -oP '"(id|name)": "([\w+-]+)"' ) ;
echo $result
"id": "7188504a-72cb-4590-a9b0-414732017837" "name": "BLUE" "id": "ed82083f-0a7c-4322-a4fb-de8db23e2bae" "name": "RED" "id": "1eb6647e-869e-4e83-9468-43e2c320bccc" "name": "public"
As a workaround (it does not resolve the "escaping \w issue)
result=$( echo $networks | grep -oE '"(id|name)": "([a-zA-Z_+-]+)"' ) ;
echo $result
Prints me:
"name": "BLUE" "name": "RED" "name": "public"
Note: prefer using $( ) syntax to execute sub shells rather than the backtick.

Twitter streaming API, where to find originator's name?

I am using Python to stream Twitter's Tweets via API. For example, the word "car" generates the following results:
{
"created_at": "Fri Sep 05 00:15:32 +0000 2014",
"id": 507683414255108096,
"id_str": "507683414255108096",
"text": "I put 'or nah' in my cousins car and Brenda & I are singing along a",
"source": "\u003ca href=\"http:\/\/www.cloudhopper.com\/\" rel=\"nofollow\"\u003eCloudhopper\u003c\/a\u003e",
"truncated": false,
"in_reply_to_status_id": null,
"in_reply_to_status_id_str": null,
"in_reply_to_user_id": null,
"in_reply_to_user_id_str": null,
"in_reply_to_screen_name": null,
"user": {
"id": 84729292,
"id_str": "84729292",
"name": "Tracy Ochoa",
"screen_name": "TracyMayy",
"location": "",
"url": "http:\/\/whythefuckinfucknot.tumblr.com",
"description": "New York 16 Instagram - TracyMayy Tumblr - http:\/\/whythefuckinfucknot.tumblr.com http:\/\/ask.fm\/tracyochoa",
"protected": false,
"verified": false,
"followers_count": 1045,
"friends_count": 453,
"listed_count": 22,
"favourites_count": 46035,
"statuses_count": 44720,
"created_at": "Sat Oct 24 00:42:23 +0000 2009",
"utc_offset": -14400,
"time_zone": "Eastern Time (US & Canada)",
"geo_enabled": true,
"lang": "en",
"contributors_enabled": false,
"is_translator": false,
"profile_background_color": "000000",
"profile_background_image_url": "http:\/\/pbs.twimg.com\/profile_background_images\/497215144607236096\/AharMORU.png",
"profile_background_image_url_https": "https:\/\/pbs.twimg.com\/profile_background_images\/497215144607236096\/AharMORU.png",
"profile_background_tile": true,
"profile_link_color": "000000",
"profile_sidebar_border_color": "FFFFFF",
"profile_sidebar_fill_color": "09B6D9",
"profile_text_color": "050505",
"profile_use_background_image": true,
"profile_image_url": "http:\/\/pbs.twimg.com\/profile_images\/504330955637919745\/JAHlbkiS_normal.jpeg",
"profile_image_url_https": "https:\/\/pbs.twimg.com\/profile_images\/504330955637919745\/JAHlbkiS_normal.jpeg",
"profile_banner_url": "https:\/\/pbs.twimg.com\/profile_banners\/84729292\/1409681919",
"default_profile": false,
"default_profile_image": false,
"following": null,
"follow_request_sent": null,
"notifications": null
},
"geo": null,
"coordinates": null,
"place": null,
"contributors": null,
"retweet_count": 0,
"favorite_count": 0,
"entities": {
"hashtags": [],
"trends": [],
"urls": [],
"user_mentions": [],
"symbols": []
},
"favorited": false,
"retweeted": false,
"possibly_sensitive": false,
"filter_level": "medium",
"lang": "en",
"timestamp_ms": "1409876132921"
}
It looks to me the Twitter user who wrote this tweet has an id "507683414255108096", is there a way to export Twitter's API Tweets with the username of the actual Twitter users who write the corresponding Tweets?
If not through API, do I need to actually follow the people to get streams of Tweets with usernames? Or is there another way around?

JSON formatting error using Boost JSON parser

I'm attempting to use Boost to read a JSON file from my Firefox configuration folder called sessionstore.js, where the information on the current/last Firefox session is saved for purposes of recovery. I've written a program based on the XML-based tutorial from the Boost website, simply swapping out the XML parts for the JSON parts, which can be seen below
#include <boost/property_tree/ptree.hpp>
#include <boost/property_tree/json_parser.hpp>
#include <boost/foreach.hpp>
#include <string>
#include <set>
#include <exception>
using boost::property_tree::ptree;
using namespace std;
const string FILENAME = "sessionstore.js";
const string WINDOW_TAG = "windows";
struct session_settings
{
void load (const string &FILENAME);
};
void session_settings::load (const string &FILENAME)
{
ptree pt;
read_json (FILENAME, pt);
}
int main()
{
try
{
session_settings Settings;
Settings.load(FILENAME);
}
catch (exception &e)
{
cout << "Error: " << e.what() << endl;
}
return 0;
}
The contents of the JSON file I'm trying to read are
{"windows":[{"tabs":[{"entries":[{"url":"about:home","title":"Mozilla Firefox Start Page","ID":5,"docshellID":11,"owner_b64":"NhAra3tiRRqhyKDUVsktxQAAAAAAAAAAwAAAAAAAAEYAAQAAAAAAAS8nfAAOr03buTZBMmukiq4HoizADOUR05MxABBLoP1AAAAAAAVhYm91dAAAAARob21l4NodcC97EdOM0ABgsPwUoweiLMAM5RHTkzEAEEug/UAAAAAADm1vei1zYWZlLWFib3V0AAAABGhvbWUAAAA=","docIdentifier":5},{"url":"http://www.google.co.uk/","title":"Google","ID":6,"docshellID":11,"docIdentifier":6,"children":[{"url":"about:blank","ID":7,"docshellID":12,"owner_b64":"NhAra3tiRRqhyKDUVsktxQAAAAAAAAAAwAAAAAAAAEYAAQAAAAAAAd6UctCANBHTk5kAEEug/UAHoizADOUR05MxABBLoP1AAAAAAv////8AAABQAQAAABhodHRwOi8vd3d3Lmdvb2dsZS5jby51ay8AAAAAAAAABAAAAAcAAAAQAAAAB/////8AAAAH/////wAAAAcAAAAQAAAAFwAAAAEAAAAXAAAAAQAAABcAAAABAAAAGAAAAAAAAAAY/////wAAABf/////AAAAF/////8AAAAX/////wEAAAAAAAAAAAABAAA=","docIdentifier":7,"scroll":"0,0"}],"formdata":{"#csi":"1","#hcache":"{\"BInSTfL-EtSt8QOl24nrCg\":[[69,{}],[14,{}],[60,{}],[81,{\"persisted\":true}],[42,{}],[43,{}],[83,{}],[95,{\"kfe\":{\"kfeHost\":\"clients1.google.co.uk\",\"kfeUrlPrefix\":\"/webpagethumbnail?r=2&f=2&s=300:585&query=&hl=en&gl=uk\",\"maxPrefetchConnections\":2,\"prefetch\":90,\"slowConnection\":false},\"logging\":{\"csiFraction\":0.05,\"gen204Fraction\":0.05},\"msgs\":{\"loading\":\"Still loading...\",\"mute\":\"Mute\",\"noPreview\":\"Preview not available\",\"sound\":\"Sound:\",\"soundOff\":\"off\",\"soundOn\":\"on\",\"unmute\":\"Unmute\"},\"pb\":{\"desiredHeight\":585,\"desiredWidth\":300,\"minHeight\":200,\"minWidth\":300},\"time\":{\"hoverClose\":300,\"hoverModeTimeout\":60,\"hoverOpen\":125,\"loading\":100,\"longHoverOpen\":725,\"prefetchOnLoad\":3000,\"timeout\":2500}}],[78,{}],[25,{\"m\":{\"bks\":true,\"blg\":true,\"dsc\":true,\"evn\":true,\"frm\":true,\"isch\":true,\"klg\":true,\"mbl\":true,\"nws\":true,\"plcs\":true,\"ppl\":true,\"prc\":true,\"pts\":true,\"rcp\":true,\"shop\":true,\"vid\":true},\"t\":null}],[64,{}],[105,{}],[22,{\"m_errors\":{\"32\":\"Sorry, no more results to show.\",\"default\":\"<font color=red>Error:</font> The server could not complete your request. Try again in 30 seconds.\"},\"m_tip\":\"Click for more information\"}],[77,{}],[84,{}],[99,{}],[29,{\"mcr\":5}],[92,{\"avgTtfc\":2000,\"fd\":1000,\"fl\":true,\"focus\":true,\"hpt\":250,\"kn\":true,\"mds\":\"clir,clue,dfn,evn,frim,klg,prc,rl,show,sp,sts,ww,mbl_he,mbl_hs,mbl_re,mbl_rs,mbl_sv,isch\",\"msg\":{\"dym\":\"Did you mean:\",\"gs\":\"Google Search\",\"kntt\":\"Use the up and down arrow keys to select each result. Press Enter to go to the selection.\",\"sif\":\"Search instead for\",\"srf\":\"Showing results for\"},\"odef\":true,\"ophe\":true,\"pq\":true,\"rpt\":41,\"tct\":\" ?\",\"tdur\":50}],[24,{}],[38,{}]]}"},"scroll":"0,0"}],"index":2,"hidden":false,"attributes":{"image":"http://www.google.co.uk/favicon.ico"},"storage":{"http://www.google.co.uk":{"web-v":"12_c9c918f0"}}}],"selected":1,"_closedTabs":[],"width":994,"height":688,"screenX":1650,"screenY":24,"sizemode":"normal","title":"Google"}],"selectedWindow":0,"_closedWindows":[{"tabs":[{"entries":[{"url":"about:home","title":"Mozilla Firefox Start Page","ID":0,"docshellID":5,"owner_b64":"NhAra3tiRRqhyKDUVsktxQAAAAAAAAAAwAAAAAAAAEYAAQAAAAAAAS8nfAAOr03buTZBMmukiq4HoizADOUR05MxABBLoP1AAAAAAAVhYm91dAAAAARob21l4NodcC97EdOM0ABgsPwUoweiLMAM5RHTkzEAEEug/UAAAAAADm1vei1zYWZlLWFib3V0AAAABGhvbWUAAAA="},{"url":"http://www.facebook.com/","title":"Welcome to Facebook - Log In, Sign Up or Learn More","ID":1,"docshellID":5,"docIdentifier":1,"formdata":{"//xhtml:div[#id='reg_form_box']/xhtml:table/xhtml:tbody/xhtml:tr[6]/xhtml:td[2]/xhtml:div/xhtml:div/xhtml:select":0,"//xhtml:div[#id='reg_form_box']/xhtml:table/xhtml:tbody/xhtml:tr[6]/xhtml:td[2]/xhtml:div/xhtml:div/xhtml:select[2]":0,"#sex":0,"#birthday_month":0,"#birthday_day":0,"#birthday_year":0},"scroll":"0,0"}],"index":2,"hidden":false,"attributes":{"image":"http://www.facebook.com/favicon.ico"}},{"entries":[{"url":"http://twitter.com/","title":"Twitter","ID":3,"docshellID":6,"docIdentifier":3,"children":[{"url":"http://api.twitter.com/receiver.html","ID":4,"docshellID":7,"referrer":"http://twitter.com/","docIdentifier":4,"scroll":"0,0"}],"formdata":{},"scroll":"0,0"}],"index":1,"hidden":false,"attributes":{"image":"http://twitter.com/phoenix/favicon.ico"}}],"selected":2,"_closedTabs":[],"width":994,"height":688,"screenX":1366,"screenY":307,"sizemode":"normal","cookies":[{"host":".facebook.com","value":"J4-69","path":"/","name":"lsd"},{"host":".facebook.com","value":"http%3A%2F%2Fwww.facebook.com%2F","path":"/","name":"reg_fb_gate"},{"host":".facebook.com","value":"http%3A%2F%2Fwww.facebook.com%2F","path":"/","name":"reg_fb_ref"},{"host":".facebook.com","value":"994x624","path":"/","name":"wd"},{"host":".twitter.com","value":"43838368","path":"/","name":"__utmc"},{"host":"twitter.com","value":"4bfz%2B%2BmebEkRkMWFCXm%2FCUOsvDoVeFTl","path":"/","name":"original_referer"},{"host":"scribe.twitter.com","value":"4bfz%2B%2BmebEkRkMWFCXm%2FCUOsvDoVeFTl","path":"/","name":"original_referer"},{"host":".twitter.com","value":"BAh7CToPY3JlYXRlZF9hdGwrCDoVZ%252F4vAToMY3NyZl9pZCIlODE2MGI1ZjJh%250AYmViNDMwODMxNDlkN2U5ZDg5Yjk4ZmU6B2lkIiU2N2I4YjdmNGExNWFkNzlk%250AODI0MDVjMGM1NmMzYjVhYSIKZmxhc2hJQzonQWN0aW9uQ29udHJvbGxlcjo6%250ARmxhc2g6OkZsYXNoSGFzaHsABjoKQHVzZWR7AA%253D%253D--8b0d751e9774c5cfaa61fdec567cb782aa8757dd","path":"/","name":"_twitter_sess","httponly":true},{"host":".twitter.com","value":"43838368","path":"/","name":"__utmc"},{"host":"twitter.com","value":"4bfz%2B%2BmebEkRkMWFCXm%2FCUOsvDoVeFTl","path":"/","name":"original_referer"},{"host":"scribe.twitter.com","value":"4bfz%2B%2BmebEkRkMWFCXm%2FCUOsvDoVeFTl","path":"/","name":"original_referer"},{"host":".twitter.com","value":"BAh7CToPY3JlYXRlZF9hdGwrCDoVZ%252F4vAToMY3NyZl9pZCIlODE2MGI1ZjJh%250AYmViNDMwODMxNDlkN2U5ZDg5Yjk4ZmU6B2lkIiU2N2I4YjdmNGExNWFkNzlk%250AODI0MDVjMGM1NmMzYjVhYSIKZmxhc2hJQzonQWN0aW9uQ29udHJvbGxlcjo6%250ARmxhc2g6OkZsYXNoSGFzaHsABjoKQHVzZWR7AA%253D%253D--8b0d751e9774c5cfaa61fdec567cb782aa8757dd","path":"/","name":"_twitter_sess","httponly":true}],"title":"Twitter"}],"session":{"state":"stopped","lastUpdate":1305658398727}}
and when I tried to load that with my program I got the error
Error: sessionstore.js(1): expected value
Since the file is formatted all on a single line, this meant the error could be anywhere in the file, so I ran it though a Javascript beautifier, keeping the default options, and pasted the results back into the original file and executed the program.
The formatted JSON is
{
"windows": [{
"tabs": [{
"entries": [{
"url": "about:home",
"title": "Mozilla Firefox Start Page",
"ID": 5,
"docshellID": 11,
"owner_b64": "NhAra3tiRRqhyKDUVsktxQAAAAAAAAAAwAAAAAAAAEYAAQAAAAAAAS8nfAAOr03buTZBMmukiq4HoizADOUR05MxABBLoP1AAAAAAAVhYm91dAAAAARob21l4NodcC97EdOM0ABgsPwUoweiLMAM5RHTkzEAEEug/UAAAAAADm1vei1zYWZlLWFib3V0AAAABGhvbWUAAAA=",
"docIdentifier": 5
}, {
"url": "http://www.google.co.uk/",
"title": "Google",
"ID": 6,
"docshellID": 11,
"docIdentifier": 6,
"children": [{
"url": "about:blank",
"ID": 7,
"docshellID": 12,
"owner_b64": "NhAra3tiRRqhyKDUVsktxQAAAAAAAAAAwAAAAAAAAEYAAQAAAAAAAd6UctCANBHTk5kAEEug/UAHoizADOUR05MxABBLoP1AAAAAAv////8AAABQAQAAABhodHRwOi8vd3d3Lmdvb2dsZS5jby51ay8AAAAAAAAABAAAAAcAAAAQAAAAB/////8AAAAH/////wAAAAcAAAAQAAAAFwAAAAEAAAAXAAAAAQAAABcAAAABAAAAGAAAAAAAAAAY/////wAAABf/////AAAAF/////8AAAAX/////wEAAAAAAAAAAAABAAA=",
"docIdentifier": 7,
"scroll": "0,0"
}],
"formdata": {
"#csi": "1",
"#hcache": "{\"BInSTfL-EtSt8QOl24nrCg\":[[69,{}],[14,{}],[60,{}],[81,{\"persisted\":true}],[42,{}],[43,{}],[83,{}],[95,{\"kfe\":{\"kfeHost\":\"clients1.google.co.uk\",\"kfeUrlPrefix\":\"/webpagethumbnail?r=2&f=2&s=300:585&query=&hl=en&gl=uk\",\"maxPrefetchConnections\":2,\"prefetch\":90,\"slowConnection\":false},\"logging\":{\"csiFraction\":0.05,\"gen204Fraction\":0.05},\"msgs\":{\"loading\":\"Still loading...\",\"mute\":\"Mute\",\"noPreview\":\"Preview not available\",\"sound\":\"Sound:\",\"soundOff\":\"off\",\"soundOn\":\"on\",\"unmute\":\"Unmute\"},\"pb\":{\"desiredHeight\":585,\"desiredWidth\":300,\"minHeight\":200,\"minWidth\":300},\"time\":{\"hoverClose\":300,\"hoverModeTimeout\":60,\"hoverOpen\":125,\"loading\":100,\"longHoverOpen\":725,\"prefetchOnLoad\":3000,\"timeout\":2500}}],[78,{}],[25,{\"m\":{\"bks\":true,\"blg\":true,\"dsc\":true,\"evn\":true,\"frm\":true,\"isch\":true,\"klg\":true,\"mbl\":true,\"nws\":true,\"plcs\":true,\"ppl\":true,\"prc\":true,\"pts\":true,\"rcp\":true,\"shop\":true,\"vid\":true},\"t\":null}],[64,{}],[105,{}],[22,{\"m_errors\":{\"32\":\"Sorry, no more results to show.\",\"default\":\"<font color=red>Error:</font> The server could not complete your request. Try again in 30 seconds.\"},\"m_tip\":\"Click for more information\"}],[77,{}],[84,{}],[99,{}],[29,{\"mcr\":5}],[92,{\"avgTtfc\":2000,\"fd\":1000,\"fl\":true,\"focus\":true,\"hpt\":250,\"kn\":true,\"mds\":\"clir,clue,dfn,evn,frim,klg,prc,rl,show,sp,sts,ww,mbl_he,mbl_hs,mbl_re,mbl_rs,mbl_sv,isch\",\"msg\":{\"dym\":\"Did you mean:\",\"gs\":\"Google Search\",\"kntt\":\"Use the up and down arrow keys to select each result. Press Enter to go to the selection.\",\"sif\":\"Search instead for\",\"srf\":\"Showing results for\"},\"odef\":true,\"ophe\":true,\"pq\":true,\"rpt\":41,\"tct\":\" ?\",\"tdur\":50}],[24,{}],[38,{}]]}"
},
"scroll": "0,0"
}],
"index": 2,
"hidden": false,
"attributes": {
"image": "http://www.google.co.uk/favicon.ico"
},
"storage": {
"http://www.google.co.uk": {
"web-v": "12_c9c918f0"
}
}
}],
"selected": 1,
"_closedTabs": [],
"width": 994,
"height": 688,
"screenX": 1650,
"screenY": 24,
"sizemode": "normal",
"title": "Google"
}],
"selectedWindow": 0,
"_closedWindows": [{
"tabs": [{
"entries": [{
"url": "about:home",
"title": "Mozilla Firefox Start Page",
"ID": 0,
"docshellID": 5,
"owner_b64": "NhAra3tiRRqhyKDUVsktxQAAAAAAAAAAwAAAAAAAAEYAAQAAAAAAAS8nfAAOr03buTZBMmukiq4HoizADOUR05MxABBLoP1AAAAAAAVhYm91dAAAAARob21l4NodcC97EdOM0ABgsPwUoweiLMAM5RHTkzEAEEug/UAAAAAADm1vei1zYWZlLWFib3V0AAAABGhvbWUAAAA="
}, {
"url": "http://www.facebook.com/",
"title": "Welcome to Facebook - Log In, Sign Up or Learn More",
"ID": 1,
"docshellID": 5,
"docIdentifier": 1,
"formdata": {
"//xhtml:div[#id='reg_form_box']/xhtml:table/xhtml:tbody/xhtml:tr[6]/xhtml:td[2]/xhtml:div/xhtml:div/xhtml:select": 0,
"//xhtml:div[#id='reg_form_box']/xhtml:table/xhtml:tbody/xhtml:tr[6]/xhtml:td[2]/xhtml:div/xhtml:div/xhtml:select[2]": 0,
"#sex": 0,
"#birthday_month": 0,
"#birthday_day": 0,
"#birthday_year": 0
},
"scroll": "0,0"
}],
"index": 2,
"hidden": false,
"attributes": {
"image": "http://www.facebook.com/favicon.ico"
}
}, {
"entries": [{
"url": "http://twitter.com/",
"title": "Twitter",
"ID": 3,
"docshellID": 6,
"docIdentifier": 3,
"children": [{
"url": "http://api.twitter.com/receiver.html",
"ID": 4,
"docshellID": 7,
"referrer": "http://twitter.com/",
"docIdentifier": 4,
"scroll": "0,0"
}],
"formdata": {},
"scroll": "0,0"
}],
"index": 1,
"hidden": false,
"attributes": {
"image": "http://twitter.com/phoenix/favicon.ico"
}
}],
"selected": 2,
"_closedTabs": [],
"width": 994,
"height": 688,
"screenX": 1366,
"screenY": 307,
"sizemode": "normal",
"cookies": [{
"host": ".facebook.com",
"value": "J4-69",
"path": "/",
"name": "lsd"
}, {
"host": ".facebook.com",
"value": "http%3A%2F%2Fwww.facebook.com%2F",
"path": "/",
"name": "reg_fb_gate"
}, {
"host": ".facebook.com",
"value": "http%3A%2F%2Fwww.facebook.com%2F",
"path": "/",
"name": "reg_fb_ref"
}, {
"host": ".facebook.com",
"value": "994x624",
"path": "/",
"name": "wd"
}, {
"host": ".twitter.com",
"value": "43838368",
"path": "/",
"name": "__utmc"
}, {
"host": "twitter.com",
"value": "4bfz%2B%2BmebEkRkMWFCXm%2FCUOsvDoVeFTl",
"path": "/",
"name": "original_referer"
}, {
"host": "scribe.twitter.com",
"value": "4bfz%2B%2BmebEkRkMWFCXm%2FCUOsvDoVeFTl",
"path": "/",
"name": "original_referer"
}, {
"host": ".twitter.com",
"value": "BAh7CToPY3JlYXRlZF9hdGwrCDoVZ%252F4vAToMY3NyZl9pZCIlODE2MGI1ZjJh%250AYmViNDMwODMxNDlkN2U5ZDg5Yjk4ZmU6B2lkIiU2N2I4YjdmNGExNWFkNzlk%250AODI0MDVjMGM1NmMzYjVhYSIKZmxhc2hJQzonQWN0aW9uQ29udHJvbGxlcjo6%250ARmxhc2g6OkZsYXNoSGFzaHsABjoKQHVzZWR7AA%253D%253D--8b0d751e9774c5cfaa61fdec567cb782aa8757dd",
"path": "/",
"name": "_twitter_sess",
"httponly": true
}, {
"host": ".twitter.com",
"value": "43838368",
"path": "/",
"name": "__utmc"
}, {
"host": "twitter.com",
"value": "4bfz%2B%2BmebEkRkMWFCXm%2FCUOsvDoVeFTl",
"path": "/",
"name": "original_referer"
}, {
"host": "scribe.twitter.com",
"value": "4bfz%2B%2BmebEkRkMWFCXm%2FCUOsvDoVeFTl",
"path": "/",
"name": "original_referer"
}, {
"host": ".twitter.com",
"value": "BAh7CToPY3JlYXRlZF9hdGwrCDoVZ%252F4vAToMY3NyZl9pZCIlODE2MGI1ZjJh%250AYmViNDMwODMxNDlkN2U5ZDg5Yjk4ZmU6B2lkIiU2N2I4YjdmNGExNWFkNzlk%250AODI0MDVjMGM1NmMzYjVhYSIKZmxhc2hJQzonQWN0aW9uQ29udHJvbGxlcjo6%250ARmxhc2g6OkZsYXNoSGFzaHsABjoKQHVzZWR7AA%253D%253D--8b0d751e9774c5cfaa61fdec567cb782aa8757dd",
"path": "/",
"name": "_twitter_sess",
"httponly": true
}],
"title": "Twitter"
}],
"session": {
"state": "stopped",
"lastUpdate": 1305658398727
}
}
The error
Error: sessionstore.js(179): expected value
now identifies the fault as being on the third-last line, the one that reads "lastUpdate": 1305658398727. From what I've read about the JSON format, this sounds to me like a comma or bracket is missing from this line, but this is a file that has been produced my Mozilla to work with Firefox, and I don't believe that they would make a mistake like that, so I am lead to believe that there is a problem with the JSON parser in Boost. Can anyone please confirm if this is the case, or if I'm the one doing something wrong?
I think the problem is this value is bigger than an int or a double. I don't know what data type uses BOOST JSON for reading numbers. To test this, just change the number to be a string and parse it again. In the standard, numbers are not limited, but you have to select a data type to represent them, and maybe they selected double, clearly not enough for this number. I'll take a look to see if you can configure the type used for numbers.
EDIT:
Looking again at the implementation, the "number" rule is implemented using Spirit as follows:
number
= strict_real_p
| int_p
;
Looking at Spirit strict_real_p uses double as the underlying type, and int_p actually uses an int.
The bad news is that, for what I see in the code, this is not configurable, so you have to change the JSON to be interpreted.
After receiving answers from Diego Sevilla and c-smile, I did a bit of Googling to figure out how I would incorporate their suggestions into Boost, since changing the JSON file unfortunately isn't an option in my case, and I came across this ticket on the Boost bug tracker that describes my exact problem. It has since been fixed and released with Boost 1.45. I, however, am using version 1.42 from the Ubuntu repositories, so will need to install the newer version manually.
As Diego said that is because 1305658398727 does not fit into neither strict_real_p nor int_p production.
I suspect you will need either other JSON parser or to modify Spirit definitions by yourself.
Either like this:
number
= strict_real_p
| int_p
| int64_p
;
or just as:
number
= real_p;
Ideally date/time in JSON should be presented by strings in ISO format. In this case you will not have such problems. I suspect that data there is just a number of milliseconds since 1970-01-01 (JavaScript Date.valueOf())