What is causing this Invalid JSON ABI structure error? - polkadot-js

I'm trying to deploy an ink! contract through polkadotJS.
var WASM = fs.readFileSync('./resources/flipper.wasm');
var ABI = fs.readFileSync('./resources/metadata.json');
const api = await ApiPromise.create();
const code = new CodePromise(api,ABI,WASM);
When executing, I get this error:
Error: Invalid JSON ABI structure supplied, expected a recent metadata version
According to Polkadot, the error is caused by a version lower than 3.0-rc1. Yet my version is ink! 3.0.0-rc3
my abi:
{
"metadataVersion": "0.1.0",
"source": {
"hash": "0x7fbad529eb12d718da29468d27aa3f7b202bec25411f58d32999166ff614cf7f",
"language": "ink! 3.0.0-rc3",
"compiler": "rustc 1.53.0-nightly"
},
"contract": {
"name": "flipper",
"version": "0.1.0",
"authors": [
"[your_name] <[your_email]>"
]
},
"spec": {
"constructors": [
{
"args": [
{
"name": "init_value",
"type": {
"displayName": [
"bool"
],
"type": 1
}
}
],
"docs": [
"Constructor that initializes the `bool` value to the given `init_value`."
],
"name": [
"new"
],
"selector": "0x9bae9d5e"
},
{
"args": [],
"docs": [
"Constructor that initializes the `bool` value to `false`.",
"",
"Constructors can delegate to other constructors."
],
"name": [
"default"
],
"selector": "0xed4b9d1b"
}
],
"docs": [],
"events": [],
"messages": [
{
"args": [],
"docs": [
" A message that can be called on instantiated contracts.",
" This one flips the value of the stored `bool` from `true`",
" to `false` and vice versa."
],
"mutates": true,
"name": [
"flip"
],
"payable": false,
"returnType": null,
"selector": "0x633aa551"
},
{
"args": [],
"docs": [
" Simply returns the current value of our `bool`."
],
"mutates": false,
"name": [
"get"
],
"payable": false,
"returnType": {
"displayName": [
"bool"
],
"type": 1
},
"selector": "0x2f865bd9"
}
]
},
"storage": {
"struct": {
"fields": [
{
"layout": {
"cell": {
"key": "0x0000000000000000000000000000000000000000000000000000000000000000",
"ty": 1
}
},
"name": "value"
}
]
}
},
"types": [
{
"def": {
"primitive": "bool"
}
}
]
}
So my ink! version is not the issue, what else is causing this? Or am I maybe doing something else wrong?

So when I imported the .contract file like this
var contract = fs.readFileSync('./resources/flipper.contract');
const json = u8aToString(contract);
const ABI = new Abi(json, api.registry.getChainProperties());
the error was solved! I used the contract blob instead of the wasm and abi json

Related

Task in ECS not being called by Step Functions

I have a DAG in Step Functions which is executed by an app within an ECS container. This DAG is scheduled to run every day. I'm seeing the following behavior:
Some random tasks fail with States. Timeout error (the timeout is set to 30 min)
I couldn't find the CloudWatch logs for these tasks that failed. This indicates that the tasks itself are never called.
Why this happens? Is there any setting I can use to prevent such behavior? Below follows the task code:
Input:
.
{
"version": "0",
"id": "ff4a2a37-7024-a213-70f0-11df9104484a",
"detail-type": "Scheduled Event",
"source": "aws.events",
"account": "",
"time": "2022-11-22T08:30:00Z",
"region": "us-east-1",
"resources": [
"arn:aws:events:rule/bi-datalake-hml"
],
"detail": {}
}
Output:
.
{
"Failures": [],
"SdkHttpMetadata": {
"AllHttpHeaders": {
"x-amzn-RequestId": [
"50737371-c24f-4901-98a5-bdef19278cf8"
],
"Content-Length": [
"2224"
],
"Date": [
"Tue, 22 Nov 2022 08:30:46 GMT"
],
"Content-Type": [
"application/x-amz-json-1.1"
]
},
"HttpHeaders": {
"Content-Length": "2224",
"Content-Type": "application/x-amz-json-1.1",
"Date": "Tue, 22 Nov 2022 08:30:46 GMT",
"x-amzn-RequestId": "50737371-c24f-4901-98a5-bdef19278cf8"
},
"HttpStatusCode": 200
},
"SdkResponseMetadata": {
"RequestId": "50737371-c24f-4901-98a5-bdef19278cf8"
},
"Tasks": [
{
"Attachments": [
{
"Details": [
{
"Name": "subnetId",
"Value": "subnet-62d7664e"
}
],
"Id": "eacd4cdf-3ce8-4927-a869-52eb71f553b3",
"Status": "PRECREATED",
"Type": "ElasticNetworkInterface"
}
],
"Attributes": [
{
"Name": "ecs.cpu-architecture",
"Value": "x86_64"
}
],
"AvailabilityZone": "us-east-1c",
"ClusterArn": "arn:aws::cluster/bi-datalake-hml",
"Containers": [
{
"ContainerArn": "arn::container/bi-datalake-hml/ad12e4d9017443d889055024d1932ddf/eb1c44d5-01b8-4c52-8950-8676b6ecd949",
"Cpu": "0",
"GpuIds": [],
"Image": ".dkr.ecr.us-east-1.amazonaws.com/bi_dbtoncloud_hml:latest",
"LastStatus": "PENDING",
"ManagedAgents": [],
"Name": "bi_dbtoncloud_hml",
"NetworkBindings": [],
"NetworkInterfaces": [],
"TaskArn": "arn:aws:ecs:us-east-1::task/bi-datalake-hml/ad12e4d9017443d889055024d1932ddf"
}
],
"Cpu": "256",
"CreatedAt": 1669105846495,
"DesiredStatus": "RUNNING",
"EnableExecuteCommand": false,
"EphemeralStorage": {
"SizeInGiB": 20
},
"Group": "family:bi_dbtoncloud_hml",
"InferenceAccelerators": [],
"LastStatus": "PROVISIONING",
"LaunchType": "FARGATE",
"Memory": "1024",
"Overrides": {
"ContainerOverrides": [
{
"Command": [],
"Environment": [
{
"Name": "DBT_MODEL_TO_RUN",
"Value": "alpha_order_address_shipping"
},
{
"Name": "TASK_TOKEN",
"Value": "AQCIAAAAKgAAAAMAAAAAAAAAAaP+z4q5FgxDHKZhHvzNF0PDV8l/5AkxTlorAGbQfnjdDJE1P1NWf+Jj1OINDelJ0RLrsAtdwIDJcAFmehGFj9mGJ905+T9sdWmKbsSjHuR0fCksAw==vIiI8wfR+LDUo1zPl03VOvkCHUeD5mzrDBoyjVRpA7QuqJ8ocA5OmVSN6MEGg3eS24H/3m3/MZRbmRNydbvI5DIB9PDD5seYIJamDTlfqEtYESgxWBoPrlVmvuphEnw5orSIeh5sZpsKm3/AlzB4OsoZaJleWBd+1WQbWclKEpV9bG3aKCsJO5rYyVaI7Ik09lTrogpL0VeulC2q/rY4cXR/r3lPA9ZL9YalUXgqij2ZuZIRby63hEjzTcoIkieMQMRcRd7XmKb1p8LsE2nSfSRnRotn9JeKVn7/4UBmWx0iQUd/14Dw1TXUSDwUx0sAPN7dtv2RdnqdLfJI6LoJnuPagkySFTaPOsJ0jpgLi9cjsJMVlWboCXlH57fj0JNosDLisxYlf5R3lB4paspZ8DHfgfAW5Saywc/KCCby7wfyYvVKpXvsJFIoBdtXaXs9tlbyTmlg9Dy9Oaol33ZFhVrOuDTzgDf4x6Mguxz1cQEaze8Ui0G2NDCoNxPU+WffSkrsb0Pflf5LsPfHAuI2"
},
{
"Name": "DBT_ENV",
"Value": "hml"
}
],
"EnvironmentFiles": [],
"Name": "bi_dbtoncloud_hml",
"ResourceRequirements": []
}
],
"InferenceAcceleratorOverrides": []
},
"PlatformFamily": "Linux",
"PlatformVersion": "1.4.0",
"Tags": [],
"TaskArn": "arn:aws:ecs:us-east-1::task/bi-datalake-hml/ad12e4d9017443d889055024d1932ddf",
"TaskDefinitionArn": "arn:aws:ecs:us-east-1::task-definition/bi_dbtoncloud_hml:2",
"Version": 1
}
]
}
If you need more information, please let me know.

How to build a multi-dimentional json native query for Druid?

I have data with multiple dimensions, stored in the Druid cluster. for example, Data of movies and the revenue they earned from each country where they were screened.
I'm trying to build a query that the answer to be returned will be a table of all the movies, the total revenue of each of them, and the revenue for each country.
I succeeded to do it in Turnilo - it generated for me the following Druid query -
[
[
{
"queryType": "timeseries",
"dataSource": "movies_source",
"intervals": "2021-11-18T00:01Z/2021-11-21T00:01Z",
"granularity": "all",
"aggregations": [
{
"name": "__VALUE__",
"type": "doubleSum",
"fieldName": "revenue"
}
]
},
{
"queryType": "topN",
"dataSource": "movies_source",
"intervals": "2021-11-18T00:01Z/2021-11-21T00:01Z",
"granularity": "all",
"dimension": {
"type": "default",
"dimension": "movie_id",
"outputName": "movie_id"
},
"aggregations": [
{
"name": "revenue",
"type": "doubleSum",
"fieldName": "revenue"
}
],
"metric": "revenue",
"threshold": 50
}
],
[
{
"queryType": "topN",
"dataSource": "movies_source",
"intervals": "2021-11-18T00:01Z/2021-11-21T00:01Z",
"granularity": "all",
"filter": {
"type": "selector",
"dimension": "movie_id",
"value": "some_movie_id"
},
"dimension": {
"type": "default",
"dimension": "country",
"outputName": "country"
},
"aggregations": [
{
"name": "revenue",
"type": "doubleSum",
"fieldName": "revenue"
}
],
"metric": "revenue",
"threshold": 5
}
]
]
But it doesn't work when I'm trying to use it as a body for a Postman query - I got
{
"error": "Unknown exception",
"errorMessage": "Unexpected token (START_ARRAY), expected VALUE_STRING: need JSON String that contains type id (for subtype of org.apache.druid.query.Query)\n at [Source: (org.eclipse.jetty.server.HttpInputOverHTTP); line: 2, column: 3]",
"errorClass": "com.fasterxml.jackson.databind.exc.MismatchedInputException",
"host": null
}
How should I build the corresponding query so that it works with Postman?
I am not familiar with Turnilo but have you tried using the Druid Console to write SQL and convert to Native request with the "Explain SQL query" option under the "Run/..." menu?
Your native queries seem to be doing a Top N instead of listing all movies, so I think the SQL might be something like:
SELECT movie_id, country_id, SUM(revenue) total_revenue
FROM movies_source
WHERE __time BETWEEN '2021-11-18 00:01:00' AND '2021-11-21 00:01:00'
GROUP BY movie_id, country_id
ORDER BY total_revenue DESC
LIMIT 50
I don't have the data source to test, but tested with sample wikipedia data with similar query structure:
SELECT namespace, cityName, sum(sum_added) total
FROM "wikipedia" r
WHERE cityName IS NOT NULL
AND __time BETWEEN '2015-09-12 00:00:00' AND '2015-09-15 00:00:00'
GROUP BY namespace, cityName
ORDER BY total DESC
limit 50
which results in the following Native query:
{
"queryType": "groupBy",
"dataSource": {
"type": "table",
"name": "wikipedia"
},
"intervals": {
"type": "intervals",
"intervals": [
"2015-09-12T00:00:00.000Z/2015-09-15T00:00:00.001Z"
]
},
"virtualColumns": [],
"filter": {
"type": "not",
"field": {
"type": "selector",
"dimension": "cityName",
"value": null,
"extractionFn": null
}
},
"granularity": {
"type": "all"
},
"dimensions": [
{
"type": "default",
"dimension": "namespace",
"outputName": "d0",
"outputType": "STRING"
},
{
"type": "default",
"dimension": "cityName",
"outputName": "d1",
"outputType": "STRING"
}
],
"aggregations": [
{
"type": "longSum",
"name": "a0",
"fieldName": "sum_added",
"expression": null
}
],
"postAggregations": [],
"having": null,
"limitSpec": {
"type": "default",
"columns": [
{
"dimension": "a0",
"direction": "descending",
"dimensionOrder": {
"type": "numeric"
}
}
],
"limit": 50
},
"context": {
"populateCache": false,
"sqlOuterLimit": 101,
"sqlQueryId": "cd5aabed-5e08-49b7-af63-fe82c125d3ee",
"useApproximateCountDistinct": false,
"useApproximateTopN": false,
"useCache": false
},
"descending": false
}

Data does not match any schemas from 'oneOf'

I am getting this error after upgrading my api from .netcore2.2 to 3.1 and trying to generate using autorest with the --v3 switch
WARNING: Schema violation: Data does not match any schemas from
'oneOf'
I have tried with and without SerializeAsV2
I see from the Autorest docs that this warning is because of an supported feature.
anyOf, oneOf are not currently supported
In services.AddSwaggerGen I have
c.ParameterFilter<SwaggerEnumParameterFilter>();
c.SchemaFilter<SwaggerEnumFilter>();
where
public void Apply(OpenApiParameter parameter, ParameterFilterContext context)
{
var type = context.ApiParameterDescription.Type;
if (type.IsEnum)
parameter.Extensions.Add("x-ms-enum", new OpenApiObject
{
["name"] = new OpenApiString(type.Name),
["modelAsString"] = new OpenApiBoolean(false)
});
}
public class SwaggerEnumFilter : ISchemaFilter
{
public void Apply(OpenApiSchema model, SchemaFilterContext context)
{
if (model == null)
throw new ArgumentNullException("model");
if (context == null)
throw new ArgumentNullException("context");
if (context.Type.IsEnum)
model.Extensions.Add(
"x-ms-enum",
new OpenApiObject
{
["name"] = new OpenApiString(context.Type.Name),
["modelAsString"] = new OpenApiBoolean(false)
}
);
}
}
[update]
After upgrading to Autorest 3.0.6244 the warnings have changed to errors and the error message ends with
post > parameters > 0)
If I don't use the v3 switch I get the error
FATAL: swagger-document/individual/schema-validator - FAILED
FATAL: Error: [OperationAbortedException] Error occurred. Exiting.
Process() cancelled due to exception : [OperationAbortedException] Error occurred. Exiting.
I can see in the swagger.json that the parameters property "name" is not generating correctly. Here it contains "body" whereas previously it contained "info"
"/api/FrameLookUp": {
"post": {
"tags": [
"Frame"
],
"operationId": "FrameLookup",
"consumes": [
"application/json-patch+json",
"application/json",
"text/json",
"application/*+json"
],
"produces": [
"application/json"
],
"parameters": [
{
"in": "header",
"name": "Authorization",
"description": "access token",
"required": true,
"type": "String"
},
{
"in": "body",
"name": "body",
"schema": {
"$ref": "#/definitions/FrameRequest"
}
}
],
"responses": {
"200": {
"description": "Success",
"schema": {
"$ref": "#/definitions/FrameResponse"
}
}
}
}
},
The controller is
[Produces("application/json")]
[Authorize(AuthenticationSchemes = JwtBearerDefaults.AuthenticationScheme)]
[Route("api")]
public class FrameController : MyController
{
[ProducesResponseType(typeof(FrameResponse), StatusCodes.Status200OK)]
[HttpPost("FrameLookUp")]
public IActionResult FrameLookup([FromBody] FrameRequest info)
{
IMyResponse MyFunc(IMyRequest x) => FrameData.FrameLookUp(info);
return InnerMethod(MyFunc, info);
}
}
Update
I have also tried using the SwaggerParameter from Swashbuckle.AspNetCore.Annotations
[Update]
I am thinking that maybe I just need to try the release for issue 1766
I tried cloning the swashbuckle.aspnetcore repo but ran into this issue
[Update]
I added c.GeneratePolymorphicSchemas(); to the AddSwaggerGen options but it has not helped.
[Update]
Here is the first error message
ERROR: Schema violation: Data does not match any schemas from 'oneOf'
- https://localhost:44348/api-docs/v1/swagger.json:1951:8 ($.paths["/api/synchronise-management/get-product-images-Ids"].post.parameters)
Investigating line 1951 in swagger.json
In the working swagger ( generated from dotnet2.2 project ) the json looks very similar however the parameter order is swapped
The other difference I can see is the generated name of the parameter
I see from this question the error occurs in the same place
[Update]
when I add the --debug switch to the autorest call I get
/configuration
DEBUG: pipeline-emitter - END
DEBUG: configuration-emitter - END
DEBUG: swagger-document-override/md-override-loader - END
DEBUG: swagger-document/loader - END
DEBUG: swagger-document/individual/transform - START
DEBUG: swagger-document/individual/transform - END
DEBUG: swagger-document/individual/schema-validator - START
ERROR: Schema violation: Data does not match any schemas from 'oneOf'
- https://localhost:44348/api/v1/swagger.json:1951:8 ($.paths["/api/synchronise-management/get-product-images-Ids"].
[Update]
Here is the cut down json
{
"swagger": "2.0",
"info": {
"title": "myapi API31",
"description": "ASP.NET Core Web API",
"version": "v1"
},
"host": "localhost:44348",
"basePath": "/v1",
"schemes": [
"https"
],
"paths": {
"/api/Test": {
"get": {
"tags": [
"Auth"
],
"operationId": "Test",
"responses": {
"200": {
"description": "Success"
}
}
}
},
"/api/RequestToken": {
"post": {
"tags": [
"Auth"
],
"operationId": "RequestToken",
"consumes": [
"application/json-patch+json",
"application/json",
"text/json",
"application/*+json"
],
"produces": [
"application/json"
],
"parameters": [
{
"in": "body",
"name": "body",
"schema": {
"$ref": "#/definitions/TokenRequest"
}
}
],
"responses": {
"200": {
"description": "Success",
"schema": {
"$ref": "#/definitions/TokenResponse"
}
}
}
}
},
"/api/FrameLookUp": {
"post": {
"tags": [
"Frame"
],
"operationId": "FrameLookup",
"consumes": [
"application/json-patch+json",
"application/json",
"text/json",
"application/*+json"
],
"produces": [
"application/json"
],
"parameters": [
{
"in": "header",
"name": "Authorization",
"description": "access token",
"required": true,
"type": "String"
},
{
"in": "body",
"name": "body",
"schema": {
"$ref": "#/definitions/FrameRequest"
}
}
],
"responses": {
"200": {
"description": "Success",
"schema": {
"$ref": "#/definitions/FrameResponse"
}
}
}
}
}
},
"definitions": {
"TokenRequest": {
"required": [
"password",
"username"
],
"type": "object",
"properties": {
"username": {
"type": "string"
},
"password": {
"type": "string"
}
}
},
"TokenResponse": {
"type": "object",
"properties": {
"tokenResult": {
"type": "string"
}
}
},
"FramePackTypeEnum": {
"enum": [
"NotApplicable",
"PipeRack",
"LwBVan",
"VanTray",
"Car",
"CarryBag"
],
"type": "string",
"x-ms-enum": {
"name": "FramePackTypeEnum",
"modelAsString": false
}
},
"FrameRequest": {
"type": "object",
"properties": {
"qCodeJobId": {
"format": "int32",
"type": "integer"
},
"quantity": {
"format": "int32",
"type": "integer"
},
"widthInMm": {
"format": "int32",
"type": "integer"
},
"heightInMm": {
"format": "int32",
"type": "integer"
},
"ePackingType": {
"$ref": "#/definitions/FramePackTypeEnum"
},
"userEmail": {
"type": "string"
}
}
},
"FrameCaseEnum": {
"enum": [
"Case0_NoBraces",
"Case1_1Vertical_0Horizontal",
"Case2_2Vertical_0Horizontal",
"Case3_NVertical_0Horizontal",
"Case4_0Vertical_1Horizontal",
"Case5_1Vertical_1Horizontal",
"Case6_2Vertical_1Horizontal",
"Case7_NVertical_1Horizontal",
"Case8_0Vertical_2Horizontal",
"Case9_1Vertical_2Horizontal",
"Case10_2Vertical_2Horizontal",
"Case11_NVertical_2Horizontal",
"Case12_0Vertical_NHorizontal",
"Case13_1Vertical_NHorizontal",
"Case14_2Vertical_NHorizontal",
"Case15_NVertical_NHorizontal"
],
"type": "string",
"x-ms-enum": {
"name": "FrameCaseEnum",
"modelAsString": false
}
},
"FrameResponse": {
"type": "object",
"properties": {
"description": {
"type": "string"
},
"caseNumber": {
"$ref": "#/definitions/FrameCaseEnum"
},
"memberPriceEachExGst": {
"format": "double",
"type": "number"
},
"retailPriceEachExGst": {
"format": "double",
"type": "number"
}
}
}
}
}
With the .netcore2.2 api the request generates as
"FrameRequest": {
"type": "object",
"properties": {
"qCodeJobId": {
"format": "int32",
"type": "integer"
},
"quantity": {
"format": "int32",
"type": "integer"
},
"widthInMm": {
"format": "int32",
"type": "integer"
},
"heightInMm": {
"format": "int32",
"type": "integer"
},
"ePackingType": {
"enum": [
"NotApplicable",
"PipeRack",
"LwBVan",
"VanTray",
"Car",
"CarryBag"
],
"type": "string",
"x-ms-enum": {
"name": "FramePackTypeEnum",
"modelAsString": false
}
},
"userEmail": {
"type": "string"
}
}
}
Here is the command line I am running
autorest --input-file=.\myswagger.json --output-folder=generated --csharp --namespace=DDD --debug
Some links which the author, Kirsten Greed, put in comments:
https://github.com/domaindrivendev/Swashbuckle.AspNetCore#schema-filters
https://github.com/domaindrivendev/Swashbuckle.AspNetCore/pull/1766
https://stackoverflow.com/questions/63857310/could-not-find-a-part-of-the-path-d-dev-swashbuckle-aspnetcore-src-swashbuckle
From your swagger.json we can see the validation shows:
https://validator.swagger.io/validator/debug?url=https://raw.githubusercontent.com/heldersepu/hs-scripts/master/swagger/63783800_swagger.json
{
"schemaValidationMessages": [
{
"level": "error",
"domain": "validation",
"keyword": "oneOf",
"message": "instance failed to match exactly one schema (matched 0 out of 2)",
"schema": {
"loadingURI": "http://swagger.io/v2/schema.json#",
"pointer": "/definitions/parametersList/items"
},
"instance": {
"pointer": "/paths/~1api~1FrameLookUp/post/parameters/0"
}
}
]
}
that lead us to your code:
that type: "String" should be: type: "string" with all lower case the error goes away

What is the replacement function for Contract.at in web3.js for including a contract?

Getting this error in the console:
Uncaught TypeError: TestContract.at is not a function
I am implementing a sample contract on a test server using this code i got from a course which I'm doing on Blockchain
var TestContract =new web3.eth.Contract([
{
"constant": false,
"inputs": [
{
"name": "_fName",
"type": "string"
},
{
"name": "_age",
"type": "uint256"
}
],
"name": "setInstructor",
"outputs": [],
"payable": false,
"stateMutability": "nonpayable",
"type": "function"
},
{
"constant": true,
"inputs": [],
"name": "getInstructor",
"outputs": [
{
"name": "",
"type": "string"
},
{
"name": "",
"type": "uint256"
}
],
"payable": false,
"stateMutability": "view",
"type": "function"
}
])
var Test = TestContract.at('0xd1d0ba6a5af6bb66490d04b99f4955eb9c9fef36');
You can just add an address as the second parameter
var TestContract =new web3.eth.Contract([
{
"constant": false,
"inputs": [
{
"name": "_fName",
"type": "string"
},
{
"name": "_age",
"type": "uint256"
}
],
"name": "setInstructor",
"outputs": [],
"payable": false,
"stateMutability": "nonpayable",
"type": "function"
},
{
"constant": true,
"inputs": [],
"name": "getInstructor",
"outputs": [
{
"name": "",
"type": "string"
},
{
"name": "",
"type": "uint256"
}
],
"payable": false,
"stateMutability": "view",
"type": "function"
}
],'0xd1d0ba6a5af6bb66490d04b99f4955eb9c9fef36')
You can read more about the available parameters there
Or you can add it via
TestContract.options.address = '0xd1d0ba6a5af6bb66490d04b99f4955eb9c9fef36'

Use regex in Powershell v2 to get values from a json file

How would I access the following values using the regex function in Powershell, and assign each one to an individual variable?:
id (i.e. get the value: TOKEN_ID) - under token
id (i.e. get the value: TENANT_ID) - under token, tenant
adminURL (i.e. get the value: http://10.100.0.222:35357/v2.0) - the first value under serviceCatalog,endpoints
As I am using Powershell v2, I can't use the ConvertFrom-Json cmdlet. So far I've tried converting the document to an xml file using the a third-party PS script, but it doesn't always get it right. I'd like to use regex, but I am not very comfortable with it.
$json =
"{
"access": {
"metadata": {
"is_admin": 0,
"roles": [
"9fe2ff9ee4384b1894a90878d3e92bab"
]
},
"serviceCatalog": [
{
"endpoints": [
{
"adminURL": "http://10.100.0.222:8774/v2/TENANT_ID",
"id": "0eb78b6d3f644438aea327d9c57b7b5a",
"internalURL": "http://10.100.0.222:8774/v2/TENANT_ID",
"publicURL": "http://8.21.28.222:8774/v2/TENANT_ID",
"region": "RegionOne"
}
],
"endpoints_links": [],
"name": "nova",
"type": "compute"
},
{
"endpoints": [
{
"adminURL": "http://10.100.0.222:9696/",
"id": "3f4b6015a2f9481481ca03dace8acf32",
"internalURL": "http://10.100.0.222:9696/",
"publicURL": "http://8.21.28.222:9696/",
"region": "RegionOne"
}
],
"endpoints_links": [],
"name": "neutron",
"type": "network"
},
{
"endpoints": [
{
"adminURL": "http://10.100.0.222:8776/v2/TENANT_ID",
"id": "16f6416588f64946bdcdf4a431a8f252",
"internalURL": "http://10.100.0.222:8776/v2/TENANT_ID",
"publicURL": "http://8.21.28.222:8776/v2/TENANT_ID",
"region": "RegionOne"
}
],
"endpoints_links": [],
"name": "cinder_v2",
"type": "volumev2"
},
{
"endpoints": [
{
"adminURL": "http://10.100.0.222:8779/v1.0/TENANT_ID",
"id": "be48765ae31e425cb06036b1ebab694a",
"internalURL": "http://10.100.0.222:8779/v1.0/TENANT_ID",
"publicURL": "http://8.21.28.222:8779/v1.0/TENANT_ID",
"region": "RegionOne"
}
],
"endpoints_links": [],
"name": "trove",
"type": "database"
},
{
"endpoints": [
{
"adminURL": "http://10.100.0.222:9292",
"id": "1adfcb5414304f3596fb81edb2dfb514",
"internalURL": "http://10.100.0.222:9292",
"publicURL": "http://8.21.28.222:9292",
"region": "RegionOne"
}
],
"endpoints_links": [],
"name": "glance",
"type": "image"
},
{
"endpoints": [
{
"adminURL": "http://10.100.0.222:8777",
"id": "350f3b91d73f4b3ab8a061c94ac31fbb",
"internalURL": "http://10.100.0.222:8777",
"publicURL": "http://8.21.28.222:8777",
"region": "RegionOne"
}
],
"endpoints_links": [],
"name": "ceilometer",
"type": "metering"
},
{
"endpoints": [
{
"adminURL": "http://10.100.0.222:8000/v1/",
"id": "2198b0d32a604e75a5cc1e13276a813d",
"internalURL": "http://10.100.0.222:8000/v1/",
"publicURL": "http://8.21.28.222:8000/v1/",
"region": "RegionOne"
}
],
"endpoints_links": [],
"name": "heat-cfn",
"type": "cloudformation"
},
{
"endpoints": [
{
"adminURL": "http://10.100.0.222:8776/v1/TENANT_ID",
"id": "7c193c4683d849ca8e8db493722a4d8c",
"internalURL": "http://10.100.0.222:8776/v1/TENANT_ID",
"publicURL": "http://8.21.28.222:8776/v1/TENANT_ID",
"region": "RegionOne"
}
],
"endpoints_links": [],
"name": "cinder",
"type": "volume"
},
{
"endpoints": [
{
"adminURL": "http://10.100.0.222:8773/services/Admin",
"id": "11fac8254be74d7d906110f0069e5748",
"internalURL": "http://10.100.0.222:8773/services/Cloud",
"publicURL": "http://8.21.28.222:8773/services/Cloud",
"region": "RegionOne"
}
],
"endpoints_links": [],
"name": "nova_ec2",
"type": "ec2"
},
{
"endpoints": [
{
"adminURL": "http://10.100.0.222:8004/v1/TENANT_ID",
"id": "38fa4f9afce34d4ca0f5e0f90fd758dd",
"internalURL": "http://10.100.0.222:8004/v1/TENANT_ID",
"publicURL": "http://8.21.28.222:8004/v1/TENANT_ID",
"region": "RegionOne"
}
],
"endpoints_links": [],
"name": "heat",
"type": "orchestration"
},
{
"endpoints": [
{
"adminURL": "http://10.100.0.222:35357/v2.0",
"id": "256cdf78ecb04051bf0f57ec11070222",
"internalURL": "http://10.100.0.222:5000/v2.0",
"publicURL": "http://8.21.28.222:5000/v2.0",
"region": "RegionOne"
}
],
"endpoints_links": [],
"name": "keystone",
"type": "identity"
}
],
"token": {
"audit_ids": [
"gsjrNoqFSQeuLUo0QeJprQ"
],
"expires": "2014-12-15T15:09:29Z",
"id": "TOKEN_ID",
"issued_at": "2014-12-15T14:09:29.794527",
"tenant": {
"description": "Auto created account",
"enabled": true,
"id": "TENANT_ID",
"name": "USERNAME"
}
},
"user": {
"id": "USER_ID",
"name": "USERNAME",
"roles": [
{
"name": "_member_"
}
],
"roles_links": [],
"username": "USERNAME"
}
}
}"
If you are using .NET 3.5 or higher on your machines with PowerShell 2.0, you can use a JSON serializer (from the linked answer):
[System.Reflection.Assembly]::LoadWithPartialName("System.Web.Extensions")
$json = "{a:1,b:2,c:{nested:true}}"
$ser = New-Object System.Web.Script.Serialization.JavaScriptSerializer
$obj = $ser.DeserializeObject($json)
This would be preferable to using regex.
For admin URL for example, you'd refer to:
$obj.access.serviceCatalog[0].endpoints[0].adminURL
Using RegEx Anyway
if ($json -match '(?s)"serviceCatalog".+?"endpoints".+?"adminURL"[^"]+"(?<adminUrl>[^"]+)".+?"token".+?"id"[^"]+"(?<tokenID>[^"]+)".+?"tenant".+?"id"[^"]+"(?<tenantID>[^"]+)') {
$Matches['adminURL']
$Matches['tokenID']
$Matches['tenantID']
}
RegEx Breakdown:
(?s) tells the regex engine that . matches anything, including newlines (by default it wouldn't).
Of course all of the "whatever" parts just match literally.
.+? matches 1 or more of any character (including newlines since we're using s), and the ? makes it non-greedy.
[^"]+ this matches 1 or more characters that are not a double quote.
() is a capturing group. By using (?<name>) we can refer back to the group later by name rather than number, just a nicety.
So the basic idea is to look for the literals, then get to a point where we can capture the values needed. After a -regex operator match in PowerShell, the $Matches variable is populated with the matches, groups, etc.
Note that this relies on the values being in the order they are in the posted JSON. If they were in a different order it would fail.
To work around that you could split this into 3 different regex matches.