Schema validation error - django

I have used a field 'timestamp' is of type 'datetime.datetime' used in models.py in one of the django project,
Here is data :
{"bedId": "5807a14", "description": "",
"timestamp": "2013-09-16T15:40:16.383133", "encounterId": null, "patientId": null, "type":
"end_of_shift_note", "triggeredBy": "abc"}
Here is schema :
schema =
{
"type":"object",
"$schema": "http://json-schema.org/draft-03/schema",
"id": "http://jsonschema.net",
"required":true,
"properties":{
"type": {
"type":"string",
"id": "http://jsonschema.net/bedNumber",
"required":true
},
"encounterId": {
"type":"string",
"id": "http://jsonschema.net/encounterId",
"required":true
},
"timestamp": {
"type" :["null","string","date-time"],
'required' :false
},
"bedId": {
"type":"string",
"id": "http://jsonschema.net/cleaningStatus",
"required":true
},
"patientId": {
"type":["string","null"],
"id": "http://jsonschema.net/facilityId",
"required":true
} ,
"id": {
"type":["string","null"],
"id": "http://jsonschema.net/id",
"required":false
},
"triggeredBy": {
"type":["string","null"],
"id": "http://jsonschema.net/lastCleanedTime",
"required":false
}
}
}
On doing schema validation : https://pypi.python.org/pypi/json-schema-validator
def assertDataMatchesSchema(self, data, schema_file_name):
with open(os.path.join("hma/resource_jsonschema", schema_file_name)) as schema_file:
try:
schema = json.load(schema_file)
validate(data, schema)
except Exception as e:
print "Schema validation Error Message :",e.message
It is giving error on terminal:
Schema validation Error Message : Expecting property name: line 19 column 5 (char 401)
Problem is simple: What would be the type format for timestamp used in above jsonschema?
Please help

Got your problem:-
You are giving type instead of format.
Following example is working for me:-
data = {"timestamp": "2013-09-16T15:40:16.21211"}
schema ="""{
"type":"object",
"$schema": "http://json-schema.org/draft-03/schema",
"id": "http://jsonschema.net",
"required":true,
"properties":{
"timestamp": {
"format" :"date-time",
"type":"string",
"required" :false
}
}}"""
jsonschema.validate(data,json.loads(schema))

Related

Data does not match any schemas from 'oneOf'

I am getting this error after upgrading my api from .netcore2.2 to 3.1 and trying to generate using autorest with the --v3 switch
WARNING: Schema violation: Data does not match any schemas from
'oneOf'
I have tried with and without SerializeAsV2
I see from the Autorest docs that this warning is because of an supported feature.
anyOf, oneOf are not currently supported
In services.AddSwaggerGen I have
c.ParameterFilter<SwaggerEnumParameterFilter>();
c.SchemaFilter<SwaggerEnumFilter>();
where
public void Apply(OpenApiParameter parameter, ParameterFilterContext context)
{
var type = context.ApiParameterDescription.Type;
if (type.IsEnum)
parameter.Extensions.Add("x-ms-enum", new OpenApiObject
{
["name"] = new OpenApiString(type.Name),
["modelAsString"] = new OpenApiBoolean(false)
});
}
public class SwaggerEnumFilter : ISchemaFilter
{
public void Apply(OpenApiSchema model, SchemaFilterContext context)
{
if (model == null)
throw new ArgumentNullException("model");
if (context == null)
throw new ArgumentNullException("context");
if (context.Type.IsEnum)
model.Extensions.Add(
"x-ms-enum",
new OpenApiObject
{
["name"] = new OpenApiString(context.Type.Name),
["modelAsString"] = new OpenApiBoolean(false)
}
);
}
}
[update]
After upgrading to Autorest 3.0.6244 the warnings have changed to errors and the error message ends with
post > parameters > 0)
If I don't use the v3 switch I get the error
FATAL: swagger-document/individual/schema-validator - FAILED
FATAL: Error: [OperationAbortedException] Error occurred. Exiting.
Process() cancelled due to exception : [OperationAbortedException] Error occurred. Exiting.
I can see in the swagger.json that the parameters property "name" is not generating correctly. Here it contains "body" whereas previously it contained "info"
"/api/FrameLookUp": {
"post": {
"tags": [
"Frame"
],
"operationId": "FrameLookup",
"consumes": [
"application/json-patch+json",
"application/json",
"text/json",
"application/*+json"
],
"produces": [
"application/json"
],
"parameters": [
{
"in": "header",
"name": "Authorization",
"description": "access token",
"required": true,
"type": "String"
},
{
"in": "body",
"name": "body",
"schema": {
"$ref": "#/definitions/FrameRequest"
}
}
],
"responses": {
"200": {
"description": "Success",
"schema": {
"$ref": "#/definitions/FrameResponse"
}
}
}
}
},
The controller is
[Produces("application/json")]
[Authorize(AuthenticationSchemes = JwtBearerDefaults.AuthenticationScheme)]
[Route("api")]
public class FrameController : MyController
{
[ProducesResponseType(typeof(FrameResponse), StatusCodes.Status200OK)]
[HttpPost("FrameLookUp")]
public IActionResult FrameLookup([FromBody] FrameRequest info)
{
IMyResponse MyFunc(IMyRequest x) => FrameData.FrameLookUp(info);
return InnerMethod(MyFunc, info);
}
}
Update
I have also tried using the SwaggerParameter from Swashbuckle.AspNetCore.Annotations
[Update]
I am thinking that maybe I just need to try the release for issue 1766
I tried cloning the swashbuckle.aspnetcore repo but ran into this issue
[Update]
I added c.GeneratePolymorphicSchemas(); to the AddSwaggerGen options but it has not helped.
[Update]
Here is the first error message
ERROR: Schema violation: Data does not match any schemas from 'oneOf'
- https://localhost:44348/api-docs/v1/swagger.json:1951:8 ($.paths["/api/synchronise-management/get-product-images-Ids"].post.parameters)
Investigating line 1951 in swagger.json
In the working swagger ( generated from dotnet2.2 project ) the json looks very similar however the parameter order is swapped
The other difference I can see is the generated name of the parameter
I see from this question the error occurs in the same place
[Update]
when I add the --debug switch to the autorest call I get
/configuration
DEBUG: pipeline-emitter - END
DEBUG: configuration-emitter - END
DEBUG: swagger-document-override/md-override-loader - END
DEBUG: swagger-document/loader - END
DEBUG: swagger-document/individual/transform - START
DEBUG: swagger-document/individual/transform - END
DEBUG: swagger-document/individual/schema-validator - START
ERROR: Schema violation: Data does not match any schemas from 'oneOf'
- https://localhost:44348/api/v1/swagger.json:1951:8 ($.paths["/api/synchronise-management/get-product-images-Ids"].
[Update]
Here is the cut down json
{
"swagger": "2.0",
"info": {
"title": "myapi API31",
"description": "ASP.NET Core Web API",
"version": "v1"
},
"host": "localhost:44348",
"basePath": "/v1",
"schemes": [
"https"
],
"paths": {
"/api/Test": {
"get": {
"tags": [
"Auth"
],
"operationId": "Test",
"responses": {
"200": {
"description": "Success"
}
}
}
},
"/api/RequestToken": {
"post": {
"tags": [
"Auth"
],
"operationId": "RequestToken",
"consumes": [
"application/json-patch+json",
"application/json",
"text/json",
"application/*+json"
],
"produces": [
"application/json"
],
"parameters": [
{
"in": "body",
"name": "body",
"schema": {
"$ref": "#/definitions/TokenRequest"
}
}
],
"responses": {
"200": {
"description": "Success",
"schema": {
"$ref": "#/definitions/TokenResponse"
}
}
}
}
},
"/api/FrameLookUp": {
"post": {
"tags": [
"Frame"
],
"operationId": "FrameLookup",
"consumes": [
"application/json-patch+json",
"application/json",
"text/json",
"application/*+json"
],
"produces": [
"application/json"
],
"parameters": [
{
"in": "header",
"name": "Authorization",
"description": "access token",
"required": true,
"type": "String"
},
{
"in": "body",
"name": "body",
"schema": {
"$ref": "#/definitions/FrameRequest"
}
}
],
"responses": {
"200": {
"description": "Success",
"schema": {
"$ref": "#/definitions/FrameResponse"
}
}
}
}
}
},
"definitions": {
"TokenRequest": {
"required": [
"password",
"username"
],
"type": "object",
"properties": {
"username": {
"type": "string"
},
"password": {
"type": "string"
}
}
},
"TokenResponse": {
"type": "object",
"properties": {
"tokenResult": {
"type": "string"
}
}
},
"FramePackTypeEnum": {
"enum": [
"NotApplicable",
"PipeRack",
"LwBVan",
"VanTray",
"Car",
"CarryBag"
],
"type": "string",
"x-ms-enum": {
"name": "FramePackTypeEnum",
"modelAsString": false
}
},
"FrameRequest": {
"type": "object",
"properties": {
"qCodeJobId": {
"format": "int32",
"type": "integer"
},
"quantity": {
"format": "int32",
"type": "integer"
},
"widthInMm": {
"format": "int32",
"type": "integer"
},
"heightInMm": {
"format": "int32",
"type": "integer"
},
"ePackingType": {
"$ref": "#/definitions/FramePackTypeEnum"
},
"userEmail": {
"type": "string"
}
}
},
"FrameCaseEnum": {
"enum": [
"Case0_NoBraces",
"Case1_1Vertical_0Horizontal",
"Case2_2Vertical_0Horizontal",
"Case3_NVertical_0Horizontal",
"Case4_0Vertical_1Horizontal",
"Case5_1Vertical_1Horizontal",
"Case6_2Vertical_1Horizontal",
"Case7_NVertical_1Horizontal",
"Case8_0Vertical_2Horizontal",
"Case9_1Vertical_2Horizontal",
"Case10_2Vertical_2Horizontal",
"Case11_NVertical_2Horizontal",
"Case12_0Vertical_NHorizontal",
"Case13_1Vertical_NHorizontal",
"Case14_2Vertical_NHorizontal",
"Case15_NVertical_NHorizontal"
],
"type": "string",
"x-ms-enum": {
"name": "FrameCaseEnum",
"modelAsString": false
}
},
"FrameResponse": {
"type": "object",
"properties": {
"description": {
"type": "string"
},
"caseNumber": {
"$ref": "#/definitions/FrameCaseEnum"
},
"memberPriceEachExGst": {
"format": "double",
"type": "number"
},
"retailPriceEachExGst": {
"format": "double",
"type": "number"
}
}
}
}
}
With the .netcore2.2 api the request generates as
"FrameRequest": {
"type": "object",
"properties": {
"qCodeJobId": {
"format": "int32",
"type": "integer"
},
"quantity": {
"format": "int32",
"type": "integer"
},
"widthInMm": {
"format": "int32",
"type": "integer"
},
"heightInMm": {
"format": "int32",
"type": "integer"
},
"ePackingType": {
"enum": [
"NotApplicable",
"PipeRack",
"LwBVan",
"VanTray",
"Car",
"CarryBag"
],
"type": "string",
"x-ms-enum": {
"name": "FramePackTypeEnum",
"modelAsString": false
}
},
"userEmail": {
"type": "string"
}
}
}
Here is the command line I am running
autorest --input-file=.\myswagger.json --output-folder=generated --csharp --namespace=DDD --debug
Some links which the author, Kirsten Greed, put in comments:
https://github.com/domaindrivendev/Swashbuckle.AspNetCore#schema-filters
https://github.com/domaindrivendev/Swashbuckle.AspNetCore/pull/1766
https://stackoverflow.com/questions/63857310/could-not-find-a-part-of-the-path-d-dev-swashbuckle-aspnetcore-src-swashbuckle
From your swagger.json we can see the validation shows:
https://validator.swagger.io/validator/debug?url=https://raw.githubusercontent.com/heldersepu/hs-scripts/master/swagger/63783800_swagger.json
{
"schemaValidationMessages": [
{
"level": "error",
"domain": "validation",
"keyword": "oneOf",
"message": "instance failed to match exactly one schema (matched 0 out of 2)",
"schema": {
"loadingURI": "http://swagger.io/v2/schema.json#",
"pointer": "/definitions/parametersList/items"
},
"instance": {
"pointer": "/paths/~1api~1FrameLookUp/post/parameters/0"
}
}
]
}
that lead us to your code:
that type: "String" should be: type: "string" with all lower case the error goes away

AWS API Gateway fails to import Swagger definition: Unsupported model type 'MapProperty'

I am currently on this screen trying to import my app's swagger definition so I can create an API Gateway instance.
Unfortunately, you can see I'm getting some errors - even though swagger seems to think it's entirely fine.
Your API was not imported due to errors in the Swagger file.
Unable to create model for 200 response to method 'GET /api/v1/courses': Validation Result: warnings : [], errors : [Invalid content type specified: */*]
Unsupported model type 'MapProperty' in 200 response to method 'GET /api/v1/courses/all'. Ignoring.
Here is my swagger definition:
{
"swagger": "2.0",
"info": {
"description": "Api Documentation",
"version": "1.0",
"title": "Api Documentation",
"termsOfService": "urn:tos",
"contact": {},
"license": {
"name": "Apache 2.0",
"url": "http://www.apache.org/licenses/LICENSE-2.0"
}
},
"host": "********.appspot.com",
"basePath": "/",
"tags": [{
"name": "course-controller",
"description": "Course Controller"
}],
"paths": {
"/api/v1/courses": {
"get": {
"tags": ["course-controller"],
"summary": "getCourses",
"operationId": "getCoursesUsingGET",
"produces": ["*/*"],
"parameters": [{
"name": "code",
"in": "query",
"description": "code",
"required": false,
"type": "string"
}],
"responses": {
"200": {
"description": "OK",
"schema": {
"type": "array",
"items": {
"$ref": "#/definitions/Course"
}
}
},
"401": {
"description": "Unauthorized"
},
"403": {
"description": "Forbidden"
},
"404": {
"description": "Not Found"
}
},
"deprecated": false
}
},
"/api/v1/courses/all": {
"get": {
"tags": ["course-controller"],
"summary": "getAllCourses",
"operationId": "getAllCoursesUsingGET",
"produces": ["*/*"],
"responses": {
"200": {
"description": "OK",
"schema": {
"type": "object",
"additionalProperties": {
"type": "object"
}
}
},
"401": {
"description": "Unauthorized"
},
"403": {
"description": "Forbidden"
},
"404": {
"description": "Not Found"
}
},
"deprecated": false
}
}
},
"definitions": {
"Course": {
"type": "object",
"properties": {
"code": {
"type": "string"
},
"credits": {
"type": "integer",
"format": "int32"
},
"id": {
"type": "integer",
"format": "int32"
},
"lastUpdated": {
"type": "string"
},
"name": {
"type": "string"
},
"prerequisites": {
"type": "string"
},
"restrictions": {
"type": "string"
},
"seats": {
"$ref": "#/definitions/Seats"
},
"waitlist": {
"$ref": "#/definitions/Seats"
}
},
"title": "Course"
},
"Seats": {
"type": "object",
"properties": {
"actual": {
"type": "integer",
"format": "int32"
},
"capacity": {
"type": "integer",
"format": "int32"
},
"remaining": {
"type": "integer",
"format": "int32"
}
},
"title": "Seats"
}
}
}
Is there any reason you can find for this swagger definition breaking in API Gateway?
AWS API Gateway has some limitations in its OpenAPI support. For example, it does not support additionalProperties in models (this keyword is used in the 200 response schema for the /api/v1/courses/all endpoint in your API).
You can click the "Import and ignore warnings" button to ignore those errors and proceed with the import.

How to fetch API data as this. props?

I am trying to fetch data from API in react component as
{this.props.buyer && this.props.buyer[0].phone_number[0].number} - it's throwing error
Cannot read property 'number' of undefined
{this.props.buyer && this.props.buyer[0].name} - it's working fine
This is the API data
Orders: {
buyer:
},
}
[
{
"id": 2,
"name": "Qi Xiang",
"type": "Consignee",
"address": {
"id": 2,
"type": "shipping",
"street": "China China",
"city": "Beijing",
"postal_code": "34343",
"province": "23232",
"country": "CN"
},
"email": null,
"phone_number": {
"number": "323232",
"type": "Phone"
},
"id_image_url": "/api/files/24e49645-df42-4984-a
}
]
},
}
Your phonenumber is not array. You must use this:
this.props.buyer[0].phone_number.number

ElasticSearch - copy_to with dynamic template

Following up my previous question: ElasticSearch overriding mapping from text to object
I have an index template:
{
"template" : "project.*",
"order" : 100,
"dynamic_templates": [
{
"message_field": {
"mapping": {
"type": "object"
},
"match": "message"
},
"message_properties": {
"path_match": "message.*",
"mapping": {
"type": "string",
"index": "not_analyzed"
}
}
}
]
}
which basically creates new fields for everything under "message" field. I am doing this because "message" field is mapped as a string in another index template and I am overriding it.
Sample document:
{
"level": "30",
...
"kubernetes": {
"container_name": "data-sync-server",
"namespace_name": "alitest03",
...
},
"message": {
"tag": "AUDIT",
"requestId": 1234,
...
},
}
...
}
This works fine, but it ends up creating top level fields like "tag" and "requestId".
I don't want to pollute the top level and would like to have fields like "audit.tag", "audit.requestId".
Tried using copy_to like this, but I don't see any "audit.*" fields:
{
"template" : "project.*",
"order" : 100,
"dynamic_templates": [
{
"message_field": {
"mapping": {
"type": "object"
},
"match": "message"
},
"message_properties": {
"path_match": "message.*",
"mapping": {
"type": "string",
"index": "not_analyzed",
"copy_to" : "audit.{name}"
}
}
}
]
}
A sample search result when using the template above with copy_to is below. I don't see any "audit.*" fields.
{
"timestamp": "October 15th 2018, 15:46:15.994",
"_id": "YmI1NDRjMTgtZTY3Ni00ZGUxLTk2NDMtOTJhZjk3ZWU1YTJj",
"_index": "project.alitestproj02.aa564e69-c643-11e8-af2a-fa163e4c9c9e.2018.10.15",
"_score": "",
"_type": "com.redhat.viaq.common",
...
"kubernetes.container_name": "data-sync-server",
"kubernetes.namespace_name": "alitestproj02",
...
"message": "{\"level\":30,\"time\":1539607575994,\"pid\":19,\"hostname\":\"data-sync-server-6-pxcsm\",\"tag\":\"AUDIT\",\"msg\":\"\",\"requestId\":20355,\"operationType\":\"query\",\"parentTypeName\":\"Meme\",\"path\":\"allMemes.866.owner\",\"success\":true,\"parent\":{\"_type\":\"meme\",\"photourl\":\"photo472\",\"owner\":\"owner35\",\"likes\":0,\"_id\":\"zzEnLAQmQeuTC1mj\",\"createdAt\":\"2018-10-15T11:58:33.896Z\",\"updatedAt\":\"2018-10-15T11:58:33.896Z\",\"id\":\"zzEnLAQmQeuTC1mj\"},\"arguments\":{},\"dataSourceType\":\"InMemory\",\"v\":1}\n",
"requestId": "20355",
"tag": "AUDIT",
...
"v": 1
}

Store list or collection in Apache Avro schema

I'm currently creating Avro schema to store twitter data streams.
My data source in JSON:
{
'id': '123456789',
'text': 'bla bla bla...',
'entities': {
'hashtags': [{'text':'hashtag1'},{'text':'hashtag2'}]
}
}
in Cassandra, I can define collection (sets or lists) to store hashtags data.
But I have no idea how to define this structure in Apache Avro.
Here's my best try:
{"namespace": "ln.twitter",
"type": "record",
"name": "main",
"fields": [
{"name": "id","type": "string"},
{"name": "text","type": "string"},
{"name": "hashtags","type": "string"} // is there any better format for this ?
]
}
Need your advice please.
Thanks,
Yusata.
The entities field needed explicit records (or maps) inside. Here's a schema that should work:
{
"type": "record",
"name": "Main",
"fields": [
{
"name": "id",
"type": "string"
},
{
"name": "text",
"type": "string"
},
{
"name": "entities",
"type": {
"type": "record",
"name": "Entities",
"fields": [
{
"name": "hashtags",
"type": {
"type": "array",
"items": {
"type": "record",
"name": "Hashtag",
"fields": [
{
"name": "text",
"type": "string"
}
]
}
}
}
]
}
}
]
}
In case it's helpful, you can use this tool to generate an (anonymous) Avro schema from any valid JSON record. You'll then just need to add names to the record types.
You can try it on your example after switching its ' to ":
{
"id": "123456789",
"text": "bla bla bla...",
"entities": {"hashtags": [{"text": "hashtag1"}, {"text": "hashtag2"}]}
}