How to index nested documents using Boto into dynamo? Or is it even possible?
I tried the following but it just hangs...
CASE A
users.put_item(data={
'username': 'johnd',
'first_name': 'John',
'last_name': 'Doe',
'account': {'type': 'standard', 'age', '5'},
})
Along the same vein of nesting:
CASE B
users.put_item(data={
'username': 'johnd',
'first_name': 'John',
'last_name': 'Doe',
'account': {'type': {'level':1, 'class':'standard''}},
})
Can it support arrays?
CASE C
users.put_item(data={
'username': 'johnd',
'first_name': 'John',
'last_name': 'Doe',
'accounts': [{'account_name': 1}, {'account_name': 2}]
})
Can it support nested arrays?
CASE D
users.put_item(data={
'username': 'johnd',
'first_name': 'John',
'last_name': 'Doe',
'accounts': [{'account_name': 1, 'types':['Checking']}, {'account_name': 2, 'types':['Savings Standard', 'Savings Plus']}]
})
If not, are there alternatives in or outside of python? Or is this not possible?
You can use the layer1 interface in boto to put nested structures. I'm not sure if it's available in the layer2 interface yet. There's an example in this thread on the AWS forums.
Related
I am trying to create a json schema and validate it using the standard validation base class. I get this error message which I can't tell what it refers to exactly.
Looking at my schema and value I don't see anywhere where I have an empty space in the parameter names and I'm not sure what the index 38 reffers to.
The schema that I provide is:
# value = {'enabled': -1, 'order_index': 0, 'width': 10}
# schema = {
'title': 'The Test column schema',
'type': 'object',
'required': ['enabled', 'order_index', 'width'],
'default': {'enabled': True, 'order_index': 0, 'width': 10},
'properties':
{'enabled':
{'title': 'Enabled',
'description': 'Display of column',
'type': 'boolean'
},
'order_index':
{'title': 'Order Index',
'description': 'Order of column to be displayed',
'type': 'number',
'minimum': 0,
'maximum': 999,
'default': 0
},
'width':
{'title': 'Width',
'description': 'Width of column to be displayed',
'type': 'number',
'minimum': 10,
'maximum': 999,
'default': 30}}}
validate_schema = JSONSchemaValidator(limit_value=schema)
validate_schema(value)
Where JSONSchemaValidator is an inheritor of the Django.BaseValidator
class JSONSchemaValidator(BaseValidator):
def compare(self, a, b):
try:
jsonschema.validate(a, b)
except jsonschema.exceptions.ValidationError as e:
raise ValidationError(
"Failed JSON schema check for %(value). {}".format(str(e)), params={"value": a}
)
As it turns out my error was not in my code but an exception in django failing to format a string.
I'm trying to create new project sheets in a folder, however I can't find a way to ensure the sheet is a project sheet.
Here is my code so far:
def create_resource_sheet(name):
""" Create a new resource sheet.
:param name:
:return:
"""
folder_id = get_resource_folder_id()
# TODO: Find and delete existing sheet with this name
resource_sheet = SS.models.Sheet({
'name': name,
# 'gantt_enabled': True, # This was added as an attempt to force it to a project sheet, but error 1032
'columns': [{
'title': 'Task',
'type': 'TEXT_NUMBER',
'primary': True,
'width': 200
}, {
'title': 'Status',
'type': 'PICKLIST',
'options': ['wtg', 'hld', 'ip', 'rdy', 'rev', 'fin', 'omt'], # TODO: Update this list
'width': 180
}, {
'title': '% Complete',
'type': 'TEXT_NUMBER',
'tag': ['GANTT_PERCENT_COMPLETE'],
'width': 85
}, {
'title': 'Assigned To',
'type': 'CONTACT_LIST',
'tag': ['GANTT_ASSIGNED_RESOURCE', 'GANTT_DISPLAY_LABEL'],
'width': 150
}, {
'title': '% Use',
'type': 'TEXT_NUMBER',
'tag': ['GANTT_ALLOCATION'],
'width': 60
}, {
'title': 'Days',
'type': 'DURATION',
'tag': ['GANTT_DURATION'],
'width': 70
}, {
'title': 'Start',
'type': 'ABSTRACT_DATETIME',
'tag': ['CALENDAR_START_DATE', 'GANTT_START_DATE'],
'width': 80
}, {
'title': 'Start',
'type': 'ABSTRACT_DATETIME',
'tag': ['CALENDAR_START_DATE', 'GANTT_START_DATE'],
'width': 80
}, {
'title': 'Finish',
'type': 'ABSTRACT_DATETIME',
'tag': ['CALENDAR_END_DATE', 'GANTT_END_DATE'],
'width': 80
}, {
'title': 'Type',
'type': 'TEXT_NUMBER',
'width': 150
}, {
'title': 'Comments',
'type': 'TEXT_NUMBER',
'width': 700
}
]
})
response = SS.Folders.create_sheet_in_folder(folder_id, resource_sheet)
new_sheet = response.result
return new_sheet
I am getting the following error code:
smartsheet.exceptions.ApiError: {"result": {"shouldRetry": false,
"code": 1142, "name": "ApiError", "errorCode": 1142, "recommendation":
"Do not retry without fixing the problem. ", "message": "Column type
DURATION is reserved for project sheets and may not be manually set on
a column.", "refId": "6gurrzzwhepe", "statusCode": 400}}
Is there a way I can create project sheets from scratch?
I have tried setting gantt_enabled to true but that just triggered a different error, so did setting 'project_settings'.
I have tried creating the sheet with just the primary column, then update_sheet to set project settings, which tells me: To set projectSettings, you must first enable dependencies on the sheet.
I have tried setting dependencies enabled, both directly in the create_sheet and in update_sheet, but both return: The attribute(s) sheet.dependenciesEnabled are not allowed for this operation.
I'm going to keep trying things, but I'm getting out of ideas.
Create the sheet from a template, either using the global project template or a user defined template if you want to customize.
templates = SS.Templates.list_public_templates()
for template in templates.data:
if template.global_template ==
smartsheet.models.enums.GlobalTemplate.PROJECT_SHEET:
break
sheet = smartsheet.models.Sheet({
'name': name,
'from_id': template.id
})
resource_sheet = SS.Folders.create_sheet_in_folder(folder_id, sheet)
If you want to customize create a project sheet using the Smartsheet web UI, make your changes and then Save As Template. Once you have the template, grab the ID from the Properties if you don't want to search for it, or SS.Templates.list_user_created_templates().
You cannot create a new dependency-enabled sheet through the API. You'll need to manually create a template and then use the API to make a copy.
I have the following string received from an ajax post in a django view (its 2 rows of a angular ui-grid):
data=[{"part_name":"33029191913","id":"5","qty":"3","ticket":"","comments":"test"},
{"part_name":"3302919777","id":"3","qty":"323","ticket":"","comments":"test"}]
Then doing json.loads(data) that gives me the following:
data={'part_name': '33029191913', 'id': '5', 'qty': '3', 'ticket': '', 'comments': 'test'}
{'part_name': '3302919777', 'id': '3', 'qty': '323', 'ticket': '', 'comments': 'test'}
Without specifying each field in order to create a dictionary, I want to create one that looks like the following in order to insert it into my model, via MyModel(**data):
{'part_name': '33029191913', 'id': '5', 'qty': '3', 'ticket': '', 'comments': 'test'}
After json.loads(data), if i try to access one element (eg. data[0]['part_name']), it correctly shows '33029191913', but if I do data[0] or data[1], it gives me only the apparent keys:
part_nameidqtyticketcomments.
How can I separate the string received by ajax into dictionaries in order to insert them into the django model?
Taking this json string from a post:
data=[{"part_name": "part 1", "part_id": "1", "quantity": "233", "ticket": "32", "comments": "32"}{"part_name": "part 1", "part_id": "1", "quantity": "233", "ticket": "32", "comments": "32"}]
grid=ast.literal_eval(data) returns:
{'part_name': 'part 1', 'part_id': '1', 'quantity': '233', 'ticket': '32', 'comments': '32'}{'part_name': 'part 1', 'part_id': '1', 'quantity': '233', 'ticket': '32', 'comments': '32'}
Assuming that the keys on the ajax json string are the same as your model fields(only the 'part_name' field is not in the model, and hardcoding the order_id foreign key for test purposes):
for d in grid:
object_insert={}
for key in d:
object_insert[key]=d[key]
object_insert['order_id']=1
del object_insert['part_name']
order=Order_Detail(**object_insert)
order.save()
This saves the dictionary in the model without having to specify each of the keys.
I followed these aws documents in order to set up my DynamoDB table:
creating table
loading the data into the table
Initially I created the table without any concern and the table structure would look like this:
Name > String
(Primary Key)
Category > String
Threads > Number
Messages > Number
Views > Number
When I tried to load the data, which was given on the 2nd hyperlink above, it throws up an exception saying:
An error occurred (ValidationException) when calling the
BatchWriteItem operation: The provided key element does not match the
schema
I did use the following command via the aws cli:
aws dynamodb batch-write-item --request-items file:///home/kula/Documents/aws/sampledata/Forum.json
This is the json file I'm trying to load, which I copied it from aws.
I also had a look into this ticket, and removed the quotes for numbers but still did not have any luck. Where am I going wrong?
Any help would be appreciated.
I apparently ended up creating python scripts to create table and load data into it.
Creating table:
import boto3
dynamodb = boto3.resource('dynamodb', region_name='us-east-1')
table = dynamodb.create_table(
TableName='users',
KeySchema=[
{
'AttributeName': 'username',
'KeyType': 'HASH'
},
{
'AttributeName': 'last_name',
'KeyType': 'RANGE'
}
],
AttributeDefinitions=[
{
'AttributeName': 'username',
'AttributeType': 'S'
},
{
'AttributeName': 'last_name',
'AttributeType': 'S'
},
],
ProvisionedThroughput={
'ReadCapacityUnits': 1,
'WriteCapacityUnits': 1
}
)
# Wait until the table exists.
table.meta.client.get_waiter('table_exists').wait(TableName='users')
Loading data into the table:
#!/usr/bin/python
import boto3
dynamodb = boto3.resource('dynamodb', region_name='us-east-1')
table = dynamodb.Table('users')
with table.batch_writer() as batch:
batch.put_item(
Item={
'account_type': 'standard_user',
'username': 'johndoe',
'first_name': 'John',
'last_name': 'Doe',
'age': 25,
'address': {
'road': '1 Jefferson Street',
'city': 'Los Angeles',
'state': 'CA',
'zipcode': 90001
}
}
)
batch.put_item(
Item={
'account_type': 'standard_user',
'username': 'bobsmith',
'first_name': 'Bob',
'last_name': 'Smith',
'age': 18,
'address': {
'road': '3 Madison Lane',
'city': 'Louisville',
'state': 'KY',
'zipcode': 40213
}
}
)
batch.put_item(
Item={
'account_type': 'super_user',
'username': 'alicedoe',
'first_name': 'Alice',
'last_name': 'Doe',
'age': 27,
'address': {
'road': '1 Jefferson Street',
'city': 'Los Angeles',
'state': 'CA',
'zipcode': 90001
}
}
)
print(table.creation_date_time)
Hope this helps someone!
I ran into a similar issue when trying to batch insert via aws dynamodb batch-write-item --request-items file://data/sample_services_data.json
It turns out that your primary partition key is case sensitive. In this scenario below, '77' will fail the import. If you change it to 'id' instead of 'Id' it will import without throwing the schema error.
{
"PutRequest": {
"Item": {
"Id": { "S": "77" },
"name": {"S":"Amazon Web Services"},
"language": {"S":"Python"},
"description": {"S":"Awesome super service"}
}
}
},
{
"PutRequest": {
"Item": {
"id": {"S":"99"},
"name": {"S":"Epic Service"},
"language": {"S":"NodeJS"},
"description": {"S":"Awesome epic service"}
}
}
}
data = [{'account_type': u'account_type', 'balance': 3484382.4899999998, 'type': 'report', 'name': u'Assets', 'level': 1}, {'account_type': u'liquidity', 'balance': 87301.78, 'type': 'account', 'name': u'100101 Cash', 'level': 4}, {'account_type': u'liquidity', 'balance': 257350.98, 'type': 'account', 'name': u'100201 HDFC Bank', 'level': 4}]
#api.multi
def account_fun():
for item in data:
return item['balance']
i'm calling that function using compute and getting the error expected singleton. but i want store all the balance from data into the database by one by one. in account.account table in odoo .
balance = field.Float(string="Balance",compute="account_fun")#creating new balance field.
how can i do that.and have to show balance field in list view. thanks,
I am not sure what you are trying to do with that logic, I have just give you idea here how you can do it.
data = [{'account_type': u'account_type', 'balance': 3484382.4899999998, 'type': 'report', 'name': u'Assets', 'level': 1}, {'account_type': u'liquidity', 'balance': 87301.78, 'type': 'account', 'name': u'100101 Cash', 'level': 4}, {'account_type': u'liquidity', 'balance': 257350.98, 'type': 'account', 'name': u'100201 HDFC Bank', 'level': 4}]
#api.multi
def account_fun(self):
for rec in self:
balance =0
for item in data:
balance += item['balance']
rec.balance = balance
balance = field.Float(string="Balance",compute="account_fun")
You are getting this error because in self list of recordset will be
there. In new api there is different way to set the functional field
which I have described above.
#api.multi
def account_fun(self):
data = [{'account_type': u'account_type', 'balance': 3484382.4899999998, 'type': 'report', 'name': u'Assets', 'level': 1}, {'account_type': u'liquidity', 'balance': 87301.78, 'type': 'account', 'name': u'100101 Cash', 'level': 4}, {'account_type': u'liquidity', 'balance': 257350.98, 'type': 'account', 'name': u'100201 HDFC Bank', 'level': 4}]
for rec in self:
for j in range(len(data)):
if rec.code in data[j]['name']:
rec.balance= data[j]['balance']
balance = fields.Float(string="Balance",compute="account_fun")
thanks #Emipro Technologies Pvt. Ltd. ,some what changed your code getting what i want.