Django Postgres DeserializationError: Problem installing fixture - django

I am trying to import my developement database fixtures into my production server. The migrations have been made and I tried slimming down the database migration to only two classes. This is the command and error I am receiving:
Error
(env) root#django-01:/home/projects/server/mysite# python3 local.py loaddata db.json
Traceback (most recent call last):
File "/home/projects/server/env/lib/python3.5/site-packages/django/core/serializers/json.py", line 68, in Deserializer
objects = json.loads(stream_or_string)
File "/usr/lib/python3.5/json/__init__.py", line 319, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.5/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.5/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 19 column 1 (char 229)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "local.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/home/projects/server/env/lib/python3.5/site-packages/django/core/management/__init__.py", line 371, in execute_from_command_line
utility.execute()
File "/home/projects/server/env/lib/python3.5/site-packages/django/core/management/__init__.py", line 365, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/projects/server/env/lib/python3.5/site-packages/django/core/management/base.py", line 288, in run_from_argv
self.execute(*args, **cmd_options)
File "/home/projects/server/env/lib/python3.5/site-packages/django/core/management/base.py", line 335, in execute
output = self.handle(*args, **options)
File "/home/projects/server/env/lib/python3.5/site-packages/django/core/management/commands/loaddata.py", line 72, in handle
self.loaddata(fixture_labels)
File "/home/projects/server/env/lib/python3.5/site-packages/django/core/management/commands/loaddata.py", line 113, in loaddata
self.load_label(fixture_label)
File "/home/projects/server/env/lib/python3.5/site-packages/django/core/management/commands/loaddata.py", line 168, in load_label
for obj in objects:
File "/home/projects/server/env/lib/python3.5/site-packages/django/core/serializers/json.py", line 73, in Deserializer
raise DeserializationError() from exc
django.core.serializers.base.DeserializationError: Problem installing fixture '/home/projects/server/mysite/db.json':
db.json
[
{
"model": "memberships.benefit",
"pk": 1,
"fields": {
"name": "SSL Certificate"
}
},
{
"model": "memberships.benefit",
"pk": 2,
"fields": {
"name": "No cancellation fee"
}
},
]

You need use proper json format, remove last (,) from your json
[
{
"model": "memberships.benefit",
"pk": 1,
"fields": {
"name": "SSL Certificate"
}
},
{
"model": "memberships.benefit",
"pk": 2,
"fields": {
"name": "No cancellation fee"
}
}
]
refer this
hope it helps

Related

Airflow 2.x bucket and range argument for ExternalTable(Ghseets) Operator from Gsheets

I am having problems using the new Airflow operator BigQueryCreateExternalTableOperator within Google-Composer:
Question 1
After creating an Airflow task this is happening :
AttributeError: 'BigQueryCreateExternalTableOperator' object has no attribute 'bucket'
However, as I am querying a gsheets file why it is looking for bucket argument? I am getting crazy trying to find what is happening! According to the docs it is optional!
Sample Code
task1 = BigQueryCreateExternalTableOperator(
task_id="task1_externaltable",
table_resource={
"tableReference": {
"projectId": projectid,
"datasetId": datasetid,
"tableId": tableid,
},
"schema": schema_fields,
"externalDataConfiguration": {
"sourceFormat": "GOOGLE_SHEETS",
"autodetect": False,
"compression": "NONE",
"googleSheetsOptions": {
"skipLeadingRows": 1,
"range": gsheets_tab_name,
},
"sourceUris": gsheets_url,
},
},
)
Following Elad's suggestion, error traceback!:
AttributeError: 'BigQueryCreateExternalTableOperator' object has no attribute 'bucket'
[2022-03-18, 14:45:38 UTC] {taskinstance.py:1268} INFO - Marking task as UP_FOR_RETRY. dag_id=trm_analytics_attribution_collision_checker_dag, task_id=create_manual_attribution_2_external_table, execution_date=20220318T144520, start_date=20220318T144536, end_date=20220318T144538
[2022-03-18, 14:45:38 UTC] {standard_task_runner.py:89} ERROR - Failed to execute job 444 for task create_manual_attribution_2_external_table
Traceback (most recent call last):
File "/opt/python3.8/lib/python3.8/site-packages/airflow/task/task_runner/standard_task_runner.py", line 85, in _start_by_fork
args.func(args, dag=self.dag)
File "/opt/python3.8/lib/python3.8/site-packages/airflow/cli/cli_parser.py", line 48, in command
return func(*args, **kwargs)
File "/opt/python3.8/lib/python3.8/site-packages/airflow/utils/cli.py", line 94, in wrapper
return f(*args, **kwargs)
File "/opt/python3.8/lib/python3.8/site-packages/airflow/cli/commands/task_command.py", line 302, in task_run
_run_task_by_selected_method(args, dag, ti)
File "/opt/python3.8/lib/python3.8/site-packages/airflow/cli/commands/task_command.py", line 107, in _run_task_by_selected_method
_run_raw_task(args, ti)
File "/opt/python3.8/lib/python3.8/site-packages/airflow/cli/commands/task_command.py", line 180, in _run_raw_task
ti._run_raw_task(
File "/opt/python3.8/lib/python3.8/site-packages/airflow/utils/session.py", line 70, in wrapper
return func(*args, session=session, **kwargs)
File "/opt/python3.8/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1330, in _run_raw_task
self._execute_task_with_callbacks(context)
File "/opt/python3.8/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1420, in _execute_task_with_callbacks
self.render_templates(context=context)
File "/opt/python3.8/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1995, in render_templates
self.task.render_template_fields(context)
File "/opt/python3.8/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 1061, in render_template_fields
self._do_render_template_fields(self, self.template_fields, context, jinja_env, set())
File "/opt/python3.8/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 1072, in _do_render_template_fields
content = getattr(parent, attr_name)
AttributeError: 'BigQueryCreateExternalTableOperator' object has no attribute 'bucket'
[2022-03-18, 14:45:38 UTC] {local_task_job.py:154} INFO - Task exited with return code 1
[2022-03-18, 14:45:38 UTC] {local_task_job.py:264} INFO - 0 downstream tasks scheduled from follow-on schedule check

MongoDB doesn't work locally because of SRV error (no answer)

Situation
We are working on the backend of our project consisting of two services written in Django and FastAPI. Both services use MongoDB as their database system.
In Django we use djongo==1.3.0 for ORM compatibility.
In FastAPI we use odmantic==0.3.4.
Both of these libraries use pymongo==3.11.3 underneath.
This MongoDB SRV error has been an issue for our Django service ever since we created it, but we managed to somehow work around it by not using the latest packages, such as:
Django==2.2.20
djongo==1.3.0
pymongo==3.11.3
Recently due to security risks we had to upgrade:
urllib3 from 1.25.8 to 1.26.5
pydantic from 1.8.1 to 1.8.2
Django from 2.2.20 to 2.2.22
Those were suggested by GitHub's dependabot.
Problem
When we run any of these services locally now they break with the following base exception:
dns.resolver.NoAnswer: The DNS response does not contain an answer to the question: _mongodb._tcp.cluster0.k1eh0.mongodb.net. IN SRV
Full log for Django:
> python manage.py test
Creating test database for alias 'default'...
Traceback (most recent call last):
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/djongo/database.py", line 10, in connect
return clients[db]
KeyError: 'djongo_test'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/dns/resolver.py", line 212, in __init__
rrset = response.find_rrset(response.answer, qname,
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/dns/message.py", line 341, in find_rrset
raise KeyError
KeyError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/dns/resolver.py", line 220, in __init__
crrset = response.find_rrset(response.answer,
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/dns/message.py", line 341, in find_rrset
raise KeyError
KeyError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/pymongo/srv_resolver.py", line 72, in _resolve_uri
results = resolver.query('_mongodb._tcp.' + self.__fqdn, 'SRV',
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/dns/resolver.py", line 1100, in query
return get_default_resolver().query(qname, rdtype, rdclass, tcp, source,
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/dns/resolver.py", line 1003, in query
answer = Answer(_qname, rdtype, rdclass, response,
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/dns/resolver.py", line 232, in __init__
raise NoAnswer(response=response)
dns.resolver.NoAnswer: The DNS response does not contain an answer to the question: _mongodb._tcp.database-test.avxhw.mongodb.net. IN SRV
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "manage.py", line 21, in <module>
main()
File "manage.py", line 17, in main
execute_from_command_line(sys.argv)
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
utility.execute()
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 375, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/django/core/management/commands/test.py", line 23, in run_from_argv
super().run_from_argv(argv)
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/django/core/management/base.py", line 323, in run_from_argv
self.execute(*args, **cmd_options)
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/django/core/management/base.py", line 364, in execute
output = self.handle(*args, **options)
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/django/core/management/commands/test.py", line 53, in handle
failures = test_runner.run_tests(test_labels)
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/django/test/runner.py", line 629, in run_tests
old_config = self.setup_databases(aliases=databases)
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/django/test/runner.py", line 552, in setup_databases
return _setup_databases(
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/django/test/utils.py", line 170, in setup_databases
connection.creation.create_test_db(
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/django/db/backends/base/creation.py", line 58, in create_test_db
self._create_test_db(verbosity, autoclobber, keepdb)
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/django/db/backends/base/creation.py", line 168, in _create_test_db
with self._nodb_connection.cursor() as cursor:
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/django/db/backends/base/base.py", line 256, in cursor
return self._cursor()
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/django/db/backends/base/base.py", line 233, in _cursor
self.ensure_connection()
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/django/db/backends/base/base.py", line 217, in ensure_connection
self.connect()
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/django/db/backends/base/base.py", line 195, in connect
self.connection = self.get_new_connection(conn_params)
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/djongo/base.py", line 185, in get_new_connection
self.client_connection = Database.connect(db=name, **connection_params)
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/djongo/database.py", line 13, in connect
clients[db] = MongoClient(**kwargs, connect=False)
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/pymongo/mongo_client.py", line 639, in __init__
res = uri_parser.parse_uri(
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/pymongo/uri_parser.py", line 500, in parse_uri
nodes = dns_resolver.get_hosts()
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/pymongo/srv_resolver.py", line 102, in get_hosts
_, nodes = self._get_srv_response_and_hosts(True)
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/pymongo/srv_resolver.py", line 83, in _get_srv_response_and_hosts
results = self._resolve_uri(encapsulate_errors)
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/pymongo/srv_resolver.py", line 79, in _resolve_uri
raise ConfigurationError(str(exc))
pymongo.errors.ConfigurationError: The DNS response does not contain an answer to the question: _mongodb._tcp.database-test.avxhw.mongodb.net. IN SRV
Full log for FastAPI:
> python -m src.main
Traceback (most recent call last):
File "/home/bk/inz/backend/testservice/venv/lib/python3.8/site-packages/pymongo/srv_resolver.py", line 72, in _resolve_uri
results = resolver.query('_mongodb._tcp.' + self.__fqdn, 'SRV',
File "/home/bk/inz/backend/testservice/venv/lib/python3.8/site-packages/dns/resolver.py", line 1321, in query
return resolve(qname, rdtype, rdclass, tcp, source,
File "/home/bk/inz/backend/testservice/venv/lib/python3.8/site-packages/dns/resolver.py", line 1305, in resolve
return get_default_resolver().resolve(qname, rdtype, rdclass, tcp, source,
File "/home/bk/inz/backend/testservice/venv/lib/python3.8/site-packages/dns/resolver.py", line 1202, in resolve
(answer, done) = resolution.query_result(response, None)
File "/home/bk/inz/backend/testservice/venv/lib/python3.8/site-packages/dns/resolver.py", line 674, in query_result
raise NoAnswer(response=answer.response)
dns.resolver.NoAnswer: The DNS response does not contain an answer to the question: _mongodb._tcp.cluster0.k1eh0.mongodb.net. IN SRV
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/bk/inz/backend/testservice/src/main.py", line 20, in <module>
client = AsyncIOMotorClient(MONGODB_URL)
File "/home/bk/inz/backend/testservice/venv/lib/python3.8/site-packages/motor/core.py", line 156, in __init__
delegate = self.__delegate_class__(*args, **kwargs)
File "/home/bk/inz/backend/testservice/venv/lib/python3.8/site-packages/pymongo/mongo_client.py", line 639, in __init__
res = uri_parser.parse_uri(
File "/home/bk/inz/backend/testservice/venv/lib/python3.8/site-packages/pymongo/uri_parser.py", line 500, in parse_uri
nodes = dns_resolver.get_hosts()
File "/home/bk/inz/backend/testservice/venv/lib/python3.8/site-packages/pymongo/srv_resolver.py", line 102, in get_hosts
_, nodes = self._get_srv_response_and_hosts(True)
File "/home/bk/inz/backend/testservice/venv/lib/python3.8/site-packages/pymongo/srv_resolver.py", line 83, in _get_srv_response_and_hosts
results = self._resolve_uri(encapsulate_errors)
File "/home/bk/inz/backend/testservice/venv/lib/python3.8/site-packages/pymongo/srv_resolver.py", line 79, in _resolve_uri
raise ConfigurationError(str(exc))
pymongo.errors.ConfigurationError: The DNS response does not contain an answer to the question: _mongodb._tcp.cluster0.k1eh0.mongodb.net. IN SRV
In our CI setup on GitHub Actions Django service tests are still run successfully. It's clear that the problem lies within the DNS aspect.
Questions
Are there any specific DNS servers which do not support MongoDB? I was using Google's 8.8.8.8 before and after it broke.
Why would the problem be caused by upgrading listed dependencies? Only urllib was upgraded for both services.
Is there any workaround?
Review https://github.com/mongodb/specifications/blob/master/source/initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.rst for how SRV URIs work, then manually issue the relevant DNS queries using a tool such as dig to verify your environment has working DNS.

DeserializationError when adding fixture on server (loaddata)

I am trying to add my data to database in my server. I use exactly the same JSON file in my local machine and it works. But when I do the same in server it gives me Deserialization Error.
The JSON file is so big that I can't show it here but I am sure there is no typo.
I did all migrations and database works - I can add object from admin dashboard. How can I solve this error?
The error I get:
Traceback (most recent call last):
File "/home/testuser/myprojectt/myvenv/lib/python3.8/site-packages/django/core/serializers/json.py", line 69, in Deserializer
objects = json.loads(stream_or_string)
File "/usr/lib/python3.8/json/__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.8/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "manage.py", line 22, in <module>
main()
File "manage.py", line 18, in main
execute_from_command_line(sys.argv)
File "/home/testuser/myprojectt/myvenv/lib/python3.8/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
utility.execute()
File "/home/testuser/myprojectt/myvenv/lib/python3.8/site-packages/django/core/management/__init__.py", line 395, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/testuser/myprojectt/myvenv/lib/python3.8/site-packages/django/core/management/base.py", line 330, in run_from_argv
self.execute(*args, **cmd_options)
File "/home/testuser/myprojectt/myvenv/lib/python3.8/site-packages/django/core/management/base.py", line 371, in execute
output = self.handle(*args, **options)
File "/home/testuser/myprojectt/myvenv/lib/python3.8/site-packages/django/core/management/commands/loaddata.py", line 72, in handle
self.loaddata(fixture_labels)
File "/home/testuser/myprojectt/myvenv/lib/python3.8/site-packages/django/core/management/commands/loaddata.py", line 114, in loaddata
self.load_label(fixture_label)
File "/home/testuser/myprojectt/myvenv/lib/python3.8/site-packages/django/core/management/commands/loaddata.py", line 172, in load_label
for obj in objects:
File "/home/testuser/myprojectt/myvenv/lib/python3.8/site-packages/django/core/serializers/json.py", line 74, in Deserializer
raise DeserializationError() from exc
django.core.serializers.base.DeserializationError: Problem installing fixture '/home/testuser/myprojectt/myprojectt/articles/fixtures/article.json':
Checked the JSON again and again but it still working on local but the same error on server.
I am using Ubuntu on the server. And didn't have git-lfs installed. To solve this issue:
Install it with sudo apt-get install git-lfs.
Then run git lfs pull on project directory.
And then I managed to work with the JSON data as I wanted.

negative timestamp in installed ssm patches list of EC2 instance

I want to get all the installed patches on an AWS EC2 instance, So I run this code in boto3:
response = client.describe_instance_patches(InstanceId=instance_id, Filters=[{'Key': 'State','Values': ['Installed',]} ])
My instance has a patch with a negative timestamp :
{
"Patches": [
{
"KBId": "KB3178539",
"Severity": "Important",
"Classification": "SecurityUpdates",
"Title": "Security Update for Windows 8.1 (KB3178539)",
"State": "Installed",
"InstalledTime": 1483574400.0
},
{
"KBId": "KB4493446",
"Severity": "Critical",
"Classification": "SecurityUpdates",
"Title": "2019-04 Security Monthly Quality Rollup for Windows 8.1 for x64-based Systems (KB4493446)",
"State": "Installed",
"InstalledTime": 1555804800.0
},
{
"KBId": "KB4487080",
"Severity": "Important",
"Classification": "SecurityUpdates",
"Title": "2019-02 Security and Quality Rollup for .NET Framework 3.5, 4.5.2, 4.6, 4.6.1, 4.6.2, 4.7, 4.7.1, 4.7.2 for Windows 8.1 (KB4487080)",
"State": "Installed",
"InstalledTime": -62135596800.0
}
]
}
So my boto3 snippet gives me this error:
response = client.describe_instance_patches(InstanceId=instance_id, Filters=[{'Key': 'State','Values': ['Installed',]}, ])
File "/usr/local/lib/python2.7/dist-packages/botocore/client.py", line 357, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/usr/local/lib/python2.7/dist-packages/botocore/client.py", line 648, in _make_api_call
operation_model, request_dict, request_context)
File "/usr/local/lib/python2.7/dist-packages/botocore/client.py", line 667, in _make_request
return self._endpoint.make_request(operation_model, request_dict)
File "/usr/local/lib/python2.7/dist-packages/botocore/endpoint.py", line 102, in make_request
return self._send_request(request_dict, operation_model)
File "/usr/local/lib/python2.7/dist-packages/botocore/endpoint.py", line 135, in _send_request
request, operation_model, context)
File "/usr/local/lib/python2.7/dist-packages/botocore/endpoint.py", line 167, in _get_response
request, operation_model)
File "/usr/local/lib/python2.7/dist-packages/botocore/endpoint.py", line 218, in _do_get_response
response_dict, operation_model.output_shape)
File "/usr/local/lib/python2.7/dist-packages/botocore/parsers.py", line 242, in parse
parsed = self._do_parse(response, shape)
File "/usr/local/lib/python2.7/dist-packages/botocore/parsers.py", line 740, in _do_parse
parsed = self._handle_json_body(response['body'], shape)
File "/usr/local/lib/python2.7/dist-packages/botocore/parsers.py", line 761, in _handle_json_body
return self._parse_shape(shape, parsed_json)
File "/usr/local/lib/python2.7/dist-packages/botocore/parsers.py", line 302, in _parse_shape
return handler(shape, node)
File "/usr/local/lib/python2.7/dist-packages/botocore/parsers.py", line 572, in _handle_structure
raw_value)
File "/usr/local/lib/python2.7/dist-packages/botocore/parsers.py", line 302, in _parse_shape
return handler(shape, node)
File "/usr/local/lib/python2.7/dist-packages/botocore/parsers.py", line 310, in _handle_list
parsed.append(self._parse_shape(member_shape, item))
File "/usr/local/lib/python2.7/dist-packages/botocore/parsers.py", line 302, in _parse_shape
return handler(shape, node)
File "/usr/local/lib/python2.7/dist-packages/botocore/parsers.py", line 572, in _handle_structure
raw_value)
File "/usr/local/lib/python2.7/dist-packages/botocore/parsers.py", line 302, in _parse_shape
return handler(shape, node)
File "/usr/local/lib/python2.7/dist-packages/botocore/parsers.py", line 589, in _handle_timestamp
return self._timestamp_parser(value)
File "/usr/local/lib/python2.7/dist-packages/botocore/utils.py", line 558, in parse_timestamp
return datetime.datetime.fromtimestamp(value, tzlocal())
File "/usr/local/lib/python2.7/dist-packages/dateutil/tz/_common.py", line 144, in fromutc
return f(self, dt)
File "/usr/local/lib/python2.7/dist-packages/dateutil/tz/_common.py", line 258, in fromutc
dt_wall = self._fromutc(dt)
File "/usr/local/lib/python2.7/dist-packages/dateutil/tz/_common.py", line 222, in _fromutc
dtoff = dt.utcoffset()
File "/usr/local/lib/python2.7/dist-packages/dateutil/tz/tz.py", line 216, in utcoffset
if self._isdst(dt):
File "/usr/local/lib/python2.7/dist-packages/dateutil/tz/tz.py", line 288, in _isdst
if self.is_ambiguous(dt):
File "/usr/local/lib/python2.7/dist-packages/dateutil/tz/tz.py", line 250, in is_ambiguous
(naive_dst != self._naive_is_dst(dt - self._dst_saved)))
OverflowError: date value out of range
I need to get the installed patches of several instances and I don't want the script to break when it finds a negative timestamp. How can workaround this ? How can I use the filters to get only valid timestamps ?
try:
response = client.describe_instance_patches(InstanceId=instance_id, Filters=[{'Key': 'State','Values': ['Installed',]} ])
except OverflowError:
//handle the exception as you want. Print a log, ignore, whatever.
I got this working by using subprocess with aws cli instead of boto3:
response = subprocess.Popen(["aws", "ssm", "describe-instance-patches", "--instance-id", instance_id, "--filters","Key=State,Values=Installed", "--profil", "prod", "--output", "json"], stdout=subprocess.PIPE).stdout.read()

Django logs errors

I'm sorry to ask about this
I'm trying to set up log to be able to test my code; Django pass variable into template
and I was following the example in https://docs.djangoproject.com/en/2.1/topics/logging/
Here the code they use
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'file': {
'level': 'DEBUG',
'class': 'logging.FileHandler',
'filename': r'C:\Users\Sk\Desktop\project\FrounterWeb\FrounterWeb\logs',
},
},
'loggers': {
'django': {
'handlers': ['file'],
'level': 'DEBUG',
'propagate': True,
},
},
}
But i run into this problem, copy and paste and answer i get is this;
Microsoft Windows [Version 10.0.10586]
(c) 2015 Microsoft Corporation. All rights reserved.
C:\Users\Sk\Desktop\project\FrounterWeb>py runserver_plus
(null): can't open file 'runserver_plus': [Errno 2] No such file or directory
C:\Users\Sk\Desktop\project\FrounterWeb>py manage.py runserver_plus
Traceback (most recent call last):
File "C:\Users\Sk\AppData\Local\Programs\Python\Python37\lib\logging\config.py", line 555, in configure
handler = self.configure_handler(handlers[name])
File "C:\Users\Sk\AppData\Local\Programs\Python\Python37\lib\logging\config.py", line 728, in configure_handler
result = factory(**kwargs)
File "C:\Users\Sk\AppData\Local\Programs\Python\Python37\lib\logging\__init__.py", line 1041, in __init__
StreamHandler.__init__(self, self._open())
File "C:\Users\Sk\AppData\Local\Programs\Python\Python37\lib\logging\__init__.py", line 1070, in _open
return open(self.baseFilename, self.mode, encoding=self.encoding)
PermissionError: [Errno 13] Permission denied: 'C:\\Users\\Sk\\Desktop\\project\\FrounterWeb\\FrounterWeb\\logs'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "manage.py", line 15, in <module>
execute_from_command_line(sys.argv)
File "C:\Users\Sk\AppData\Local\Programs\Python\Python37\lib\site-packages\django\core\management\__init__.py", line 381, in execute_from_command_line
utility.execute()
File "C:\Users\Sk\AppData\Local\Programs\Python\Python37\lib\site-packages\django\core\management\__init__.py", line 357, in execute
django.setup()
File "C:\Users\Sk\AppData\Local\Programs\Python\Python37\lib\site-packages\django\__init__.py", line 19, in setup
configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
File "C:\Users\Sk\AppData\Local\Programs\Python\Python37\lib\site-packages\django\utils\log.py", line 76, in configure_logging
logging_config_func(logging_settings)
File "C:\Users\Sk\AppData\Local\Programs\Python\Python37\lib\logging\config.py", line 792, in dictConfig
dictConfigClass(config).configure()
File "C:\Users\Sk\AppData\Local\Programs\Python\Python37\lib\logging\config.py", line 563, in configure
'%r' % name) from e
ValueError: Unable to configure handler 'files'
I believe this problem with the dictionary, so I remove the 'File':{} & 'django':{} dictionary and i receive another problems
the new code:
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'file': {
'level': 'DEBUG',
'class': 'logging.FileHandler',
'filename': r'C:\Users\Sk\Desktop\project\FrounterWeb\FrounterWeb\logs',
},
},
'loggers': {
'django': {
'handlers': ['file'],
'level': 'DEBUG',
'propagate': True,
},
},
}
and this is problems;
Microsoft Windows [Version 10.0.10586]
(c) 2015 Microsoft Corporation. All rights reserved.
C:\Users\Sk\Desktop\project\FrounterWeb>py runserver_plus
(null): can't open file 'runserver_plus': [Errno 2] No such file or directory
C:\Users\Sk\Desktop\project\FrounterWeb>py manage.py runserver_plus
Traceback (most recent call last):
File "C:\Users\Sk\AppData\Local\Programs\Python\Python37\lib\logging\config.py", line 555, in configure
handler = self.configure_handler(handlers[name])
File "C:\Users\Sk\AppData\Local\Programs\Python\Python37\lib\logging\config.py", line 728, in configure_handler
result = factory(**kwargs)
File "C:\Users\Sk\AppData\Local\Programs\Python\Python37\lib\logging\__init__.py", line 1041, in __init__
StreamHandler.__init__(self, self._open())
File "C:\Users\Sk\AppData\Local\Programs\Python\Python37\lib\logging\__init__.py", line 1070, in _open
return open(self.baseFilename, self.mode, encoding=self.encoding)
PermissionError: [Errno 13] Permission denied: 'C:\\Users\\Sk\\Desktop\\project\\FrounterWeb\\FrounterWeb\\logs'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "manage.py", line 15, in <module>
execute_from_command_line(sys.argv)
File "C:\Users\Sk\AppData\Local\Programs\Python\Python37\lib\site-packages\django\core\management\__init__.py", line 381, in execute_from_command_line
utility.execute()
File "C:\Users\Sk\AppData\Local\Programs\Python\Python37\lib\site-packages\django\core\management\__init__.py", line 357, in execute
django.setup()
File "C:\Users\Sk\AppData\Local\Programs\Python\Python37\lib\site-packages\django\__init__.py", line 19, in setup
configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
File "C:\Users\Sk\AppData\Local\Programs\Python\Python37\lib\site-packages\django\utils\log.py", line 76, in configure_logging
logging_config_func(logging_settings)
File "C:\Users\Sk\AppData\Local\Programs\Python\Python37\lib\logging\config.py", line 792, in dictConfig
dictConfigClass(config).configure()
File "C:\Users\Sk\AppData\Local\Programs\Python\Python37\lib\logging\config.py", line 563, in configure
'%r' % name) from e
ValueError: Unable to configure handler 'files'
C:\Users\Sk\Desktop\project\FrounterWeb>py manage.py runserver_plus
Traceback (most recent call last):
File "C:\Users\Sk\AppData\Local\Programs\Python\Python37\lib\logging\config.py", line 555, in configure
handler = self.configure_handler(handlers[name])
File "C:\Users\Sk\AppData\Local\Programs\Python\Python37\lib\logging\config.py", line 688, in configure_handler
config_copy = dict(config) # for restoring in case of error
ValueError: dictionary update sequence element #0 has length 1; 2 is required
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "manage.py", line 15, in <module>
execute_from_command_line(sys.argv)
File "C:\Users\Sk\AppData\Local\Programs\Python\Python37\lib\site-packages\django\core\management\__init__.py", line 381, in execute_from_command_line
utility.execute()
File "C:\Users\Sk\AppData\Local\Programs\Python\Python37\lib\site-packages\django\core\management\__init__.py", line 357, in execute
django.setup()
File "C:\Users\Sk\AppData\Local\Programs\Python\Python37\lib\site-packages\django\__init__.py", line 19, in setup
configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
File "C:\Users\Sk\AppData\Local\Programs\Python\Python37\lib\site-packages\django\utils\log.py", line 76, in configure_logging
logging_config_func(logging_settings)
File "C:\Users\Sk\AppData\Local\Programs\Python\Python37\lib\logging\config.py", line 792, in dictConfig
dictConfigClass(config).configure()
File "C:\Users\Sk\AppData\Local\Programs\Python\Python37\lib\logging\config.py", line 563, in configure
'%r' % name) from e
ValueError: Unable to configure handler 'class'
This problem means it can read it own library
I'm sorry trouble you, but pls help me, i really don't know what to since is Django is tell is having problem with own build in library(i think)
You need to add formatter to your LOGGING:
LOGGING = {
'formatters': {
'verbose': {
'format': '{levelname} {message}',
'style': '{',
},
},
# Rest of the Logging Config
}
The answer was very simple, like Burhan Khalid said the logs path to the site not tied a path for the file to logs to.
All i did was add this to my handler
'filename': 'zigview\logging_files\debug.log',
and is working
I feel so embarrass, to make such a simple mistakes