Does anyone use structlog with Django? I'm looking for a code sample how can I integrate Django logging (which is done via standard library), and structlog.
I've tried the code from the "Rendering Using structlog-based Formatters Within logging" example, with only slightest modifications:
# From my settings.py, basically the same code as in the linked example
timestamper = structlog.processors.TimeStamper(fmt="%Y-%m-%d %H:%M:%S")
pre_chain = [
structlog.stdlib.add_log_level,
timestamper,
]
LOGGING = {
"version": 1,
"disable_existing_loggers": False,
"formatters": { ... }, # Exactly like in the linked example
"handlers": { ... }, # Ditto, but only "default" handler (no files)
"loggers": {
"django": {
"handlers": ["default"],
"level": "INFO",
},
# I also had "" logger here, with the same config as "django",
# but it's irrelevant for the example purposes.
}
}
# Same as in the example
structlog.configure(
processors=[
structlog.stdlib.add_log_level,
structlog.stdlib.PositionalArgumentsFormatter(),
timestamper,
structlog.processors.StackInfoRenderer(),
structlog.processors.format_exc_info,
structlog.stdlib.ProcessorFormatter.wrap_for_formatter,
],
context_class=dict,
logger_factory=structlog.stdlib.LoggerFactory(),
wrapper_class=structlog.stdlib.BoundLogger,
cache_logger_on_first_use=True,
)
However, I end up with logging errors. This is an excerpt of what happens on a simple GET request that ends up with 404
TypeError: not all arguments converted during string formatting
...
File "/usr/local/lib/python3.6/site-packages/django/core/handlers/base.py", line 152, in get_response
extra={'status_code': 404, 'request': request},
Message: '\x1b[2m2017-05-08 18:34:53\x1b[0m [\x1b[33m\x1b[1mwarning \x1b[0m] \x1b[1mNot Found: /favicon.ico\x1b[0m'
Arguments: ('/favicon.ico',)
I've tried to figure out what exactly goes on but lost my way in the debugger.
Of course, I can use structlog just for application logging, and keep the standard library loggers like they are. However, I want all logging unified, so my application's output would be uniform, ready for parsing.
I'd greatly appreciate a code snippet that shows how to integrate structlog with Django correctly.
It’s most likely this bug that will be fixed in structlog 17.2 that should be released soonish: https://github.com/hynek/structlog/pull/117 (feel free to comment or to try out if it fixes your problem).
Related
https://cloud.google.com/dialogflow/es/docs/reference/rest/v2beta1/projects.knowledgeBases.documents/import
Consider I'm having an csv file to be imported in a cloud storage, How exactly do I execute this above API request and import the knowledge base qna's, I've added the documentation link above, I'm getting the below error too
Change the parent to projects/your-project-id/knowledgeBases/xxxxxx and import should accept it.
But I suggest to use projects.knowledgeBases.documents.create if you are planning to create a knowledge base from scratch using a CSV file. See sample request via projects.knowledgeBases.documents.create:
parent: projects/your-project-id/knowledgeBases/xxxxx
importGcsCustomMetadata: false
Request Body:
{
"contentUri": "gs://my-bucket/faq.csv",
"displayName": "test_csv",
"knowledgeTypes": [
"FAQ"
],
"mimeType": "text/csv"
}
Returns HTTP 200:
{
"name": "projects/your-project-id/locations/us/operations/document-create-20210829-21261630297603-6127fbb9-0000-21dc-bec9-240588717654"
}
Created knowledge base in Dialogflow console:
Background: I am building a cloud function using python to take a .csv uploaded to our bucket and upload its rows as entities into Datastore.
When trying to deploy the Python function, I got an error where the function would fail to deploy due to the following error code:
Build failed: {
"cacheStats": [{
"status": "MISS",
"hash": "SOME HASH",
"type": "docker_layer_cache",
"level": "global"
}, {
"status": "MISS",
"hash": "SOME HASH",
"type": "docker_layer_cache",
"level": "project"
}
]
}
To try to diagnose the problem, I attempted deploying a template Python function, and got the same error. (Code included below just in case)
def hello_world(request):
"""Responds to any HTTP request.
Args:
request (flask.Request): HTTP request object.
Returns:
The response text or any set of values that can be turned into a
Response object using
`make_response <http://flask.pocoo.org/docs/1.0/api/#flask.Flask.make_response>`.
"""
request_json = request.get_json()
if request.args and 'message' in request.args:
return request.args.get('message')
elif request_json and 'message' in request_json:
return request_json['message']
else:
return f'Hello World!'
From my understanding of the error, it implies that there is an issue with the hashing to my project specifically, but it is not clear to me how to fix this, or what resulted in this error occurring.
https://cloud.google.com/functions/docs/troubleshooting has no information about an error of this type, and other errors that I have seen similar to this error are due to situations such as the entry function parameters being incorrect, which is not the case here.
Cloud functions are running as normal according to google (https://status.cloud.google.com/).
Is this a genuine problem on google's end? Or is there some setting that I am not aware of that would have caused this issue?
Update: Node.js functions are working fine, python functions still do not work. In addition, Go functions result in an error where the logs fail on build step #5 due to Token exchange failed for project: 'project-id'
declareUpdate();
//get Docs
myDoc = cts.doc("/heal/scripts/Test.json").toObject();
//add Data
myDoc.prescribedPlayer =
[
{
"default": "http://www.youtube.com/watch?vu003dhYB0mn5zh2c"
}
]
//persist
xdmp.documentInsert("/heal/scripts/Test.json",myDoc,null,"scripts")
You're looking to add a new JSON property. You can do that using a REST Client API request, sending a PATCH command. Use an insert instruction in the patch.
See the note in Specifying Position in JSON, which indicates that
You cannot use last-child to insert a property as an immediate child of the root node of a document. Use before or after instead. For details, see Limitations of JSON Path Expressions.
Instead, your patch will look something like:
{
"insert": {
"context": "/topProperty",
"position": "after",
"content":
[
{
"default": "http://www.youtube.com/watch?vu003dhYB0mn5zh2c"
}
],
}
}
where topProperty is a JSON property that is part of the root node of the JavaScript object you want to update.
If that approach is problematic (for instance, if there is no topProperty that's reliably available), you could also do a sequence of operations:
retrieve the document
edit the content in Python
update the document in the database
With this approach, there is the possibility that some other process may update the document while you're working on it. You can either rely on optimistic locking or a multi-statement transaction to work around that, depending on the potential consequences of someone else doing a write.
Hey #Ankur Please check below python method,
def PartialUpdateData(self,filename, content, context):
self.querystring = {"uri": "/" + self.collection + "/" + filename}
url = self.baseUri
self.header = {'Content-Type': "application/json"}
mydata = {
"patch":[{ "insert": {
"context": context,
"position": "before",
"content": content
}}]}
resp = requests.patch(url + "/documents", data=json.dumps(mydata),
headers=self.header, auth=self.auth, params=self.querystring)
return resp.content
I hope this can solve your problem.
I'm using Google Place API for Web Service, in Python.
And I'm trying to add places like the tutorial here
My code is here:
from googleplaces import GooglePlaces, types, lang, GooglePlacesError, GooglePlacesAttributeError
API_KEY = "[Google Place API KEY]"
google_places = GooglePlaces(API_KEY)
try:
added_place = google_places.add_place(
name='Mom and Pop local store',
lat_lng={'lat': 51.501984, 'lng': -0.141792},
accuracy=100,
types=types.TYPE_HOME_GOODS_STORE,
language=lang.ENGLISH_GREAT_BRITAIN)
except GooglePlacesError as error_detail:
print error_detail
But I kept getting this error:
I tried to change the input into Json format or Python dictionary format, then it gave the error "google_places.add_place() only accept 1 parameter, 2 give"......
Is there any right way to use Google Place API Add Place method in Python?
Oh, finally I found the solution, it's so simple... I am not familiar with Python POST requests, in fact everything is easy.
Just need the code here, and we will be able to add a Place in Google Place API, with Python:
import requests
post_url = "https://maps.googleapis.com/maps/api/place/add/json?key=" + [YOUR_API_KEY]
r = requests.post(post_url, json={
"location": {
"lat": -33.8669710,
"lng": 151.1958750
},
"accuracy": 50,
"name": "Google Shoes!",
"phone_number": "(02) 9374 4000",
"address": "48 Pirrama Road, Pyrmont, NSW 2009, Australia",
"types": ["shoe_store"],
"website": "http://www.google.com.au/",
"language": "en-AU"
})
To check the results:
print r.status_code
print r.json()
Currently, when an error occurs in my Django application this is what is returned as an error:
{
"name": [
"This field is required."
]
}
I saw that i can specify custom error message like this
Field(error_messages = { 'required': '12' })
I would like to know if it's possible to tell Django to return error codes along plain text error? And not replace it with error_messages. If it's possible, what's the best way to do it?
Something like this would be nice:
{
"name": [
"This field is required."
],
"error_code": 12
}
This way I could translate the various errors in different languages in my client and not on the server.
This question is tagged django-rest-framework. If indeed you're using DRF, you can do this with a custom exception handler.