I've got a doozy. I have a protobuf timestamp message that I'm trying to parse from a Python dictionary.
The target Protobuf message looks like this:
message LogStatusSnapshot {
google.protobuf.Timestamp time = 1;
LogStatus.LogStatusEnum status = 2;
LogStatusLevel.LogStatusLevelEnum level = 3;
string hostname = 4;
}
and a Python dictionary looks like this:
tl_snapshot = {u'status': u'FAILED', u'level': u'TEST_COMPUTER', u'hostname': u'mfg-line12', u'time': u'2022-10-26T13:14:07.831052'}
I'm parsing like so:
from google.protobuf.json_format import ParseDict
ParseDict(tl_snapshot, log_pb2.LogStatusSnapshot())
When I remove the time field, all other fields parse just fine. I'm getting a crazy-seeming error from the ParseDict internal library saying:
ParseError: Failed to parse time field: time data '2022-10' does not match format '%Y-%m-%dT%H:%M:%S'.
As I approach insanity, I've printed out the tl_snapshot["time"] and verified that it is an acceptable RFC 3339 date string of length 26 characters. Unless I'm misusing the API, it very much seems like Google's Protobuf ParseDict functionality is somehow cutting off most of the characters of the date string I pass.
Has anyone else come across this before?? Software versions:
Python 2.7
Protobuf 3.6.1
Your dict's time value is not an RFC3339 compliant value because it does not include a timezone offset.
RFC3339 is required by Timestamp.
If you append Z (for Zulu i.e UTC+0) to the time value i.e. 2022-10-26T13:14:07.831052Z, it will work.
You'll want to ensure that the values you generate for time are RFC3339 compliant. The Python 3.x libraries support this.
Related
I have a one to many relation between session and camp. Now I have to get the max and min dates of all camps combined for a particular session.
I am able to do it like this:
sess = Session.objects.last()
max_min_dates = sess.camp.aggregate(Min('start_date'), Max('end_date'))
But if I try to send this from HttpResponse then I am getting this error:
TypeError: Object of type 'date' is not JSON serializable
So I need to send the formatted date values in that. How can I modify the above code to get the same?
The default encoder for json.dumps() does not support date encoding (ie. can't convert date into str). You can use django encoder instead, it supports a few more data types see the link for more info.
Django Ref
import json
from django.core.serializers.json import DjangoJSONEncoder
json_str = json.dumps(max_min_dates, cls=DjangoJSONEncoder)
When I POST my form, I receive the following exception:
act_date: ["Date has wrong format. Use one of these formats instead: YYYY-MM-DD."]
And the same validation error for other DateFields.
I changed the default date format in extjs (Ext.util.Format.defaultDateFormat= 'Y-m-d') which did not work.
So next I define date format in Django setting:
'DATE_FORMATS': [("%Y-%m-%d"),],
This also hasn't worked.
When you said I changed the default date format in extjs (Ext.util.Format.defaultDateFormat= 'Y-m-d') which did not work. it's the date format don't work or again you'r server validation ?
Cause actually if i do this :
var d = new Date();
Ext.Date.format(d, 'Y-m-d');
That give me : "2019-03-05" and it's seem to be correct.
Have you check inside of you'r POST request the format of the sending date ?
You maybe have a unexpected date format before sending you'r request.
Problem:
I'm currently trying to insert a date time object into my Cassandra database using the following code:
dt_str = '2016-09-01 12:00:00.00000'
dt_thing = datetime.datetime.strptime(dt_str, '%Y-%m-%d %H:%M:%S.%f')
def insert_record(session):
session.execute(
"""
INSERT INTO record_table (last_modified)
VALUES(dt_thing)
"""
)
However, I'm receiving the following error:
cassandra.protocol.SyntaxException: <Error from server: code=2000 [Syntax error in CQL query] message="line 3:17 no viable alternative at input ')' (...record_table (last_modified) VALUES([dt_thing])...)">
Background Info
I'm relatively new to Cassandra and I'm not sure how to go about this. What I'm basically trying to do is add an existing date time value in my database since an earlier version of the code is looking for one but it does not exist yet (hence, why I'm manually adding one).
I'm using Python 2.7 and Cassandra 3.0.
Any input or how to go about this would be great!
I answered a similar question yesterday. But the general idea, is that you'll want to define a prepared statement. Then bind your dt_thing variable to it, like this:
dt_str = '2016-09-01 12:00:00.00000'
dt_thing = datetime.datetime.strptime(dt_str, '%Y-%m-%d %H:%M:%S.%f')
def insert_record(session):
preparedInsert = session.prepare(
"""
INSERT INTO record_table (last_modified)
VALUES (?)
"""
)
session.execute(preparedInsert,[dt_thing])
Also, I don't recommend using a timestamp as a lone PRIMARY KEY (which is the only model for which that INSERT would work).
I am new to coding with python, learning it step by step. I have an assignment to parse text file and update database. The file would have some status like
ticket summary:
Frequency :
Action taken: < something like "restarted server">
Status:
I want to parse this file, fetch the values for the fields like "ticket summary","frequency" etc. and put them in the database, where columns for them are defined. I am reading through python regex and sub string parsing, but not finding how to start. Need help
since you haven't included an example, I will provide one to use here. I am assuming you have already installed mongodb, pymongo and have an instance of it running locally. as an example I am choosing my file to have the following formatting:
Frequency: restarted server, Action taken- none, Status: active
the code will use regex to extract the fields. for demos have a look here: regex demo
import pymongo
import re
client = pymongo.MongoClient()
db = client['some-db']
freqgroup = re.compile(r"(?P<Frequency>Frequency[\:\-]\s?).*?(?=,)", flags=re.I)
actgroup = re.compile(r"(?P<ActionTaken>Action\sTaken[\:\-]\s?).*?(?=,)", flags=re.I)
statgroup = re.compile(r"(?P<Status>Status[\:\-]\s?).*", flags=re.I)
with open("some-file.txt", "rb") as f:
for line in f:
k = re.search(freqgroup, line)
db.posts.insert_one({"Frequency": line[k.end("Frequency"):k.end()]})
k = re.search(actgroup, line)
db.posts.insert_one({"ActionTaken": line[k.end("ActionTaken"):k.end()]})
k = re.search(statgroup, line)
db.posts.insert_one({"Status": line[k.end("Status"):k.end()]})
I'm working on a Google App Engine / Django app and I encountered the following problem:
In my html I have an input for time. The input is free text - the user types "in 1 hour" or "tomorrow at 11am". The text is then sent to the server in AJAX, which parses it using this python library: http://code.google.com/p/parsedatetime/. Once parsed, the server returns an epoch timestamp of the time.
Here is the problem - Google App Engine always runs on UTC. Therefore, lets say that the local time is now 11am and the UTC time is 2am. When I send "now" to the server it will return "2am", which is good because I want the date to be received in UTC time. When I send "in 1 hour" the server will return "3am" which is good, again. However, when I send "at noon" the server will return "12pm" because it thinks that I'm talking about noon UTC - but really I need it to return 3am, which is noon for the request sender.. I can pass on the TZ of the browser that sends the request, but that wont really help me - the parsedatetime library wont take a timezone argument (correct me if I'm wrong). Is there a walk around this? Maybe setting the environments TZ somehow?
Thanks!
What you could do is add the difference using a timedelta object (http://docs.python.org/library/datetime.html)
The offset
here's some (very rough) code to give you the idea:
import parsedatetime
import datetime
my_users_timezone = whatever #replace this with a string that will make sense in the offsets dictionary
utc_timezone_offsets_in_hours = {
'utc' : 0,
'pacific' : -8,
# etc
}
parsed_time = parsedatetime.whatever(input_string)
offset_hours = utc_utc_timezone_offsets_in_hours[my_users_timezone]
final_time = parsed_time + datetime.timedelta(hours=offset_hours)
return final_time
parsedatetime's parse routine expects a timetuple() as the sourceTime parameter and should carry over any timezone information you include in it as I don't recall writing any code that overrode it. If it doesn't then it's a bug and let me know.
You can use code like how the answer above suggested for now to add the TZ offset after the parse() routine returns what it has determined the datetime to be:
import parsedatetime as pdt
cal = pdt.Calendar()
start = datetime.datetime.now().timetuple()
parsed, flag = cal.parse('in 1 hr', start)
then you can take the timetuple value of parsed and use timedelta to add your offset hours