I'm struggling to see how I can add a datetime field with an infinity end date.
Setting the default to 'infinity' results in a Django.core exception
django.core.exceptions.ValidationError: ["'infinity' value has an invalid format. It must be in YYYY-MM-DD HH:MM[:ss[.uuuuuu]][TZ] format."]
NB: This is not the same as defining it Null (None). It's specially supported in postgres as just a string condition check for example
SELECT * FROM table WHERE dt = 'infinity'; // or
SELECT * FROM table WHERE NOT isfinite(dt);
Nulls won't show in this, nor can you do a between condition on a Null value unless you COALESCE it, but that will result in a sequential scan.
Any ideas?
Django ORM converts infinity to datetime.max, so consider using datetime.max instead:
$ psql -d yourdb
yourdb=# UPDATE app_yourmodel SET last_login = 'infinity' WHERE id = 1;
$ python3 manage.py shell
>>> from app.models import YourModel
>>> from datetime import datetime
>>>
>>> YourModel.objects.get(id=1).last_login
datetime.datetime(9999, 12, 31, 23, 59, 59, 999999, tzinfo=<UTC>)
>>>
>>> datetime.max
datetime.datetime(9999, 12, 31, 23, 59, 59, 999999)
Related
I have a date of birth (dob) DateField in my Django Model (Author). I tried to annotate age parameter. I searched for many possible ways to do it and each procedure generated some kind of error.
Here I tried in python console first to make sure that the expression would be a valid one:
>>> from datetime import datetime
>>> (datetime.now() - datetime(2000,1,1)).days #output: 7506
First Try:
>>> from django.db.models import F
>>> authors = Author.objects.annotate(age = (datetime.now()-F('dob')).days) #no-error here
>>> print(authors) # Exception thrown here
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/home/puru/Documents/Python/django/prac1/django-master/lib/python3.7/site-packages/Django-3.2-py3.7.egg/django/db/models/query.py", line 324, in __getitem__
qs._fetch_all()
File "/home/puru/Documents/Python/django/prac1/django-master/lib/python3.7/site-packages/Django-3.2-py3.7.egg/django/db/models/query.py", line 1305, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/home/puru/Documents/Python/django/prac1/django-master/lib/python3.7/site-packages/Django-3.2-py3.7.egg/django/db/models/query.py", line 70, in __iter__
for row in compiler.results_iter(results):
File "/home/puru/Documents/Python/django/prac1/django-master/lib/python3.7/site-packages/Django-3.2-py3.7.egg/django/db/models/sql/compiler.py", line 1100, in apply_converters
value = converter(value, expression, connection)
File "/home/puru/Documents/Python/django/prac1/django-master/lib/python3.7/site-packages/Django-3.2-py3.7.egg/django/db/backends/sqlite3/operations.py", line 291, in convert_datefield_value
value = parse_date(value)
File "/home/puru/Documents/Python/django/prac1/django-master/lib/python3.7/site-packages/Django-3.2-py3.7.egg/django/utils/dateparse.py", line 75, in parse_date
match = date_re.match(value)
TypeError: expected string or bytes-like object
Second time I used ExpressionWrapper to define that output type will be DateTimeField
>>> from django.db.models import DateTimeField, ExpressionWrapper
>>> authors = Author.objects.annotate(age = ExpressionWrapper(timezone.now() - F('dob'), output_field=DateTimeField()).days)
AttributeError: 'ExpressionWrapper' object has no attribute 'days'
I have also tried the RawSQL. I am using Sqlite database here. so date('now') would provide me current date.
>>> from django.db.models.expressions import RawSQL
>>> authors = Author.objects.annotate(age=RawSQL("date('now')-dob"))
Traceback (most recent call last):
File "<console>", line 1, in <module>
TypeError: __init__() missing 1 required positional argument: 'params'
So is there any way, I could solve the issue?
You were almost there, the issue was the output_field value. It should be DurationField instead of DateTimeField
age_expr = ExpressionWrapper(timezone.now() - F('dob'), output_field=DateTimeField())
queryset = Author.objects.annotate(age=age_expr)
NOTE: You can't use .days along with ExpressionWrapper since the annotate operation performs in the DB level.
To calculate the age, you may need to use the .total_seconds() method
print(queryset[0].age.total_seconds() / 60 / 60 / 24 / 365.25)
Empty string or list params worked for the RawSQL command.
authors = Author.objects.annotate(age=RawSQL("date('now')-dob", params=""))
I am using zipline to backtest with the local data, but it seems unsuccessful.
from datetime import datetime
import pytz
import pandas as pd
from zipline.algorithm import TradingAlgorithm
import zipline.utils.factory as factory
class BuyApple(TradingAlgorithm):
def handle_data(self, data):
self.order('AAPL', 1)
if __name__ == '__main__':
data = pd.read_csv('AAPL.csv')
simple_algo = BuyApple()
results = simple_algo.run(data)
above is my code, When I run this script, I got the message:
[2015-04-03 01:41:53.712035] WARNING: Loader: No benchmark data found for date range.
start_date=2015-04-03 00:00:00+00:00, end_date=2015-04-03 01:41:53.632300, url=http://ichart.finance.yahoo.com/table.csv?a=3&c=2015&b=3&e=3&d=3&g=d&f=2015&s=%5EGSPC
Traceback (most recent call last):
File "bollinger.py", line 31, in <module>
results = simple_algo.run(data)
File "/home/xinzhou/.local/lib/python2.7/site-packages/zipline-0.7.0-py2.7.egg/zipline/algorithm.py", line 372, in run
source = DataFrameSource(source)
File "/home/xinzhou/.local/lib/python2.7/site-packages/zipline-0.7.0-py2.7.egg/zipline/sources/data_frame_source.py", line 42, in __init__
assert isinstance(data.index, pd.tseries.index.DatetimeIndex)
AssertionError
Then I change my code to below:
from datetime import datetime
import pytz
import pandas as pd
from zipline.algorithm import TradingAlgorithm
import zipline.utils.factory as factory
class BuyApple(TradingAlgorithm):
def handle_data(self, data):
self.order('AAPL', 1)
if __name__ == '__main__':
start = datetime(2000, 1, 9, 14, 30, 0, 0, pytz.utc)
end = datetime(2001, 1, 10, 21, 0, 0, 0, pytz.utc)
data = pd.read_csv('AAPL.csv', parse_dates=True, index_col=0)
sim_params = factory.create_simulation_parameters(
start=start, end=end, capital_base=10000)
sim_params.data_frequency = '1d'
sim_params.emission_rate = '1d'
simple_algo = BuyApple()
results = simple_algo.run(data)
The
assert isinstance(data.index, pd.tseries.index.DatetimeIndex)
AssertionError
is gone. But in my terminal, it keeps in this message:
[2015-04-03 01:44:28.141657] WARNING: Loader: No benchmark data found for date range.
start_date=2015-04-03 00:00:00+00:00, end_date=2015-04-03 01:44:28.028243, url=http://ichart.finance.yahoo.com/table.csv?a=3&c=2015&b=3&e=3&d=3&g=d&f=2015&s=%5EGSPC
How to solve this problem? Thanks.
data.index=pd.to_datetime(data.index)
data.index=data.index.tz_localize(pytz.utc)
The next code works for me.Is a version of the tutorial example "My first Algorithm" (http://www.zipline.io/tutorial/) .Data must be in ascending order by date. Run as a normal python program( python yourfilename.py):
import pytz
from datetime import datetime
from zipline.algorithm import TradingAlgorithm
from zipline.api import order, record, symbol
import pandas as pd
# Load data manually csv
#Date,Open,High,Low,Close,Volume,Adj Close
#1984-09-07,26.5,26.87,26.25,26.5,2981600,3.02
#...
parse = lambda x: pytz.utc.localize(datetime.strptime(x, '%Y-%m-%d'))
data=pd.read_csv('aapl.csv', parse_dates=['Date'], index_col=0,date_parser=parse)
# Define algorithm
def initialize(context):
pass
def handle_data(context, data):
order('Close',10)
record(AAPL=data['Close'])
# Create algorithm object passing in initialize and
# handle_data functions
algo_obj = TradingAlgorithm(initialize=initialize,
handle_data=handle_data)
# Run algorithm
perf_manual = algo_obj.run(data)
# Print
perf_manual.to_csv('output.csv'
I'm using Django 1.4 and PostgreSQL.
In settings.py I have:
USE_TZ = True
Creation of dates always makes offset-aware timezones:
>>> from django.utils import timezone
>>> timezone.now()
datetime.datetime(2014, 9, 1, 7, 48, 12, 636318, tzinfo=<UTC>)
Models from my own app return timezone aware dates.
>>> from fundedbyme.campaign.models import *
>>> e = EquityCampaign.objects.all()[0].created
datetime.datetime(2013, 9, 19, 11, 29, 57, 844642, tzinfo=<UTC>)
From some other third party apps I return offset aware dates.
>>> from paypaladaptive.models import Payment
>>> Payment.objects.all()[0].created_date
datetime.datetime(2013, 5, 14, 11, 58, 47, 713878, tzinfo=<UTC>)
However the User model does not have a timezone aware date.
>>> u = User.objects.all()[0]
>>> u.date_joined
datetime.datetime(2014, 5, 13, 16, 20, 37, 709550)
Why would only the django user model not return offset aware datetimes? This is causing me a serious bug when I use django-registration. In the line linked to below, a native and aware date get compared.
https://bitbucket.org/ubernostrum/django-registration/src/8f242e35ef7c004e035e54b4bb093c32bf77c29f/registration/models.py?at=default#cl-218 it
I've looked in the auth_user table in postgres and the date_joined field is a "timestamp with time zone" field.
Why would I get native dates returned only on the User model?
I m trying to send a simple mail using IPython. I have not set up any models still getting this error. What can be done?
Error :
/home/sourabh/Django/learn/local/lib/python2.7/site-packages/django/db/models/fields/init.py:827: RuntimeWarning: DateTimeField received a naive datetime (2013-09-04 14:14:13.698105) while time zone support is active.
RuntimeWarning)
Tried : The first step is to add USE_TZ = True to your settings file and install pytz (if possible).
Error changed:
(learn)sourabh#sL:~/Django/learn/event$ python manage.py shell
/home/sourabh/Django/learn/local/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py:53: RuntimeWarning: SQLite received a naive datetime (2013-09-05 00:59:32.181872) while time zone support is active.
RuntimeWarning)
The problem is not in Django settings, but in the date passed to the model. Here's how a timezone-aware object looks like:
>>> from django.utils import timezone
>>> import pytz
>>> timezone.now()
datetime.datetime(2013, 11, 20, 20, 8, 7, 127325, tzinfo=pytz.UTC)
And here's a naive object:
>>> from datetime import datetime
>>> datetime.now()
datetime.datetime(2013, 11, 20, 20, 9, 26, 423063)
So if you are passing email date anywhere (and it eventually gets to some model), just use Django's now(). If not, then it's probably an issue with an existing package that fetches date without timezone and you can patch the package, ignore the warning or set USE_TZ to False.
Use django.utils.timezone.make_aware function to make your naive datetime objects timezone aware and avoid those warnings.
It converts naive datetime object (without timezone info) to the one that has timezone info (using timezone specified in your django settings if you don't specify it explicitly as a second argument):
import datetime
from django.conf import settings
from django.utils.timezone import make_aware
naive_datetime = datetime.datetime.now()
naive_datetime.tzinfo # None
settings.TIME_ZONE # 'UTC'
aware_datetime = make_aware(naive_datetime)
aware_datetime.tzinfo # <UTC>
Just to fix the error to set current time
from django.utils import timezone
import datetime
datetime.datetime.now(tz=timezone.utc) # you can use this value
Quick and dirty - Turn it off:
USE_TZ = False
in your settings.py
make sure settings.py has
USE_TZ = True
In your python file:
from django.utils import timezone
timezone.now() # use its value in model field
One can both fix the warning and use the timezone specified in settings.py, which might be different from UTC.
For example in my settings.py I have:
USE_TZ = True
TIME_ZONE = 'Europe/Paris'
Here is a solution; the advantage is that str(mydate) gives the correct time:
>>> from datetime import datetime
>>> from django.utils.timezone import get_current_timezone
>>> mydate = datetime.now(tz=get_current_timezone())
>>> mydate
datetime.datetime(2019, 3, 10, 11, 16, 9, 184106,
tzinfo=<DstTzInfo 'Europe/Paris' CET+1:00:00 STD>)
>>> str(mydate)
'2019-03-10 11:16:09.184106+01:00'
Another equivalent method is using make_aware, see dmrz post.
If you are trying to transform a naive datetime into a datetime with timezone in django, here is my solution:
>>> import datetime
>>> from django.utils import timezone
>>> t1 = datetime.datetime.strptime("2019-07-16 22:24:00", "%Y-%m-%d %H:%M:%S")
>>> t1
datetime.datetime(2019, 7, 16, 22, 24)
>>> current_tz = timezone.get_current_timezone()
>>> t2 = current_tz.localize(t1)
>>> t2
datetime.datetime(2019, 7, 16, 22, 24, tzinfo=<DstTzInfo 'Asia/Shanghai' CST+8:00:00 STD>)
>>>
t1 is a naive datetime and t2 is a datetime with timezone in django's settings.
You can also override settings, particularly useful in tests:
from django.test import override_settings
with override_settings(USE_TZ=False):
# Insert your code that causes the warning here
pass
This will prevent you from seeing the warning, at the same time anything in your code that requires a timezone aware datetime may give you problems. If this is the case, see kravietz answer.
In the model, do not pass the value:
timezone.now()
Rather, remove the parenthesis, and pass:
timezone.now
If you continue to get a runtime error warning, consider changing the model field from DateTimeField to DateField.
If you need to convert the actual date string to date object, I have got rid of the warning by simply using astimezone:
>>> from datetime import datetime, timezone
>>> datetime_str = '2013-09-04 14:14:13.698105'
>>> datetime_object = datetime.strptime(datetime_str, "%Y-%m-%d %H:%M:%S.%f")
>>> datetime_object.astimezone(timezone.utc)
datetime.datetime(2013, 9, 4, 6, 14, 13, 698105, tzinfo=datetime.timezone.utc)
I encountered this warning when using the following model.
from datetime import datetime
class MyObject(models.Model):
my_date = models.DateTimeField(default=datetime.now)
To fix it, I switched to the following default.
from django.utils import timezone
class MyObject(models.Model):
my_date = models.DateTimeField(default=timezone.now)
This might be a bit of a trivial question, but can't I get django now() to be in the time zone defined in settings.TIME_ZONE?
This is what is actually happening:
>>> from django.utils import timezone
>>> timezone.now()
datetime.datetime(2012, 5, 30, 16, 30, 0, 782087, tzinfo=<UTC>)
>>> timezone.get_default_timezone()
<DstTzInfo 'Asia/Singapore' SMT+6:55:00 STD>
Django's source code (as displayed in the chosen answer) explains the concept of timezone.now():
datetime.now() yields the current time (in your active timezone!) without timezone information ("naive datetime"), whereas ...
timezone.now() always yields the current time in UTC (!) with timezone information.
This is irritating at first sight, yes. They could have decided to yield the current time of the active timezone, but they didn't. You can still use timezone.localtime(timezone.now()) to get what you want:
from django.utils import timezone
from datetime import datetime
timezone.get_current_timezone()
# <DstTzInfo 'Antarctica/McMurdo' LMT+11:39:00 STD>
datetime.now()
# datetime.datetime(2014, 8, 19, 20, 8, 8, 440959)
timezone.localtime(timezone.now())
# datetime.datetime(2014, 8, 19, 20, 8, 14, 889429, tzinfo=<DstTzInfo 'Antarctica/McMurdo' NZST+12:00:00 STD>)
timezone.now()
# datetime.datetime(2014, 8, 19, 8, 8, 22, 273529, tzinfo=<UTC>)
datetime.utcnow()
# datetime.datetime(2014, 8, 19, 8, 8, 29, 769312)
For newcomers and ordinary users timezone.localtime(timezone.now()) is probably the most intuitive. A local time which still retains timezone information.
EDIT: The standard library equivalent for a timezone-aware local time is datetime.now().astimezone(). astimezone applies the system local timezone by default, which allows you to correctly convert any timezone-aware datetime to your local time. To set an arbitrary timezone using timezone names you need the pytz package.
from datetime import datetime, timezone
import pytz
datetime.now()
# datetime.datetime(2022, 9, 14, 5, 26, 35, 146551)
datetime.now().astimezone()
# datetime.datetime(2022, 9, 14, 5, 26, 44, 19645, tzinfo=datetime.timezone(datetime.timedelta(seconds=7200), 'CEST'))
datetime.now(tz=timezone.utc)
# datetime.datetime(2022, 9, 14, 3, 26, 51, 203917, tzinfo=datetime.timezone.utc)
datetime.now(tz=timezone.utc).astimezone()
# datetime.datetime(2022, 9, 14, 5, 26, 58, 546724, tzinfo=datetime.timezone(datetime.timedelta(seconds=7200), 'CEST'))
datetime.now(tz=pytz.timezone('Europe/Kiev'))
# datetime.datetime(2022, 9, 14, 6, 27, 38, 714633, tzinfo=<DstTzInfo 'Europe/Kiev' EEST+3:00:00 DST>)
Or I could just read the source:
def now():
"""
Returns an aware or naive datetime.datetime, depending on settings.USE_TZ.
"""
if settings.USE_TZ:
# timeit shows that datetime.now(tz=utc) is 24% slower
return datetime.utcnow().replace(tzinfo=utc)
else:
return datetime.now()
Answer is nope, I have to adjust it myself.
It depends
now()
Returns an aware or naive datetime that represents the current point in time when USE_TZ is True or False respectively.
https://docs.djangoproject.com/en/dev/ref/utils/#django-utils-timezone
So all would indicate that USE_TZ is false in your case, and it's not taking the TZ in consideration.