I'm having a problem when building an exe from my scripts that use the pytz lib. I'm constantly getting the error:
File "pytz\__init__.pyc", line 180, in timezone UnknownTimeZoneError: 'Europe\Ljubljana'
No matter how I build the exe I get this error. Running the script works. I tried all suggestions that were posted here and on other sites.
My setup.py file:
from distutils.core import setup
import os.path
try:
import py2exe
has_py2exe = True
except ImportError, e:
has_py2exe = False
myScript=__import__("myScript")
options = {'py2exe': {'packages': ['pytz']}}
options['py2exe'] = {'dist_dir': 'dist'}
setup(
name="myScript",
version=myScript.CONST_VERSION,
console=[{
'script': 'myScript.py',
'copyright': 'None',
'company_name': 'None'
}],
options=options,
)
if has_py2exe:
import zipfile
zipfile_path = os.path.join(options['py2exe']['dist_dir'], 'library.zip')
z = zipfile.ZipFile(zipfile_path, 'a')
import pytz
assert (pytz.__file__.endswith('__init__.pyc') or pytz.__file__.endswith('__init__.py')), pytz.__file__
zoneinfo_dir = os.path.join(os.path.dirname(pytz.__file__), 'zoneinfo')
disk_basedir = os.path.dirname(os.path.dirname(pytz.__file__))
for absdir, directories, filenames in os.walk(zoneinfo_dir):
assert absdir.startswith(disk_basedir), (absdir, disk_basedir)
zip_dir = absdir[len(disk_basedir):]
for f in filenames:
z.write(os.path.join(absdir, f), os.path.join(zip_dir, f))
z.close()
This builds the exe and includes the zoneinfo dir and all timezone files to library.zip.
I only use the pytz in one function where I convert the CEST timestamp to UTC timestamp to store on the server.
The function:
from datetime import datetime
import pytz
def date_time_utc(date_str):
date_tz = pytz.timezone("Europe/Ljubljana")
fmt = '%Y-%m-%d %H:%M:%S'
date_str_dt_object = datetime.strptime(date_str, fmt)
date_str_dt_object = date_tz.localize(date_str_dt_object)
date_utc = date_str_dt_object.astimezone(pytz.timezone('UTC'))
return date_utc.strftime(fmt)
The setup script for the pytz is from here. All other things like this script and any other suggested fixes raised the exactly same error.
Has any one figured out how to get this working?
Related
I was running this file fine yesterday and now I'm getting an AttributeError when I tried running it today. Here is the code I'm trying to run:
from openpyxl import load_workbook
def read_in():
wb = load_workbook('GenerateModel.xlsx')
ws = wb.get_sheet_by_name('Sheet1')
da_name = []
for i in range(1, ws.max_row+1):
if ws.cell(row=i,column=3).value != None and (
ws.cell(row=i,column=3).value != u'DA Name'):
da_name.append(ws.cell(row=i,column=3).value.encode('ascii'))
start_date = ws.cell(row=4, column=4).value
end_date = ws.cell(row=4, column=5).value
if start_date == None or end_date == None:
raise ValueError('Date cannot be left blank')
if start_date > end_date:
raise ValueError('Start Date must be less than End Date')
And here is the error that I get:
line 28, in read_in
for i in range(1, ws.max_row+1):
AttributeError: 'Worksheet' object has no attribute 'max_row'
I tried running another python script and I'm getting an ImportError that says
from openpyxl.styles import PatternFill
ImportError: cannot import name PatternFill
This makes me think that there is something wrong with the openpyxl module. I installed it using pip and I'm using the Spyder IDE. Thanks for the help.
It says '1.8.5'
Your openpyxl version is seriously outdated - hence the absence of max_row attribute on a WorkSheet class.
You need to figure out what Python executable is used when you run your script from the Spyder IDE and then update openpyxl in that environment:
pip install --upgrade openpyxl
(currently latest openpyxl is 2.4.8)
I would like to change the timestamp in the log file so that it reflects my current time zone so that i can debug errors at a faster rate,
is it possible that i can change the time zone in the log file ?
currently my config is:
logging.basicConfig(filename='audit.log',
filemode='w',
level=logging.INFO,
format='%(asctime)s %(message)s',
datefmt='%m/%d/%Y %I:%M:%S %p')
How to log the timezone
%Z from strftime format
Windows
>>> import logging
>>> logging.basicConfig(format="%(asctime)s %(message)s", datefmt="%m/%d/%Y %I:%M:%S %p %Z")
>>> logging.error('test')
11/03/2017 02:29:54 PM Mountain Daylight Time test
Linux
>>> import logging
>>> logging.basicConfig(format="%(asctime)s %(message)s", datefmt="%m/%d/%Y %I:%M:%S %p %Z")
>>> logging.error('test')
11/03/2017 02:30:50 PM MDT test
If the question is
How do I log in a different timezone than the local time on the server?
part of the answer is logging.Formatter.converter, however, you have to understand naive and aware datetime objects. Unless you want to write your own timezone module, I highly suggest the pytz library (pip install pytz). Python 3 includes a UTC and UTC offset timezone, but there's rules you'll have to implement for daylight saving or other offsets, so I would suggest the pytz library, even for python 3.
For example,
>>> import datetime
>>> utc_now = datetime.datetime.utcnow()
>>> utc_now.isoformat()
'2019-05-21T02:30:09.422638'
>>> utc_now.tzinfo
(None)
If I apply a timezone to this datetime object, the time won't change (or will issue a ValueError for < python 3.7ish).
>>> mst_now = utc_now.astimezone(pytz.timezone('America/Denver'))
>>> mst_now.isoformat()
'2019-05-21T02:30:09.422638-06:00'
>>> utc_now.isoformat()
'2019-05-21T02:30:09.422638'
However, if instead, I do
>>> import pytz
>>> utc_now = datetime.datetime.now(tz=pytz.timezone('UTC'))
>>> utc_now.tzinfo
<UTC>
now we can create a properly translated datetime object in whatever timezone we wish
>>> mst_now = utc_now.astimezone(pytz.timezone('America/Denver'))
>>> mst_now.isoformat()
'2019-05-20T20:31:44.913939-06:00'
Aha! Now to apply this to the logging module.
Epoch timestamp to string representation with timezone
The LogRecord.created attribute is set to the time when the LogRecord was created (as returned by time.time()), from the time module. This returns a timestamp (seconds since the epoch). You can do your own translation to a given timezone, but again, I suggest pytz, by overriding the converter.
import datetime
import logging
import pytz
class Formatter(logging.Formatter):
"""override logging.Formatter to use an aware datetime object"""
def converter(self, timestamp):
dt = datetime.datetime.fromtimestamp(timestamp)
tzinfo = pytz.timezone('America/Denver')
return tzinfo.localize(dt)
def formatTime(self, record, datefmt=None):
dt = self.converter(record.created)
if datefmt:
s = dt.strftime(datefmt)
else:
try:
s = dt.isoformat(timespec='milliseconds')
except TypeError:
s = dt.isoformat()
return s
Python 3.5, 2.7
>>> logger = logging.root
>>> handler = logging.StreamHandler()
>>> handler.setFormatter(Formatter("%(asctime)s %(message)s"))
>>> logger.addHandler(handler)
>>> logger.setLevel(logging.DEBUG)
>>> logger.debug('test')
2019-05-20T22:25:10.758782-06:00 test
Python 3.7
>>> logger = logging.root
>>> handler = logging.StreamHandler()
>>> handler.setFormatter(Formatter("%(asctime)s %(message)s"))
>>> logger.addHandler(handler)
>>> logger.setLevel(logging.DEBUG)
>>> logger.debug('test')
2019-05-20T22:29:21.678-06:00 test
Substitute America/Denver with America/Anchorage for the posix timezone as defined by pytz
>>> next(_ for _ in pytz.common_timezones if 'Alaska' in _)
'US/Alaska'
US/Alaska is deprecated
>>> [_ for _ in pytz.all_timezones if 'Anchorage' in _]
['America/Anchorage']
Local
If you got to this question and answers looking for how to log the local timezone, then instead of hardcoding the timezone, get tzlocal (pip install tzlocal) and replace
tzinfo = pytz.timezone('America/Denver')
with
tzinfo = tzlocal.get_localzone()
Now it will work on whatever server runs the script, with the timezone on the server.
Caveat when not logging UTC
I should add, depending on the application, logging in local time zones can create ambiguity or at least confusion twice a year, where 2 AM is skipped or 1 AM repeats, and possibly others.
#!/usr/bin/python
from datetime import datetime
from pytz import timezone
import logging
def timetz(*args):
return datetime.now(tz).timetuple()
tz = timezone('Asia/Shanghai') # UTC, Asia/Shanghai, Europe/Berlin
logging.Formatter.converter = timetz
logging.basicConfig(
format="%(asctime)s %(levelname)s: %(message)s",
level=logging.INFO,
datefmt="%Y-%m-%d %H:%M:%S",
)
logging.info('Timezone: ' + str(tz))
Using pytz to define a timezone relative to UTC.
Based on the example by: secsilm
#!/usr/bin/env python
from datetime import datetime
import logging
import time
from pytz import timezone, utc
def main():
logging.basicConfig(format="%(asctime)s %(message)s",
datefmt="%Y-%m-%d %H:%M:%S")
logger = logging.getLogger(__name__)
logger.error("default")
logging.Formatter.converter = time.localtime
logger.error("localtime")
logging.Formatter.converter = time.gmtime
logger.error("gmtime")
def customTime(*args):
utc_dt = utc.localize(datetime.utcnow())
my_tz = timezone("US/Eastern")
converted = utc_dt.astimezone(my_tz)
return converted.timetuple()
logging.Formatter.converter = customTime
logger.error("customTime")
# to find the string code for your desired tz...
# print(pytz.all_timezones)
# print(pytz.common_timezones)
if __name__ == "__main__":
main()
Ostensibly the pytz package is the blessed way of converting time zones in Python. So we start with datetime, convert, then get the (immutable) time_tuple to match return type of the time methods
Setting the logging.Formatter.converter function is recommended by this answer: (Python logging: How to set time to GMT).
Find your favorite TZ code by uncommenting the end lines
just add this pythonic line to your code (using pytz and datetime):
from pytz import timezone
from datetime import datetime
import logging
logging.Formatter.converter = lambda *args: datetime.now(tz=timezone('tz string name')).timetuple()
# quoting Ryan J McCall: to find the string name for your desired timezone...
# print(pytz.all_timezones)
# or print(pytz.common_timezones)
An alternative solution if you want to use logging configuration function:
import pytz
import logging
import logging.config
from datetime import datetime
tz = pytz.timezone('Asia/Tokyo')
class TokyoFormatter(logging.Formatter):
converter = lambda *args: datetime.now(tz).timetuple()
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'Tokyo': {
'()': TokyoFormatter,
'format': '%(asctime)s %(levelname)s: %(message)s',
'datefmt': '%Y-%m-%d %H:%M:%S'
},
},
'handlers': {
'console': {
'class': 'logging.StreamHandler',
'formatter': 'Tokyo'
},
},
'loggers': {
'foo': {
'handlers': ['console'],
'level': 'INFO'
},
}
}
logging.config.dictConfig(LOGGING)
logger = logging.getLogger('foo')
logger.info('Just a test.')
Define the logging formatter, e.g., "TokyoFormatter". It has an attibute "converter", finishing the job of converting the time zone.
For more details, please refer to Customizing handlers with dictConfig().
import logging, time
from datetime import datetime, timedelta
logger = logging.getLogger(__name__)
converter = lambda x, y: (datetime.utcnow() - timedelta(
hours=7 if time.localtime().tm_isdst else 6)
).timetuple()
logging.Formatter.converter = converter
Edited as Elias points out the original answer didn't check for DST.
If you know your utc offset, you can define a function to correct the time and then pass it to logging.Formatter.converter.
For example, you want to convert the time to UTC+8 timezone, then:
import logging
import datetime
def beijing(sec, what):
'''sec and what is unused.'''
beijing_time = datetime.datetime.now() + datetime.timedelta(hours=8)
return beijing_time.timetuple()
logging.Formatter.converter = beijing
logging.basicConfig(
format="%(asctime)s %(levelname)s: %(message)s",
level=logging.INFO,
datefmt="%Y-%m-%d %H:%M:%S",
)
Just change the hours in datetime.timedelta(hours=8) depending on your situation.
Reference: https://alanlee.fun/2019/01/06/how-to-change-logging-date-timezone/
I am using zipline to backtest with the local data, but it seems unsuccessful.
from datetime import datetime
import pytz
import pandas as pd
from zipline.algorithm import TradingAlgorithm
import zipline.utils.factory as factory
class BuyApple(TradingAlgorithm):
def handle_data(self, data):
self.order('AAPL', 1)
if __name__ == '__main__':
data = pd.read_csv('AAPL.csv')
simple_algo = BuyApple()
results = simple_algo.run(data)
above is my code, When I run this script, I got the message:
[2015-04-03 01:41:53.712035] WARNING: Loader: No benchmark data found for date range.
start_date=2015-04-03 00:00:00+00:00, end_date=2015-04-03 01:41:53.632300, url=http://ichart.finance.yahoo.com/table.csv?a=3&c=2015&b=3&e=3&d=3&g=d&f=2015&s=%5EGSPC
Traceback (most recent call last):
File "bollinger.py", line 31, in <module>
results = simple_algo.run(data)
File "/home/xinzhou/.local/lib/python2.7/site-packages/zipline-0.7.0-py2.7.egg/zipline/algorithm.py", line 372, in run
source = DataFrameSource(source)
File "/home/xinzhou/.local/lib/python2.7/site-packages/zipline-0.7.0-py2.7.egg/zipline/sources/data_frame_source.py", line 42, in __init__
assert isinstance(data.index, pd.tseries.index.DatetimeIndex)
AssertionError
Then I change my code to below:
from datetime import datetime
import pytz
import pandas as pd
from zipline.algorithm import TradingAlgorithm
import zipline.utils.factory as factory
class BuyApple(TradingAlgorithm):
def handle_data(self, data):
self.order('AAPL', 1)
if __name__ == '__main__':
start = datetime(2000, 1, 9, 14, 30, 0, 0, pytz.utc)
end = datetime(2001, 1, 10, 21, 0, 0, 0, pytz.utc)
data = pd.read_csv('AAPL.csv', parse_dates=True, index_col=0)
sim_params = factory.create_simulation_parameters(
start=start, end=end, capital_base=10000)
sim_params.data_frequency = '1d'
sim_params.emission_rate = '1d'
simple_algo = BuyApple()
results = simple_algo.run(data)
The
assert isinstance(data.index, pd.tseries.index.DatetimeIndex)
AssertionError
is gone. But in my terminal, it keeps in this message:
[2015-04-03 01:44:28.141657] WARNING: Loader: No benchmark data found for date range.
start_date=2015-04-03 00:00:00+00:00, end_date=2015-04-03 01:44:28.028243, url=http://ichart.finance.yahoo.com/table.csv?a=3&c=2015&b=3&e=3&d=3&g=d&f=2015&s=%5EGSPC
How to solve this problem? Thanks.
data.index=pd.to_datetime(data.index)
data.index=data.index.tz_localize(pytz.utc)
The next code works for me.Is a version of the tutorial example "My first Algorithm" (http://www.zipline.io/tutorial/) .Data must be in ascending order by date. Run as a normal python program( python yourfilename.py):
import pytz
from datetime import datetime
from zipline.algorithm import TradingAlgorithm
from zipline.api import order, record, symbol
import pandas as pd
# Load data manually csv
#Date,Open,High,Low,Close,Volume,Adj Close
#1984-09-07,26.5,26.87,26.25,26.5,2981600,3.02
#...
parse = lambda x: pytz.utc.localize(datetime.strptime(x, '%Y-%m-%d'))
data=pd.read_csv('aapl.csv', parse_dates=['Date'], index_col=0,date_parser=parse)
# Define algorithm
def initialize(context):
pass
def handle_data(context, data):
order('Close',10)
record(AAPL=data['Close'])
# Create algorithm object passing in initialize and
# handle_data functions
algo_obj = TradingAlgorithm(initialize=initialize,
handle_data=handle_data)
# Run algorithm
perf_manual = algo_obj.run(data)
# Print
perf_manual.to_csv('output.csv'
I have installed django 1.6 using pip on Windows 7. When I try to run django-admin.py, I get this error
C:\django-admin.py startproject test007
Traceback (most recent call last):
File "C:\Python27\Scripts\django-admin.py", line 2 in ?
from django.core import management
File "C:\Python27\Lib\site-packages\django\core\management\__init__.py", line 55
except ImportError as e:
^
SyntaxError: invalid syntax
in the \management__init__.py, the imports are
import collections
import os
import sys
from optparse import OptionParser, NO_DEFAULT
import imp
from django.core.exceptions import ImproperlyConfigured
from django.core.management.base import BaseCommand, CommandError, handle_default_options
from django.core.management.color import color_style
from django.utils.importlib import import_module
from django.utils import six
# For backwards compatibility: get_version() used to be in this module.
from django import get_version
Here's the relevant block from the same file
parts = app_name.split('.')
parts.append('management')
parts.reverse()
part = parts.pop()
path = None
# When using manage.py, the project module is added to the path,
# loaded, then removed from the path. This means that
# testproject.testapp.models can be loaded in future, even if
# testproject isn't in the path. When looking for the management
# module, we need look for the case where the project name is part
# of the app_name but the project directory itself isn't on the path.
try:
f, path, descr = imp.find_module(part, path)
except ImportError as e:
if os.path.basename(os.getcwd()) != part:
raise e
else:
if f:
f.close()
while parts:
part = parts.pop()
f, path, descr = imp.find_module(part, [path] if path else None)
if f:
f.close()
return path
in which the same line 55 from the traceback is in the try/except block towards the bottom. I've uninstalled and reinstalled but, to no avail.
It works when I give the/full/path/to/django-admin.py but it shouldn't be required.
I am trying to have a Django script run every 5 minutes via cron on my dev laptop (Mac OS X). Here is the code in the script:
import sys
import os
def setup_environment():
pathname = os.path.dirname(sys.argv[0])
sys.path.append(os.path.abspath(pathname))
sys.path.append(os.path.normpath(os.path.join(os.path.abspath(pathname), '../')))
os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'
setup_environment()
from common.models import TweetCache
import datetime
def main():
print "(%s) Caching tweets..." % str(datetime.datetime.now())
previous_tweets = TweetCache.objects.filter(username='atmospherian')
for prev in previous_tweets:
prev.delete()
import twitter
api = twitter.Api()
tweets = api.GetUserTimeline('atmospherian')
for t in tweets:
tc = TweetCache(username='atmospherian', date=t.created_at, text=t.text)
tc.save()
if __name__ == '__main__':
main()
crontab:
*/5 * * * * python /absolute/path/to/tweet_cache.py
error from system mail:
X-Cron-Env: <SHELL=/bin/sh>
X-Cron-Env: <PATH=/usr/bin:/bin>
X-Cron-Env: <LOGNAME=jason>
X-Cron-Env: <USER=jason>
X-Cron-Env: <HOME=/Users/jason>
Date: Tue, 16 Feb 2010 17:45:00 -0500 (EST)
Traceback (most recent call last):
File "/Users/jason/Development/bandistry/tweet_cache.py", line 22, in <module>
from common.models import TweetCache
File "/Users/jason/Development/bandistry/common/models.py", line 1, in <module>
from django.db import models
ImportError: No module named django.db
can anyone tell me what im doing wrong?
sys.argv[0] isn't always the most reliable way to get the current file's path. I recommend this modification:
pathname = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, pathname)
sys.path.insert(0, os.path.abspath(os.path.join(pathname, '..')))
Note the use of sys.path.insert instead of sys.path.append, and the use of file. Also, using abspath on file -before- dirname reduces the chance you reduce the entire filename down to the empty string or simply '.' which may not even be accurate.
Also, is the django package installed in one of those two paths you add? If not, you sill need to add that path
Finally, a minor quip, probably unrelated to your django import issue, but you really should be doing:
os.environ['DJANGO_SETTINGS_MODULE'] = 'bandistry.settings'
It may work without, but it's better if you put your entire django application in a package. This reduces the chance of you obscuring other package names.