I'm trying to accumulate CellRanges that meet certain condition so that I can set a property on that entire range in one go
rng=None
for c in Cell(1,1).vertical_range:
if c.value and c.value.endswith(' Total'):
rng=rng+c.horizontal_range if rng is not None else c.horizontal_range
rng.font.bold=True
I'm getting error below for ranges accumulating more 30 areas or so. It's not always the same number of areas that cause the error. So I can't put my finger on a specific limit. I do use a work around whereby I set the desired property on the range after say 20 areas were accumulated and then reset the CellRange but it would be good to be able to accumulate all the areas I need subject to whatever contrains on the number of areas excel has
Traceback (most recent call last):
File "27/scriptStarter.py", line 128, in <module>
File "C:\Users\xxxx\rangebug.py", line 3, in <module>
rng=rng+c.horizontal_range if rng is not None else c.horizontal_range
File "27/basic_io.py", line 546, in __add__
File "27/basic_io.py", line 465, in __init__
File "27/basic_io.py", line 1022, in _cell_name_parser
File "27/basic_io.py", line 1136, in _named_range_parser
File "27/iron.py", line 139, in getNamedRange
File "27/dnparser.py", line 95, in checkForErrors
dntypes.NitroException: Exception from HRESULT: 0x800A03EC
This is a bug in our software - we'll look into it and release a fix.
In the meantime, you could do this:
for c in Cell(1,1).vertical_range:
if c.value and c.value.endswith(' Total'):
c.horizontal_range.font.bold=True
Related
I'm training a unet on a google cloud tensorflow vm instance. When I run fit_generator, I get a MemoryError.
When I run the same code locally on tensorflow (cpu version), this does not happen. I have tried increasing the RAM to 13GB on the VM instance (larger than my local machine).
model = unet()
model_checkpoint = ModelCheckpoint('unet_membrane.hdf5', monitor='loss',verbose=1, save_best_only=True)
model.fit_generator(myGene,steps_per_epoch=300,epochs=1,callbacks=[model_checkpoint])
I expect the model to train, but instead I get a MemoryError with the following Traceback
Epoch 1/1
Found 30 images belonging to 1 classes.
Found 30 images belonging to 1 classes.
Traceback (most recent call last):
File "main.py", line 18, in <module>
model.fit_generator(myGene,steps_per_epoch=300,epochs=1,callbacks=[model_checkpoint])
File "/usr/local/lib/python2.7/dist-packages/keras/legacy/interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 1418, in fit_generator
initial_epoch=initial_epoch)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/training_generator.py", line 181, in fit_generator
generator_output = next(output_generator)
File "/usr/local/lib/python2.7/dist-packages/keras/utils/data_utils.py", line 709, in get
six.reraise(*sys.exc_info())
File "/usr/local/lib/python2.7/dist-packages/keras/utils/data_utils.py", line 685, in get
inputs = self.queue.get(block=True).get()
File "/usr/lib/python2.7/multiprocessing/pool.py", line 567, in get
raise self._value
MemoryError
it seems like your machine has run out of memory and can't train while storing all the arrays at the same time. Try to optimize your code to maybe save arrays of data, then loading them when needed so you don't have to store them in your RAM.
I am using pandas v0.14.1 with python 2.7
I have a groupby object and I am trying to pull out a group identified by particular key. The key is in fact in the group:
>>> key in key_groups.groups.keys()
True
but when I try to make the get_group call it fails with a memory error:
>>>> key_groups.get_group(key)
*** MemoryError:
The full stacktrace is:
Traceback (most recent call last):
File "main.py", line 141, in <module>
main(num_days=arguments.days, num_variants=arguments.variants)
File "main.py", line 76, in main
problem, solution = Solver.Solve(request, num_variants)
File "/srv/compunctuator/src/Solver.py", line 49, in Solve
solution = attempt_minimization(t)
File "/srv/compunctuator/src/Solver.py", line 41, in attempt_minimization
t.scruple()
File "/srv/compunctuator/src/Compunctuator.py", line 136, in scruple
self.__iterate__()
File "/srv/compunctuator/src/Compunctuator.py", line 95, in __iterate__
self.__maximize_impressions__()
File "/srv/compunctuator/src/Compunctuator.py", line 583, in __maximize_impressions__
df = key_groups.get_group(key)
File "/srv/compunctuator/.virtualenvs/compunctuator/local/lib/python2.7/site-packages/pandas/core/groupby.py", line 573, in get_group
inds = self._get_index(name)
File "/srv/compunctuator/.virtualenvs/compunctuator/local/lib/python2.7/site-packages/pandas/core/groupby.py", line 429, in _get_index
sample = next(iter(self.indices))
File "/srv/compunctuator/.virtualenvs/compunctuator/local/lib/python2.7/site-packages/pandas/core/groupby.py", line 414, in indices
return self.grouper.indices
File "properties.pyx", line 34, in pandas.lib.cache_readonly.__get__ (pandas/lib.c:36380)
File "/srv/compunctuator/.virtualenvs/compunctuator/local/lib/python2.7/site-packages/pandas/core/groupby.py", line 1253, in indices
return _get_indices_dict(label_list, keys)
File "/srv/compunctuator/.virtualenvs/compunctuator/local/lib/python2.7/site-packages/pandas/core/groupby.py", line 3474, in _get_indices_dict
np.prod(shape))
File "algos.pyx", line 1997, in pandas.algos.groupsort_indexer (pandas/algos.c:37521) MemoryError
If I actually use the dictionary lookup I can get the indices out:
>>>> key_groups.groups[key]
[0, 2]
It seems like everything should work here.
I realize a similar question was asked here pandas get_group causes memory error
but it was never resolved and I thought I could give more details if necessary.
So i have a django site that is giving me this AmbiguousTimeError. I have a job activates when a product is saved that is given a brief timeout before updating my search index. Looks like an update was made in the Daylight Savings Time hour, and pytz cannot figure out what to do with it.
How can i prevent this from happening the next time the hour shifts for DST?
[2012-11-06 14:22:52,115: ERROR/MainProcess] Unrecoverable error: AmbiguousTimeError(datetime.datetime(2012, 11, 4, 1, 11, 4, 335637),)
Traceback (most recent call last):
File "/usr/local/lib/python2.6/dist-packages/celery/worker/__init__.py", line 353, in start
component.start()
File "/usr/local/lib/python2.6/dist-packages/celery/worker/consumer.py", line 369, in start
self.consume_messages()
File "/usr/local/lib/python2.6/dist-packages/celery/worker/consumer.py", line 842, in consume_messages
self.connection.drain_events(timeout=10.0)
File "/usr/local/lib/python2.6/dist-packages/kombu/connection.py", line 191, in drain_events
return self.transport.drain_events(self.connection, **kwargs)
File "/usr/local/lib/python2.6/dist-packages/kombu/transport/virtual/__init__.py", line 760, in drain_events
self._callbacks[queue](message)
File "/usr/local/lib/python2.6/dist-packages/kombu/transport/virtual/__init__.py", line 465, in _callback
return callback(message)
File "/usr/local/lib/python2.6/dist-packages/kombu/messaging.py", line 485, in _receive_callback
self.receive(decoded, message)
File "/usr/local/lib/python2.6/dist-packages/kombu/messaging.py", line 457, in receive
[callback(body, message) for callback in callbacks]
File "/usr/local/lib/python2.6/dist-packages/celery/worker/consumer.py", line 560, in receive_message
self.strategies[name](message, body, message.ack_log_error)
File "/usr/local/lib/python2.6/dist-packages/celery/worker/strategy.py", line 25, in task_message_handler
delivery_info=message.delivery_info))
File "/usr/local/lib/python2.6/dist-packages/celery/worker/job.py", line 120, in __init__
self.eta = tz_to_local(maybe_iso8601(eta), self.tzlocal, tz)
File "/usr/local/lib/python2.6/dist-packages/celery/utils/timeutils.py", line 52, in to_local
dt = make_aware(dt, orig or self.utc)
File "/usr/local/lib/python2.6/dist-packages/celery/utils/timeutils.py", line 211, in make_aware
return localize(dt, is_dst=None)
File "/usr/local/lib/python2.6/dist-packages/pytz/tzinfo.py", line 349, in localize
raise AmbiguousTimeError(dt)
AmbiguousTimeError: 2012-11-04 01:11:04.335637
EDIT: I fixed it temporarily with this code in celery:
celery/worker/job.py # line 120
try:
self.eta = tz_to_local(maybe_iso8601(eta), self.tzlocal, tz)
except:
self.eta = None
I don't want to have changes in a pip installed app, so i need to fix what i can in my code:
This runs when i save my app:
self.task_cls.apply_async(
args=[action, get_identifier(instance)],
countdown=15
)
I'm assuming that i need to somehow detect if i'm in the ambiguous time and adjust countdown.
I think i'm going to have to clear the tasks to fix this, but how can i prevent this from happening the next time the hour shifts for DST?
It's not clear what you're doing (you haven't shown any code), but basically you need to take account for the way the world works. You can't avoid having ambiguous times when you convert from local time to UTC (or to a different zone's local time) when the time goes back an hour.
Likewise you ought to be aware that there are "gap" or "impossible" times, where a reasonable-sounding local time simply doesn't occur.
I don't know what options Python gives you, but ideally an API should let you resolve ambiguous times however you want - whether that's throwing an error, giving you the earlier occurrence, the later occurrence, or something else.
Apparently, Celery solved this issue:
https://github.com/celery/celery/issues/1061
I have a function that will read data from a website, process it, and then load it into MongoDB. When I run this without threading it works fine but as soon as I set up celery tasks that just call this one function I frequently get the following error: "OperationFailure: database error: unauthorized db:dbname lock type:-1"
It's somewhat odd because if I run the non-celery version on multiple terminals, I do not get this error at all.
I suspect it has something to do with there not being an open connection to Mongo although in my code I'm opening one up right before every Mongo call.
The exact exception is below:
Task twitter[a974bfcc-d6ca-4baf-b36f-cae9143ce2d9] raised exception: OperationFailure(u'database error: unauthorized db:data lock type:-1 client:68.193.49.9',)
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/celery/execute/trace.py", line 36, in trace
return cls(states.SUCCESS, retval=fun(*args, **kwargs))
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/celery/app/task/__init__.py", line 232, in __call__
return self.run(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/celery/app/__init__.py", line 172, in run
return fun(*args, **kwargs)
File "/djangoblog/network/tasks.py", line 40, in twitter
n_twitter.GetTweetsTwitter(user)
File "/djangoblog/network/twitter.py", line 255, in GetTweetsTwitter
id = SaveTweet(user, network, tweet)
File "/djangoblog/network/twitter.py", line 150, in SaveTweet
if mmo.Moment.objects(user=user.id,source_id=id,network=network.id).count() == 0:
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/mongoengine/queryset.py", line 933, in count
return self._cursor.count(with_limit_and_skip=True)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/mongoengine/queryset.py", line 563, in _cursor
self._cursor_obj = self._collection.find(self._query,
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/mongoengine/queryset.py", line 493, in _collection
if self._collection_obj.name not in db.collection_names():
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/pymongo/database.py", line 361, in collection_names
names = [r["name"] for r in results]
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/pymongo/cursor.py", line 703, in next
if len(self.__data) or self._refresh():
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/pymongo/cursor.py", line 666, in _refresh
self.__uuid_subtype))
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/pymongo/cursor.py", line 628, in __send_message self.__tz_aware)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/pymongo/helpers.py", line 101, in _unpack_response error_object["$err"])
OperationFailure: database error: unauthorized db:data lock type:-1 client:68.193.49.9
Sorry for the formatting but if you look at the line that starts with mmo.Moment there's a connection being opened right before that's called.
Doing a bit of research it looks as if it has something to do with the way threading is handled in PyMongo - http://api.mongodb.org/python/1.5.1/faq.html#how-does-connection-pooling-work-in-pymongo - I may need to start closing the connections but I'd expect MongoEngine to be doing this..
This is likely due to the fact that you are not calling db.authenticate() when you start the new connection and are using auth on MongoDB.
Regarding the closing of threads, I would recommend making sure you are using connection pooling and letting the driver manage the pools (calling close() or similar manually can lead to a lot of pain).
For more info see the note in the pymongo documentation about using authenticate() in a multi-threaded environment.
Mac OSX 10.7
Python 2.7
Django 1.3
Markdown 2.0.3
I am trying to migrate from a wordpress blog to a django-powered one. I am using pigmonkey's django-basic-apps for it's xml parsing script, but every time I attempt it, markdown keeps raising "markdown.MarkdownException: UnicodeDecodeError: Markdown only accepts unicode or ascii input."
Full traceback:
Traceback (most recent call last):
File "manage.py", line 14, in <module>
execute_manager(settings)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/management/__init__.py", line 438, in execute_manager
utility.execute()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/management/__init__.py", line 379, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/management/base.py", line 191, in run_from_argv
self.execute(*args, **options.__dict__)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/management/base.py", line 220, in execute
output = self.handle(*args, **options)
File "/Users/mycomputer/.virtualenvs/zachswhite/lib/python2.7/site-packages/django_basic_apps-0.8-py2.7.egg/basic/blog/management/commands/wordpress_import.py", line 100, in handle
post.save()
File "/Users/mycomputer/.virtualenvs/zachswhite/lib/python2.7/site-packages/django_basic_apps-0.8-py2.7.egg/basic/blog/models.py", line 77, in save
self.body_rendered = mark_safe(formatter(self.body_rendered, filter_name=self.markup))
File "/Users/mycomputer/.virtualenvs/zachswhite/lib/python2.7/site-packages/django_markup-0.3-py2.7.egg/django_markup/markup.py", line 90, in __call__
return filter_class().render(text, **filter_kwargs)
File "/Users/mycomputer/.virtualenvs/zachswhite/lib/python2.7/site-packages/django_markup-0.3-py2.7.egg/django_markup/filter/markdown_filter.py", line 21, in render
text = markdown(text, **kwargs)
File "/Users/mycomputer/.virtualenvs/zachswhite/lib/python2.7/site-packages/markdown/__init__.py", line 598, in markdown
return md.convert(text)
File "/Users/mycomputer/.virtualenvs/zachswhite/lib/python2.7/site-packages/markdown/__init__.py", line 377, in convert
message(CRITICAL, 'UnicodeDecodeError: Markdown only accepts unicode or ascii input.')
File "/Users/mycomputer/.virtualenvs/zachswhite/lib/python2.7/site-packages/markdown/__init__.py", line 112, in message
raise MarkdownException, text
markdown.MarkdownException: UnicodeDecodeError: Markdown only accepts unicode or ascii input.
There's nothing outside unicode in there, that I've found maybe you guys would have better luck.
I can upload the xml file too, if that would help
This should be fixed as of my most recent commit to my fork of django-basic-apps. The error was caused by passing the post's body and tease through the inline parser before passing it to Markdown (which is necessary for other reasons). I don't fully understand why the error was being thrown, but you can read today's commit message for what I do know:
https://github.com/pigmonkey/django-vellum/commit/1d5fb4743a7fc34e4b6cd415144ad232bbc3f0ad
By the way, I just noticed this question today as I was doing some Googling, trying to understand this error. For future reference, if you contact me through Github, or through my website, or create an Issue for the project on Github, or email me, I'll probably respond a lot faster! I'm always interested to hear about any bugs found in my projects.