Tensorboad - add_summary makes my code crashes - python-2.7

I am very new to tensorflow, and I try to display my first tensorboard.
I downloaded and executed ok the board for the example given here
https://www.tensorflow.org/versions/r0.7/how_tos/summaries_and_tensorboard/index.html
Following the method, I have in my code:
weights_hidden = tf.Variable(tf.truncated_normal([image_size * image_size, 1024]), name='weights_hidden')
_ = tf.histogram_summary('weights_hidden', weights_hidden)
and when I run the session
with tf.Session(graph=graph) as session:
merged = tf.merge_all_summaries()
writer = tf.train.SummaryWriter("/tmp/test", session.graph_def)
tf.initialize_all_variables().run()
for step in range(num_steps):
summary_str, l, predictions = session.run(
[optimizer, loss, train_prediction], feed_dict=feed_dict)
if (step % 500 == 0):
writer.add_summary(summary_str, step)
The process crashes with the following error
Traceback (most recent call last):
File "/home/xxx/Desktop/xxx/xxx.py", line 108, in <module>
writer.add_summary(summary_str, step)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/summary_io.py", line 128, in add_summary
event = event_pb2.Event(wall_time=time.time(), summary=summary)
File "/usr/local/lib/python2.7/dist-packages/google/protobuf/internal/python_message.py", line 522, in init
_ReraiseTypeErrorWithFieldName(message_descriptor.name, field_name)
File "/usr/local/lib/python2.7/dist-packages/google/protobuf/internal/python_message.py", line 453, in _ReraiseTypeErrorWithFieldName
six.reraise(type(exc), exc, sys.exc_info()[2])
File "/usr/local/lib/python2.7/dist-packages/google/protobuf/internal/python_message.py", line 520, in init
copy.MergeFrom(new_val)
File "/usr/local/lib/python2.7/dist-packages/google/protobuf/internal/python_message.py", line 1237, in MergeFrom
"expected %s got %s." % (cls.__name__, type(msg).__name__))
TypeError: Parameter to MergeFrom() must be instance of same class: expected Summary got NoneType. for field Event.summary
What am I missing ?
Any help/comment would be very welcome
Thank you very much for the help
K.

You should write:
_, summary_str, l, predictions = session.run(
[optimizer, merged, loss, train_prediction], feed_dict=feed_dict)
I added a 4th argument merged which corresponds to the summary you are trying to get (you were only getting the result of the optimization step).

Related

Getting AttributeError: sender while iterating to get specific values

Getting "AttributeError: sender" is thrown as the exchange query iterates. Same with other values (message_id, etc) too. My only option at this point is to put a try/catch around it and need to refactor a lot of content under the loop. However, I would think the query should not be crashing under normal circumstances due to any data issue. Please let me know what could be going wrong. There appears to be a 'bad' email object that causes it?
kwargs = {"is_read": False}
kwargs["datetime_received__gt"] = some_date_time
filtered_items = my_exchange._service_account.inbox.filter(**kwargs)
filtered_items.page_size = 20
print(filtered_items.count())
3 <-- 3 objects
for sender_obj, msg_id, msg_subj, msg_text, msg_size in filtered_items.values_list("sender", "message_id", "subject", "text_body", "size").iterator():
print(sender_obj)
count = count + 1
print(count)
Mailbox(name='Some User1', email_address='someuser1#myemail.acme', routing_type='SMTP', mailbox_type='Mailbox')
1
Mailbox(name='Some User2', email_address='someuser2#myemail.acme', routing_type='SMTP', mailbox_type='OneOff')
2
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/usr/local/lib/python3.6/site-packages/exchangelib/queryset.py", line 273, in __iter__
yield from self._format_items(items=self._query(), return_format=self.return_format)
File "/usr/local/lib/python3.6/site-packages/exchangelib/queryset.py", line 352, in _item_yielder
yield item_func(i)
File "/usr/local/lib/python3.6/site-packages/exchangelib/queryset.py", line 380, in <lambda>
item_func=lambda i: tuple(f.get_value(i) for f in self.only_fields),
File "/usr/local/lib/python3.6/site-packages/exchangelib/queryset.py", line 380, in <genexpr>
item_func=lambda i: tuple(f.get_value(i) for f in self.only_fields),
File "/usr/local/lib/python3.6/site-packages/exchangelib/fields.py", line 189, in get_value
return getattr(item, self.field.name)
AttributeError: sender
It looks like you are trying to get the sender field of something that is not a Message. Probably your inbox contains a meeting request or some other non-message object.
I'm not sure this is a bug. What did you expect to be the result of getting the sender attribute of something that does not have a sender field?
If you want only Message objects in your list, you can try adding a filter on item_class='IPF.Note'.

Seaborn KDE visualisation, value error on dataset

I am attempting to visualise a KDE plot in Seaborn, but am encountering an error on entering data.
The data is a set of scores ranging from 1-13 and is in the form of a numpy array.
Below is the section of code I'm using.
query_CNM = 'SELECT SCORE from CNMATCH LIMIT 2000'
df = pd.read_sql(query_CNM, conn, index_col = None)
yy = np.array(df)
plot = sns.kdeplot(yy)
Below is the full error that I'm receiving.
Traceback (most recent call last):
File "/Applications/PyCharm CE.app/Contents/helpers/pydev/pydevd.py", line 1758, in <module>
main()
File "/Applications/PyCharm CE.app/Contents/helpers/pydev/pydevd.py", line 1752, in main
globals = debugger.run(setup['file'], None, None, is_module)
File "/Applications/PyCharm CE.app/Contents/helpers/pydev/pydevd.py", line 1147, in run
pydev_imports.execfile(file, globals, locals) # execute the script
File "/Users/uni/Desktop/Proof_Of_Concept/PYQTKDE.py", line 66, in <module>
plot = sns.kdeplot(yy)
File "/Users/uni/.conda/envs/fing.py/lib/python2.7/site-packages/seaborn/distributions.py", line 664, in kdeplot
x, y = data.T
ValueError: need more than 1 value to unpack
I can't seem to find exactly how the data needs to be formatted for sea-born in order to fit a KDE, if any insights can be provided on this it would be greatly appreciated.

TensorVariable to Array

I'm trying to evaluate a theano TensorValue expression:
import pymc3
import numpy as np
with pymc3.Model():
growth = pymc3.Normal('growth_%s' % 'some_name', 0, 10)
x = np.arange(4)
(x * growth).eval()
but get the error
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/danna/.virtualenvs/lib/python2.7/site-packages/theano/gof/graph.py", line 522, in eval
self._fn_cache[inputs] = theano.function(inputs, self)
File "/home/danna/.virtualenvs/lib/python2.7/site-packages/theano/compile/function.py", line 317, in function
output_keys=output_keys)
File "/home/danna/.virtualenvs/lib/python2.7/site-packages/theano/compile/pfunc.py", line 486, in pfunc
output_keys=output_keys)
File "/home/danna/.virtualenvs/lib/python2.7/site-packages/theano/compile/function_module.py", line 1839, in orig_function
name=name)
File "/home/danna/.virtualenvs/lib/python2.7/site-packages/theano/compile/function_module.py", line 1487, in __init__
accept_inplace)
File "/home/danna/.virtualenvs/lib/python2.7/site-packages/theano/compile/function_module.py", line 181, in std_fgraph
update_mapping=update_mapping)
File "/home/danna/.virtualenvs/lib/python2.7/site-packages/theano/gof/fg.py", line 175, in __init__
self.__import_r__(output, reason="init")
File "/home/danna/.virtualenvs/lib/python2.7/site-packages/theano/gof/fg.py", line 346, in __import_r__
self.__import__(variable.owner, reason=reason)
File "/home/danna/.virtualenvs/lib/python2.7/site-packages/theano/gof/fg.py", line 391, in __import__
raise MissingInputError(error_msg, variable=r)
theano.gof.fg.MissingInputError: Input 0 of the graph (indices start from 0), used to compute InplaceDimShuffle{x}(growth_some_name), was not provided and not given a value. Use the Theano flag exception_verbosity='high', for more information on this error.
I tried
Can someone please help me see what the theano variables actually output?
Thank you!
I'm using Python 2.7 and theano 1.0.3
While PyMC3 distributions are TensorVariable objects, they don't technical have any values to be evaluated outside of sampling. If you want values, you have to at least run sampling on the model:
with pymc3.Model():
growth = pymc3.Normal('growth', 0, 10)
trace = pymc3.sample(10)
x = np.arange(4)
x[:, np.newaxis]*trace['growth']
If you want to view node values during sampling, you'd need to use theano.tensor.printing.Print objects. For more info, see the PyMC3 debugging tips.

python script library error while continuously executing script

Error while running python script to get pixel points from google static map image. I got the python script from Google maps - how to get building's polygon coordinates from address?
I use python2.7 to execute the script
Initially when i was running the script i was not getting any error, but after continuous running for 3-4 hours i am getting the following error
Traceback (most recent call last):
File "pyscript.py", line 19, in <module>
imgBuildings = io.imread(urlBuildings)
File "/usr/local/lib/python2.7/dist-packages/skimage/io/_io.py", line 60, in i
with file_or_url_context(fname) as fname:
File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
return self.gen.next()
File "/usr/local/lib/python2.7/dist-packages/skimage/io/util.py", line 29, in
u = urlopen(resource_name)
File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python2.7/urllib2.py", line 435, in open
response = meth(req, response)
File "/usr/lib/python2.7/urllib2.py", line 548, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python2.7/urllib2.py", line 473, in error
return self._call_chain(*args)
File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
result = func(*args)
File "/usr/lib/python2.7/urllib2.py", line 556, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 403: Forbidden
AS i am new to python i am not sure how to fix it ? Is this some kind of cache issue?
Help very much appreciated.
I've seen this problem quite a lot and its due to intermittent network drop errors. There is a recursive trick with try/catch exception handling that will avoid this ever happening, even if your network goes down for hours.
To explain: You attempt a download. If it fails, the download will attempt a recursive retry again 1/4,1/2,1,2,4,8,... seconds later, waiting up to 1 hour to get the next download. If you are working in a company for instance, the network might go down over the weekend, but your code will just poll for 1 hour (maximum) and then recover again when the network is fixed.
import time
def recursiveBuildingGetter( urlBuildings, waitTime=0.25 ):
try:
imgBuildings = io.imread(urlBuildings)
except:
print "Warning: Failure at time %f secs for %s" % ( waitTime, str(urlBuildings) )
waitTime = waitTime * 2.0
if ( waitTime > 3600.0 ):
waitTime = 3600.0
time.sleep(waitTime)
imgBuildings = recursiveBuildingGetter( urlBuilding, waitTime )
if ( waitTime == 3600.0 ):
waitTime = 0.25
return imgBuildings

Celery task errors when getting facebook picture

I have the following celery task:
#task
def get_users_facebook_as_profile_icon(user_id, facebook_id):
logger.info('Grabbing users facebook picture')
url = "http://graph.facebook.com/%s/picture?type=large" % facebook_id
import requests
response = requests.get(url)
if response.status_code != 200:
raise Exception("Could not get facebook profile picture")
...
I have more after this, but I keep getting the following error:
"AssertionError('PID check failed. RNG must be re-initialized after fork(). Hint: Try Random.atfork()',)"
Task was called with args: (3246, 17500596) kwargs: {}.
The contents of the full traceback was:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/celery/app/trace.py", line 240, in trace_task
R = retval = fun(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/celery/app/trace.py", line 437, in __protected_call__
return self.run(*args, **kwargs)
File "/home/ubuntu/mounzawebsite/mounza/celery_tasks/login_registration.py", line 42, in get_users_facebook_as_profile_icon
hashname = user.generate_picture_name()
File "/home/ubuntu/mounzawebsite/mounza/web/models.py", line 515, in generate_picture_name
return generate_random_name(None)
File "/home/ubuntu/mounzawebsite/mounza/web/models.py", line 40, in generate_random_name
str(random.randint(1, 99982098098908237)) +
File "/usr/lib/python2.7/dist-packages/Crypto/Random/__init__.py", line 41, in get_random_bytes
return _UserFriendlyRNG.get_random_bytes(n)
File "/usr/lib/python2.7/dist-packages/Crypto/Random/_UserFriendlyRNG.py", line 213, in get_random_bytes
return _get_singleton().read(n)
File "/usr/lib/python2.7/dist-packages/Crypto/Random/_UserFriendlyRNG.py", line 163, in read
return _UserFriendlyRNG.read(self, bytes)
File "/usr/lib/python2.7/dist-packages/Crypto/Random/_UserFriendlyRNG.py", line 122, in read
self._check_pid()
File "/usr/lib/python2.7/dist-packages/Crypto/Random/_UserFriendlyRNG.py", line 138, in _check_pid
raise AssertionError("PID check failed. RNG must be re-initialized after fork(). Hint: Try Random.atfork()")
AssertionError: PID check failed. RNG must be re-initialized after fork(). Hint: Try Random.atfork()
I tried digging into this online, not able to find the root cause. but this is the only task where this error occurs. The only difference is that i'm downloading an image from Facebook, but I never see this issue anywhere else, including other tasks where I download images.
The URL works perfectly if I do it through a web browser, but it's only via this task it fails. Is there anything else that could contribute to this??
I have exhausted all attempts in fixing this :(
Here is why:
http://comments.gmane.org/gmane.comp.python.amqp.celery.user/3664
always run teh below:
Crypto.Random.atfork()
When a new worker process is initialized. Done and done.