Tensorboard callback with keras fit_generator, 'Function' has no attribute 'fetch_callbacks' - python-2.7

I am trying to run a model using keras's fit_generator with a tensorboard_callback for profiling a specific epoch. I am running the following code for the generator:
def gen(source):
loopable = iter(source)
for batch in loopable:
yield (batch[0], batch[1])
In the main training script I am instantiating the generator and using the model with a tensorboard callback as follows:
train_gen = gen(train_datasource)
log_dir="logs/profile/" + datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1, profile_batch = 3)
m.fit_generator(train_gen, epochs=5, steps_per_epoch=500, use_multiprocessing=True, workers=32, callbacks=[tensorboard_callback])
The main issue I am facing is that the training always halts with the error 'Function' has no attribute 'fetch_callbacks' with the following stack trace:
m.fit_generator(train_gen, epochs=5, steps_per_epoch=500, use_multiprocessing=True, workers=32, callbacks=[tensorboard_callback])
File "/usr/local/lib/python2.7/dist-packages/keras/legacy/interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 1418, in fit_generator
initial_epoch=initial_epoch)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/training_generator.py", line 177, in fit_generator
callbacks.on_epoch_begin(epoch)
File "/usr/local/lib/python2.7/dist-packages/keras/callbacks.py", line 65, in on_epoch_begin
callback.on_epoch_begin(epoch, logs)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/keras/callbacks_v1.py", line 386, in on_epoch_begin
self.merged] = self._fetch_callback
AttributeError: 'Function' object has no attribute 'fetch_callbacks'
I am using tensorflow 1.15 also tried downgrading to 1.14 but still no success. I am trying to train with the tensorboard callback to debug the performance for a specific epoch other than the first one. But so far my attempts have failed to make the callback function correctly. I made sure the GPU is running and detected correctly too.
Any help would be much appreciated.

I ended up using tf.keras fit function instead of fit generator and it worked correctly as expected:
m.fit(x=train_gen, epochs=5, steps_per_epoch=500, use_multiprocessing=True, workers=8, callbacks=[tensorboard_callback])

Related

pyomo mindtpy example program when run becomes unfeasible for binary variable

So I installed pyomo, glpk, and ipopt with anaconda,
When I run the example code here: https://pyomo.readthedocs.io/en/stable/contributed_packages/mindtpy.html
from pyomo.environ import *
model = ConcreteModel()
model.x = Var(bounds=(1.0,10.0),initialize=5.0)
model.y = Var(within=Binary)
model.c1 = Constraint(expr=(model.x-3.0)**2 <= 50.0*(1-model.y))
model.c2 = Constraint(expr=model.x*log(model.x)+5.0 <= 50.0*(model.y))
model.objective = Objective(expr=model.x, sense=minimize)
SolverFactory('mindtpy').solve(model, mip_solver='glpk', nlp_solver='ipopt',tee=True)
model.objective.display()
model.display()
model.pprint()
I get the output that the binary variable has apparently become infeasible:
python minlpex.py
INFO: ---Starting MindtPy---
INFO: Original model has 2 constraints (2 nonlinear) and 0 disjunctions, with
2 variables, of which 1 are binary, 0 are integer, and 1 are continuous.
INFO: NLP 1: Solve relaxed integrality
INFO: NLP 1: OBJ: 1.0 LB: 1.0 UB: inf
INFO: ---MindtPy Master Iteration 0---
INFO: MIP 1: Solve master problem.
WARNING: Empty constraint block written in LP format - solver may error
Traceback (most recent call last):
File "minlpex.py", line 13, in <module>
op.SolverFactory('mindtpy').solve(model, mip_solver='glpk', nlp_solver='ipopt',tee=True)
File "/anaconda3/envs/py36/lib/python3.6/site-packages/pyomo/contrib/mindtpy/MindtPy.py", line 370, in solve
MindtPy_iteration_loop(solve_data, config)
File "/anaconda3/envs/py36/lib/python3.6/site-packages/pyomo/contrib/mindtpy/iterate.py", line 30, in MindtPy_iteration_loop
handle_master_mip_optimal(master_mip, solve_data, config)
File "/anaconda3/envs/py36/lib/python3.6/site-packages/pyomo/contrib/mindtpy/mip_solve.py", line 62, in handle_master_mip_optimal
config)
File "/anaconda3/envs/py36/lib/python3.6/site-packages/pyomo/contrib/gdpopt/util.py", line 199, in copy_var_list_values
v_to.set_value(value(v_from, exception=False))
File "/anaconda3/envs/py36/lib/python3.6/site-packages/pyomo/core/base/var.py", line 173, in set_value
if valid or self._valid_value(val):
File "/anaconda3/envs/py36/lib/python3.6/site-packages/pyomo/core/base/var.py", line 185, in _valid_value
"domain %s" % (val, type(val), self.domain))
ValueError: Numeric value `0.22709088987977885` (<class 'float'>) is not in domain Binary
So I was a little confused, since this was a code provided, I would not expect it to error like this. So I feel like I'm messing something up or I am missing some required library?
Thanks a lot.
Looks like something must be wrong with the conda pyomo install or ipopt install.
When I reinstalled using pip for ipopt and compiling pyomo from github source everything works fine.

py2neo 2.0 bind and push node errors

Using Py2Neo 2.0 and Pycharm Community Edition 4
I'm trying to update a node. First I get the node object, change a node property, bind to the database, and then push the node. I get a slew of errors. Here is the code.
user_node = Graph().find_one('USER',
property_key='email',
property_value='marnee#marnee.com')
user_properties['mission_statement'] = 'New mission statement'
user_node.bind(uri=Graph().uri)
user_node.push()
The node is found, it does have a mission_statement property. The exception seems to happen on .push(). The Graph() uri is good, too.
Below are the errors.
I had been able to do this successfully about a week ago. I have not updated any packages recently.
The really weird part is that if I have a breakpoint and run this in debug mode I do not get any errors and the node is updated successfully.
Traceback (most recent call last):
File "C:/Users/Marnee Dearman/PycharmProjects/AgoraDev/py2neo_2.0_tests/create_rel_int_loc.py", line 27, in <module>
user_node.push()
File "C:\Users\Marnee Dearman\PycharmProjects\VirtualEnvs\AgoraDev\lib\site-packages\py2neo\core.py", line 1519, in push
batch.push()
File "C:\Users\Marnee Dearman\PycharmProjects\VirtualEnvs\AgoraDev\lib\site-packages\py2neo\batch\push.py", line 73, in push
self.graph.batch.run(self)
File "C:\Users\Marnee Dearman\PycharmProjects\VirtualEnvs\AgoraDev\lib\site-packages\py2neo\batch\core.py", line 99, in run
response = self.post(batch)
File "C:\Users\Marnee Dearman\PycharmProjects\VirtualEnvs\AgoraDev\lib\site-packages\py2neo\batch\core.py", line 88, in post
data.append(dict(job, id=i))
File "C:\Users\Marnee Dearman\PycharmProjects\VirtualEnvs\AgoraDev\lib\site-packages\py2neo\batch\core.py", line 232, in __iter__
yield "to", self.target.uri_string
File "C:\Users\Marnee Dearman\PycharmProjects\VirtualEnvs\AgoraDev\lib\site-packages\py2neo\batch\core.py", line 180, in uri_string
uri_string = self.entity.ref
File "C:\Users\Marnee Dearman\PycharmProjects\VirtualEnvs\AgoraDev\lib\site-packages\py2neo\core.py", line 1421, in ref
return "node/%s" % self._id
File "C:\Users\Marnee Dearman\PycharmProjects\VirtualEnvs\AgoraDev\lib\site-packages\py2neo\core.py", line 1412, in _id
self.__id = int(self.uri.path.segments[-1])
ValueError: invalid literal for int() with base 10: ''
Using Nigel's advice below, I got this to work. It was a usage error on my part:
user_node = Graph().find_one('USER',
property_key='email',
property_value='marnee#email.com')
user_node.properties['mission_statement'] = 'New mission statement'
user_node.push()
There are a couple of problems with your code so I will try to clarify the correct usage of these methods.
The bind method is used to connect a local entity (in this case, Node) to a corresponding remote equivalent. You should generally never need to use this method explicitly as entities are typically bound automatically on creation or retrieval. In your case, the find_one method does exactly this and constructs a client-side node that is bound to a corresponding server-side node; an explicit bind is not required.
The second issue is with your usage of bind. The URI taken by this method is that of a specific remote resource. You have passed the URI of the Graph itself (probably http://localhost:7474/db/data/) instead of that of the Node (such as http://localhost:7474/db/data/node/2345). The actual error that you see is caused by py2neo attempting to strip the ID from the URI and failing.
The simple solution is to remove the bind call.

Web2py application: How to reference .yaml file in controller?

I have an app running online under web2py. Now, i am adding names.yml file which i need to call in my controller file (default.py) on web2py server. where should I keep the .yml/.yaml files. Currently I have kept them in views with default/names.yml but when I call it in default.py like:
dicttagger = DictionaryTagger([ 'default/names.yml', 'default/surname.yml'])
i get no such file error.
Also tried below:
dicttagger = DictionaryTagger([ 'views/default/names.yml', 'views/default/surname.yml'])
same error
class snapshot as under:
class DictionaryTagger(object):
def __init__(self, dictionary_paths):
files = [open(path, 'r') for path in dictionary_paths]
dictionaries = [yaml.load(dict_file) for dict_file in files]
map(lambda x: x.close(), files)
Any suggestions as how to do this or am I making mistake of using yaml/yml file in we2py and it doesn't work in web2py app hosted online?
question 2
thank you. it resolved an error but I am not sure how to add nltk.download() into my hosted app. I keep getting the below error. Can you pls have a look:
Traceback
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
Traceback (most recent call last):
File "/home/prakashsukhwal/web2py/gluon/restricted.py", line 220, in restricted
exec ccode in environment
File "/home/prakashsukhwal/web2py/applications/Sensiva/controllers/default.py", line 4, in
nltk.download()
File "/usr/local/lib/python2.7/dist-packages/nltk/downloader.py", line 644, in download
self._interactive_download()
File "/usr/local/lib/python2.7/dist-packages/nltk/downloader.py", line 958, in _interactive_download
DownloaderShell(self).run()
File "/usr/local/lib/python2.7/dist-packages/nltk/downloader.py", line 981, in run
user_input = raw_input('Downloader> ').strip()
EOFError: EOF when reading a line
Error snapshot help
(EOF when reading a line)
inspect attributes
Frames
File /home/prakashsukhwal/web2py/gluon/restricted.py in restricted at line 220 code arguments variables
File /home/prakashsukhwal/web2py/applications/Sensiva/controllers/default.py in at line 4 code arguments variables
File /usr/local/lib/python2.7/dist-packages/nltk/downloader.py in download at line 644 code arguments variables
File /usr/local/lib/python2.7/dist-packages/nltk/downloader.py in _interactive_download at line 958 code arguments variables
File /usr/local/lib/python2.7/dist-packages/nltk/downloader.py in run at line 981 code arguments variables
Function argument list
(self=)
Code listing
def run(self):
print 'NLTK Downloader'
while True:
self._simple_interactive_menu(
'd) Download', 'l) List', ' u) Update', 'c) Config', 'h) Help', 'q) Quit')
user_input = raw_input('Downloader> ').strip()
if not user_input: print; continue
command = user_input.lower().split()[0]
args = user_input.split()[1:]
try:
Variables
user_input undefined
builtinraw_input
).strip undefined
Context
You can store the files wherever you want, but if you're using the Python open function, you'll need to give it full paths, not paths relative to the web2py application folder. Instead, try:
import os
dicttagger = DictionaryTagger([os.path.join(request.folder, 'views',
'default', 'names.yml'),
...])

Creating a single executable .exe from Python script that uses PuLP

I have been struggling with this for a while. I have used py2exe and cx_freeze to package everything. I am using a 32 bit machine and Everything works fine and the interface opens up and everything just that I know the entire puLP package is not being copied correctly into the package. I know this because the solver does not work. Inside both library zips in the packages created by py2exe and cx_freeze, there are only .pyc files included where PuLP has cbc.exe and other file types that make the solver work.
Is there any work around this? I have tried copying the actual PuLP package into the library.zip as well as into the dist folder and that didn't work.
Here is the setup I used for py2exe:
import sys
from cx_Freeze import setup, Executable
# Dependencies are automatically detected, but it might need fine tuning.
build_exe_options = {"packages": ["pulp"],
"icon": "icon.ico",
"include_files": ["icon.ico","cbc.exe","cbc-32","cbc-64","cbc-osx-64","CoinMP.dll"]}
# GUI applications require a different base on Windows (the default is for a
# console application).
base = None
if sys.platform == "win32":
base = "Win32GUI"
setup( name = "my_app",
version = "0.1",
options = {"build_exe": build_exe_options},
executables = [Executable("my_app.py", base=base)])
I received the following error:
Exception in Tkinter callback
Traceback (most recent call last):
File "Tkinter.pyc", line 1470, in __call__
File "my_app.py", line 796, in <lambda>
File "my_app.py", line 467, in branchAndBound
File "pulp\pulp.pyc", line 1619, in solve
AttributeError: 'NoneType' object has no attribute 'actualSolve'
EDIT
I tried to change the paths to cbc.exe and CoinMP.dll but that didn't really work either. I am probably missing something.
I changed the following inside solvers.py in the PuLP package:
try:
coinMP_path = config.get("locations", "CoinMPPath").split(', ')
except configparser.Error:
coinMP_path = ['/Users/al/Desktop/my_app/build/exe.win32-2.7']
try:
cbc_path = config.get("locations", "CbcPath")
except configparser.Error:
cbc_path = '/Users/al/Desktop/my_app/build/exe.win32-2.7'
try:
pulp_cbc_path = config.get("locations", "PulpCbcPath")
except configparser.Error:
pulp_cbc_path = '/Users/al/Desktop/my_app/build/exe.win32-2.7'
What am I missing or doing wrong?

GNU radio:error in running modtool for making new modules

hi i am trying to add new blocks in gnuradio . I tried to follow the the procedure mentioned in gnuradio website
http://gnuradio.org/redmine/projects/gnuradio/wiki/OutOfTreeModules?version=24
but when in terminal window i write "gr_modtool newmod howto"
it gives following error
File "/usr/local/bin/gr_modtool", line 41, in <module>
main()
File "/usr/local/bin/gr_modtool", line 36, in main
modtool.setup()
File "/usr/local/lib/python2.7/dist-packages/gnuradio/modtool/modtool_newmod.py", line 70, in setup
self._srcdir = gr.prefs().get_string('modtool', 'newmod_path', options.srcdir)
File "/usr/local/lib/python2.7/dist-packages/gnuradio/gr/gnuradio_core_general.py", line 18953, in get_string
return _gnuradio_core_general.gr_prefs_get_string(self, *args, **kwargs)
TypeError: in method 'gr_prefs_get_string', argument 4 of type 'std::string const'
please tell me how to fix it.
To create your own modules in GNU Radio Companion using the gr_modtool try this:
$ gr_modtool newmod
Name of the new module: myPlugin
To add processing blocks to your newly created module use:
../gr-myPlugin$ gr_modtool add
You can choose if you want a cpp or python block by adding the "-l" specificier:
../gr-myPlugin$ gr_modtool add -l python
More info:
http://lists.gnu.org/archive/html/discuss-gnuradio/2013-03/msg00216.html