The following code (taken from - https://github.com/dennybritz/tf-rnn/blob/master/bidirectional_rnn.ipynb)
import tensorflow as tf
import numpy as np
tf.reset_default_graph()
# Create input data
X = np.random.randn(2, 10, 8)
# The second example is of length 6
X[1,6:] = 0
X_lengths = [10, 6]
cell = tf.contrib.rnn.LSTMCell(num_units=64, state_is_tuple=True)
outputs, states = tf.nn.bidirectional_dynamic_rnn(
cell_fw=cell,
cell_bw=cell,
dtype=tf.float64,
sequence_length=X_lengths,
inputs=X)
output_fw, output_bw = outputs
states_fw, states_bw = states
is giving the following error for
tensorflow - 1.1 for both 2.7 and 3.5
ValueError: Attempt to reuse RNNCell <tensorflow.contrib.rnn.python.ops.core_rnn_cell_impl.LSTMCell object at 0x10ce0c2b0>
with a different variable scope than its first use. First use of cell was with scope
'bidirectional_rnn/fw/lstm_cell', this attempt is with scope 'bidirectional_rnn/bw/lstm_cell'.
Please create a new instance of the cell if you would like it to use a different set of weights.
If before you were using: MultiRNNCell([LSTMCell(...)] * num_layers), change to:
MultiRNNCell([LSTMCell(...) for _ in range(num_layers)]). If before you were using the same cell
instance as both the forward and reverse cell of a bidirectional RNN, simply create two instances
(one for forward, one for reverse). In May 2017, we will start transitioning this cell's behavior to use
existing stored weights, if any, when it is called with scope=None (which can lead to silent model degradation,
so this error will remain until then.)
But it is working in
tensorflow - 1.0.1 for python 3.5 (did not test on python - 2.7)
I tried with multiple code examples I found online but
tf.nn.bidirectional_dynamic_rnn
is giving the same error with tensorflow - 1.1
Is there a bug in tensorflow 1.1 or am i just missing something?
Sorry you ran into this. I can confirm that the error appears in 1.1 (docker run -it gcr.io/tensorflow/tensorflow:1.1.0 python) but not in 1.2 RC0 (docker run -it gcr.io/tensorflow/tensorflow:1.2.0-rc0 python).
So it looks like either 1.2-rc0 or 1.0.1 are your options for the moment.
Related
Using interactive Glue Sessions in a Jupyter Notebook was working correctly with the aws-glue-sessions package version 0.32 installed. After upgrading with pip3 install --upgrade jupyter boto3 aws-glue-sessions to version 0.35, the kernel would not start. Gave an error message in GlueKernel.py line 443 in set_glue_version Exception: Valid Glue versions are {'3.0', '2,0} and the Kernel won't start.
Reverting to version 0.32 resolves the issue. Tried installing 0.35, 0.34, 0.33 and get the error, which makes me think it's something I'm doing wrong or don't understand and not something in the product. Is there anything additional I need to do to upgrade the version of the aws-glue-sessions?
Obviously this is not a good workaround - but it worked for me.
I went into the file GlueKernel.py in the directory: \site-packages\aws_glue_interactive_sessions_kernel\glue_pyspark
and hard-coded the 2nd line of this function to set the version to "3.0"
I'm on windows
def set_glue_version(self, glue_version):
glue_version = str("3.0")
if glue_version not in VALID_GLUE_VERSIONS:
raise Exception(f"Valid Glue versions are {VALID_GLUE_VERSIONS}")
self.glue_version = glue_version
I am a bit lost here as well -- and confused. I will add that I am a python newbie. I am running the whole thing on Windows. AWS has an article that describes the installation. So, I am assuming it's supported. I get the same error as #theOtherOne.
line 443 in set_glue_version Exception: Valid Glue versions are {'3.0', '2,0}
I checked GlueKernel.py of glue_pyspark, and found this code:
def _retrieve_os_env_variable(self, key):
_, output = subprocess.getstatusoutput(f"echo ${key}")
return output or os.environ.get(key)
When I run the code below manually, I get $GLUE_VERSION as final result. That obviously doesn't match '2.0' or '3.0'. The command for retrieving environment variables on Windows is a different one. If my understanding is correct, then this whole thing will never work on Windows. Maybe I am the only one who wants to run it on Windows and no one else cares? I got it to work on WSL, but still. I lost quite some time to fix something that cannot be fixed (or can it?)
import subprocess
import os
_, output = subprocess.getstatusoutput(f"echo $GLUE_VERSION")
osoutput = os.environ.get("GLUE_VERSION")
print(output) #$GLUE_VERSION
print (osoutput) #'3.0'
print(output or osoutput) #$GLUE_VERSION
enter image description here
So the issue seems to be that GLUE_VERSION is not set in the environment variables. Once this is set - it works
I use C++ code to read pictures from WMTS server using DGAL.
First I initialize GDAL once:
...
OGRRegisterAll();
etc.
But new connection is opened every time I want to read new image (different urls):
gdalDataset = GDALOpen(my_url, GA_ReadOnly);
URL example: https://sampleserver6.arcgisonline.com/arcgis/rest/services/Toronto/ImageServer/tile/12/1495/1145
Unfortunately I didn't find a way to read multiply images by same connection.
Is there such option in GDAL or in WMTS?
Are there other ways to improve timing (I read thousands of images)?
While GDAL can read PNG files, it doesn't add much since those lack any geographical metadata.
You probably want to interact with the WMS server instead, not the images directly. You can for example run gdalinfo on the main url to see the subdatasets:
gdalinfo https://sampleserver6.arcgisonline.com/arcgis/services/Toronto/ImageServer/WMSServer?request=GetCapabilities&service=WMS
The first layer seems to have an issue, I'm not sure, but the other ones seem to behave fine.
I hope you don't mind me using some Python code, but the c++ api should be similar. Or you could try using the command-line utilities first (gdal_translate), to get familiar with the service.
See the WMS driver for more information and examples:
https://gdal.org/drivers/raster/wms.html
You can for example retrieve a subset and store it with:
from osgeo import gdal
url = r"WMS:https://sampleserver6.arcgisonline.com:443/arcgis/services/Toronto/ImageServer/WMSServer?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=Toronto%3ANone&SRS=EPSG:4326&BBOX=-79.454856,43.582524,-79.312167,43.711781"
bbox = [-79.35, 43.64, -79.32, 43.61]
filename = r'D:\Temp\toronto_subset.tif'
ds = gdal.Translate(filename, url, xRes=0.0001, yRes=0.0001, projWin=bbox)
ds = None
Which looks like:
import numpy as np
import matplotlib.pyplot as plt
ds = gdal.OpenEx(filename)
img = ds.ReadAsArray()
ds = None
mpl_extent = [bbox[i] for i in [0,2,3,1]]
fig, ax = plt.subplots(figsize=(5,5), facecolor="w")
ax.imshow(np.moveaxis(img, 0, -1), extent=mpl_extent)
Note that the data in native resolution for these type of services is often ridiculously large, so usually you want to specify a subset and/or limited resolution as the output.
I keep on getting this error when trying to view objects in the Debugger in PyCharm:
Unable to display children:Attribute not found: value
I have deduced that it is an error with Pycharm itself, not my code
(I get the same error on multiple scripts, but no error on with an older version of Pycharm on 2 different computers)
I'm on PyCharm Community 2017.3.4
Any ideas for workarounds, other than installing an older version?
I am finding similar issues. I too think there is something up with PyCharm it does not work as expected or previous version as you mention. I also found Pyscripter to debug as expected.
In some instances I would rely on the result object, result[0] or result.getOutput(0) to pass to next tool. Instead one can use a variable for the "output" or use the string (name) directly as input for the next tool.
For example,
facility_staging_polygons = os.path.join(outGDB, 'facility_staging_polygons\Polygon_1')
result = arcpy.MakeFeatureLayer_management(facility_staging_polygons, 'facility_staging_polygons_Layer')
# Process: Update Attributes
arcpy.AddField_management('facility_staging_polygons_Layer', "area_calc", "LONG")
My project requests a new layer, which needs the new operator of Tensor to compute bitwiseXOR between input x and constant Key k.
E.g. x = 4 (bit form: 100), k = 7 (111), the bitwiseXOR(x, k) expects as 3 (011).
As far as I know, Tensor only has LogicXOR operator for bool type. Luckily, Tensorflow has the extended ability to have a new Op. However, I read the document in https://www.tensorflow.org/extend/adding_an_op, I can get the basic idea, but that is far from the implementation, maybe because of the lack of c++ knowledge. Any suggestions for implementation the new operator that will be helpful. Then I can use that new Op of Tensor to build the new layers.
If you don't want to implement your own C++ op, you can try with tf.py_func which allows you to define a python function that operates on numpy arrays and is then used as a Tensorflow operation in the graph.
For your problem you can use numpy's bitwise_xor():
import tensorflow as tf
import numpy as np
t1 = tf.constant([2,4,6], dtype=tf.int64)
t2 = tf.constant([1,3,5], dtype=tf.int64)
t_xor = tf.py_func(np.bitwise_xor, [t1, t2], tf.int64, stateful=False)
with tf.Session() as sess:
val = sess.run(t_xor)
print(val)
which prints [3,7,3] as expected.
Please take care of the known limitations of this function (taken from the link above):
N.B. The tf.py_func() operation has the following known limitations:
The body of the function (i.e. func) will not be serialized in a
GraphDef. Therefore, you should not use this function if you need to
serialize your model and restore it in a different environment.
The operation must run in the same address space as the Python program
that calls tf.py_func(). If you are using distributed TensorFlow, you
must run a tf.train.Server in the same process as the program that
calls tf.py_func() and you must pin the created operation to a device
in that server (e.g. using with tf.device():).
I am playing around with pyglet 1.2alpha-1 and Python 3.3. I have the following (extremely simple) application and cannot figure out what my issue is:
import pyglet
window = pyglet.window.Window()
#image = pyglet.resource.image('img1.jpg')
image = pyglet.image.load('img1.jpg')
label = pyglet.text.Label('Hello, World!!',
font_name='Times New Roman',
font_size=36,
x=window.width//2, y=window.height//2,
anchor_x='center', anchor_y='center')
#window.event
def on_draw():
window.clear()
label.draw()
# image.blit(0,0)
pyglet.app.run()
With the above code, my text label will appear as long as image.blit(0, 0) is commented out. However, if I try to display the image, the program crashes with the following error:
File "C:\Python33\lib\site-packages\pyglet\gl\lib.py", line 105, in errcheck
raise GLException(msg)
pyglet.gl.lib.GLException: b'invalid value'
I also get the above error if I try to use pyglet.resource.image instead of pyglet.image.load (the image and py file are in the same directory).
Any one know how I can fix this issue?
I am using Python 3.3, pyglet 1.2alpha-1, and Windows 8.
The code -including the image.blit- runs fine for me. I'm using python 2.7.3, pyglet 1.1.4
There's nothing wrong with the code. You might consider trying other python and pyglet versions for the time being (until pyglet has a new stable release)
This isn't a "fix", but might at least determine if it's fixable or not (mine was not). (From the Pyglet mailing group.)
You can verify whether the system does not even support Textures greater than 1024, by running this code (Python 3+):
from ctypes import c_long
from pyglet.gl import glGetIntegerv, GL_MAX_TEXTURE_SIZE
i = c_long()
glGetIntegerv(GL_MAX_TEXTURE_SIZE, i)
print (i) # output: c_long(1024) (or higher)
That is the maximum texture size your system supports. If it's 1024, then any larger pictures will raise an Exception. (And the only fix is, get a better system).