Got the following code:
import time
from google.protobuf.timestamp_pb2 import Timestamp
from google.cloud import bigquery_datatransfer_v1
def runQuery (parent, requested_run_time):
client = bigquery_datatransfer_v1.DataTransferServiceClient()
projectid = '[enter your projectId here]' # Enter your projectID here
transferid = '[enter your transferId here]' # Enter your transferId here
parent = client.project_transfer_config_path(projectid, transferid)
start_time = bigquery_datatransfer_v1.types.Timestamp(seconds=int(time.time() + 10))
response = client.start_manual_transfer_runs(parent, requested_run_time=start_time)
print(response)
We used it in few different projects and cases and everything works fine. Today I deployed another function using this code and keep getting the following error:
AttributeError: 'DataTransferServiceClient' object has no attribute
'project_transfer_config_path'
What am I missing?
Thank you!
You are probably using a newer version (2.0.0 or 2.1.0) of the google-cloud-bigquery-datatransfer client library. In these versions, most utility methods have been removed, one of them being project_transfer_config_path.
You can use the method transfer_config_path of the client to achieve the same result.
I would strongly suggest that you study the Migration Guide to 2.0.0 as there might be other changes that you need to make too.
In case you are using version 2.0.0 and not 2.1.0, I would recommend upgrading to the latest since there are breaking changes between them, for example the import paths that were changed in 2.0.0 have been reverted in 2.1.0.
Versions
Python : 2.7.14
six : 1.9.0 & 1.11.0(tried on both)
OS : mac(10.13.3) & ubuntu(16.04) [tried on both]
Error
from six.moves import http_client
ImportError: No module named moves
Description
In flask application which is running on google app engine while running it on local system using dev_appserver.py getting above error while importing from six.moves import http_client
What I have tried
After importing six have tried dir(six) which shows that moves is there in list but it's not able to import it which is very strange.
Output of six.__version__: 1.11.0
Output if dir(six)
['/opt/tribes-backend', '/opt/tribes-backend/lib1', '/usr/lib/google-cloud-sdk/platform/google_appengine', '/usr/lib/google-cloud-sdk/platform/google_appengine', '/usr/lib/python2.7', '/usr/lib/python2.7/lib-dynload', '/usr/lib/google-cloud-sdk/platform/google_appengine/lib/ssl-2.7.11', '/usr/lib/google-cloud-sdk/platform/google_appengine/lib/grpcio-1.0.0', '/usr/lib/google-cloud-sdk/platform/google_appengine/lib/six-1.9.0', '/usr/lib/google-cloud-sdk/platform/google_appengine/lib/protobuf-3.0.0', '/usr/lib/google-cloud-sdk/platform/google_appengine/lib/enum-0.9.23', '/usr/lib/google-cloud-sdk/platform/google_appengine/lib/futures-3.0.5', '/usr/lib/google-cloud-sdk/platform/google_appengine/lib/setuptools-36.6.0', '/usr/lib/google-cloud-sdk/platform/google_appengine/lib/protorpc-1.0', '/usr/lib/google-cloud-sdk/platform/google_appengine/lib/pytz-2017.2', '/usr/lib/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.3', '/usr/lib/google-cloud-sdk/platform/google_appengine/lib/webob-1.1.1', '/usr/lib/google-cloud-sdk/platform/google_appengine/lib/werkzeug-0.11.10', '/usr/lib/google-cloud-sdk/platform/google_appengine/lib/yaml-3.10', '/usr/local/lib/python2.7/dist-packages/enum', '/usr/lib/google-cloud-sdk/platform/google_appengine/lib/concurrent/concurrent', '/usr/local/lib/python2.7/dist-packages/concurrent', '/usr/local/lib/python2.7/dist-packages/google', '/usr/lib/google-cloud-sdk/platform/google_appengine/google']
['BytesIO', 'Iterator', 'MAXSIZE', 'Module_six_moves_urllib', 'Module_six_moves_urllib_error', 'Module_six_moves_urllib_parse', 'Module_six_moves_urllib_request', 'Module_six_moves_urllib_response', 'Module_six_moves_urllib_robotparser', 'MovedAttribute', 'MovedModule', 'PY2', 'PY3', 'StringIO', '_LazyDescr', '_LazyModule', '_MovedItems', '_SixMetaPathImporter', 'author', 'builtins', 'doc', 'file', 'name', 'package', 'path', 'version', '_add_doc', '_assertCountEqual', '_assertRaisesRegex', '_assertRegex', '_func_closure', '_func_code', '_func_defaults', '_func_globals', '_import_module', '_importer', '_meth_func', '_meth_self', '_moved_attributes', '_print', '_urllib_error_moved_attributes', '_urllib_parse_moved_attributes', '_urllib_request_moved_attributes', '_urllib_response_moved_attributes', '_urllib_robotparser_moved_attributes', 'absolute_import', 'add_metaclass', 'add_move', 'advance_iterator', 'assertCountEqual', 'assertRaisesRegex', 'assertRegex', 'b', 'binary_type', 'byte2int', 'callable', 'class_types', 'create_bound_method', 'exec_', 'functools', 'get_function_closure', 'get_function_code', 'get_function_defaults', 'get_function_globals', 'get_method_function', 'get_method_self', 'get_unbound_function', 'indexbytes', 'int2byte', 'integer_types', 'iterbytes', 'iteritems', 'iterkeys', 'iterlists', 'itertools', 'itervalues', 'moves', 'next', 'operator', 'print_', 'python_2_unicode_compatible', 'raise_from', 'remove_move', 'reraise', 'string_types', 'sys', 'text_type', 'types', 'u', 'unichr', 'viewitems', 'viewkeys', 'viewvalues', 'with_metaclass', 'wraps'].
As it can be seen from above output moves inside six still giving error while importing it.
Spent a lot of time on this and no solution till now any help would be greatly appreciated here. :(
UPDATE1
Error stacktrace :-
from google.cloud.datastore import helpers
File "/opt/tribes-backend/denv/local/lib/python2.7/site-packages/google/cloud/datastore/helpers.py", line 27, in <module>
from google.cloud._helpers import _datetime_to_pb_timestamp
File "/opt/tribes-backend/denv/local/lib/python2.7/site-packages/google/cloud/_helpers.py", line 30, in <module>
from six.moves import http_client
ImportError: No module named moves
For me the issue was resolved by following it on https://github.com/googleapis/python-ndb/issues/249
andrewsg commented 11 days ago:
I think we've identified an issue with devappserver related to the six library specifically. Could you please try a workaround? Add the line: import six; reload(six) to the top of your app, before NDB is loaded and let me know if that works.
Based on follow up with google support team have figured out that communicating with datastore using google-cloud-datastore is deprecated
instead using ndb for communicating datastore is the way to go.
Updated documentation stating deprecation using client datastore library is documented here
Documentation to getting started with ndb client library in python is documented here
In some cases if you want to use some python pure libraries, like six, available in your applications you will need to use third-party libraries.
Follow the instructions here, to add a third-party library, until the command pip install -t lib -r requirements.txt. In your requirements.txt file just add six==1.11.0.
It solved the problem for me.
During the test for my Django 1.9 project I get an error:
Python swears on this code:
def test_students_list(self):
# make request to the server to get homepage page
response = self.client.get(self.url)
# do we have student name on a page?
self.assertIn('Vitaliy', response.content)
How to set the same encoding for the arguments in function assertIn?
I tried so:
self.assertIn(u"Vitaliy", response.content.decode('utf8'))
The result is the same...
P.S. I have Python 2.7.6 on Ubuntu 14.04
Did you try to define your Python source code encoding using:
# -- coding: utf-8 --
As suggest in PEP 0263.
See https://docs.djangoproject.com/en/1.9/ref/request-response/#django.http.HttpResponse.content
The HTTPResponse.content is noted in the docs as being a bytestring (https://github.com/django/django/blob/master/django/http/response.py#L225), it should be encoded as DEFAULT_CHARSET which by default is utf-8 but in both our cases this doesn't seem to get through to the test.
My solution is to tell Python request.content should have unicode encoding:
def test_students_list(self):
# make request to the server to get homepage page
response = self.client.get(self.url)
# do we have student name on a page?
self.assertIn('Vitaliy', unicode(response.content, encoding='utf-8'))
Use self.assertContains(response, 'Vitaliy') instead.
I collected user data using a wx python gui and than I used uno to fill this data into an openoffice document under ubuntu 10.xx
user + my-script ( +empty document ) --> prefilled document
After upgrading to ubuntu 14.04 uno doesn't work with python 2.7 anymore and now we have libreoffice instead of openoffice in ubuntu. when I try to run my python2.7 code, it says:
ImportError: No module named uno
How could I bring it back to work?
what I tried:
installed https://pypi.python.org/pypi/unotools v0.3.3
sudo apt-get install libreoffice-script-provider-python
converted the code to python3 and got uno importable, but wx is not importable in python3 :-/
ImportError: No module named 'wx'
googled and read python3 only works with wx phoenix
so tried to install: http://wxpython.org/Phoenix/snapshot-builds/
but wasn't able to get it to run with python3
is there a way to get the uno bridge to work with py2.7 under ubuntu 14.04?
Or how to get wx to run with py3?
what else could I try?
Create a python macro in LibreOffice that will do the work of inserting the data into LibreOffice and then in your python 2.7 code envoke the macro.
As the macro is running from with LibreOffice it will use python3.
Here is an example of how to envoke a LibreOffice macro from the command line:
#!/usr/bin/python3
# -*- coding: utf-8 -*-
##
# a python script to run a libreoffice python macro externally
# NOTE: for this to run start libreoffice in the following manner
# soffice "--accept=socket,host=127.0.0.1,port=2002,tcpNoDelay=1;urp;" --writer --norestore
# OR
# nohup soffice "--accept=socket,host=127.0.0.1,port=2002,tcpNoDelay=1;urp;" --writer --norestore &
#
import uno
from com.sun.star.connection import NoConnectException
from com.sun.star.uno import RuntimeException
from com.sun.star.uno import Exception
from com.sun.star.lang import IllegalArgumentException
def uno_directmacro(*args):
localContext = uno.getComponentContext()
localsmgr = localContext.ServiceManager
resolver = localsmgr.createInstanceWithContext("com.sun.star.bridge.UnoUrlResolver", localContext )
try:
ctx = resolver.resolve("uno:socket,host=localhost,port=2002;urp;StarOffice.ComponentContext")
except NoConnectException as e:
print ("LibreOffice is not running or not listening on the port given - ("+e.Message+")")
return
msp = ctx.getValueByName("/singletons/com.sun.star.script.provider.theMasterScriptProviderFactory")
sp = msp.createScriptProvider("")
scriptx = sp.getScript('vnd.sun.star.script:directmacro.py$directmacro?language=Python&location=user')
try:
scriptx.invoke((), (), ())
except IllegalArgumentException as e:
print ("The command given is invalid ( "+ e.Message+ ")")
return
except RuntimeException as e:
print("An unknown error occurred: " + e.Message)
return
except Exception as e:
print ("Script error ( "+ e.Message+ ")")
print(e)
return
return(None)
uno_directmacro()
And this is the corresponding macro code within LibreOffice called "directmacro.py" and stored in the User area for libreOffice macros (which would normally be $HOME/.config/libreoffice/4/user/Scripts/python :
#!/usr/bin/python
from com.sun.star.awt.MessageBoxButtons import BUTTONS_OK, BUTTONS_OK_CANCEL, BUTTONS_YES_NO, BUTTONS_YES_NO_CANCEL, BUTTONS_RETRY_CANCEL, BUTTONS_ABORT_IGNORE_RETRY
from com.sun.star.awt.MessageBoxButtons import DEFAULT_BUTTON_OK, DEFAULT_BUTTON_CANCEL, DEFAULT_BUTTON_RETRY, DEFAULT_BUTTON_YES, DEFAULT_BUTTON_NO, DEFAULT_BUTTON_IGNORE
from com.sun.star.awt.MessageBoxType import MESSAGEBOX, INFOBOX, WARNINGBOX, ERRORBOX, QUERYBOX
def directmacro(*args):
import socket, time
class FontSlant():
from com.sun.star.awt.FontSlant import (NONE, ITALIC,)
#get the doc from the scripting context which is made available to all scripts
desktop = XSCRIPTCONTEXT.getDesktop()
model = desktop.getCurrentComponent()
text = model.Text
tRange = text.End
cursor = desktop.getCurrentComponent().getCurrentController().getViewCursor()
doc = XSCRIPTCONTEXT.getDocument()
parentwindow = doc.CurrentController.Frame.ContainerWindow
# your cannot insert simple text and text into a table with the same method
# so we have to know if we are in a table or not.
# oTable and oCurCell will be null if we are not in a table
oTable = cursor.TextTable
oCurCell = cursor.Cell
insert_text = "This is text inserted into a LibreOffice Document\ndirectly from a macro called externally"
Text_Italic = FontSlant.ITALIC
Text_None = FontSlant.NONE
cursor.CharPosture=Text_Italic
if oCurCell == None: # Are we inserting into a table or not?
text.insertString(cursor, insert_text, 0)
else:
cell = oTable.getCellByName(oCurCell.CellName)
cell.insertString(cursor, insert_text, False)
cursor.CharPosture=Text_None
return None
You will of course need to adapt the code to either accept data as arguments, read it from a file or whatever.
Ideally I would say use python 3, because python 2 is becoming outdated. The switch requires quite a bit of new coding changes, but better sooner than later. So I tried:
sudo pip3 install -U --pre \
-f http://wxpython.org/Phoenix/snapshot-builds/ \
wxPython_Phoenix
However this gave me errors, and I didn't want to spend the next couple of days working through them. Probably the pre-release versions are not ready for prime time yet.
So instead, what I recommend is to switch to AOO for now. See https://stackoverflow.com/a/27980255/5100564 for instructions. AOO does not have all the latest features that LO has, but it is a good solid Office product.
Apparently it is also possible to rebuild LibreOffice with python 2 using this script: https://gist.github.com/hbrunn/6f4a007a6ff7f75c0f8b
Code:
#!/usr/bin/env python
import boto.ec2
conn_ec2 = boto.ec2.connect_to_region('us-east-1') # access keys are environment vars
my_code = """#!/usr/bin/env python
import sys
sys.stdout = open('file', 'w')
print 'test'
"""
reservation = conn_ec2.run_instances(image_id = 'ami-a73264ce',
key_name = 'backendkey',
instance_type = 't1.micro',
security_groups = ['backend'],
instance_initiated_shutdown_behavior = 'terminate',
user_data = my_code)
The instance is initiated with the proper settings (it's the public Ubuntu 12.04, 64-bit, image) and I can SSH into it normally. The user-data script seems to be loaded correctly: I can see it in /var/lib/cloud/instance/user-data.txt (and also in /var/lib/cloud/instance/scripts/part-001) and on the EC2 console.
But that's it, the script doesn't seem to be executed. Following this answer I checked the /var/log/cloud-init.log file but it doesn't seem to contain any error messages related to my script (well, maybe I'm missing something - here is a gist with the contents of cloud-init.log).
What am I missing?
This is probably not relevant anymore, but yet.
I've just used boto with ubuntu and user data, although the documenation says that the user data has to be base64 encoded, it only worked for me if I pass the 64 bit paramter as regular string.
I read the content of user data from file (using fh.read()) and then just pass this as the user_data paramter to run_instances.
I think it's not working for you because user data can't use any shebang like you used "#!/usr/bin/env python"
On the help page http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html there are two examples one is the standard "#!/bin/bash", and another one looks artificial "#cloud-config". Probably it's only 2 available shebangs. The bash one works for me.