When attempting to use the AWS CLI for the EC2 instance I'm working with, I receive the following error.
[ec2-user#ip-xxx-xxx-xxx-xxx ~]$ aws
Traceback (most recent call last):
File "/usr/bin/aws", line 27, in <module>
sys.exit(main())
File "/usr/bin/aws", line 23, in main
return awscli.clidriver.main()
File "/usr/lib/python2.7/dist-packages/awscli/clidriver.py", line 54, in main
return driver.main()
File "/usr/lib/python2.7/dist-packages/awscli/clidriver.py", line 186, in main
command_table = self._get_command_table()
File "/usr/lib/python2.7/dist-packages/awscli/clidriver.py", line 96, in _get_command_table
self._command_table = self._build_command_table()
File "/usr/lib/python2.7/dist-packages/awscli/clidriver.py", line 116, in _build_command_table
command_object=self)
File "/usr/local/lib/python2.7/site-packages/botocore-1.4.8-py2.7.egg/botocore/session.py", line 680, in emit
return self._events.emit(event_name, **kwargs)
File "/usr/local/lib/python2.7/site-packages/botocore-1.4.8-py2.7.egg/botocore/hooks.py", line 226, in emit
return self._emit(event_name, kwargs)
File "/usr/local/lib/python2.7/site-packages/botocore-1.4.8-py2.7.egg/botocore/hooks.py", line 209, in _emit
response = handler(**kwargs)
File "/usr/lib/python2.7/dist-packages/awscli/customizations/opsworkscm.py", line 21, in alias_opsworks_cm
alias_command(command_table, 'opsworkscm', 'opsworks-cm')
File "/usr/lib/python2.7/dist-packages/awscli/customizations/utils.py", line 71, in alias_command
current = command_table[existing_name]
KeyError: 'opsworkscm'
I am not quite sure why this is happening. I am working with other ec2 instances setup similar to this one that work, but I am not sure what difference may be causing this error.
I ran across this issue in the aws-cli GH repo. I ran sudo pip install awscli and it updated botocore to version 1.4.86 which fixed my issue.
Issue in aws-cli GH repo
I was using Ubuntu Xenial and needed to have awscli newer than 1.4.38 so I was using awscli from Ubuntu / Zesty.
As with pip, you need to upgrade python3-botocore so this worked for me:
apt-get install awscli python3-botocore
(from zesty repository).
Your /usr/bin/aws must be an old executable.
Run whereis aws. You will get a list of aws executables.
Find the most recent one by running aws --version.
Remove the corrupted executable. In your case sudo rm /usr/bin/aws
Related
I have a python 2.7 script I run locally to launch a Apache Beam / Google Dataflow job (SDK 2.12.0). The job takes a csv file from a Google storage bucket, processes it and then creates an entity in Google Datastore for each row. The script ran fine for years ...but now it is failing:
INFO:root:2019-05-15T22:07:11.481Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:root:2019-05-15T21:47:13.370Z: JOB_MESSAGE_ERROR: Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py", line 773, in run
self._load_main_session(self.local_staging_directory)
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py", line 489, in _load_main_session
pickler.load_session(session_file)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/internal/pickler.py", line 280, in load_session
return dill.load_session(file_path)
File "/usr/local/lib/python2.7/dist-packages/dill/_dill.py", line 410, in load_session
module = unpickler.load()
File "/usr/lib/python2.7/pickle.py", line 864, in load
dispatch[key](self)
File "/usr/lib/python2.7/pickle.py", line 1139, in load_reduce
value = func(*args)
File "/usr/local/lib/python2.7/dist-packages/dill/_dill.py", line 827, in _import_module
return __import__(import_name)
ImportError: No module named idlelib
I believe this error is happening at the worker level (not locally). I don't make reference to it in my script. To make sure it wasn't me I have installed updates for all google-cloud packages, apache-beam[gcp] etc locally -just in case. I tried importing idlelib into my script I get the same error. Any suggestions?
It has been fine for years and started failing from SDK 2.12.0 release.
What was the last release that this script succeeding on? 2.11?
I am switching to a new computer with a fresh install of Windows 10 Pro and am having a very strange issue with the EB CLI. I am not able to run 'eb deploy' using Windows Power Shell, I get the following error:
ERROR: OSError - [WinError 145] The directory is not empty: '.elasticbeanstalk\\app_versions'
I and uninstalled/reinstalled Python along with the EB CLI but with the same result.
Note: I am able to run all other EB commands like eb ssh or eb logs with no issues.
An observation I was able to make while watching the '.elasticbeanstalk' folder, I see the 'app_versions' folder being created along with the application zip in that folder. Once the command fails the ZIP file remains in the 'app_versions' folder for about 10 to 15 seconds before it is removed. I checked S3 and the zip file is uploaded...
I have reviewed this other Stack Overflow issue: AWS Elastic Beanstalk deploy not working
I do not have Google/Dropbox or OneDrive running on the directory I am working in. Just to be safe I paused OneDrive but still nothing.
Please, any help would be amazing!
UPDATE:
Ran eb deploy --debug
There is no error until AFTER the upload is completed, confirmed this by checking the S3 bucket and seeing the latest upload.
2019-02-04 14:50:06,522 (INFO) eb : Traceback (most recent call last):
File "C:\Users\winng\AppData\Roaming\Python\Python37\site-packages\ebcli\core\ebrun.py", line 62, in run_app
app.run()
File "C:\Users\winng\AppData\Roaming\Python\Python37\site-packages\cement\core\foundation.py", line 797, in run
return_val = self.controller._dispatch()
File "C:\Users\winng\AppData\Roaming\Python\Python37\site-packages\cement\core\controller.py", line 472, in _dispatch
return func()
File "C:\Users\winng\AppData\Roaming\Python\Python37\site-packages\cement\core\controller.py", line 478, in _dispatch
return func()
File "C:\Users\winng\AppData\Roaming\Python\Python37\site-packages\ebcli\core\abstractcontroller.py", line 94, in default
self.do_command()
File "C:\Users\winng\AppData\Roaming\Python\Python37\site-packages\ebcli\controllers\deploy.py", line 78, in do_command
staged=self.staged, timeout=self.timeout, source=self.source)
File "C:\Users\winng\AppData\Roaming\Python\Python37\site-packages\ebcli\operations\deployops.py", line 59, in deploy
build_config=build_config
File "C:\Users\winng\AppData\Roaming\Python\Python37\site-packages\ebcli\operations\commonops.py", line 538, in create_app_version
fileoperations.delete_app_versions()
File "C:\Users\winng\AppData\Roaming\Python\Python37\site-packages\ebcli\core\fileoperations.py", line 432, in delete_app_versions
delete_directory(app_version_folder)
File "C:\Users\winng\AppData\Roaming\Python\Python37\site-packages\ebcli\core\fileoperations.py", line 425, in delete_directory
shutil.rmtree(location)
File "c:\users\winng\appdata\local\programs\python\python37\lib\shutil.py", line 513, in rmtree
return _rmtree_unsafe(path, onerror)
File "c:\users\winng\appdata\local\programs\python\python37\lib\shutil.py", line 401, in _rmtree_unsafe
onerror(os.rmdir, path, sys.exc_info())
File "c:\users\winng\appdata\local\programs\python\python37\lib\shutil.py", line 399, in _rmtree_unsafe
os.rmdir(path)
OSError: [WinError 145] The directory is not empty: '.elasticbeanstalk\\app_versions'
2019-02-04 14:50:06,526 (INFO) eb : OSError - [WinError 145] The directory is not empty: '.elasticbeanstalk\\app_versions'
I tried to install awscli after ebcli, and they both broke. Currently, if I type aws s3 ls, it just hangs with no response, and if I try to use eb, I get this error:
Traceback (most recent call last):
File "/usr/local/bin/eb", line 11, in <module>
load_entry_point('awsebcli==3.8.4', 'console_scripts', 'eb')()
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 565, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2631, in load_entry_point
return ep.load()
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2291, in load
return self.resolve()
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2297, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "/usr/local/lib/python2.7/dist-packages/ebcli/core/ebcore.py", line 43, in <module>
from . import ebglobals, base, io, hooks
File "/usr/local/lib/python2.7/dist-packages/ebcli/core/base.py", line 19, in <module>
from ebcli import __version__
ImportError: cannot import name __version__
I basically need to have command line tools for s3 and elastic beanstalk, but I apparently have no luck, and will be spending my entire day googling the universe, and combing through error codes to try and fix this : (
I'm on Ubuntu 14.04 on a Thinkpad.
It is quite common for different Python libraries to install over each other, causing problems like this.
A popular fix is to use a the virtualenv tool to create isolated Python environments.
The AWS documentation for awsebcli has a page showing how: Install the EB CLI in a Virtual Environment
Alternatively, keep using the AWS Command-Line Interface (CLI) since it works across all AWS services, rather than using service-specific command sets like awsebcli (which pre-date the CLI).
I am starting with webapp2. I have created an application at following directory.
/home/github_projects/hellowebapp2
But when I try to fire up a server using:
/usr/lib/google-cloud-sdk/bin/dev_appserver.py github_projects/hellowebapp2
I get following error:
This action requires the installation of components: [app-engine-python]
You cannot perform this action because this Cloud SDK installation is
managed by an external package manager. If you would like to get the
latest version, please see our main download page at:
https://cloud.google.com/sdk/
Traceback (most recent call last):
File "/usr/lib/google-cloud-sdk/bin/dev_appserver.py", line 35, in <module>
main()
File "/usr/lib/google-cloud-sdk/bin/dev_appserver.py", line 22, in main
command=__file__)
File "/usr/lib/google-cloud-sdk/lib/googlecloudsdk/core/updater/update_manager.py", line 189, in EnsureInstalledAndRestart
return manager._EnsureInstalledAndRestart(components, msg, command)
File "/usr/lib/google-cloud-sdk/lib/googlecloudsdk/core/updater/update_manager.py", line 1139, in _EnsureInstalledAndRestart
restart_args=restart_args):
File "/usr/lib/google-cloud-sdk/lib/googlecloudsdk/core/updater/update_manager.py", line 660, in Install
restart_args=restart_args)
File "/usr/lib/google-cloud-sdk/lib/googlecloudsdk/core/updater/update_manager.py", line 690, in Update
self._EnsureNotDisabled()
File "/usr/lib/google-cloud-sdk/lib/googlecloudsdk/core/updater/update_manager.py", line 357, in _EnsureNotDisabled
'The component manager is disabled for this installation')
googlecloudsdk.core.updater.update_manager.UpdaterDisableError: The component manager is disabled for this installation
P.S I have already installed sdk from https://cloud.google.com/sdk/docs/#deb
Ok I solved this by installing specific python package from :
https://cloud.google.com/sdk/downloads#apt-get
sudo apt-get install google-cloud-sdk-app-engine-python
I just installed the AWS CLI following this installation guide for Windows:
http://docs.aws.amazon.com/cli/latest/userguide/installing.html#install-msi-on-windows
However, when I execute aws --version in my cli the following error is thrown:
C:\Users\René Winkler>aws --version
Traceback (most recent call last):
File "aws", line 27, in <module>
File "aws", line 23, in main
File "awscli\clidriver.pyc", line 49, in main
File "awscli\clidriver.pyc", line 57, in create_clidriver
File "botocore\session.pyc", line 368, in full_config
File "botocore\configloader.pyc", line 105, in load_config
File "botocore\configloader.pyc", line 126, in raw_config_parse
File "botocore\exceptions.pyc", line 27, in __init__
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe9 in position 12: ordinal not in range(128)
What could be the cause of this problem?
It seemed to be an encoding problem with my windows user name as it contains "é". Finally, I uninstalled AWSCli, then created a second windows user account named "aws" and installed AWSCli under this account. In my main account, when I type in
runas /user:*computer_name*\aws cmdin the cmd, it opens a second cmd which runs under the aws account. You can create an alias for this command by doskey awscli=runas /user:*computer_name*\aws cmd. In this manner, I'm able to use AWSCli under my main account.
For anyone who is still facing this problem, this issue has been fixed in later AWS CLI versions. (My version is aws-cli/1.16.65 Python/3.6.0 Windows/10 botocore/1.12.55)
If you are passing JSON files to the CLI tool, also make sure they are encoded in UTF-8.