Aws cli not installing in cygwin throwing error - amazon-web-services

These are the error details. Trying to install the awscli through pip. tried wget method as well.
$ pip install awscli
Collecting awscli
Using cached awscli-1.25.53-py3-none-any.whl (3.9 MB)
Collecting PyYAML<5.5,>=3.10
Using cached PyYAML-5.4.1.tar.gz (175 kB)
0 [main] python 2066 child_copy: stack write copy failed, 0xFFFF5480..0x100000000, done 4294923504, windows pid 16392, Win32 error 5
5496 [main] python 2066 dofork: child
2067 - pid 11768, exitval 0x103, errno 11
ERROR: Error [Errno 11] Resource
temporarily unavailable while executing command pip subprocess to install build dependencies
Installing build dependencies ... error
ERROR: Could not install packages due to an OSError: [Errno 11] Resource temporarily unavailable
Any help will be appreciated.

From Installing or updating the latest version of the AWS CLI - AWS Command Line Interface:
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

Related

yarn on AWS Lambda?

Did anyone manage to get yarn installed and working on AWS Lambda?
(I'm trying to create a function that can build small React projects)
Currently I have yarn installed and working in a Docker image (based on public.ecr.aws/lambda/provided:al2), but while running it in the Lambda environment I'm getting permission errors when yarn tries to access the read-only file system.
Dockerfile snippet:
FROM public.ecr.aws/lambda/provided:al2
...
# Update yum
RUN yum -y update
# Install NodeJS
RUN curl -fsSL https://rpm.nodesource.com/setup_16.x \
| bash - && yum -y install nodejs
# Install Yarn
RUN curl -fsSL https://dl.yarnpkg.com/rpm/yarn.repo \
| tee /etc/yum.repos.d/yarn.repo && \
rpm --import https://dl.yarnpkg.com/rpm/pubkey.gpg && \
yum -y install yarn
# Copy Binaries
RUN cp -r \
/usr/share/yarn/bin/* \
/usr/bin/corepack /usr/bin/node /usr/bin/npm /usr/bin/npx \
/opt/bin/
# Copy Libraries
RUN cp -r \
/usr/share/yarn/lib/* \
/usr/lib/node_modules \
/opt/lib/
# Configure NPM & Yarn
RUN npm config set prefix /tmp/npm/
RUN yarn config set prefix /tmp/yarn/
...
Errors:
>yarn install
warning Cannot find a suitable global folder. Tried these: "/usr/local, /home/sbx_user1051/.yarn"
ovl: Error while doing RPMdb copy-up:
[Errno 30] Read-only file system: '/var/lib/rpm/.dbenv.lock'
Could not set cachedir: [Errno 30] Read-only file system: '/var/tmp/yum-sbx_user1051-eC5Oqk'
Error making cache directory: /var/cache/yum/x86_64/2/amzn2-core error was: [Errno 30] Read-only file system: '/var/cache/yum'
error Command failed with exit code 1.
*** edit ***
With global flag
>yarn --global-folder /tmp/yarn/ install
warning Skipping preferred cache folder "/home/sbx_user1051/.cache/yarn" because it is not writable.
warning Selected the next writable cache folder in the list, will be "/tmp/.yarn-cache-993".
warning Cannot find a suitable global folder. Tried these: "/usr/local, /home/sbx_user1051/.yarn"
ovl: Error while doing RPMdb copy-up:
[Errno 30] Read-only file system: '/var/lib/rpm/.dbenv.lock'
Could not set cachedir: [Errno 30] Read-only file system: '/var/tmp/yum-sbx_user1051-3FcZxd'
Error making cache directory: /var/cache/yum/x86_64/2/amzn2-core error was: [Errno 30] Read-only file system: '/var/cache/yum'
error Command failed with exit code 1.

Getting error "post-link script failed for package defaults ::ipykernel-4.10.0-py27_0" while installing ipykernel in anaconda with python 2.7

I am trying to install ipykernel in a separate python 2.7 environment (py27) for use of jupyter notebook. My base environment uses python3.9 that works great. When trying to use conda install ipykernel, I get the error below. Yes, this was done in a separate environment from my base and I am not aware of any virus softwares that are impeding this from successfully completing. Those are the only answers that I've seemed to find online. I also tried reinstalling anaconda. I did not attach the output from conda install -v ipykernel, but saw an error saying invalid syntax in decorator.py on line 162. It was pointing to the file=stderr in a print function call.
Similar to this problem https://github.com/conda/conda/issues/7351, but was unable to solve the problem with the suggestions.
(py27) C:\Users\Heath>conda install ipykernel
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: done
## Package Plan ##
environment location: C:\Users\Heath\anaconda3\envs\py27
added / updated specs:
- ipykernel
The following NEW packages will be INSTALLED:
backports pkgs/main/noarch::backports-1.0-pyhd3eb1b0_2
backports.shutil_~ pkgs/main/noarch::backports.shutil_get_terminal_size-1.0.0-pyhd3eb1b0_3
backports_abc pkgs/main/noarch::backports_abc-0.5-py_1
colorama pkgs/main/noarch::colorama-0.4.4-pyhd3eb1b0_0
decorator pkgs/main/noarch::decorator-5.1.0-pyhd3eb1b0_0
enum34 pkgs/main/win-64::enum34-1.1.6-py27_1
futures pkgs/main/win-64::futures-3.3.0-py27_0
ipykernel pkgs/main/win-64::ipykernel-4.10.0-py27_0
ipython pkgs/main/win-64::ipython-5.8.0-py27_0
ipython_genutils pkgs/main/noarch::ipython_genutils-0.2.0-pyhd3eb1b0_1
jupyter_client pkgs/main/win-64::jupyter_client-5.3.4-py27_0
jupyter_core pkgs/main/win-64::jupyter_core-4.6.1-py27_0
libsodium pkgs/main/win-64::libsodium-1.0.16-h8b3e59e_0
pathlib2 pkgs/main/win-64::pathlib2-2.3.5-py27_0
pickleshare pkgs/main/win-64::pickleshare-0.7.5-py27_0
prompt_toolkit pkgs/main/win-64::prompt_toolkit-1.0.15-py27h3a8ec6a_0
pygments pkgs/main/noarch::pygments-2.5.2-py_0
python-dateutil pkgs/main/noarch::python-dateutil-2.8.2-pyhd3eb1b0_0
pywin32 pkgs/main/win-64::pywin32-227-py27h0c8e037_1
pyzmq pkgs/main/win-64::pyzmq-18.1.0-py27hc56fc5f_0
scandir pkgs/main/noarch::scandir-1.10.0-pyh5d7bf9c_3
simplegeneric pkgs/main/win-64::simplegeneric-0.8.1-py27_2
singledispatch pkgs/main/noarch::singledispatch-3.7.0-pyhd3eb1b0_1001
six pkgs/main/noarch::six-1.16.0-pyhd3eb1b0_0
tornado pkgs/main/win-64::tornado-5.1.1-py27h0c8e037_0
traitlets pkgs/main/win-64::traitlets-4.3.3-py27_0
wcwidth pkgs/main/noarch::wcwidth-0.2.5-pyhd3eb1b0_0
win_unicode_conso~ pkgs/main/win-64::win_unicode_console-0.5-py27hc037021_0
zeromq pkgs/main/win-64::zeromq-4.3.1-h2880e7c_3
Proceed ([y]/n)? y
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
ERROR conda.core.link:_execute(699): An error occurred while installing package 'defaults::ipykernel-4.10.0-py27_0'.
Rolling back transaction: done
LinkError: post-link script failed for package defaults::ipykernel-4.10.0-py27_0
location of failed script: C:\Users\Heath\anaconda3\envs\py27\Scripts\.ipykernel-post-link.bat
==> script messages <==
<None>
==> script output <==
stdout:
stderr:
return code: 1
()```

Custom Lambda image - getting aws-lambda-ric error when trying to run Lambda

I've build a NodeJS project, which I need to run on a custom Docker image.
This is my Dockerfile:
FROM public.ecr.aws/lambda/nodejs:14-x86_64
# Create app directory
WORKDIR /usr/src/
RUN yum update && yum install -y git openssh-client vim python py-pip pip jq
RUN yum update && yum install -y automake autoconf libtool dpkg pkgconfig nasm libpng cmake
RUN pip install awscli
# RUN apk --purge -v del py-pip
# RUN rm /var/cache/apk/*
RUN npm install -g yarn
RUN yarn install --frozen-lockfile
# Bundle app source
COPY . .
RUN yarn build
ENTRYPOINT ["npx", "aws-lambda-ric"]
CMD [ "src/executionHandler.runner" ]
But when I call docker run <imagename>
I get the following errors:
tar: curl-7.78.0/tests/data/test1131: Cannot open: No such file or directory
tar: curl-7.78.0: Cannot mkdir: Permission denied
tar: curl-7.78.0/tests/data/test971: Cannot open: No such file or directory
tar: curl-7.78.0: Cannot mkdir: Permission denied
tar: Exiting with failure status due to previous errors
./scripts/preinstall.sh: line 28: cd: curl-7.78.0: No such file or directory
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! aws-lambda-ric#2.0.0 preinstall: `./scripts/preinstall.sh`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the aws-lambda-ric#2.0.0 preinstall script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2021-10-13T09_11_31_626Z-debug.log
Install for [ 'aws-lambda-ric#latest' ] failed with code 1
The base image I use was taken from official AWS images repository.
How can I resolve this permissions issue?
This isn't really a permissions issue.
According to https://gallery.ecr.aws/lambda/nodejs the base image you're using public.ecr.aws/lambda/nodejs should come with the entire runtime pre-installed. I think the issue is that your entrypoint uses npx, which is a tool for running local npm packages, and the base image can only have the packages installed globally. npx, if it can't find the package in local package.json, tries to install the package. This is both unnecessary, since it's already installed globally, and not possible in the stripped down public image in to which you have installed some of the prereqs like cmake, autoconf, etc, but not libcurl.
I suspect
ENTRYPOINT ["aws-lambda-ric"]
without npx and without all the extraneous development packages will work fine with this image.

Error whenn installing awsebcli via Gitlab CLI

I have springboot project in gitlab that I try to deploy to Amazon Elastic Beanstalk environment that I created via Amazon web console.
In gitlab I am using Default shared Runners.
to deploy to AWS I must use 'eb' cli, for this I also need python
I tired to use this in my .gitlab-ci.yml
deploy:
stage: deploy
image: python:3.7.2-alpine3.7
script:
- pip install -r awsebcli
- eb use myenv-env -v
- eb deploy myenv-env -v
But got errors:
Pulling docker image python:3.7.2-alpine3.7 ...
Using docker image sha256:00be2573e9f79754b17954ba7a310a5f70c25b6f5bb78375e27e9e86d874877e for python:3.7.2-alpine3.7 ...
Running on runner-0c303413-project-4590-concurrent-0 via ed06f0845ef4...
Fetching changes...
Removing .m2/
Removing target/
HEAD is now at 5c19020 aws1
Checking out 5c190200 as master...
Skipping Git submodules setup
Checking cache for default...
Successfully extracted cache
Downloading artifacts for build (14719)...
Downloading artifacts from coordinator... ok id=14719 responseStatus=200 OK token=Yy7j2TGy
$ pip install -r awsebcli
Could not open requirements file: [Errno 2] No such file or directory: 'awsebcli'
ERROR: Job failed: exit code 1
Update
when I changed the line into:
pip3 install awsebcli
got:
Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7fe6611ff2e8>: Failed to establish a new connection: [Errno -2] Name does not resolve')': /simple/awsebcli/
Could not find a version that satisfies the requirement awsebcli (from versions: )
No matching distribution found for awsebcli
The problem was proxy.
I added this:
pip3 --proxy=myProxy:myPort install awsebcli --upgrade

Django on Elastic Beanstalk "requirements.txt" is invalid

I'm trying to deploy a django project with Elastic Beanstalk cli. I followed the Official Amazon Tutorial as well as this Tutorial on realpython.com.
The Deploymentprocess works fine as long as I stick to a newly created Django Project with just a view dependencies for pip.
When try it with my Django Project The Deployment fails because the requirements.txt is invalid. I created it inside my virutalenv like this: pip freeze > requirements.txt
braintree==3.32.0
cffi==1.9.1
cryptography==1.7
Django==1.10.4
django-allauth==0.29.0
django-betterforms==1.1.4
django-contrib-comments==1.7.3
django-formtools==1.0
django-payments==0.9.6
django-tinymce==2.4.0
enum34==1.1.6
idna==2.1
ipaddress==1.0.17
oauthlib==2.0.1
Pillow==3.4.2
pyasn1==0.1.9
pycparser==2.17
PyJWT==1.4.2
python-openid==2.2.5
requests==2.12.3
requests-oauthlib==0.7.0
six==1.10.0
stripe==1.43.0
suds-jurko==0.6
xmltodict==0.10.2
In my despair I tried to activate the dependencies one by one. The Deployment process is so slow that I gave up after a while. But now I know now that Django==1.10.4, django-allauth==0.29.0, django-betterforms==1.1.4, django-contrib-comments==1.7.3, django-formtools==1.0 and django-tinymce==2.4.0 are not the problem. With django-payments==0.9.6 it fails but its not the only dependency that causes the problem.
This is in my activity.log:
error: command 'gcc' failed with exit status 1
----------------------------------------
Command "/opt/python/run/venv/bin/python2.7 -c "import setuptools, tokenize;
__file__='/tmp/pip-build-BF9Oen/cffi/setup.py';
exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))"
install --record /tmp/pip-BsXWzo-record/install-record.txt --single-version-externally-managed --compile --install-headers
/opt/python/run/venv/include/site/python2.7/cffi" failed with error code 1 in /tmp/pip-build-BF9Oen/cffi
You are using pip version 7.1.2, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
2016-12-13 14:49:05,155 ERROR
Error installing dependencies: Command '/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status 1
Traceback (most recent call last):
File "/opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py", line 22, in main
install_dependencies()
File "/opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py", line 18, in install_dependencies
check_call('%s install -r %s' % (os.path.join(APP_VIRTUAL_ENV, 'bin', 'pip'), requirements_file), shell=True)
File "/usr/lib64/python2.7/subprocess.py", line 541, in check_call
raise CalledProcessError(retcode, cmd)
CalledProcessError: Command
'/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements.txt'
returned non-zero exit status 1 (Executor::NonZeroExitStatus)
I't could have something to do with missing Software. I know how to use the .ebextensions .config files but I don't know whats missing. Is there a descent way to debug this locally?
You seem to have problems with both FFI and Python. You have Cryptography in use but don't probably have all the required libraries Install Python and FFI requirements and OpenSSL dependencies for Cryptography.
$ eb ssh # SSH into your EB environment
$ sudo su # gain root privileges
$ yum install python-devel # or python27-devel
$ yum install libffi-devel # CFFI / FFI requirements
$ yum install openssl-devel # Cryptography requirements
If Pip complains, make sure you have the latest pip installed:
# repeat SSH and root privilege steps
$ source /opt/python/run/venv/bin/activate # activate Python environment
$ pip install --upgrade pip # upgrade to latest pip
To make the changes permanent, append them to your EB configuration. This also works if you do not have direct SSH access.
# .ebextensions/10_packages.config excerpt
packages:
yum:
gcc: []
libffi-devel: []
openssl-devel: []
python-devel: [] # or python27-devel
.ebextensions folder files with the .config suffix are state declarations for the Elastic Beanstalk environment. That is, they are instructions that are run on each Elastic Beanstalk deployment when you run eb deploy or your CI environment deploys code to EB.
Amazon Linux is an RPM based Linux that uses YUM for package management. You will find RHEL / CentOS / OpenSUSE instructions for Python package installation helpful for the Amazon Linux distribution.
There typically isn’t a way to debug these issues locally, because people don’t normally run Amazon linux on their local machines, and EB has a very specific environment setup that is not easy to reproduce (at least it wasn’t for me).
If you want to do it fast, just ssh to one of your EB/EC2 instances and try to install the pip requirements one-by-one by hand. If you want to do it right, you do have to go through this list of requirements one-by-one. (i.e., start with one requirement in the requirements file, deploy, then two requirements, then three requirements, etc.). You may want to start with upgrading pip by adding
pip==9.0.1
At the top of your requirements file and also making sure the following Amazon Linux (if that is what you are using) are installed:
python-devel
libffi
libffi-devel