expected <block end>, but found '<scalar>' in 'reader' in .ebextensions error - amazon-web-services

My command in .ebextensions is failing with the following error:
The configuration file .ebextensions/setvars.config in application version code-pipeline-1674590101261-00ce5911c1ae7ca5d24c81474ba76352008bc540 contains invalid YAML or JSON. YAML exception: Invalid Yaml: while parsing a block mapping in 'reader', line 3, column 5: command: "/opt/elasticbeanstalk/ ... ^ expected <block end>, but found '<scalar>' in 'reader', line 3, column 93: ... nt | jq -r 'to_entries | .[] | "export \(.key)=\"\(.value)\""' > ... ^ , JSON exception: Invalid JSON: Unexpected character (c) at position 0.. Update the configuration file.
The .ebextensions/setvars.config contains:
commands:
setvars:
command: "/opt/elasticbeanstalk/bin/get-config environment | jq -r 'to_entries | .[] | "export \(.key)=\"\(.value)\""' >> ~/.bash_profile"
packages:
yum:
jq: []
The snippet itself has been copied from was docs. Can someone please help me fixing this issue?
I want to run the command inside the double quotes in the elastic beanstalk when the application is starting.
EDIT: I added a backslash before the first quote which was making the YAML valid but then I got this error in cfn-init.log
2023-01-24 20:45:59,593 [ERROR] Command setvars (/"/opt/elasticbeanstalk/bin/get-config environment | jq -r 'to_entries | .[] | "export \(.key)=\"\(.value)\""' >> ~/.bash_profile") failed
2023-01-24 20:45:59,593 [ERROR] Error encountered during build of prebuild_0_checked_backend: Command setvars failed
Traceback (most recent call last):
File "/usr/lib/python3.7/site-packages/cfnbootstrap/construction.py", line 579, in run_config
CloudFormationCarpenter(config, self._auth_config, self.strict_mode).build(worklog)
File "/usr/lib/python3.7/site-packages/cfnbootstrap/construction.py", line 278, in build
self._config.commands)
File "/usr/lib/python3.7/site-packages/cfnbootstrap/command_tool.py", line 127, in apply
raise ToolError(u"Command %s failed" % name)
cfnbootstrap.construction_errors.ToolError: Command setvars failed
2023-01-24 20:45:59,596 [ERROR] -----------------------BUILD FAILED!------------------------
2023-01-24 20:45:59,596 [ERROR] Unhandled exception during build: Command setvars failed
Traceback (most recent call last):
File "/opt/aws/bin/cfn-init", line 181, in <module>
worklog.build(metadata, configSets, strict_mode)
File "/usr/lib/python3.7/site-packages/cfnbootstrap/construction.py", line 137, in build
Contractor(metadata, strict_mode).build(configSets, self)
File "/usr/lib/python3.7/site-packages/cfnbootstrap/construction.py", line 567, in build
self.run_config(config, worklog)
File "/usr/lib/python3.7/site-packages/cfnbootstrap/construction.py", line 579, in run_config
CloudFormationCarpenter(config, self._auth_config, self.strict_mode).build(worklog)
File "/usr/lib/python3.7/site-packages/cfnbootstrap/construction.py", line 278, in build
self._config.commands)
File "/usr/lib/python3.7/site-packages/cfnbootstrap/command_tool.py", line 127, in apply
raise ToolError(u"Command %s failed" % name)
cfnbootstrap.construction_errors.ToolError: Command setvars failed

You do not need quotations at the beginning and the end:
commands:
setvars:
command: /opt/elasticbeanstalk/bin/get-config environment | jq -r 'to_entries | .[] | "export \(.key)=\"\(.value)\""' >> ~/.bash_profile
packages:
yum:
jq: []

Related

FileNotFoundError: [Errno 2] No such file or directory: 'auto-restart'

Tried use watchdog to auto-restart celery worker/beat in docker container when code changed, and get error:
# watchmedo auto-restart -d . -R -- celery -A main_app_name beat -l info
Traceback (most recent call last):
File "/usr/local/bin/watchmedo", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.9/site-packages/watchdog/watchmedo.py", line 641, in main
args.func(args)
File "/usr/local/lib/python3.9/site-packages/watchdog/watchmedo.py", line 625, in auto_restart
handler.start()
File "/usr/local/lib/python3.9/site-packages/watchdog/tricks/__init__.py", line 174, in start
self.process = subprocess.Popen(self.command, preexec_fn=getattr(os, 'setsid', None))
File "/usr/local/lib/python3.9/subprocess.py", line 951, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "/usr/local/lib/python3.9/subprocess.py", line 1821, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'auto-restart'
#
all dependencies installed from requirements.txt:
***
pyyaml~=6.0
watchdog~=2.1.6
auto-restart~=2.2
***
When search 'auro-restart' file in container, got:
# find / -type f -name '*auto*restart*'
/root/.cache/pip/wheels/40/11/c1/9e570f5aac7910cf701243cda8b5fc58e56b329a3d73b4840b/auto_restart-2.2-py3-none-any.whl
/usr/local/bin/auto_restart
/usr/local/bin/auto_restart_tool
/usr/local/lib/python3.9/site-packages/auto_restart/auto_restart_when_git_changed.py
/usr/local/lib/python3.9/site-packages/auto_restart/__pycache__/auto_restart_when_git_changed.cpython-39.pyc
#
Anybody knows the reason of this issue?
Reverting to 2.1.5 fixed the issue.
https://github.com/gorakhargosh/watchdog/issues/855
Use this command instead
watchmedo auto-restart --patterns="*" --recursive -- celery -A main_app_name beat -l info

Getting errors when launching Elastic Beanstalk instance in AWS for Django app

I'm using Python 3.9.5 for my Django app and I am trying to deploy it on AWS in Elastic Beanstalk. The platform I chose was AWS Linux 2 with Python 3.8. When I try to do an eb create, I get this stacktrace error:
----------------------------------------
/var/log/eb-engine.log
----------------------------------------
cmd_obj.run()
File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 79, in run
_build_ext.run(self)
File "/usr/lib64/python3.8/distutils/command/build_ext.py", line 340, in run
self.build_extensions()
File "/usr/lib64/python3.8/distutils/command/build_ext.py", line 449, in build_extensions
self._build_extensions_serial()
File "/usr/lib64/python3.8/distutils/command/build_ext.py", line 474, in _build_extensions_serial
self.build_extension(ext)
File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 196, in build_extension
_build_ext.build_extension(self, ext)
File "/usr/lib64/python3.8/distutils/command/build_ext.py", line 528, in build_extension
objects = self.compiler.compile(sources,
File "/usr/lib64/python3.8/distutils/ccompiler.py", line 574, in compile
self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
File "/usr/lib64/python3.8/distutils/unixccompiler.py", line 129, in _compile
raise CompileError(msg)
distutils.errors.CompileError: command 'gcc' failed with exit status 1
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-v7gtd0uf/cld2-cffi_20eacbc571034a60bc5a47d71b1737a7/setup.py", line 154, in <module>
setup(
File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/usr/lib64/python3.8/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib64/python3.8/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/usr/lib64/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 299, in run
self.find_sources()
File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 306, in find_sources
mm.run()
File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 541, in run
self.add_defaults()
File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 578, in add_defaults
sdist.add_defaults(self)
File "/usr/lib64/python3.8/distutils/command/sdist.py", line 226, in add_defaults
self._add_defaults_python()
File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/setuptools/command/sdist.py", line 107, in _add_defaults_python
build_py = self.get_finalized_command('build_py')
File "/usr/lib64/python3.8/distutils/cmd.py", line 299, in get_finalized_command
cmd_obj.ensure_finalized()
File "/usr/lib64/python3.8/distutils/cmd.py", line 107, in ensure_finalized
self.finalize_options()
File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/setuptools/command/build_py.py", line 37, in finalize_options
orig.build_py.finalize_options(self)
File "/usr/lib64/python3.8/distutils/command/build_py.py", line 43, in finalize_options
self.set_undefined_options('build',
File "/usr/lib64/python3.8/distutils/cmd.py", line 287, in set_undefined_options
src_cmd_obj.ensure_finalized()
File "/usr/lib64/python3.8/distutils/cmd.py", line 107, in ensure_finalized
self.finalize_options()
File "/tmp/pip-install-v7gtd0uf/cld2-cffi_20eacbc571034a60bc5a47d71b1737a7/setup.py", line 143, in finalize_options
self.distribution.ext_modules = get_ext_modules()
File "/tmp/pip-install-v7gtd0uf/cld2-cffi_20eacbc571034a60bc5a47d71b1737a7/setup.py", line 128, in get_ext_modules
import cld2
File "/tmp/pip-install-v7gtd0uf/cld2-cffi_20eacbc571034a60bc5a47d71b1737a7/cld2/__init__.py", line 187, in <module>
_full_cld2 = _full_ffi.verify('#include <binding_decls.h>',
File "/tmp/pip-install-v7gtd0uf/cld2-cffi_20eacbc571034a60bc5a47d71b1737a7/.eggs/cffi-1.14.5-py3.8-linux-x86_64.egg/cffi/api.py", line 468, in verify
lib = self.verifier.load_library()
File "/tmp/pip-install-v7gtd0uf/cld2-cffi_20eacbc571034a60bc5a47d71b1737a7/.eggs/cffi-1.14.5-py3.8-linux-x86_64.egg/cffi/verifier.py", line 104, in load_library
self._compile_module()
File "/tmp/pip-install-v7gtd0uf/cld2-cffi_20eacbc571034a60bc5a47d71b1737a7/.eggs/cffi-1.14.5-py3.8-linux-x86_64.egg/cffi/verifier.py", line 201, in _compile_module
outputfilename = ffiplatform.compile(tmpdir, self.get_extension())
File "/tmp/pip-install-v7gtd0uf/cld2-cffi_20eacbc571034a60bc5a47d71b1737a7/.eggs/cffi-1.14.5-py3.8-linux-x86_64.egg/cffi/ffiplatform.py", line 22, in compile
outputfilename = _build(tmpdir, ext, compiler_verbose, debug)
File "/tmp/pip-install-v7gtd0uf/cld2-cffi_20eacbc571034a60bc5a47d71b1737a7/.eggs/cffi-1.14.5-py3.8-linux-x86_64.egg/cffi/ffiplatform.py", line 58, in _build
raise VerificationError('%s: %s' % (e.__class__.__name__, e))
cffi.VerificationError: CompileError: command 'gcc' failed with exit status 1
----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/52/6d/044789e730141bcda2a7368836f714684a7d13bd44a2a33b387cb31b4335/cld2-cffi-0.1.4.tar.gz#sha256=3a29948364ed1e426c5bf542832eee208b1c70c0ad512b1a99bec0e6486f6c67 (from https://pypi.org/simple/cld2-cffi/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement cld2-cffi==0.1.4 (from versions: 0.1.1, 0.1.3, 0.1.4)
ERROR: No matching distribution found for cld2-cffi==0.1.4
2021/06/28 05:05:10.028254 [INFO] Executing cleanup logic
2021/06/28 05:05:10.028356 [INFO] CommandService Response: {"status":"FAILURE","api_version":"1.0","results":[{"status":"FAILURE","msg":"Engine execution has encountered an error.","returncode":1,"events":[{"msg":"Instance deployment failed to install application dependencies. The deployment failed.","timestamp":1624856710,"severity":"ERROR"},{"msg":"Instance deployment failed. For details, see 'eb-engine.log'.","timestamp":1624856710,"severity":"ERROR"}]}]}
2021/06/28 05:05:10.028538 [INFO] Platform Engine finished execution on command: app-deploy
2021/06/28 05:09:53.602699 [INFO] Starting...
2021/06/28 05:09:53.602746 [INFO] Starting EBPlatform-PlatformEngine
2021/06/28 05:09:53.602761 [INFO] reading event message file
2021/06/28 05:09:53.602865 [INFO] no eb envtier info file found, skip loading env tier info.
2021/06/28 05:09:53.602925 [INFO] Engine received EB command cfn-hup-exec
2021/06/28 05:09:53.668640 [INFO] Running command /bin/sh -c /opt/aws/bin/cfn-get-metadata -s arn:aws:cloudformation:us-west-1:465435786833:stack/awseb-e-53i6aevdmm-stack/176ff4a0-d7ce-11eb-a76f-02cd38bb34df -r AWSEBAutoScalingGroup --region us-west-1
2021/06/28 05:09:53.981851 [INFO] Running command /bin/sh -c /opt/aws/bin/cfn-get-metadata -s arn:aws:cloudformation:us-west-1:465435786833:stack/awseb-e-53i6aevdmm-stack/176ff4a0-d7ce-11eb-a76f-02cd38bb34df -r AWSEBBeanstalkMetadata --region us-west-1
2021/06/28 05:09:54.274593 [INFO] checking whether command tail-log is applicable to this instance...
2021/06/28 05:09:54.274608 [INFO] this command is applicable to the instance, thus instance should execute command
2021/06/28 05:09:54.274612 [INFO] Engine command: (tail-log)
2021/06/28 05:09:54.274662 [INFO] Executing instruction: GetTailLogs
2021/06/28 05:09:54.274666 [INFO] Tail Logs...
2021/06/28 05:09:54.274907 [INFO] Running command /bin/sh -c tail -n 100 /var/log/eb-engine.log
It appears that some library in my Python code isn't being read or found correctly, but I have no idea how to debug it. Does anybody know how to approach this problem?

AWS ElasticBeanstalk ebextensions command fails during deployment but works fine when I run it manually

I'm using .ebextensions to run some automation commands, but some commands are failing without a explaining the cause issue or printing any informative stacktrace, but when I run the same command manually it works like a charm with no issues:
the command:
aws logs put-retention-policy --log-group-name `{"Fn::Join":["/", ["/aws/eb", { "Ref":"AWSEBEnvironmentName" }, "var/log/app-log"]]}` --retention-in-days 7 --region `{"Ref":"AWS::Region"}`
the stack trace:
[ERROR] Command 03 (aws logs put-retention-policy --log-group-name /aws/log-group-name --retention-in-days 7 --region us-west-2) failed
2021-04-19 21:33:50,248 [ERROR] Error encountered during build of prebuild_2_squirrel: Command 03 failed
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/cfnbootstrap/construction.py", line 542, in run_config
CloudFormationCarpenter(config, self._auth_config).build(worklog)
File "/usr/lib/python2.7/site-packages/cfnbootstrap/construction.py", line 260, in build
changes['commands'] = CommandTool().apply(self._config.commands)
File "/usr/lib/python2.7/site-packages/cfnbootstrap/command_tool.py", line 117, in apply
raise ToolError(u"Command %s failed" % name)
ToolError: Command 03 failed
2021-04-19 21:33:50,250 [ERROR] -----------------------BUILD FAILED!------------------------
2021-04-19 21:33:50,250 [ERROR] Unhandled exception during build: Command 03 failed
Traceback (most recent call last):
File "/opt/aws/bin/cfn-init", line 171, in <module>
worklog.build(metadata, configSets)
File "/usr/lib/python2.7/site-packages/cfnbootstrap/construction.py", line 129, in build
Contractor(metadata).build(configSets, self)
File "/usr/lib/python2.7/site-packages/cfnbootstrap/construction.py", line 530, in build
self.run_config(config, worklog)
File "/usr/lib/python2.7/site-packages/cfnbootstrap/construction.py", line 542, in run_config
CloudFormationCarpenter(config, self._auth_config).build(worklog)
File "/usr/lib/python2.7/site-packages/cfnbootstrap/construction.py", line 260, in build
changes['commands'] = CommandTool().apply(self._config.commands)
File "/usr/lib/python2.7/site-packages/cfnbootstrap/command_tool.py", line 117, in apply
raise ToolError(u"Command %s failed" % name)
Anyone seen this before or knows what it could be the issue ?
Thanks in advance.
after checking the /var/log/cfn-init-cmd.log, I saw details about the issue, the log group gets created few seconds after, so I just need to wait

why murano-test-runner connecte to keystone?

I add some unit tests for my package "kubernetes-cluster" follow Tomcat Package(see: https://github.com/openstack/murano-apps/blob/master/Tomcat/package/Classes/TomcatTest.yaml).
and then, run it with command(without --config-file,--os-auth-url, without murano.conf in path /etc/murano/):
murano-test-runner -v io.murano.apps.docker.kubernetes.KuryrCluster io.murano.test.KuryrClusterTest -l /my-packages and core-library/local/path
I got the error message:
2017-03-16 07:39:49.978 | 2017-03-16 07:42:50.773 1697 ERROR murano.cmd.test_runner [-] Command failed: 'NoneType' object has no attribute 'replace'
2017-03-16 07:39:49.978 | Traceback (most recent call last):
2017-03-16 07:39:49.978 | File "/home/jenkins/workspace/murano-programming-language-unit-test/.tox/murano-test-runner/lib/python2.7/site-packages/murano/cmd/test_runner.py", line 374, in main
2017-03-16 07:39:49.978 | exit_code = test_runner.run_tests()
2017-03-16 07:39:49.978 | File "/home/jenkins/workspace/murano-programming-language-unit-test/.tox/murano-test-runner/lib/python2.7/site-packages/murano/cmd/test_runner.py", line 213, in run_tests
2017-03-16 07:39:49.978 | ks_opts = self._validate_keystone_opts(self.args)
2017-03-16 07:39:49.978 | File "/home/jenkins/workspace/murano-programming-language-unit-test/.tox/murano-test-runner/lib/python2.7/site-packages/murano/cmd/test_runner.py", line 195, in _validate_keystone_opts
2017-03-16 07:39:49.979 | ks_opts[param] = ks_opts[param].replace('v2.0', 'v3')
2017-03-16 07:39:49.979 | AttributeError: 'NoneType' object has no attribute 'replace'
2017-03-16 07:39:49.979 |
2017-03-16 07:39:49.979 | Command failed: 'NoneType' object has no attribute 'replace'
I just want to check my unit tests write in muranoPL right or not, like pep8 or py27.
Can I run tox -e murano-test-runner like tox -e py27 without connect to keystone server and neutron server? If can not , why? if I can, how? mock keystoneclient and neutronclient? or other ways?
some one can help me? thanks.
I mock:
keystoneclient.v3.Client(and deal with auth_uri.replace('v2.0', 'v3') AttributeError Exception)
murano.engine.system.net_explorer.NetworkExplorer._get_client
then murano-test-runner can run without config-opts os-* or config-file

Elastic Beanstalk: dictionary update sequence element #0 has length 1; 2 is required

We had a working Elastic Beanstalk environment, but after adding a config file, we've started getting this error during deploy:
Error occurred during build: dictionary update sequence element #0 has length 1; 2 is required
The config file looks like this:
# .ebextentions/seed.config
container_commands:
01_set_tenant_gateway:
command: rake db:seed:set_tenant_gateway
leader_only: true
env: our-env
The log gives us this:
2015-04-09 18:27:48,220 [DEBUG] Running command seed_tenant_gateway
2015-04-09 18:27:48,220 [DEBUG] Generating defaults for command
seed_tenant_gateway <<<
2015-04-09 18:27:48,405 [DEBUG] Running test for command
seed_tenant_gateway 2015-04-09 18:27:48,406 [ERROR] Unhandled
exception during build: dictionary update sequence element #0 has
length 1; 2 is required Traceback (most recent call last): File
"/opt/aws/bin/cfn-init", line 122, in
worklog.build(detail.metadata, configSets) File "/usr/lib/python2.6/site-packages/cfnbootstrap/construction.py", line
117, in build
Contractor(metadata).build(configSets, self) File "/usr/lib/python2.6/site-packages/cfnbootstrap/construction.py", line
502, in build
self.run_config(config, worklog) File "/usr/lib/python2.6/site-packages/cfnbootstrap/construction.py", line
511, in run_config
CloudFormationCarpenter(config, self._auth_config).build(worklog) File "/usr/lib/python2.6/site-packages/cfnbootstrap/construction.py",
line 247, in build
changes['commands'] = CommandTool().apply(self._config.commands) File "/usr/lib/python2.6/site-packages/cfnbootstrap/command_tool.py",
line 84, in apply
testResult = ProcessHelper(test, env=env, cwd=cwd).call() File "/usr/lib/python2.6/site-packages/cfnbootstrap/util.py", line 397, in
init
self._env = dict(env) ValueError: dictionary update sequence element #0 has length 1; 2 is required
What might be going on here?
1) Syntax for env key:
env:
<variable name>: <variable value>
2) Also, you need to enclose command in quotes:
command: "<command to run>"
from http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-ec2.html