Test an AWS lambda locally using Docker container image - amazon-web-services

i have been having a hard time getting this thing to work right
i am trying to invoke the Lambda locally which is running on docker container but i have following issue
Docker File
FROM public.ecr.aws/lambda/python:3.8
COPY myfunction.py ./
CMD ["myfunction.lambda_handler"]
Python File
myfunction.py
import json
import sys
def lambda_handler(event, context):
print("Hello AWS!")
print("event = {}".format(event))
return {
'statusCode': 200,
}
Step 1:
docker build --tag custom .
output
C:\Users\s.shah\IdeaProjects\YoutubeVideos\Learn>docker build --tag custom .
[+] Building 0.5s (7/7) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 31B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for public.ecr.aws/lambda/python:3.8 0.5s
=> [internal] load build context 0.0s
=> => transferring context: 35B 0.0s
=> [1/2] FROM public.ecr.aws/lambda/python:3.8#sha256:d5a4b8f3f7394358bfe2cb51677f3d14af59c08adf831332cb4501f56dfd64cc 0.0s
=> CACHED [2/2] COPY myfunction.py ./ 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:6a339ad8416cd93632ae4418e89409dae2e8a684de7990746b613b223a974899 0.0s
=> => naming to docker.io/library/custom
Step 2:
docker run -p 9000:8080 random-letter:latest
output
INFO[0000] exec '/var/runtime/bootstrap' (cwd=/var/task, handler=)
Step 3:
curl -XPOST "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{}'
{"errorMessage": "Unable to unmarshal input: Expecting value: line 1 column 1 (char 0)", "errorType": "Runtime.UnmarshalError", "stackTrace": []}
INFO[0057] extensionsDisabledByLayer(/opt/disable-extensions-jwigqn8j) -> stat /opt/disable-extensions-jwigqn8j: no such file or directory
WARN[0057] Cannot list external agents error="open /opt/extensions: no such file or directory"
START RequestId: d49a7179-7ec5-4122-933f-be04abfed953 Version: $LATEST
Traceback (most recent call last):able to unmarshal input: Expecting value: line 1 column 1 (char 0)
END RequestId: d49a7179-7ec5-4122-933f-be04abfed953
REPORT RequestId: d49a7179-7ec5-4122-933f-be04abfed953 Init Duration: 0.23 ms Duration: 65.39 ms Billed Duration: 100 ms Memory Size: 3008 MB Max Memory Used: 3008 MB
Any help would be great

Answer
https://github.com/lambci/docker-lambda/issues/208
curl -XPOST "http://localhost:9000/2015-03-31/functions/function/invocations" -d "{"""msg""":"""hello"""}"
this works

For anyone using the Postman for invoking the endpoint, you can try the following:

Related

[Driver Manager]Can't open lib '/opt/microsoft/msodbcsql17/lib64/libmsodbcsql-17.9.so.1.1'

When I run
$ python manage.py inspectdb --database=mssql_database
I have the following error
django.db.utils.Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib '/opt/microsoft/msodbcsql17/lib64/libmsodbcsql-17.9.so.1.1' : file not found (0) (SQLDriverConnect)")
but the file libmsodbcsql-17.9.so.1.1 is there.
$ cat /etc/odbcinst.ini
[ODBC]
Trace=Yes
TraceFile=/tmp/odbc.log
[FreeTDS]
Description=TDS driver (Sybase/MS SQL)
Driver=libtdsodbc.so
Setup=libtdsS.so
CPTimeout=
CPReuse=
UsageCount=2
[ODBC Driver 17 for SQL Server]
Description=Microsoft ODBC Driver 17 for SQL Server
Driver=/opt/microsoft/msodbcsql17/lib64/libmsodbcsql-17.9.so.1.1
UsageCount=1
$ odbcinst -j
unixODBC 2.3.7
odbcinst: symbol lookup error: odbcinst: undefined symbol: odbcinst_system_file_name
$ ldd /opt/microsoft/msodbcsql17/lib64/libmsodbcsql-17.9.so.1.1
linux-vdso.so.1 (0x00007fff545c4000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f9f85470000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f9f85268000)
libodbcinst.so.2 => /home/pd/sibp/env/lib/python3.6/site-packages/sqlanydb-1.0.11.dist-info/lib64/libodbcinst.so.2 (0x00007f9f84fcc000)
libkrb5.so.3 => /usr/lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007f9f84cf6000)
libgssapi_krb5.so.2 => /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007f9f84aab000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f9f84722000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f9f84384000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f9f8416c000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f9f83f4d000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9f83b5c000)
/lib64/ld-linux-x86-64.so.2 (0x00007f9f85a80000)
libdbtasks17_r.so => /home/pd/sibp/env/lib/python3.6/site-packages/sqlanydb-1.0.11.dist-info/lib64/libdbtasks17_r.so (0x00007f9f83912000)
libnsl.so.1 => /lib/x86_64-linux-gnu/libnsl.so.1 (0x00007f9f836f8000)
libk5crypto.so.3 => /usr/lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007f9f834c6000)
libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007f9f832c2000)
libkrb5support.so.0 => /usr/lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007f9f830b7000)
libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007f9f82eb3000)
libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f9f82c99000)
OS: Ubuntu 18.04.6 LTS
I started fresh with clean system and I'm creating snapshots of my virtualbox now. This happens as a result of installing SQL Anywhere Database Client.
See here. I need SQL Anywhere Database Client to work with Sybase. After finishing this installation I have the above error.
$ pip list
Package Version
------------- -------
Django 1.8
django-pyodbc 1.1.3
pip 21.3.1
pyodbc 4.0.32
setuptools 59.6.0
sqlany-django 1.13
sqlanydb 1.0.11
wheel 0.37.1
While not a complete answer for this particular case, the following information may be helpful for others having difficulty using both sqlanydb (for SAP SQL Anywhere) and pyodbc (for Microsoft SQL Server) on the same machine.
The SQL Anywhere setup instructions cited in the question instruct us to
source "/opt/sqlanywhere17/bin64/sa_config.sh"
in .bashrc. That script includes, in part
LD_LIBRARY_PATH="$SQLANY17/lib32:${LD_LIBRARY_PATH:-}"
LD_LIBRARY_PATH="$SQLANY17/lib64:${LD_LIBRARY_PATH:-}"
export LD_LIBRARY_PATH
which prepends the SQLANY17/ directories to any existing LD_LIBRARY_PATH. On a vanilla install of Xubuntu 20.04 there is no LD_LIBRARY_PATH defined, so the net result is
$ echo $LD_LIBRARY_PATH
/opt/sqlanywhere17/lib64:/opt/sqlanywhere17/lib32:
That works fine for sqlanydb
# any.py
import sqlanydb
# Initiate connection to the database
DB_PARAMS = {"HOST": "192.168.0.199",
"USER": "dba",
"PASSWORD":"sql",
"DB":""}
conn = sqlanydb.connect(host=DB_PARAMS['HOST'], uid=DB_PARAMS['USER'], pwd=DB_PARAMS['PASSWORD'], dbn=DB_PARAMS['DB'])
# Instantiate cursor
curs = conn.cursor()
# Execute a query
curs.execute("SELECT 'success' as connection_status FROM SYS.DUMMY")
result = curs.fetchall()
print(result)
$ python3 any.py
[('success',)]
but it causes pyodbc to fail
# pyo.py
import pyodbc
cnxn = pyodbc.connect(
"Driver=ODBC Driver 17 for SQL Server;"
"Server=192.168.0.199;"
"Database=test;"
"UID=scott;PWD=tiger^5HHH;"
)
crsr = cnxn.cursor()
print(crsr.execute("SELECT 'success' AS connection_status").fetchall())
$ python3 pyo.py
Traceback (most recent call last):
File "pyo.py", line 3, in <module>
cnxn = pyodbc.connect(
pyodbc.Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib '/opt/microsoft/msodbcsql17/lib64/libmsodbcsql-17.9.so.1.1' : file not found (0) (SQLDriverConnect)")
If we add a third LD_LIBRARY_PATH tweak to sa_config.sh
LD_LIBRARY_PATH="$SQLANY17/lib32:${LD_LIBRARY_PATH:-}"
LD_LIBRARY_PATH="$SQLANY17/lib64:${LD_LIBRARY_PATH:-}"
LD_LIBRARY_PATH="/lib/x86_64-linux-gnu/:${LD_LIBRARY_PATH:-}"
export LD_LIBRARY_PATH
and restart the machine to apply the changes then any.py (above) continues to work, but pyo.py (above, unmodified) works, too:
$ python3 pyo.py
[('success', )]
Looks like you have driver issue, run the following command and it should work
sudo apt-get install tdsodbc
update content of odbcinst.ini
$ sudo nano /etc/odbcinst.ini
[FreeTDS]
Description = TDS Driver for MSSQL
driver = path/to/libtdsodbc.so
setup = path/to/libtdsS.so
[EDIT]
Kindly share output after running this
import pyodbc
print(pyodbc.drivers())
Also kindly check if followed all steps here for target OS:
https://learn.microsoft.com/en-us/sql/connect/odbc/linux-mac/installing-the-microsoft-odbc-driver-for-sql-server?view=sql-server-2017

how to parse Http json response and fail or pass job based on that?

I have an gitlab ci yaml file. and 2 jobs. My .gitlab-ci.yaml file is:
variables:
MSBUILD_PATH: 'C:\Program Files (x86)\MSBuild\14.0\Bin\msbuild.exe'
SOLUTION_PATH: 'Source/NewProject.sln'
stages:
- build
- trigger_IT_service
build_job:
stage: build
script:
- '& "$env:MSBUILD_PATH" "$env:SOLUTION_PATH" /nologo /t:Rebuild /p:Configuration=Debug'
trigger_IT_service_job:
stage: trigger_IT_service
script:
- 'curl http://webapps.xxx.com.tr/dataBus/runTransfer/ctDigiTransfer'
And It's my trigger_IT_service job report:
Running on DIGITALIZATION...
00:00
Fetching changes with git depth set to 50...
00:05
Reinitialized existing Git repository in D:/GitLab-Runner/builds/c11pExsu/0/personalname/newproject/.git/
Checking out 24be087a as master...
Removing Output/
git-lfs/2.5.2 (GitHub; windows amd64; go 1.10.3; git 8e3c5c93)
Skipping Git submodules setup
$ curl http://webapps.xxx.com.tr/dataBus/runTransfer/ctDigiTransfer
00:02
StatusCode : 200
StatusDescription : 200
Content : {"status":200,"message":"SAP transfer started. Please
check in db","errorCode":0,"timestamp":"2020-03-25T13:53:05
.722+0300","responseObject":null}
RawContent : HTTP/1.1 200 200
Keep-Alive: timeout=10
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: application/json;charset=UTF-8
Date: Wed, 25 Mar 2020 10:53:05 GMT
Server: Apache
I have to control the this report "Content" part in gitlab ci yaml
If "message" is "SAP transfer started. Please check in db" the pipeline should pass otherwise must be failed.
Actually my question is:
how to parse Http json response and fail or pass job based on that
Thank you for all your helps.
Best way would be to install some tool to parse json and use it, different examples here
Given json example from comment:
{
"status": 200,
"message": "SAP transfer started. Please check in db",
"errorCode": 0,
"timestamp": "2020-03-25T17:06:43.430+0300",
"responseObject": null
}
If you can install python3 on your runner you could achieve it all with script:
import requests; # note this might require additional install with pip install requests
message = requests.get('http://webapps.xxx.com.tr/dataBus/runTransfer/ctDigiTransfer').json()['message']
if message != 'SAP transfer started. Please check in db':
print('Invalid message: ' + message)
exit(1)
else:
print('Message ok')
So trigger_IT_service stage in your yaml would be:
trigger_IT_service_job:
stage: trigger_IT_service
script: >
python -c "import requests; message = requests.get('http://webapps.xxx.com.tr/dataBus/runTransfer/ctDigiTransfer').json()['message']; (print('Invalid message: ' + message), exit(1)) if message != 'SAP transfer started. Please check in db' else (print('Message ok'), exit(0))"

Curl Exec lein with Puppet on Centos Vagrant no such file or directory

EDIT
This sites.pp seems to work.
class lein {
$lein = "/usr/local/bin/lein"
$url = 'https://raw.githubusercontent.com/technomancy/leiningen/stable/bin/lein'
exec { 'download lein':
command => "/usr/bin/curl -sL -o ${lein} ${url}",
creates => $lein,
}
file { $lein:
mode => '0755',
require => Exec['download lein'],
}
exec { 'install lein':
command => $lein,
require => File[$lein],
}
}
I'm trying to install lein on a Centos Vagrant using Puppet.
To install lein you just run the script. I'm trying to dowload the script with curl, make it executable and then executing it but I'm getting no such file or directory. I have verified that the file /usr/local/bin/lein exists, so not sure why I'm getting the error.
So what I'm trying to accomplish is the puppet equivalent of this shell:
curl -sL -o /usr/local/bin/lein https://raw.githubusercontent.com/technomancy/leiningen/stable/bin/lein
chmod 0755 /usr/bin/local/lein
/usr/local/bin/lein
Also on a side note, is it possible to run exec as a non-privileged user?
puppet/manifests/site.pp
class lein {
exec { 'download lein':
command => 'curl -sL -o /usr/local/bin/lein https://raw.githubusercontent.com/technomancy/leiningen/stable/bin/lein',
path => '/usr/bin',
}
file { '/usr/local/bin/lein':
mode => '0755',
require => Exec['download lein'],
}
exec { 'install lein':
command => 'lein',
path => '/usr/local/bin',
require => File['/usr/local/bin/lein'],
}
}
Vagrantfile
# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure(2) do |config|
config.vm.box = "puphpet/centos65-x64"
config.vm.provision "puppet" do |p|
p.module_path = "puppet/modules"
p.manifests_path = "puppet/manifests"
p.manifest_file = "site.pp"
end
end
Also on a side note, is it possible to run exec as a non-privileged user?
Yes, you can, add the user in your block
class lein {
$lein = "/usr/local/bin/lein"
$url = 'https://raw.githubusercontent.com/technomancy/leiningen/stable/bin/lein'
exec { 'download lein':
command => "/usr/bin/curl -sL -o ${lein} ${url}",
creates => $lein,
user => "vagrant";
}
file { $lein:
mode => '0755',
require => Exec['download lein'],
}
exec { 'install lein':
command => $lein,
require => File[$lein],
user => "vagrant";
}
}
include lein
On a much simpler note: have you tried piping?
class lein {
$command = 'curl -sL https://raw.githubusercontent.com/technomancy/leiningen/stable/bin/lein | bash -s install'
package { 'curl':
ensure => 'installed',
}
exec { 'download lein':
command => $command,
path => $::path,
require => Package['curl'],
}
}
I got it to work by removing the path to both execs and using absolute paths for the commands. Also lein needs the HOME environment set or else it will try and download files to /.lein and it needs to be ran as vagrant
Successful sites.pp
class { 'jdk_oracle': }class lein {
$lein = "/usr/local/bin/lein"
$url = 'https://raw.githubusercontent.com/technomancy/leiningen/stable/bin/lein'
$leinhome = '/home/vagrant'
exec { 'download lein':
command => "/usr/bin/curl -sL -o ${lein} ${url}",
creates => $lein,
require => Class['jdk_oracle']
}
file { $lein:
mode => '0755',
require => Exec['download lein'],
}
exec { 'install lein':
environment => ["HOME=${leinhome}"],
command => $lein,
require => File[$lein],
creates => "${leinhome}/.lein/self-installs/leiningen-2.5.3-standalone.jar",
user => 'vagrant',
}
}
include jdk_oracle
include lein

puppet file function doesn't load contents

I am trying to use the puppet file function (not the type) in the following way
class iop_users {
include 's3file::curl'
include 'stdlib'
$secretpath=file('/etc/secret','dev/null')
notify { 'show secretpath':
message =>"secretpath is $secretpath"
}
s3file { '/opt/utab.yaml':
source => "mybucket/$secretpath/utab.yaml",
ensure => 'latest',
}
exec { 'fix perms':
command => '/bin/chmod 600 /opt/utab.yaml',
require => S3file['/opt/utab.yaml']
}
if ( $::virtual == 'xenhvm' and defined(S3file['/opt/utab.yaml']) ) {
$uhash=loadyaml('/opt/utab.yaml')
create_resources(iop_users::usercreate, $uhash)
}
}
If I run this then here is some typical output. The manifest fails as the initial "secret" used to find the path is not loaded
https_proxy=https://puppet:3128 puppet agent -t
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Caching catalog for ip-10-40-1-68.eu-west-1.compute.internal
Info: Applying configuration version '1431531382'
Notice: /Stage[main]/Iop_users/S3file[/opt/utab.yaml]/Exec[fetch /opt/utab.yaml]/returns: % Total % Received % Xferd Average Speed Time Time Time Current
Notice: /Stage[main]/Iop_users/S3file[/opt/utab.yaml]/Exec[fetch /opt/utab.yaml]/returns: Dload Upload Total Spent Left Speed
Notice: /Stage[main]/Iop_users/S3file[/opt/utab.yaml]/Exec[fetch /opt/utab.yaml] 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
Notice: /Stage[main]/Iop_users/S3file[/opt/utab.yaml]/Exec[fetch /opt/utab.yaml]/returns: curl: (56) Received HTTP code 404 from proxy after CONNECT
Error: curl -L -o /opt/utab.yaml https://s3-eu-west.amazonaws.com/mybucket//utab.yaml returned 56 instead of one of [0]
Error: /Stage[main]/Iop_users/S3file[/opt/utab.yaml]/Exec[fetch /opt/utab.yaml]/returns: change from notrun to 0 failed: curl -L -o /opt/utab.yaml https://s3-eu-west.amazonaws.com/mybucket//utab.yaml returned 56 instead of one of [0]
Notice: /Stage[main]/Iop_users/Exec[fix perms]: Dependency Exec[fetch /opt/utab.yaml] has failures: true
Warning: /Stage[main]/Iop_users/Exec[fix perms]: Skipping because of failed dependencies
Notice: secretpath is
Notice: /Stage[main]/Iop_users/Notify[show secretpath]/message: defined 'message' as 'secretpath is '
Notice: Finished catalog run in 1.28 seconds
However on the same host that the above puppet agent run fails on, if I use "apply" to try it outside of the context of a manifest, it works fine
puppet apply -e '$z=file("/etc/secret") notify { "z": message => $z}'
Notice: Compiled catalog for ip-x.x.x.x.eu-west-1.compute.internal in environment production in 0.02 seconds
Notice: wombat
Notice: /Stage[main]/Main/Notify[z]/message: defined 'message' as 'wombat
'
Notice: Finished catalog run in 0.03 seconds
What am I doing wrong? Are there any better alternative approaches I could make?
As usual I was confused about the way puppet works
Apparently, functions are always executed on the master
So any files being loaded in this way must be on the master
As soon as I added a "/etc/secret" file to the puppetmaster it all worked

calabash-ios simulator stuck at 'launching took xxx seconds'

My simulator is starting up, and showing what looks like a successful launch:
Launching took 6.652245 seconds
Except the feature file isn't running ... it just sticks there. Could this be in simlauncher? Or maybe I have a config messed up?
It was running ok earlier ... then I ran a few other simulator tasks and it seems to be out of sync.
Thanks in advance !
debug info:
18:34 $ ./go.sh
Feature: getting started with ios android automation
Scenario: debug # features/getting_started.feature:3
INFO: Using uia strategy: 'preferences'
Preparation took 0.110554 seconds
Simulator instruction set 'x86_64' is compatible with ["i386", "x86_64"]
{
:app => "/Users/qa/repos/Foo-ios/test_harness/Build/Products/Debug-iphonesimulator/Foo-cal.app",
:args => [],
:bundle_dir_or_bundle_id => "/Users/qa/repos/Foo/test_harness/Build/Products/Debug-iphonesimulator/Foo-cal.app",
:bundle_id => "com.foo.bar.Foo-cal",
:device => "iphone",
:device_target => "simulator",
:launch_method => :instruments,
:launch_retries => 5,
:log_file => "/var/folders/6f/tmnbb60j62715sj_n1xt_crr0000gr/T/run_loop20150310-51884-g6i9qx/run_loop.out",
:no_launch => false,
:no_stop => false,
:reset => false,
:results_dir => "/var/folders/6f/tmnbb60j62715sj_n1xt_crr0000gr/T/run_loop20150310-51884-g6i9qx",
:results_dir_trace => "/var/folders/6f/tmnbb60j62715sj_n1xt_crr0000gr/T/run_loop20150310-51884-g6i9qx/trace",
:script => "/var/folders/6f/tmnbb60j62715sj_n1xt_crr0000gr/T/run_loop20150310-51884-g6i9qx/_run_loop.js",
:sdk_version => nil,
:udid => "iPhone 5s (8.1 Simulator)",
:uia_strategy => :preferences,
:xcode => "6.1.1",
:xcode_path => "/Applications/Xcode.app/Contents/Developer"
}
### Starting on simulator App: /Users/qa/repos/foo/test_harness/Build/Products/Debug-iphonesimulator/Foo-cal.app ###
2015-03-10 18:35:25 -0700 xcrun instruments -w iPhone 5s (8.1 Simulator) -D /var/folders/6f/tmnbb60j62715sj_n1xt_crr0000gr/T/run_loop20150310-51884-g6i9qx/trace -t Automation /Users/qa/repos/foo/test_harness/Build/Products/Debug-iphonesimulator/Foo-cal.app -e UIARESULTSPATH /var/folders/6f/tmnbb60j62715sj_n1xt_crr0000gr/T/run_loop20150310-51884-g6i9qx -e UIASCRIPT /var/folders/6f/tmnbb60j62715sj_n1xt_crr0000gr/T/run_loop20150310-51884-g6i9qx/_run_loop.js >& /var/folders/6f/tmnbb60j62715sj_n1xt_crr0000gr/T/run_loop20150310-51884-g6i9qx/run_loop.out
Launching took 6.652245 seconds