I have need to run a custom file parser to parse incoming files live for further upstream use. The parser is an executable and takes some arguments. I have written the following lambda test function:
import subprocess, sys
import os
def lambda_handler(event, context):
path = os.environ['LAMBDA_TASK_ROOT']
subprocess.call([path+'/*exefile*',*args*])
However I get an error, and this is even without any input, meaning it's not related to the writing of new files in the lambda environment:
PermissionError errno13 Permission denied
It seems to me that it's simply not allowed to run the executable. For the IAM role I have given it the full lambda access. The custom exe file is not compiled for linux, but wouldn't I get another error if that was the issue?
Help appreciated!
Per this AWS blog post it needs to compiled for Amazon's Linux:
If you compile your own binaries, ensure that they’re either
statically linked or built for the matching version of Amazon Linux.
The current version of Amazon Linux in use within AWS Lambda can
always be found on the Supported Versions page of the Lambda docs.
Related
It's the first time I use Google Cloud Platform, so please be understanding!
I've built a scheduled workflow that simply runs a Batch job. The job runs Python code and uses the standard logging library for logging. When the job is executed, I can correctly see all the entries in Cloud Logging, but all the entries have severity ERROR although they're all INFO.
One possible reason I've been thinking about is that I haven't used the setup_logging function as described in the documentation here. The thing is, I didn't want to run the Cloud Logging setup when I run the code locally.
The questions I have are:
why does logging "work" (in the sense that logs end up in Cloud Logging) even if I did not use the setup_logging function? What is it's real role?
why do my INFO entries show up with ERROR severity?
if I include that snippet and that snippet solves this issue, should I include an if statement in my code that detects if I am running the code locally and skips that Cloud Logging setup step?
According to the documentation, you have to use a setup to send correctly logs to Cloud Logging.
This setup allows then to use the Python logging standard library.
Once installed, this library includes logging handlers to connect
Python's standard logging module to Logging, as well as an API client
library to access Cloud Logging manually.
# Imports the Cloud Logging client library
import google.cloud.logging
# Instantiates a client
client = google.cloud.logging.Client()
# Retrieves a Cloud Logging handler based on the environment
# you're running in and integrates the handler with the
# Python logging module. By default this captures all logs
# at INFO level and higher
client.setup_logging()
Then you can use the Python standard library to add logs to Cloud Logging.
# Imports Python standard library logging
import logging
# The data to log
text = "Hello, world!"
# Emits the data using the standard logging module
logging.warning(text)
why does logging "work" (in the sense that logs end up in Cloud Logging) even if I did not use the setup_logging function? What is
it's real role?
Without the setup, the log will be added to Cloud Logging but not with the correct type and as expected. It's better to use the setup.
why do my INFO entries show up with ERROR severity?
The same reason explained above
if I include that snippet and that snippet solves this issue, should I include an if statement in my code that detects if I am running the
code locally and skips that Cloud Logging setup step?
I think no need to add a if statement you run the code locally. In this case, the logs should be printed in the console even if the setup is present.
I have been trying to look at the code that is deployed in an aws lambda.
There is an existing go function that is running in the go lambda.
However, I am not able to. AWS docs says we can look at the code through the visual config view, where is this view? This is the screen that I see, where is the view to see the code?
Please help.
Or is it because we are using a go server, only the executable which is a binary is running in the lambda and hence we are not able to see the code?
Code inline is supported only for interpreted languages (js for example) and not compiled languages.
Beside below lamda limitation, It seems console lamda editor does not support go.
However the documentation suggest to use code star.
You can also get started with AWS Lambda Go support through AWS
CodeStar. AWS CodeStar lets you quickly launch development projects
that include a sample application, source control and release
automation. With this announcement, AWS CodeStar introduced new
project templates for Go running on AWS Lambda. Select one of the
CodeStar Go project templates to get started. CodeStar makes it easy
to begin editing your Go project code in AWS Cloud9, an online IDE,
with just a few clicks.
announcing-go-support-for-aws-lambda
Q: How do I create an AWS Lambda function using the Lambda console?
If you are using Node.js or Python, you can author the code for your
function using code editor in the AWS Lambda console which lets you
author and test your functions, and view the results of function
executions in a robust, IDE-like environment
lambda-faqs
Deployment package size
50 MB (zipped, for direct upload)
250 MB (unzipped, including layers)
3 MB (console editor)
lambda limits
lambda-go-how-to-create-deployment-package
Based on the code size the AWS code editor will display the code. Since the size of the code/package is large AWS Code editor can't display the same.
But you can download the package from AWS Lambda function using export.
Kindly follow the below steps:
Go to the lambda
Select the lambda
Click on Actions
Select Export function
You will get few options. Select Download deployment package.
I am working on an AWS Lambda where I need to use AWS Textract, for this, I have used AWS-SDK, with AWS-SDK I was able to import s3, but AWS Textract is not working, when I deploy and test it shows,
Unable to import module 'src/functions/routes/handler': Error
I think it has something to do with packaging, like may be It is not packaging the relevant file, but not sure and I don't know how to make it package if it is the problem, any suggestions are appreciated.
Since you have not given any code to go on, I will consider the possibility that you have properly written your import code and this error is a valid response to an issue. In that case, you should note that there are issues with importing Textract in Lambda via the aws-sdk. See this reported issue:
https://github.com/aws/aws-sdk-js/issues/2728
For a thorough and thoughtful example of using Textract with a Lambda service, consider this excellent article posted with complete details:
https://medium.com/#sumindaniro/aws-textract-with-lambda-walkthrough-ed4473aedd9d
If I try to use boto3 Lambda create_function() to create a Lambda function, and I try to include Layers via Layers=['string'] parameter, I get the following error message:
Unknown parameter in input: "Layers", must be one of: FunctionName, Runtime, Role, Handler, Code, Description, Timeout, MemorySize, Publish, VpcConfig, DeadLetterConfig, Environment, KMSKeyArn, TracingConfig, Tags
... any ideas? The documentation suggests that this should work, but something is clearly off here. NOTE: I also have a similar problem with "Layers" in update_function_configuration() as well.
My guess is that the version of boto3 that the AWS Lambda console uses has not been updated/refreshed yet to support Layers. Because when I run the same code locally on a machine with a fairly recent version of boto3, it runs without any problems. I have already tried using both listed Python runtimes of 3.6 and 3.7 that in the AWS console, but neither worked. These runtimes have respective versions of boto3 of 1.7.74 and 1.9.42. But my local machine has 1.9.59. So perhaps the addition of Lambda Layers occurred between 1.9.42 and 1.9.59.
My guess is that the version of boto3 that the AWS Lambda console uses has not been updated/refreshed yet to support Layers.
That's completely right. AWS usually updates the available libraries on AWS Lambda regularly, but hasn't updated them for several months now for unknown reasons.
The supported API endpoints are actually not defined in boto3, but in botocore.
Currently botocore 1.10.74 is available on AWS Lambda, while support for AWS Lambda Layers got added in botocore 1.12.56.
To avoid such incompatibilities between your code and the versions of available libraries, you should create a deployment package containing boto3 and botocore in addition to your AWS Lambda function code, so your code uses your bundled versions instead the ones AWS provides. That's what AWS suggests as part of their best practices as well:
Control the dependencies in your function's deployment package.
The AWS Lambda execution environment contains a number of libraries such as the AWS SDK for the Node.js and Python runtimes (a full list can be found here: Lambda Execution Environment and Available Libraries). To enable the latest set of features and security updates, Lambda will periodically update these libraries. These updates may introduce subtle changes to the behavior of your Lambda function. To have full control of the dependencies your function uses, we recommend packaging all your dependencies with your deployment package.
when using cost explorer client in lambda function , 'ce' is not supported. but it may available in new version of botocore. so is it possible to import the new version of botocore in lambda python function ? if yes, how we can implement it. please provide some example
AWS Lambda provides a way to to use a version of Boto3 other than the one included by default, you can include it in your deployment package.
From Creating a Deployment Package (Python)
AWS Lambda includes the AWS SDK for Python (Boto 3), so you don't need
to include it in your deployment package. However, if you want to use
a version of Boto3 other than the one included by default, you can
include it in your deployment package.