Lambda function package still large despite using a layer for dependencies - amazon-web-services

Question

I have a Python Lambda and since I started using AWS X-Ray the package size has ballooned from 445KB to 9.5MB.
To address this and speed up deployments of my code, I have packaged my requirements separately and added a layer to my template. The documentation suggests that this approach should work.
Packaging dependencies in a layer reduces the size of the deployment package that you upload when you modify your code.
pip install --target ../package/python -r requirements.txt
Resources:
...
ProxyFunction:
Type: AWS::Serverless::Function
Properties:
Architectures:
- x86_64
CodeUri: proxy/
Handler: app.lambda_handler
Layers:
- !Ref ProxyFunctionLibraries
Role: !GetAtt ProxyFunctionRole.Arn
Runtime: python3.8
Tracing: Active
ProxyFunctionLibraries:
Type: AWS::Serverless::LayerVersion
Properties:
LayerName: proxy-function-lib
Description: Dependencies for the ProxyFunction.
ContentUri: package/.
CompatibleRuntimes:
- python3.8
However, this doesn't seem to have prevented the Lambda from still packaging everything in the top layer, and every time I deploy the package is still 9.5MB. The new layer for some reason is 11MB in size, but that is only being deployed when a change is made.
How can I reduce the size of the Lambda function package?

Answer 1

Actually the solution here was quite simple, although not obvious to non-Lambda experts.
As described in the question, the first step was to build the package library.
pip install --target ../package/python -r requirements.txt
However, when building the Lambda using sam build -u the same 'requirements.txt' file is used and the required dependencies were again being installed, this time as part of the app.
So all I had to do was remove the requirements that I wish packaged in a separate layer and rebuild. It does mean that I have to maintain 2x 'requirements.txt' but that is entirely manageable.
I've opened an issue and hopefully AWS will update their documentation.

Answer 2

I am struggling with the same problem and solved it differently. Leaving this answer as a note to future question-seekers.
My current lambda structure is the following:
├── events
│ └── event.json
├── ingress
│ ├── app.py # lambda code
│ └── __init__.py
├── __init__.py
├── lib_layer # contains self written helpers and python dependencies
│ ├── helper.py
│ └── requirements.txt
├── samconfig.toml
├── template.yaml
└── tests
├── ...
In my template.yaml I have the following code snippets:
SQSIngestion:
Type: AWS::Serverless::Function
Properties:
CodeUri: ingress/
Handler: app.lambda_handler
Runtime: python3.9
Layers:
- !Ref PythonLibLayer
Architectures:
- x86_64
PythonLibLayer:
Type: AWS::Serverless::LayerVersion
Properties:
ContentUri: lib_layer
CompatibleRuntimes:
- python3.9
Metadata:
BuildMethod: python3.9
Since the Lambda Function already has the layer containing all dependencies + my helpers attached, the requirements.txt in /ingress can be omitted but works nevertheless when invoking the function in AWS. By calling
sam build will automatically build the dependency layer and ignore the function folder for building the dependencies.
As a tip, if working with VSCode, add in your settings.json the following line to fix pylances import error for self written packages.
{
"python.analysis.extraPaths": ["lib_layer"]
}