Im looking for lambda that can trigger Gitlab-ci pipeline to deploy specific branches and send results to slack.
Thx.
Trigger a pipeline
As per GitLab Trigger API manual:
To trigger a job you need to send a POST request to GitLab’s API endpoint:
curl -X POST <API url>/projects/<your_awesome_gitlab_project>/trigger/pipeline
The required parameters are the trigger’s token and the Git ref on which the trigger will be performed. Valid refs are the branch and the tag. The :id of a project can be found by querying the API or by visiting the CI/CD settings page which provides self-explanatory examples.
Watching a pipeline
To check pipeline results, use CloudWatch Events:
You can set up a rule to run an AWS Lambda function on a schedule. This tutorial shows how to use the AWS Management Console or the AWS CLI to create the rule. If you would like to use the AWS CLI but have not installed it, see the AWS Command Line Interface User Guide.
To check jobs status, use: Get a single pipeline or List project pipelines API calls.
curl --header "PRIVATE-TOKEN: " "https://gitlab.example.com/api/v4/projects/1/pipelines/46"
Inform on Slack
To send Slack notifications with lambda, use this tutorial:
Creating an AWS Lambda Function and API Endpoint | Slack
Two cents about endpoint security
CI Trigger is secured by token. In general, it's enough for securing your endpoints.
But, if the approach isn't enough, there are some techniques to "hide" endpoints:
client IP whitelisting with GitLab
AWS Security Groups for Lambda or for EC2
Securing URLs with Nginx or with HAProxy
Related
I have a template that creates a few resources in GCP, and I want it to either call an HTTP endpoint or publish a message to a topic whenever the deployment completes. I've been checking different services all day, and couldn't find anything about it.
In AWS, it is quite easy to deploy an SNS message that is published to an SNS topic, which is subscribed to SQS Queue, and that triggers a lambda function. In Azure ARM templates, we can use az CLI to invoke a web request and call an endpoint directly.
I couldn't find any similar in GCP. Is there any way of either calling an HTTP endpoint, Cloud Function or perhaps publishing a message to a topic whenever a deployment is finished?
I really appreciate any help.
The best approach in GCP is to Create a Logging Sink using a filter and the Logging query language to only send the Deployment Manager logs to a PubSub topic.
Here is an example of a sink sending deployment manager logs to a PubSub topic previously created:
gcloud logging sinks create $SINK_NAME pubsub.googleapis.com/projects/$YOUR_PROJECT/topics/$TOPIC \
--log-filter='resource.type="deployment"' --description="my sink"
Be careful to Set the destination permissions or will not see the logs in the PubSub topic.
Once you are getting the logs in the PubSub topic, you can configure Cloud Pub/Sub Triggers to fire up an HTTP call based on content of the log.
For the AWS marketplace integration, we have checked the sample code of Serverless integration for SaaS products. The samples provided, it is based on AWS Lambda functions for registering new subscribers, subscribing to SQS etc.
Can we implement all these functionalities with our database, functions etc, instead of using AWS lambda and dynamo DB?
Also in the examples, some lambda functions are there like stream handler, entitlement-SQS, subscription-SQS-handler, grant-revoke-access-to-product etc. How can we trigger if we use our APIS?
Yes, the AWS Marketplace SaaS integration documentation states:
When a customer subscribes to your product, they are redirected to your registration URL which is an HTTP POST request with a temporary x-amzn-marketplace-token token.
All you need is an endpoint that can receive that POST in order to integrate. If you want to trigger any additional AWS API calls, your endpoint could make use of an AWS SDK. There are also requirements surrounding what your POST endpoint must do in order to be approved by the AWS Marketplace team. I suggest reviewing the documentation above or this AWS Marketplace
SaaS Listing Process & Integration Guide.
Suppose multiple aws services are running (like EC2,S3) and If someone changes the configuration of these services , I want to know immediately and want to pull the updated configuration. So how can i listen to these services to pull configuration immediately . I went through couple of options like aws SNS and all. But i don't want to implement this using any aws services for this.
I am using spring boot application. Any help is appreciated , Thanks in advance.
The two services you should be interested in are:
AWS CloudTrail:
AWS CloudTrail is an AWS service that helps you enable governance, compliance, and operational and risk auditing of your AWS account. Actions taken by a user, role, or an AWS service are recorded as events in CloudTrail. Events include actions taken in the AWS Management Console, AWS Command Line Interface, and AWS SDKs and APIs.
AWS Config:
AWS Config provides a detailed view of the configuration of AWS resources in your AWS account. This includes how the resources are related to one another and how they were configured in the past so that you can see how the configurations and relationships change over time.
You can also configure Amazon CloudWatch Events rules that trigger when a particular API call is made (eg when a user changes some configuration). This could trigger a notification, a Lambda function, etc.
See:
Creating a CloudWatch Events Rule That Triggers on an AWS API Call Using AWS CloudTrail - Amazon CloudWatch Events
How to monitor AWS account activity with Cloudtrail, Cloudwatch Events and Serverless
I'm working with an on-premise older version of BitBucket Server v5.15.1 that does not have the Bitbucket Pipelines feature and I need how to get the webhooks to notify AWS Lambda via HTTPS POST via AWS API Gateway after a commit is made to master branch...then Lambda downloads a copy of the repo, zips it up and places it into an S3 bucket...and of course this is where CodePipeline can finally be triggered...But I'm having issues getting this on-premise BitBucket Server located within my AWS account to connect its webhook to Lambda.
I tried following this documentation below and launched the CloudFormation template with all the needed resources but I'm assuming it is for BitBucket Cloud not Bitbucket Server OP.
https://aws.amazon.com/blogs/devops/integrating-git-with-aws-codepipeline/
Anyones help with this would be really appreciated.
I suppose you are following this below blog from AWS :
https://aws.amazon.com/blogs/devops/integrating-codepipeline-with-on-premises-bitbucket-server/
We had also implemented it. If the event is coming to Lambda, then make sure your Lambda is within a VPC and it has correct outbound(read as inbound) rules to connect the Bitbucket server over HTTPS. Also the Bitbucket server accepts the VPC IP range.
I am interested in building bot using AWS Lex but I don't want to use the AWS Lambda for interacting With DB for fetching results,
For Example. If we Ask, "Can you show me the sales for the last month" I want the bot to respond with an Answer " Sales for the last month $1.2 Million"; the Simplest way to achieve this to write an AWS Lambda function to get the details, but can we use an API Endpoint of a web app hosted on Ec2 Instance or AWS ELB
Any thoughts on this?
Surya
Unfortunately no, you cannot use any form of integration for a Lex bot without going through Lambda. You can build Lex bots without Lambda, but they are only able to give static responses and can't call outside of the Lex service.
You can still use your own API endpoint by going via Lambda. Remember that if the resource your calling is in an AWS VPC but not publicly available, you'll need to add extra config for the Lambda to access it. Example of config required for Lambda to call a private AWS endpoint can be see here: AWS: Lambda function cannot call rest api using private API of EC2 instance.
From the Lex FAQs:
Q. How is an action fulfilled?
Amazon Lex integrates with AWS Lambda for ‘fulfillment’ of the action
or business logic. Alternately, you can configure Amazon Lex to return
parsed intent and slot values to the client for action fulfillment.