Connect AWS CodeDeploy to Github without using my Github user? - amazon-web-services

AWS documentation describes how you authenticate to Github using your browser, and that you're currently logged into Github as a valid user with permission to the repository you want to deploy from:
http://docs.aws.amazon.com/codedeploy/latest/userguide/github-integ.html#github-integ-behaviors-auth
Is there any way to setup CodeDeploy without linking my user and having a browser? I'd love to do this using webhooks on each repository and AWS API calls, but I'll make a Github 'service user' if I have to.
More examples:
http://blogs.aws.amazon.com/application-management/post/Tx33XKAKURCCW83/Automatically-Deploy-from-GitHub-Using-AWS-CodeDeploy
I'd love to use webhooks on my repo, or set them up myself, than permit AWS access to every repository on my Github account.

There does not appear to be an alternative to doing the OAuth flow in your browser at this point. If you're concerned about opening your whole Github account up to Amazon, creating a service user is probably the best approach, unfortunately it seems this user still needs administrative access to your repos to set up the integration.

After more research I realized my first answer is wrong, you can use AWS CLI to create a CodePipeline using a Github OAuth token. Then you can plug in your CodeDeploy deployment from there. Here's an example configuration:
{
"pipeline": {
"roleArn": "arn:aws:iam::99999999:role/AWS-CodePipeline-Service",
"stages": [
{
"name": "Source",
"actions": [
{
"inputArtifacts": [],
"name": "Source",
"actionTypeId": {
"category": "Source",
"owner": "ThirdParty",
"version": "1",
"provider": "GitHub"
},
"outputArtifacts": [
{
"name": "MyApp"
}
],
"configuration": {
"Owner": "myusername",
"Repo": "myrepo",
"Branch": "master",
"OAuthToken": "**************"
},
"runOrder": 1
}
]
},
{
"name": "Beta",
"actions": [
{
"inputArtifacts": [
{
"name": "MyApp"
}
],
"name": "CodePipelineDemoFleet",
"actionTypeId": {
"category": "Deploy",
"owner": "AWS",
"version": "1",
"provider": "CodeDeploy"
},
"outputArtifacts": [],
"configuration": {
"ApplicationName": "CodePipelineDemoApplication",
"DeploymentGroupName": "CodePipelineDemoFleet"
},
"runOrder": 1
}
]
}
],
"artifactStore": {
"type": "S3",
"location": "codepipeline-us-east-1-99999999"
},
"name": "MySecondPipeline",
"version": 1
}
}
You can create the pipeline using the command:
aws codepipeline create-pipeline --cli-input-json file://input.json
Make sure that the Github OAuth token has permissions admin:repo_hook and repo.
Reference: http://docs.aws.amazon.com/cli/latest/reference/codepipeline/create-pipeline.html

CodeDeploy and Github integration works based on Github Oauth. So to use the CodeDeploy and Github integration, you will have to trust CodeDeploy github application using your github account. Currently this integration will only work in your browser with a valid github account cause CodeDeploy application will always redirect back to CodeDeploy console to verify&finish the OAuth authentication process.

You can do it using this bash command
FROM LOCAL TO REMOTE
rsync --delete -azvv -e "ssh -i /path/to/pem" /path/to/local/code/* ubuntu#66.66.66.66:/path/to/remote/code
FROM REMOTE TO LOCAL
rsync --delete -azvv -e "ssh -i /path/to/pem" ubuntu#66.66.66.66:/path/to/remote/code/* /path/to/local/code
rsync checks file versions and updates the files that need to be update

Related

Cloudformation template properties documentation discrepancy

I'm creating my first Cloudformation template using an archived Github project from an AWS Blog:
https://aws.amazon.com/blogs/devops/part-1-develop-deploy-and-manage-for-scale-with-elastic-beanstalk-and-cloudformation-series/
https://github.com/amazon-archives/amediamanager
The template amm-elasticbeanstalk.cfn.json declares an Elastic Beanstalk resource, outlined here:
"Resources": {
"Application": {
"Type": "AWS::ElasticBeanstalk::Application",
"Properties": {
"ConfigurationTemplates": [{...}],
"ApplicationVersions": [{...}]
}
}
}
From the documentation I'm under the impression that AWS::ElasticBeanstalk::ApplicationVersion and AWS::ElasticBeanstalk::ConfigurationTemplate must be defined as separate resources, yet the example I'm working from is using the same AWSTemplateFormatVersion as the documentation. Is this a "shorthand" where namespaces can be nested if they have the same parent (i.e. AWS::ElasticBeanstalk)? Is it documented somewhere?
In the same file AWS::ElasticBeanstalk::Environment is defined as a separate resource - is this just a stylistic choice, perhaps because the environment configuration is so long?
Elastic Beanstalk consists of Applications and Environments components. Basically each environment runs only one application version at a time, however, you can run the same application version in many environments at the same time. Application versions and Saved configurations are part of the Application resource that's why it's possible to define it within the AWS::ElasticBeanstalk::Application resource properties. Environment however is a separate logical component of Elastic Beanstalk so it's impossible to declare it from within the Application resource.
For better readability I would suggest declaring all the resources separately as per this example. Also when using this approach you can directly reference the TemplateName and VersionLabel in the AWS::ElasticBeanstalk::Environment resource.
Alternatively if you want to stick to the github example you can adjust the above example to look like this:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Resources": {
"sampleApplication": {
"Type": "AWS::ElasticBeanstalk::Application",
"Properties": {
"Description": "AWS Elastic Beanstalk Sample Application",
"ApplicationVersions": [{
"VersionLabel": "Initial Version",
"Description": "Initial Version",
"SourceBundle": {
"S3Bucket": {
"Fn::Sub": "elasticbeanstalk-samples-${AWS::Region}"
},
"S3Key": "php-newsample-app.zip"
}
}],
"ConfigurationTemplates": [{
"TemplateName": "DefaultConfiguration",
"Description": "AWS ElasticBeanstalk Sample Configuration Template",
"OptionSettings": [
{
"Namespace": "aws:autoscaling:asg",
"OptionName": "MinSize",
"Value": "2"
},
{
"Namespace": "aws:autoscaling:asg",
"OptionName": "MaxSize",
"Value": "6"
},
{
"Namespace": "aws:elasticbeanstalk:environment",
"OptionName": "EnvironmentType",
"Value": "LoadBalanced"
},
{
"Namespace": "aws:autoscaling:launchconfiguration",
"OptionName": "IamInstanceProfile",
"Value": {
"Ref": "MyInstanceProfile"
}
}
],
"SolutionStackName": "64bit Amazon Linux 2018.03 v2.9.11 running PHP 5.5"
}]
}
},
"sampleEnvironment": {
"Type": "AWS::ElasticBeanstalk::Environment",
"Properties": {
"ApplicationName": {
"Ref": "sampleApplication"
},
"Description": "AWS ElasticBeanstalk Sample Environment",
"TemplateName": "DefaultConfiguration",
"VersionLabel": "Initial Version"
}
},
"MyInstanceRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"ec2.amazonaws.com"
]
},
"Action": [
"sts:AssumeRole"
]
}
]
},
"Description": "Beanstalk EC2 role",
"ManagedPolicyArns": [
"arn:aws:iam::aws:policy/AWSElasticBeanstalkWebTier",
"arn:aws:iam::aws:policy/AWSElasticBeanstalkMulticontainerDocker",
"arn:aws:iam::aws:policy/AWSElasticBeanstalkWorkerTier"
]
}
},
"MyInstanceProfile": {
"Type": "AWS::IAM::InstanceProfile",
"Properties": {
"Roles": [
{
"Ref": "MyInstanceRole"
}
]
}
}
}
}
Just want to point out that AWS silently phased out the option of having the ApplicationVerions key under an AWS::ElasticBeanstalk::Application's Properties. It was still working in July 2022 but I noticed it stopped some time in August 2022, giving the error in the CloudFormation stack's Event tab:
Properties validation failed for resource TheEBAppResName with message: #: extraneous key [ApplicationVersions] is not permitted
where TheEBAppResName is the name of your AWS::ElasticBeanstalk::Application resource.
The only solution now is to follow the current AWS example and use a separate AWS::ElasticBeanstalk::ApplicationVersion resource.
Interestingly, I can't seem to find any documentation on the obsolete ApplicationVerions property anymore and the AWS blog that you linked to is no longer available, but I did find it cached on the Wayback machine. Even the earliest AWS doc on GitHub for AWS::ElasticBeanstalk::Application doesn't mention the ApplicationVerions property. Seems like AWS silently deprecated it sometime between when the blog was posted in April 2014 and that earliest GitHub doc page in December 2017, but didn't actually remove the option until last month, August 2022.

Using ARM template to deploy WebJob

I am currently attempting to deploy a webjob as part of my app service through an ARM template. Does anyone have a sample of doing this through a zip deploy?
I have seen here and the linked question there about creating an arm template for a scheduled job but it was not completely clear on how to do it for continuous webjobs/how one would utilize zip deploy to deploy their files.
I currently have a section like this
{
"comments": "Defines an Azure Web App.",
"type": "Microsoft.Web/sites",
"name": "[parameters('siteName')]",
"apiVersion": "2015-08-01",
"location": "[variables('siteLocation')]",
"kind": "appservice",
"dependsOn": [
"[resourceId('Microsoft.Web/serverfarms', parameters('appServicePlanName'))]",
"ConfigJson",
"MdsConfigXml",
],
"tags": {
"[concat('hidden-related:', resourceGroup().id, '/providers/Microsoft.Web/serverfarms/', parameters('appServicePlanName'))]": "empty"
},
"properties": {
"sourcePlatform": "Unknown",
"targetPlatform": "Unknown",
"serverFarmId": "[concat('/subscriptions/', parameters('subscriptionId'),'/resourcegroups/', parameters('appServicePlanResourceGroup'), '/providers/Microsoft.Web/serverfarms/', parameters('appServicePlanName'))]",
"siteConfig": {
"phpVersion": "off",
"scmType": null,
"netFrameworkVersion": "v4.0",
"use32BitWorkerProcess": false,
"webSocketsEnabled": false,
"alwaysOn": "[parameters('alwaysOn')]",
"httpLoggingEnabled": true,
"logsDirectorySizeLimit": 50,
"appSettings": [
{
"name": "APPINSIGHTS_PROFILERFEATURE_VERSION",
"value": "1.0.0",
"slotSetting": true
}
]
}
},
"resources": [
{
"apiVersion": "2018-02-01",
"dependsOn": [
"[resourceId('Microsoft.Web/Sites', parameters('siteName'))]"
],
"name": "ZipDeploy",
"properties": {
"packageUri": "[parameters('deployPackageUrl')]"
},
"type": "extensions"
}
]
}
With the deployPackageURL linked to my zip file, which contains my webjob project. However when checking the app service resource, no webjob has been deployed. Can anyone point out possible errors in my ARM template? Thanks!
We can not deploy webjobs using ARM template.
If you just want to deploy Web App with a Scheduler Job, here is a sample for Scheduler Job Collection.
Here is the ways you could create WebJobs inside Web App.
Using the Azure portal.
Copying files directly in their proper location.
Deploying .NET Console WebJobs alongside an ASP.NET application
Deploying a WebJob using PowerShell ARM Cmdlets

AWS Lambda API Application .Net Core Not responding

I created an AWS Lambda Application API using AWS Toolkit for .Net Core 3.1. It has 2 Get request that expecting text JSON in a request body and returning text JSON as an output. It does not require any database connection or any other AWS resources. Locally everything works fine, all tests are passing. I publish my app to AWS account using AWS Toolkit which runs Cloud Formation setting file, again no problems, all passing. This creates my AWS Lambda API app with my API endpoint. However, when I try to use I am getting "403 Forbidden" errors:
Other thing I notice is that the default API Gateway type is Edge, I am unsure if that's making a problem. I would like to set it up to Private in cloud formation stuck from .Net Core level. I assume it is something to be change here:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Transform": "AWS::Serverless-2016-10-31",
"Description": "An AWS Serverless Application that uses the ASP.NET Core framework running in Amazon Lambda.",
"Resources": {
"AspNetCoreFunction": {
"Type": "AWS::Serverless::Function",
"Properties": {
"Handler": "AES.Protocol::AES.Protocol.LambdaEntryPoint::FunctionHandlerAsync",
"Runtime": "dotnetcore3.1",
"CodeUri": "",
"MemorySize": 256,
"Timeout": 30,
"Role": null,
"Policies": [
"AWSLambdaFullAccess"
],
"Events": {
"ProxyResource": {
"Type": "Api",
"Properties": {
"Path": "/{proxy+}",
"Method": "ANY"
}
},
"RootResource": {
"Type": "Api",
"Properties": {
"Path": "/",
"Method": "ANY"
}
}
}
}
}
},
"Outputs": {
"ApiURL": {
"Description": "API endpoint URL for Prod environment",
"Value": {
"Fn::Sub": "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/"
}
}
}
}
The previous question related to this API might be helpful.
I manage to find a solution. It seems like CloudFront is not supporting GET request with "body". So changing "GET" to "POST" request fix the problem.

Use S3 target in cloudformation for a codepipeline deploy

In CodePipeline in the AWS console its possible to specify a S3 deploy step https://docs.aws.amazon.com/codepipeline/latest/userguide/tutorials-s3deploy.html I'd like to be able to do this exactly, but in cloudformation. I'm missing something obvious here. Any help appreciated.
I was able to get a source and build step in cloudformation for the pipeline, but not the deploy step. The provider for that step I would think would be s3, but I can't seem to get it to work.
Here is a sample deploy step for S3
{
"Name": "Deploy",
"Actions": [
{
"Name":"Push-Lambda-Artifacts",
"ActionTypeId": {
"Category": "Deploy",
"Owner": "AWS",
"Provider": "S3",
"Version": "1"
},
"InputArtifacts": [
{
"Name": "lambda"
}
],
"Configuration":{
"BucketName": {
"Ref": "BucketName"
},
"Extract": true
},
"RunOrder": 1
}
]
}
I think that will get you most of the way there. InputArtifacts is from an output from my codebuild step.

What's the API to read database config details in elastic beanstalk environment?

AWS command line provides API to read elastic beanstalk environment information:
aws elasticbeanstalk describe-environments --environment-ids 'e-438kcmnae5'
However, I didn't find documentation to read the detailed database configuration (as shown in the browser console below) from an API.
What you are looking for is the DescribeEnvironmentResources Beanstalk API:
aws elasticbeanstalk describe-environment-resources --environment-name <environment-name>
Result:
{
"EnvironmentResources": {
...
"Resources": [
...
{
"Type": "AWS::RDS::DBInstance",
"LogicalResourceId": "...",
"PhysicalResourceId": "...",
"Properties": [
{
"Name": "Endpoint",
"Value": "..."
},
{
"Name": "Port",
"Value": "3306"
}
}
]
},
...
}