We need to get the service name under which a fargate task runs so we can perform some per service configuration (we have one service per customer, and use the service name to identify them).
By knowing the service discovery namespace for our cluster and the Task IP address, we are able to do find out the service by doing the following.
Get the task ip address, eaither by calling http://169.254.170.2/v2/metadata endpoint or by using the ECS_ENABLE_CONTAINER_METADATA method in my follow-up answer.
With the cluster namespace we call AWS.ServiceDiscovery.listNamespaces
From there we extract the nameSpace id.
We pass that to AWS.ServiceDiscovery.listServices
We pass the id of each service to AWS.ServiceDiscovery.listInstances
We flat map the results of that and look for an instance that matches our IP address above.
VoilĂ ! that record gives us the service name.
Works fine, it just seems like a super circuitous path! I'm just wondering whether there is some shorter way to get this information.
Here's a working C# example in two steps. It gets the taskARN from the metadata to retrieve the task description, and then reads its Group property, which contains the name of the service. It uses AWSSDK.ECS to get the task description and Newtonsoft.Json to parse the JSON.
private static string getServiceName()
{
// secret keys, should be encoded in license configuration object
var ecsClient = new AmazonECSClient( ACCESS_KEY, SECRET_KEY );
var request = new DescribeTasksRequest();
// need cluster here if not default
request.Cluster = NAME_OF_CLUSTER;
request.Tasks.Add( getTaskArn() );
var asyncResponse = ecsClient.DescribeTasksAsync( request );
// probably need this synchronously for application to proceed
asyncResponse.Wait();
var response = asyncResponse.Result;
string group = response.Tasks.Single().Group;
// group returned in the form "service:[NAME_OF_SERVICE]"
return group.Remove( 0, 8 );
}
private static string getTaskArn()
{
// special URL for fetching internal Amazon information for ECS instances
string url = #"http://169.254.170.2/v2/metadata";
string metadata = getWebRequest( url );
// use JObject to read the JSON return
return JObject.Parse( metadata )[ "TaskARN" ].ToString();
}
private static string getWebRequest( string url )
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create( url );
request.AutomaticDecompression = DecompressionMethods.GZip;
using HttpWebResponse response = (HttpWebResponse)request.GetResponse();
using Stream stream = response.GetResponseStream();
using StreamReader reader = new StreamReader( stream );
return reader.ReadToEnd();
}
You can get the service name from startedBy property of the task definition. Using boto sdk you can call a describe_tasks (or its equivalent in aws-cli: aws ecs describe-tasks) which will provide a
'startedBy': 'string'
The tag specified when a task is started. If the task is started by an Amazon ECS service, then the startedBy parameter contains the deployment ID of the service that starts it.
From:
boto3 ecs client
aws-cli ecs client
Hope it helps.
The answer above requires reading the container metadata that appears if you set the ECS_ENABLE_CONTAINER_METADATA environment variable in the task. The work flow is then:
Read the container metadata file ecs-container-metadata.json to get the taskArn
Call the aws.ecs.describe-tasks function to get the startedBy property
Call aws.servicediscover.get-service.
Two steps instead of three, if you don't count reading the metadata file. Better, to be sure, but I'm probably not going to change the way we do it for now.
Related
I want to setup a simple Amazon Connect call-flow that dials back any customer who leaves behind a phone number on my website. I am a rank beginner at Amazon Connect and cannot find any example code showing how to setup outbound calling to dynamically supplied phone numbers via a web-client.
Can someone point me to any example code. I have seen documentation of AWS Connect APIs including those for StartOutboundCall etc but am looking for some example code if possible.
https://blogs.perficient.com/2018/11/19/ac-outbound-api/ has a good example, that I've followed successfully.
The call is:
let params = {
"InstanceId" : '12345l-abcd-1234-abcde-123456789bcde',
"ContactFlowId" : '987654-lkjhgf-9875-abcde-poiuyt0987645',
"SourcePhoneNumber" : '+1xxxxxxxxx',
"DestinationPhoneNumber" : customerPhoneNumber,
"Attributes" : {
'name' : customerName,
'dayOfWeek' : dayOfWeek
}
}
let connect = new AWS.Connect();
connect.startOutboundVoiceContact(params, function (error, response) { ... });
Given a contact flow (of type "Contact Flow"), with arn: arn:aws:connect:us-east-1:xxxxxxxx:instance/12345l-abcd-1234-abcde-123456789bcde/contact-flow/987654-lkjhgf-9875-abcde-poiuyt0987645
The SourcePhoneNumber is required and must be one of those in your Amazon Connect. Or use a queue number if you have any defined.
The Attributes property will be passed as-is and will be available in text-to-speech, in your Contact Flow, with a form similar to $.Attributes.dayOfWeek.
The Contact Flow can be as simple as one Start, connected to one 'Play Prompt', connected to 'Disconnect/Hang up'.
All props go to https://blogs.perficient.com/author/dhodanic/
I am developing an ASP.NET Core Web API project. In my project, I am using Hangfire to run the background task. Therefore, I am configuring the Hangfire to use the database like this.
public void ConfigureServices(IServiceCollection services)
{
services.AddHangfire(configuration =>
{
configuration.UseSqlServerStorage("Server=(LocalDB)\\MSSQLLocalDB;Integrated Security=true;");
});
//
}
In the above code, I am using Local DB. Now, I am trying to use AWS RDS database since I am deploying my application on the AWS Elastic Beanstalks. I created a function for getting the connection
public static string GetRDSConnectionString()
{
string dbname = "ebdb";
if(string.IsNullOrEmpty(dbname)) return null;
string username = "admin";
string password = "password";
string hostname = "cxcxcxcx.xcxcxcxc.eu-west-2.rds.amazonaws.com:1234";
string port = "1234";
return "Data Source=" + hostname + ";Initial Catalog=" + dbname + ";User ID=" + username + ";Password=" + password + ";";
}
I got the above code from the official AWS documentation. In the above code, what I am not clear is the database name, is the database name always be "ebdb"? I tried to find out the database name. But could not. In the tutorial, it is saying to use ebdb. So, I used it.
Then in configuration, I changed to this.
configuration.UseSqlServerStorage(AppConfig.GetRDSConnectionString());
When I run the code, it is giving me this error.
Win32Exception: The parameter is incorrect
Unknown location
SqlException: A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: SQL Network Interfaces, error: 25 - Connection string is not valid)
System.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, object providerInfo, bool redirectedUserInstance, SqlConnectionString userConnectionOptions, SessionData reconnectSessionData, bool applyTransientFaultHandling)
Win32Exception: The parameter is incorrect
Basically, it cannot connect to the database when I run my application. But I set the correct credentials. the only thing I double is the database name (ebdb). What is wrong with my configuration and what is wrong? How can I fix it?
Calling a few things out here just incase...
You have your port specified both in your host variable and as a separate port variable...but never use port.
Can you confirm that you are able to access your SQLServer via another means, such as from SQL Management Studio?
RDS uses SSL by default now for connections, my .NET is rusty but would you need to inform the connection string that it needs to run over a secure protocol?
& finally, regarding the AWS Security Group on your RDS instance. Have you opened up the correct port to your machine/network/IP?
This is the screenshot of the RDS db instance security group section in the console.
I am using IdentityServer4 with .NET Core 2.0 on AWS's ElasticBeanstalk. I have a certificate for signing tokens. What's the best way to store this certificate and retrieve it from the application? Should I just stick it with the application files? Throw it in an environment variable somehow?
Edit: just to be clear, this is a token signing certificate, not an SSL certificate.
I don't really like the term 'token signing certificate' because it sounds so benign. What you have is a private key (as part of the certificate), and everyone knows you should secure your private keys!
I wouldn't store this in your application files. If someone gets your source code, they shouldn't also get the keys to your sensitive data (if someone has your signing cert, they can generate any token they like and pretend to be any of your users).
I would consider storing the certificate in AWS parameter store. You could paste the certificate into a parameter, which can be encrypted at rest. You then lock down the parameter with an AWS policy so only admins and the application can get the cert - your naughty Devs dont need it! Your application would pull the parameter string when needed and turn it into your certificate object.
This is how I store secrets in my application. I can provide more examples/details if required.
Edit -- This was the final result from Stu's guidance
The project needs 2 AWS packages from Nuget to the project
AWSSDK.Extensions.NETCORE.Setup
AWSSDK.SimpleSystemsManagement
Create 2 parameters in the AWS SSM Parameter Store like:
A plain string named /MyApp/Staging/SigningCertificate and the value is a Base64 encoded .pfx file
An encrypted string /MyApp/Staging/SigningCertificateSecret and the value is the password to the above .pfx file
This is the relevant code:
// In Startup class
private X509Certificate2 GetSigningCertificate()
{
// Configuration is the IConfiguration built by the WebHost in my Program.cs and injected into the Startup constructor
var awsOptions = Configuration.GetAWSOptions();
var ssmClient = awsOptions.CreateServiceClient<IAmazonSimpleSystemsManagement>();
// This is blocking because this is called during synchronous startup operations of the WebHost-- Startup.ConfigureServices()
var res = ssmClient.GetParametersByPathAsync(new Amazon.SimpleSystemsManagement.Model.GetParametersByPathRequest()
{
Path = "/MyApp/Staging",
WithDecryption = true
}).GetAwaiter().GetResult();
// Decode the certificate
var base64EncodedCert = res.Parameters.Find(p => p.Name == "/MyApp/Staging/SigningCertificate")?.Value;
var certificatePassword = res.Parameters.Find(p => p.Name == "/MyApp/Staging/SigningCertificateSecret")?.Value;
byte[] decodedPfxBytes = Convert.FromBase64String(base64EncodedCert);
return new X509Certificate2(decodedPfxBytes, certificatePassword);
}
public void ConfigureServices(IServiceCollection servies)
{
// ...
var identityServerBuilder = services.AddIdentityServer();
var signingCertificate = GetSigningCertificate();
identityServerBuilder.AddSigningCredential(signingCertificate);
//...
}
Last, you may need to set an IAM role and/or policy to your EC2 instance(s) that gives access to these SSM parameters.
Edit: I have been moving my web application SSL termination from my load balancer to my elastic beanstalk instance this week. This requires storing my private key in S3. Details from AWS here: Storing Private Keys Securely in Amazon S3
I am trying to automate the process of creating application version for an existing elastic beanstalk application through java api and command line arguments.
while implementing createApplicationVersion() of AWSElasticBeanstalkClient I am getting error for the below code snipplet.
Note: I am passing the endpoint for AWSElasticBeanstalkClient as US East-1 (N.Virginia) or the environment url for the existing application.
ArrayList<String> s3SourceBundleList = AmazonS3BucketUploadApp.doBucketUploadFromLocal(sourceLocation);
String bucketName = s3SourceBundleList.get(0);
String keyName = java.net.URLEncoder.encode(s3SourceBundleList.get(1), "UTF-8");
//String keyName = s3SourceBundleList.get(1);
S3Location s3SourceBundle = new S3Location();
s3SourceBundle.setS3Bucket(bucketName);
s3SourceBundle.setS3Key(keyName);
createApplicationVersionRequest.setSourceBundle(s3SourceBundle);
createApplicationVersionRequest.setDescription("New version");
appVersionResultObject = awsBeanstalkclient.createApplicationVersion(createApplicationVersionRequest);
Error:
com.amazonaws.AmazonClientException: Unable to unmarshall response (ParseError at [row,col]:[6,1]
and one more error is
AWS service: AmazonElasticBeanstalk AWS Request ID: null AWS service unavailable.
Please suggest any solution for this.
How are you initializing the client (check logs output - enabling logger org.apache.http.wire to TRACE could help)?
If you want an idea, peek at this source:
https://github.com/jenkinsci/awseb-deployment-plugin/blob/master/src/main/java/br/com/ingenieux/jenkins/plugins/awsebdeployment/Deployer.java
It contains all you need to build and deploy into AWS EB :)
I wrote these code lines for accessing and modifying programmatically the load balanced endpoint configurations saved in my esb (4.7.0) local registry. [in a few words i add a new address endpoint to the load balance endpoint list]
SynapseConfiguration sc = synapseMsgContext.getConfiguration();
LoadbalanceEndpoint le =(LoadbalanceEndpoint) sc.getEndpoint("test");
List<Endpoint>list = le.getChildren();
AddressEndpoint ad = new AddressEndpoint();
EndpointDefinition def = new EndpointDefinition();
def.setAddress("http://172.17.54.101:8083/RestService/rest/servizio");
def.setAddressingOn(false);
def.setTimeoutAction(100);
ad.setDefinition(def);
list.add(ad);
le.setChildren(list);
sc.updateEndpoint("test", le);
synapseMsgContext.setConfiguration(sc);
By this code lines the endpoint's updates are held in memory and lost when i restart the ESB. So this update lastes only till the esb is stopped.
How can i make these updates persistent? I mean an effective update on the endpoint xml configuration file?
You have to check endpoint serilaizer and factory.
http://svn.wso2.org/repos/wso2/carbon/platform/branches/turing/dependencies/synapse/2.1.2-wso2v3/modules/core/src/main/java/org/apache/synapse/config/xml/endpoints/