The Java Code example is all over the web. But where is an example of the properties file contents AwsCredentials.properties?
public AmazonSESMailer(Environment env) throws IOException {
this.env = env;
InputStream resource = AmazonSESMailer.class
.getResourceAsStream("/AwsCredentials.properties");
PropertiesCredentials credentials = new PropertiesCredentials(resource);
this.client = new AmazonSimpleEmailServiceClient(credentials);
Properties props = new Properties();
props.setProperty("mail.transport.protocol", "aws");
props.setProperty("mail.aws.user", credentials.getAWSAccessKeyId());
props.setProperty("mail.aws.password", credentials.getAWSSecretKey());
this.session = Session.getInstance(props);
}
private final static Logger logger = LoggerFactory
.getLogger(AmazonSESMailer.class);
private final static String FROM = "vraptor.simplemail.main.from";
private final static String REPLY_TO = "vraptor.simplemail.main.replyTo";
private final static String SEND_REAL_EMAIL = "vraptor.simplemail.send_real_email";
The getResourceAsStream("/AwsCredentials.properties") file is being loaded by the class loader, so put it in the "classes" folder of the web app. Then the contents of the file should be only two lines:
accessKey = UTZA...
secretKey = Alj...
enjoy
Related
I have a Spring Boot application that has a POST end-point that accepts 2 types of files. Based on the file category, I need to write them to S3 buckets which are in different regions. Example: Category 1 file should be written to Frankfurt (eu-central-1) and Category 2 file should be written to Ohio (us-east-2) S3 buckets. Spring boot accepts a static region (cloud.aws.region.static=eu-central-1) through property configuration and the connection is established when starting the Spring boot so the AmazoneS3 Client Bean is already created with a connection to Frankfurt itself.
I need to containerize this entire setup and deploy it in a K8 Pod.
What is the recommendation for establishing connections and writing objects to different regional buckets? How do I need to implement this? Looking for a dynamic region finding solution rather statically created Bean per region.
Below is a working piece of code that connects to Frankfurt bucket and PUT the object.
#Service
public class S3Service {
#Autowired
private AmazonS3 amazonS3Client;
public void putObject(MultipartFile multipartFile) {
ObjectMetadata objectMetaData = new ObjectMetadata();
objectMetaData.setContentType(multipartFile.getContentType());
objectMetaData.setContentLength(multipartFile.getSize());
try {
PutObjectRequest putObjectRequest = new PutObjectRequest("example-bucket", multipartFile.getOriginalFilename(), multipartFile.getInputStream(), objectMetaData);
this.amazonS3Client.putObject(putObjectRequest);
} catch (IOException e) {
/* Handle Exception */
}
}
}
Updated Code (20/08/2021)
#Component
public class AmazoneS3ConnectionFactory {
private static final Logger LOGGER = LoggerFactory.getLogger(AmazoneS3ConnectionFactory.class);
#Value("${example.aws.s3.regions}")
private String[] regions;
#Autowired
private DefaultListableBeanFactory beanFactory;
#Autowired
private AWSCredentialsProvider credentialProvider;
#PostConstruct
public void init() {
for(String region: this.regions) {
String amazonS3BeanName = region + "_" + "amazonS3";
if (!this.beanFactory.containsBean(amazonS3BeanName)) {
AmazonS3ClientBuilder builder = AmazonS3ClientBuilder.standard().withPathStyleAccessEnabled(true)
.withCredentials(this.credentialProvider).withRegion(region).withChunkedEncodingDisabled(true);
AmazonS3 awsS3 = builder.build();
this.beanFactory.registerSingleton(amazonS3BeanName, awsS3);
LOGGER.info("Bean " + amazonS3BeanName + " - Not exist. Created a bean and registered the same");
}
}
}
/**
* Returns {#link AmazonS3} for a region. Uses the default {#link AWSCredentialsProvider}
*/
public AmazonS3 getConnection(String region) {
String amazonS3BeanName = region + "_" + "amazonS3";
return (AmazonS3Client)this.beanFactory.getBean(amazonS3BeanName, AmazonS3.class);
}
}
My Service layer will call the "getConnection()" and get the AmazonS3 Object to operate on it.
The only option that I am aware is to create different S3Client with S3ClientBuilder, one for each different region. You would need to register them as Spring Beans with different names so that you can later autowire them.
Update (19/08/2021)
The following should work (sorry for the Kotlin code but it is faster to write):
Class that may contain your configuration for each region.
class AmazonS3Properties(val accessKeyId: String,
val secretAccessKey: String,
val region: String,
val bucket: String)
Configuration for S3 that will create 2 S3Clients and stored the buckets for each region (later needed).
#Configuration
class AmazonS3Configuration(private val s3Properties: Map<String, AmazonS3Properties>) {
lateinit var buckets: Map<String, String>
#PostConstruct
fun init() {
buckets = s3Properties.mapValues { it.bucket }
}
#Bean(name = "regionA")
fun regionA(): S3Client {
val regionAProperties = s3Properties["region-a"]
val awsCredentials = AwsBasicCredentials.create(regionAProperties.accessKeyId, regionAProperties.secretAccessKey)
return S3Client.builder().region(Region.of(regionAProperties.region)).credentialsProvider { awsCredentials }.build()
}
#Bean(name = "regionB")
fun regionB(): S3Client {
val regionBProperties = s3Properties["region-b"]
val awsCredentials = AwsBasicCredentials.create(regionBProperties.accessKeyId, regionBProperties.secretAccessKey)
return S3Client.builder().region(Region.of(regionBProperties.region)).credentialsProvider { awsCredentials }.build()
}
}
Service that will target one of the regions (Region A)
#Service
class RegionAS3Service(private val amazonS3Configuration: AmazonS3Configuration,
#field:Qualifier("regionA") private val amazonS3Client: S3Client) {
fun save(region: String, byteArrayOutputStream: ByteArrayOutputStream) {
val inputStream = ByteArrayInputStream(byteArrayOutputStream.toByteArray())
val contentLength = byteArrayOutputStream.size().toLong()
amazonS3Client.putObject(PutObjectRequest.builder().bucket(amazonS3Configuration.buckets[region]).key("whatever-key").build(), RequestBody.fromInputStream(inputStream, contentLength))
}
}
I am using ASP.NET Core and AWSSDK.S3 nuget package.
I am able to upload file by providing accessKeyID, secretKey, bucketName and region
Like this:
var credentials = new BasicAWSCredentials(accessKeyID, secretKey);
using (var client = new AmazonS3Client(credentials, RegionEndpoint.USEast1))
{
var request = new PutObjectRequest
{
AutoCloseStream = true,
BucketName = bucketName,
InputStream = storageStream,
Key = fileName
}
}
But I am given only an URL to upload file
11.11.11.111:/aa-bb-cc-dd-useast1
How to upload file through the URL? I am new to AWS,I will be grateful to get some help.
using Amazon.S3;
using Amazon.S3.Transfer;
using System;
using System.IO;
using System.Threading.Tasks;
namespace Amazon.DocSamples.S3
{
class UploadFileMPUHighLevelAPITest
{
private const string bucketName = "*** provide bucket name ***";
private const string filePath = "*** provide the full path name of the file to upload ***";
// Specify your bucket region (an example region is shown).
private static readonly RegionEndpoint bucketRegion = RegionEndpoint.USWest2;
private static IAmazonS3 s3Client;
public static void Main()
{
s3Client = new AmazonS3Client(bucketRegion);
UploadFileAsync().Wait();
}
private static async Task UploadFileAsync()
{
try
{
var fileTransferUtility =
new TransferUtility(s3Client);
// Option 1. Upload a file. The file name is used as the object key name.
await fileTransferUtility.UploadAsync(filePath, bucketName);
Console.WriteLine("Upload 1 completed");
}
}
}
}
https://docs.aws.amazon.com/AmazonS3/latest/dev/HLuploadFileDotNet.html
You can use the provided access point in place of the bucket name.
https://docs.aws.amazon.com/sdkfornet/v3/apidocs/items/S3/TPutObjectRequest.html
I am a salesforce developer, our company are planning to extend the service for global users, so we decided to use google translate to improve our customers' experience.
I have read the google api document, however, we met an issue when requesting GCP access token, the tutorial google provided in document which supports different languages, like java, c#, python etc. Since we are suing apex (a type of salesforce platform script), we weren't able to use your library to get GCP token.
Instead, we also checked "OAuth 2.0 for Server Accounts", unfortunately, neither worked from me.
Is there any suggestions?
I got token by below code, and the error I met was caused by special characters encode.
public with sharing class SwitchLanguageByGoogleAPIController {
private static final String ENDPOINT = '**';
private static final String TOKEN_ENDPOINT = 'https://accounts.google.com/o/oauth2/token';
private static final String SCOPE = 'https://www.googleapis.com/auth/cloud-platform';
private static final String PROJECT_ID = '**';
private static final String GLOSSARY_ID = '88';
private static final String LOCATION_ID = 'us-central1';
private static final String CLIENT_SECRET = '**';
private static final String PRIVATE_KEY = '**';
private static final String CLIENT_EMAIL = '**';
public static void translateByGlossary() {
Token token = getToken();
HttpRequest request = new HttpRequest();
request.setHeader('Content-Type', 'text/plain');
request.setEndpoint(ENDPOINT + PROJECT_ID + '/locations/' + LOCATION_ID + ':translateText?access_token=' + token.access_token);
request.setMethod('POST');
String contents = 'Personal Information, Middle Name e, first First Name';
String sourceLanguageCode = 'en';
String targetLanguageCode = 'zh';
request.setBody('{"sourceLanguageCode":"' + sourceLanguageCode + '","targetLanguageCode":"' + targetLanguageCode + '","contents":"'+ contents +'","glossaryConfig":{"glossary":"projects/' + PROJECT_ID +'/locations/' + LOCATION_ID + '/glossaries/' + GLOSSARY_ID + '"}}');
HTTP http = new HTTP();
HttpResponse reponse = http.send(request);
System.debug(reponse.getBody());
}
private static Token getToken() {
Http http = new Http();
HttpRequest req = new HttpRequest();
HttpResponse res = new HttpResponse();
//Making the call out
req.setEndpoint(TOKEN_ENDPOINT);
req.setMethod('POST');
req.setHeader('Content-Type','application/x-www-form-urlencoded');
string URLEncodedGrantType = encodingUtil.urlEncode('urn:ietf:params:oauth:grant-type:jwt-bearer','UTF-8');
string jwtSigned = generateJWT();
req.setBody('grant_type='+URLEncodedGrantType+'&assertion='+jwtSigned);
res = http.send(req);
system.debug('Response : '+res.getBody());
return (Token)JSON.deserialize(res.getBody(), Token.Class);
}
private static String generateJWT() {
Http http = new Http();
HttpRequest req = new HttpRequest();
HttpResponse res = new HttpResponse();
String JWTHeader = '{"typ":"JWT","alg":"RS256"}';
String Base64EncodedJWTHeader = EncodingUtil.base64Encode(Blob.valueOf(JWTHeader));
long issued_at = datetime.now().getTime()/1000;
long expires_at = datetime.now().addHours(1).getTime()/1000;
JWTClaimSet claimSet = new JWTClaimSet();
claimSet.iss = CLIENT_EMAIL;
claimSet.scope = SCOPE;
claimSet.aud = TOKEN_ENDPOINT;
claimSet.iat = issued_at;
claimSet.exp = expires_at;
String strClaimSetJSON = JSON.Serialize(claimSet);
String Base64EncodedClaimset = EncodingUtil.base64Encode(Blob.valueOf(strClaimSetJSON));
system.debug('Base64 Encoded Claimset::'+Base64EncodedClaimset);
Base64EncodedClaimset = PerformPostBase64Encode(Base64EncodedClaimset);
system.debug('persorm post Claimset::'+Base64EncodedClaimset);
string Base64EncodedString = Base64EncodedJWTHeader + '.' + Base64EncodedClaimset;
String algorithmName = 'RSA-SHA256';
Blob privateKey = EncodingUtil.base64Decode(PRIVATE_KEY);
Blob input = Blob.valueOf(Base64EncodedString);
Blob Blobsign = Crypto.sign(algorithmName, input , privateKey);
String base64EncodedSignature = EncodingUtil.base64Encode(Blobsign);
base64EncodedSignature = PerformPostBase64Encode(base64EncodedSignature);
system.debug('Base 64 encoded signature ::'+base64EncodedSignature );
system.debug('Encoded assertion : ' + Base64EncodedString+'.'+base64EncodedSignature);
string URLEncodedUTF8Assertion = encodingUtil.urlEncode(Base64EncodedString+'.'+base64EncodedSignature,'UTF-8');
return URLEncodedUTF8Assertion;
}
public static String PerformPostBase64Encode(String s)
{
s = s.Replace('+', '-');
s = s.Replace('/', '_');
s = s.Split('=')[0];
return s;
}
public class JWTClaimSet
{
public string iss {get;set;}
public string scope {get;set;}
public string aud {get;set;}
public Long exp {get;set;}
public Long iat {get;set;}
}
private class Token{
private String access_token;
private String token_type;
private String expires_in;
}
}
I'm trying to execute query in AWS Athena using Java API:
public class AthenaClientFactory
{
String accessKey = "access";
String secretKey = "secret";
BasicAWSCredentials awsCredentials = new
BasicAWSCredentials(accessKey, secretKey);
private final AmazonAthenaClientBuilder builder = AmazonAthenaClientBuilder.standard()
.withRegion(Regions.US_WEST_1)
.withCredentials(new AWSStaticCredentialsProvider(awsCredentials))
.withClientConfiguration(new ClientConfiguration().withClientExecutionTimeout(10));
public AmazonAthena createClient()
{
return builder.build();
}
}
private static String submitAthenaQuery(AmazonAthena client) {
QueryExecutionContext queryExecutionContext = new QueryExecutionContext().withDatabase("my_db");
ResultConfiguration resultConfiguration = new ResultConfiguration().withOutputLocation("my_bucket");
StartQueryExecutionRequest startQueryExecutionRequest = new StartQueryExecutionRequest()
.withQueryString("select * from my_db limit 3;")
.withQueryExecutionContext(queryExecutionContext)
.withResultConfiguration(resultConfiguration);
StartQueryExecutionResult startQueryExecutionResult = client.startQueryExecution(startQueryExecutionRequest);
return startQueryExecutionResult.getQueryExecutionId();
}
public void run() throws InterruptedException {
AthenaClientFactory factory = new AthenaClientFactory();
AmazonAthena client = factory.createClient();
String queryExecutionId = submitAthenaQuery(client);
}
But I get an exception from startQueryExecutionResult.
The exception is:
Client execution did not complete before the specified timeout
configuration.
Has anyone encountered something similar?
The problem was in withClientExecutionTimeout(10).
Increasing this number to 5000 solved the issue
I am having trouble fetching result from my amazon elastic search cluster using the amazon java SDK and an IAm user credential. Now the issue is that when the PATH string is equal to "/" then I am able to fetch the result correctly but when I try with a different path for e.g "/private-search" then I get a 403 forbidden error. Even when for the path that has public access I am getting a 403 forbidden error for this IAm user but it works if I remove "signer.sign(requestToSign, credentials);" line in performSigningSteps method(for public resource only).
My policy in AWS gives this IAM user access to everything in my elastic search service. And also what can I do to avoid hard-coding the access key and secret key in source code?
private static final String SERVICE_NAME = "es";
private static final String REGION = "region-name";
private static final String HOST = "host-name";
private static final String ENDPOINT_ROOT = "http://" + HOST;
private static final String PATH = "/private-search";
private static final String ENDPOINT = ENDPOINT_ROOT + PATH;
private static String accessKey = "IAmUserAccesskey"
private static String secretKey = "IAmUserSecretkey"
public static void main(String[] args) {
// Generate the request
Request<?> request = generateRequest();
// Perform Signature Version 4 signing
performSigningSteps(request);
// Send the request to the server
sendRequest(request);
}
private static Request<?> generateRequest() {
Request<?> request = new DefaultRequest<Void>(SERVICE_NAME);
request.setContent(new ByteArrayInputStream("".getBytes()));
request.setEndpoint(URI.create(ENDPOINT));
request.setHttpMethod(HttpMethodName.GET);
return request;
}
private static void performSigningSteps(Request<?> requestToSign) {
AWS4Signer signer = new AWS4Signer();
signer.setServiceName(requestToSign.getServiceName());
signer.setRegionName(REGION);
AWSCredentials credentials = new BasicAWSCredentials(accessKey, secretKey);
signer.sign(requestToSign, credentials);
}
private static void sendRequest(Request<?> request) {
ExecutionContext context = new ExecutionContext();
ClientConfiguration clientConfiguration = new ClientConfiguration();
AmazonHttpClient client = new AmazonHttpClient(clientConfiguration);
MyHttpResponseHandler<Void> responseHandler = new MyHttpResponseHandler<Void>();
MyErrorHandler errorHandler = new MyErrorHandler();
Void response = client.execute(request, responseHandler, errorHandler, context);
}
public static class MyHttpResponseHandler<T> implements HttpResponseHandler<AmazonWebServiceResponse<T>> {
#Override
public AmazonWebServiceResponse<T> handle(com.amazonaws.http.HttpResponse response) throws Exception {
InputStream responseStream = response.getContent();
String responseString = convertStreamToString(responseStream);
System.out.println(responseString);
AmazonWebServiceResponse<T> awsResponse = new AmazonWebServiceResponse<T>();
return awsResponse;
}
#Override
public boolean needsConnectionLeftOpen() {
return false;
}
}
public static class MyErrorHandler implements HttpResponseHandler<AmazonServiceException> {
#Override
public AmazonServiceException handle(com.amazonaws.http.HttpResponse response) throws Exception {
System.out.println("In exception handler!");
AmazonServiceException ase = new AmazonServiceException("exception.");
ase.setStatusCode(response.getStatusCode());
ase.setErrorCode(response.getStatusText());
return ase;
}
#Override
public boolean needsConnectionLeftOpen() {
return false;
}
}
public static String convertStreamToString(InputStream is) throws IOException {
// To convert the InputStream to String we use the
// Reader.read(char[] buffer) method. We iterate until the
// Reader return -1 which means there's no more data to
// read. We use the StringWriter class to produce the string.
if (is != null) {
Writer writer = new StringWriter();
char[] buffer = new char[1024];
try {
Reader reader = new BufferedReader(new InputStreamReader(is, "UTF-8"));
int n;
while ((n = reader.read(buffer)) != -1) {
writer.write(buffer, 0, n);
}
}
finally {
is.close();
}
return writer.toString();
}
return "";
}