I created a sftp server to upload files to S3, i have setup ssh authentication and username/password for the server.
Now i want to automate the process of creating users for the server using a lambda function, either by automating the secrets manager part or the aws transfer family user, is there a way to do this or what i am looking for is not yet possible
Java example using the AWS transfer family SDK:
public byte[] createUser(String sftpBaseFolderPath, String username) throws NoSuchAlgorithmException, IOException, JSchException {
CreateUserRequest createUserRequest = new CreateUserRequest();
createUserRequest.setServerId(serverId);
createUserRequest.setHomeDirectoryType("LOGICAL");
createUserRequest.setRole(userRoleArn);
createUserRequest.setUserName(username);
List<HomeDirectoryMapEntry> mapEntries = new ArrayList<>();
HomeDirectoryMapEntry homeDirectoryMapEntry = new HomeDirectoryMapEntry();
homeDirectoryMapEntry.setEntry("/");
homeDirectoryMapEntry.setTarget(sftpBaseFolderPath);
mapEntries.add(homeDirectoryMapEntry);
createUserRequest.setHomeDirectoryMappings(mapEntries);
KeyPair keyPair = KeyPair.genKeyPair(new JSch(), KeyPair.RSA, 2048);
ByteArrayOutputStream privateKey = new ByteArrayOutputStream();
OutputStream publicKey = new ByteArrayOutputStream();
keyPair.writePrivateKey(privateKey);
keyPair.writePublicKey(publicKey, "Key description/comment");
privateKey.close();
publicKey.close();
createUserRequest.setSshPublicKeyBody(publicKey.toString());
transferClient.createUser(createUserRequest);
return privateKey.toByteArray();
}
Transfer client initialization will be something like this:
public AWSStaticCredentialsProvider awsCredentialsProvider() {
return new AWSStaticCredentialsProvider(new BasicAWSCredentials(accessKey, secretKey));
}
public void initTransferClient() {
this.transferClient = AWSTransferClientBuilder
.standard()
.withCredentials(awsCredentialsProvider())
.withRegion(Regions.fromName(region))
.build();
}
SDK version
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-transfer</artifactId>
<version>1.12.173</version>
</dependency>
Related
I need to list folders using service account credentials without using the GCP Java client library.
I need to call using REST API as there is restriction on making use of GCP Java client library.
Is there any sample or example I can refer to make REST API calls in Java/Spring?
You can use RestTemplate and this dependency
<dependency>
<groupId>com.auth0</groupId>
<artifactId>java-jwt</artifactId>
<version>3.14.0</version>
</dependency>
Then, read your service account JSON file (here my key.json file) and enjoy! Let me know if some part are not clear!
InputStream resource = new ClassPathResource("key.json").getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(resource));
String keyFileJson = reader.lines().collect(Collectors.joining("\n"));
Map keyFile = new ObjectMapper().readValue(keyFileJson, Map.class);
String privKeyPEM = keyFile.get("private_key").toString().replace("-----BEGIN PRIVATE KEY-----","")
.replace("-----END PRIVATE KEY-----","")
.replaceAll("\n","");
byte [] encoded = Base64.decode(privKeyPEM);
PKCS8EncodedKeySpec keySpec = new PKCS8EncodedKeySpec(encoded);
KeyFactory kf = KeyFactory.getInstance("RSA");
PrivateKey privKey = kf.generatePrivate(keySpec);
String tokenUrl = "https://oauth2.googleapis.com/token";
long now = System.currentTimeMillis();
Algorithm algorithm = Algorithm.RSA256(null, (RSAPrivateKey)privKey);
String signedJwt = JWT.create()
.withKeyId(keyFile.get("private_key_id").toString())
.withIssuer(keyFile.get("client_email").toString())
.withAudience(tokenUrl)
.withClaim("scope","openid https://www.googleapis.com/auth/userinfo.email https://www.googleapis.com/auth/cloud-platform https://www.googleapis.com/auth/appengine.admin https://www.googleapis.com/auth/compute")
.withIssuedAt(new Date(now))
.withExpiresAt(new Date(now + 3600 * 1000L))
.sign(algorithm);
// System.out.println(signedJwt);
RestTemplate restTemplate = new RestTemplate();
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_FORM_URLENCODED);
MultiValueMap<String, String> map = new LinkedMultiValueMap<>();
map.add("assertion", signedJwt);
map.add("grant_type", "urn:ietf:params:oauth:grant-type:jwt-bearer");
HttpEntity<MultiValueMap<String, String>> entity = new HttpEntity<>(map, headers);
ResponseEntity<String> responseEntity = restTemplate.postForEntity(tokenUrl, entity, String.class);
Map result = new ObjectMapper().readValue(responseEntity.getBody(), Map.class);
System.out.println(result.get("access_token"));
EDIT 1
To generate and get an ID_Token, required if you want to generate a token to call Cloud Run, Cloud Functions or App Engine deployed in secured mode. You need to replace the scope claim by the target_audience URL. And at the end, print the id_token and not the access token
String signedJwt = JWT.create()
.withKeyId(keyFile.get("private_key_id").toString())
.withIssuer(keyFile.get("client_email").toString())
.withAudience(tokenUrl)
.withClaim("target_audience", "https://target-log-fqffbf2xsq-uc.a.run.app")
.withIssuedAt(new Date(now))
.withExpiresAt(new Date(now + 3600 * 1000L))
.sign(algorithm);
...
...
...
System.out.println(result.get("id_token"));
How can I access configurations from aws appconfig, in my spring boot application?
Since appconfig is a new service, is there any java sdk that we can use, cos i dont see anything for appconfig yet in https://github.com/aws/aws-sdk-java/tree/master/src/samples
Here's how I've integrated AWS AppConfig into my Spring Boot project.
First, let’s make sure we have this dependency in our pom.xml:
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-appconfig</artifactId>
<version>1.12.134</version>
</dependency>
Next, let’s create a simple configuration class of our own AWS AppConfig Client:
#Configuration
public class AwsAppConfiguration {
private static final Logger LOGGER = LoggerFactory.getLogger(AwsAppConfiguration.class);
private final AmazonAppConfig appConfig;
private final GetConfigurationRequest request;
public AwsAppConfiguration() {
appConfig = AmazonAppConfigClient.builder().build();
request = new GetConfigurationRequest();
request.setClientId("clientId");
request.setApplication("FeatureProperties");
request.setConfiguration("JsonProperties");
request.setEnvironment("dev");
}
public JSONObject getConfiguration() throws UnsupportedEncodingException {
GetConfigurationResult result = appConfig.getConfiguration(request);
String message = String.format("contentType: %s", result.getContentType());
LOGGER.info(message);
if (!Objects.equals("application/json", result.getContentType())) {
throw new IllegalStateException("config is expected to be JSON");
}
String content = new String(result.getContent().array(), "ASCII");
return new JSONObject(content).getJSONObject("feature");
}
}
Lastly, let’s create a scheduled task that polls the configuration from AWS AppConfig:
#Configuration
#EnableScheduling
public class AwsAppConfigScheduledTask {
private static final Logger LOGGER = LoggerFactory.getLogger(AwsAppConfigScheduledTask.class);
#Autowired
private FeatureProperties featureProperties;
#Autowired
private AwsAppConfiguration appConfiguration;
#Scheduled(fixedRate = 5000)
public void pollConfiguration() throws UnsupportedEncodingException {
LOGGER.info("polls configuration from aws app config");
JSONObject externalizedConfig = appConfiguration.getConfiguration();
featureProperties.setEnabled(externalizedConfig.getBoolean("enabled"));
featureProperties.setLimit(externalizedConfig.getInt("limit"));
}
}
I came across this question, as I was also trying to figure out how to best integrate AWS AppConfig into Spring Boot.
Here's an article I created. You can visit it here: https://levelup.gitconnected.com/create-features-toggles-using-aws-appconfig-in-spring-boot-7454b122bf91
Also, the source code is available on github: https://github.com/emyasa/medium-articles/tree/master/aws-spring-boot/app-config
First I added dependencies
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>appconfig</artifactId>
<version>2.18.41</version>
</dependency>
<!-- https://mvnrepository.com/artifact/software.amazon.awssdk/appconfigdata -->
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>appconfigdata</artifactId>
<version>2.19.4</version>
</dependency>
Then build the client
client = AppConfigDataClient.builder()
.credentialsProvider(() -> AwsBasicCredentials.create("<your id>", "your secret key"))
.region(Region.<your region>)
.build();
Use the client to start the configuration session
StartConfigurationSessionRequest startConfigurationSessionRequest = StartConfigurationSessionRequest.builder()
.applicationIdentifier("<your application id>")
.environmentIdentifier("your environment id")
.configurationProfileIdentifier("your config id")
.build();
Get the session token in the beginning and use it for the initial call.
String sessionToken = client.startConfigurationSession(startConfigurationSessionRequest).initialConfigurationToken();
GetLatestConfigurationRequest latestConfigurationRequest = GetLatestConfigurationRequest.builder()
.configurationToken(sessionToken)
.build();
GetLatestConfigurationResponse latestConfigurationResponse = client.getLatestConfiguration(latestConfigurationRequest);
String response = latestConfigurationResponse.configuration().asUtf8String();
You can use the next token available in the response to make the next call. The token can be cached as required.
2023, use aws-java-sdk-appconfigdata
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-appconfigdata</artifactId>
<version>1.12.394</version>
</dependency>
example:
#Slf4j
#Configuration
#EnableScheduling
public class AWSAppConfig {
private String token;
private final AWSAppConfigData client;
public AWSAppConfig() {
log.info("init app config");
var client = AWSAppConfigDataClient.builder().build();
var request = new StartConfigurationSessionRequest();
request.setEnvironmentIdentifier("prod");
request.setApplicationIdentifier("my-app");
request.setConfigurationProfileIdentifier("my-config");
request.setRequiredMinimumPollIntervalInSeconds(15);
var result = client.startConfigurationSession(request);
this.client = client;
this.token = result.getInitialConfigurationToken();
}
#Scheduled(fixedRate = 20000)
public void pollConfiguration() {
var request = new GetLatestConfigurationRequest();
request.setConfigurationToken(token);
var result = client.getLatestConfiguration(request);
this.token = result.getNextPollConfigurationToken();
var configuration = StandardCharsets.UTF_8.decode(result.getConfiguration()).toString();
log.info("content type: {}", result.getContentType());
log.info("configuration: {}", configuration);
}
}
I am using Lambda proxy integration to execute lambda from API Gateway. Following is my code for the same in Java. My lambda is executing properly from Eclipse and from AWS console. I assume this should be with the configuration of my API gateway. I have created a POST method and have defined a model as well.
But when I am trying the "Test" option from API Gateway, I am getting "Internal Error": {"errorMessage":"Input value must not be null","errorType":"java.lang.IllegalArgumentException"} and with "Execution failed due to configuration error: Malformed Lambda proxy response".
public class SavePersonHandler implements RequestHandler<PersonRequest, JSONObject> {
private DynamoDB dynamoDb;
JSONParser parser = new JSONParser();
private String DYNAMODB_TABLE_NAME = "XXXX";
private Regions REGION = Regions.US_WEST_2;
public JSONObject handleRequest(UserRequest userRequest, Context context) {
this.initDynamoDbClient();
LambdaLogger logger = context.getLogger();
String responseCode = "200";
JSONObject responseJson = new JSONObject();
saveData(personRequest); // inserting data into DynamoDB
UserResponse userResponse = new UserResponse();
try {
//Getting the recently inserted data back from DynamoDB
Item item = this.dynamoDb.getTable(DYNAMODB_TABLE_NAME).getItem("id", personRequest.getId(), "id, firstName, lastName, age, address", null);
personResponse.setMessage("Saved Successfully-"+item.toJSON());
JSONObject responseBody = new JSONObject();
responseBody.put("message", item.toJSON());
JSONObject headerJson = new JSONObject();
headerJson.put("x-custom-header", "custom header value");
responseJson.put("isBase64Encoded", false);
responseJson.put("statusCode", responseCode);
responseJson.put("headers", headerJson);
responseJson.put("body", responseBody.toString());
}
catch (Exception e) {
System.err.println("GetItem failed.");
System.err.println(e.getMessage());
}
return responseJson;
}
Unable to integrate API gateway with aws lambda helped solve the issue.
The response had to be sent back as a POJO object directly rather than serializing the POJO and sending it back as a String. This is how I got it to work.
i try to register a user in my amazon cognito user pool with username and password from my java backend but i always get the error:
Unable to verify secret hash for client
in the documentation i don't found any information how to pass the clientSecret in the register request and i don't like to create an (backend) app without a clientSecret.
My code looks like this
identityProvider = AWSCognitoIdentityProviderClientBuilder.standard().withCredentials(new AWSStaticCredentialsProvider(awsCreds)).withRegion(Regions.EU_CENTRAL_1).build();
Map<String, String> authParameters = new HashMap<>();
authParameters.put("USERNAME", "username");
authParameters.put("PASSWORD", "password");
authParameters.put("SECRET_HASH", "secret copy and paste from the aws console"); // i read in a forum post, that this should work
AdminInitiateAuthRequest authRequest = new AdminInitiateAuthRequest();
authRequest.withAuthFlow(AuthFlowType.ADMIN_NO_SRP_AUTH);
authRequest.setAuthParameters(authParameters);
authRequest.setClientId("clientId");
authRequest.setUserPoolId("userPoolId");
AdminInitiateAuthResult authResponse = identityProvider.adminInitiateAuth(authRequest);
Thanks
Marcel
To register users you should use the SignUp API. The secret hash can be calculated as follows in Java:
public String calculateSecretHash(String userPoolclientId, String userPoolclientSecret, String userName) {
if (userPoolclientSecret == null) {
return null;
}
SecretKeySpec signingKey = new SecretKeySpec(
userPoolclientSecret.getBytes(StandardCharsets.UTF_8),
HMAC_SHA256_ALGORITHM);
try {
Mac mac = Mac.getInstance(HMAC_SHA256_ALGORITHM);
mac.init(signingKey);
mac.update(userName.getBytes(StandardCharsets.UTF_8));
byte[] rawHmac = mac.doFinal(userPoolclientId.getBytes(StandardCharsets.UTF_8));
return Encoding.encodeBase64(rawHmac);
} catch (Exception e) {
throw new RuntimeException("Error while calculating ");
}
}
Can you please elaborate your use case of creating users from your backend instead of directly calling Amazon Cognito from your clients?
Edit: We have updated our documentation to include a section about how to compute the secret hash.
The following code works perfectly:
AdminInitiateAuthRequest adminInitiateAuthRequest = new AdminInitiateAuthRequest().withAuthFlow(AuthFlowType.ADMIN_NO_SRP_AUTH).withClientId("<ID of your client application>").withUserPoolId("<your user pool ID>")
.addAuthParametersEntry("USERNAME", "<your user>").addAuthParametersEntry("PASSWORD", "<your password for the user>");
AdminInitiateAuthResult adminInitiateAuth = identityProvider.adminInitiateAuth(adminInitiateAuthRequest);
System.out.println(adminInitiateAuth.getAuthenticationResult().getIdToken());
Im totally new to AWS. Im trying to use the AWS S3 notification API's. Im receiving the following error.
com.amazonaws.services.sns.model.AmazonSNSException: The security token included in the request is invalid. (Service: AmazonSNS; Status Code: 403; Error Code: InvalidClientTokenId; ...
I have NO idea what's wrong. For my accessID and secretID. Im using the main AWS codes for authentication. Am I supposed to use the main AWS credentials, or something else. Im not using any type of certificate. I dont know if they are even required.
Im using the example code supplied by AWS with some modifications to read a property file instead of hard coding the accessID and secretID.
Can someone please steer me in the right direction? I am completely confused.
public class AmazonSNSReceiver {
// AWS credentials -- replace with your credentials
static String ACCESS_KEY;
static String SECRET_KEY;
// Shared queue for notifications from HTTP server
static BlockingQueue<Map<String, String>> messageQueue = new LinkedBlockingQueue<Map<String, String>>();
// Receiver loop
public static void main(String[] args) throws Exception {
AmazonSNSReceiver sns = new AmazonSNSReceiver();
sns.getPropertyValues();
if (args.length == 1) {
sns.SNSClient(args[0]);
} else {
sns.SNSClient("8989");
}
}
// Create a client
public void SNSClient(String thisport) throws Exception{
AmazonSNSClient service = new AmazonSNSClient(new BasicAWSCredentials(ACCESS_KEY, SECRET_KEY));
// Create a topic
CreateTopicRequest createReq = new CreateTopicRequest().withName("MyTopic");
CreateTopicResult createRes = service.createTopic(createReq);
// Get an HTTP Port
int port = thisport == null ? 8989 : Integer.parseInt(thisport);
// Create and start HTTP server
Server server = new Server(port);
server.setHandler(new AmazonSNSHandler());
server.start();
// Subscribe to topic
SubscribeRequest subscribeReq = new SubscribeRequest()
.withTopicArn(createRes.getTopicArn())
.withProtocol("http")
.withEndpoint("http://" + InetAddress.getLocalHost().getHostAddress() + ":" + port);
service.subscribe(subscribeReq);
for (;;) {
// Wait for a message from HTTP server
Map<String, String> messageMap = messageQueue.take();
// Look for a subscription confirmation Token
String token = messageMap.get("Token");
if (token != null) {
// Confirm subscription
ConfirmSubscriptionRequest confirmReq = new ConfirmSubscriptionRequest()
.withTopicArn(createRes.getTopicArn())
.withToken(token);
service.confirmSubscription(confirmReq);
continue;
}
// Check for a notification
String message = messageMap.get("Message");
if (message != null) {
System.out.println("Received message: " + message);
}
}
}
public void getPropertyValues() throws IOException {
Properties prop = new Properties();
InputStream properties = getClass().getClassLoader().getResourceAsStream("SNS.properties");
prop.load(properties);
ACCESS_KEY = prop.getProperty("ACCESS_KEY");
SECRET_KEY = prop.getProperty("SECRET_KEY");
}
// HTTP handler
static class AmazonSNSHandler extends AbstractHandler {
// Handle HTTP request
public void handle(String target, HttpServletRequest request, HttpServletResponse response, int dispatch) throws IOException {
// Scan request into a string
Scanner scanner = new Scanner(request.getInputStream());
StringBuilder sb = new StringBuilder();
while (scanner.hasNextLine()) {
sb.append(scanner.nextLine());
}
// Build a message map from the JSON encoded message
InputStream bytes = new ByteArrayInputStream(sb.toString().getBytes());
Map<String, String> messageMap = new ObjectMapper().readValue(bytes, Map.class);
// Enqueue message map for receive loop
messageQueue.add(messageMap);
// Set HTTP response
response.setContentType("text/html");
response.setStatus(HttpServletResponse.SC_OK);
((Request) request).setHandled(true);
}
}
}
Your application needs to provide AWS credentials. These credentials can be obtained by several methods:
Create an IAM User and generate an Access Key and Secret Key. Include the credentials in a configuration file (it is not a good idea to put them in your application, since they could accidentally be published elsewhere).
If running the code on an Amazon EC2 instance, create an IAM Role and assign the role to the instance when it is launched. Credentials will then be automatically provided to applications running on that instance.
It is also necessary to assign permissions to the IAM User/Role. These permissions grant the right to call various AWS API calls. The fact that you receive an AuthorizationError suggests that the credentials in use do not have sufficient permissions.
See: Managing Access to Your Amazon SNS Topics