Listing AWS S3 bucket content from Springboot With Apache Camel - amazon-web-services

Question

I have a sprintboot project running v2.5.4 which works fine.
I have access to S3 and im able to list the content of a bucket i have created. So i wanted to experiment with Apache Camel to try to just list the content of a bucket which should be pretty simple according to the examples. But i keep getting errors.
I added 2 dependencies to my build.gradle
implementation group: 'org.apache.camel.springboot', name: 'camel-core-starter', version: '3.13.0'
implementation group: 'org.apache.camel.springboot', name: 'camel-aws2-s3-starter', version: '3.13.0'
and then i simply created a SimpleRouteBuilder.java
#Component
public class SimpleRouteBuilder extends RouteBuilder {
#Override
public void configure() throws Exception {
from("aws2-s3://bucketName?amazonS3Client=#createS3Client&operation=listObjects&accessKey=xxxAccessKeyxxx&secretKey=xxxSecretKeyxxx")
.log("Received body: ");
}
And i keep getting this stacktrace
On my aws s3 client factory i have set the bean name
#Slf4j
#Configuration
public class S3ClientBeanFactory {
#Bean(name = "s3Client")
and this seems to work - when i change the name to something else i get
an error about this :
No bean could be found in the registry for:S3Client
But with the "s3client" set in the camel endpoint url i get this all the time
2021-12-13 12:23:25.036 INFO [,,] 28267 --- [ main] o.a.c.impl.engine.AbstractCamelContext : Apache Camel 3.13.0 (camel-1) shutdown in 4ms (uptime:511ms)
2021-12-13 12:23:25.045 INFO [,,] 28267 --- [ main] o.apache.catalina.core.StandardService : Stopping service [Tomcat]
2021-12-13 12:23:25.069 INFO [,,] 28267 --- [ main] ConditionEvaluationReportLoggingListener :
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2021-12-13 12:23:25.085 ERROR [,,] 28267 --- [ main] o.s.boot.SpringApplication : Application run failed
org.apache.camel.FailedToStartRouteException: Failed to start route route1 because of null
2021-12-13 12:23:25.036 INFO [,,] 28267 --- [ main] o.a.c.impl.engine.AbstractCamelContext : Apache Camel 3.13.0 (camel-1) shutdown in 4ms (uptime:511ms)
2021-12-13 12:23:25.045 INFO [,,] 28267 --- [ main] o.apache.catalina.core.StandardService : Stopping service [Tomcat]
2021-12-13 12:23:25.069 INFO [,,] 28267 --- [ main] ConditionEvaluationReportLoggingListener :
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2021-12-13 12:23:25.085 ERROR [,,] 28267 --- [ main] o.s.boot.SpringApplication : Application run failed
org.apache.camel.FailedToStartRouteException: Failed to start route route1 because of null
at org.apache.camel.impl.engine.RouteService.warmUp(RouteService.java:123)
at org.apache.camel.impl.engine.InternalRouteStartupManager.doWarmUpRoutes(InternalRouteStartupManager.java:306)
at org.apache.camel.impl.engine.InternalRouteStartupManager.safelyStartRouteServices(InternalRouteStartupManager.java:189)
at org.apache.camel.impl.engine.InternalRouteStartupManager.doStartOrResumeRoutes(InternalRouteStartupManager.java:147)
at org.apache.camel.impl.engine.AbstractCamelContext.doStartCamel(AbstractCamelContext.java:3201)
at org.apache.camel.impl.engine.AbstractCamelContext.doStartContext(AbstractCamelContext.java:2863)
at org.apache.camel.impl.engine.AbstractCamelContext.doStart(AbstractCamelContext.java:2814)
at org.apache.camel.spring.boot.SpringBootCamelContext.doStart(SpringBootCamelContext.java:43)
at org.apache.camel.support.service.BaseService.start(BaseService.java:119)
at org.apache.camel.impl.engine.AbstractCamelContext.start(AbstractCamelContext.java:2510)
at org.apache.camel.impl.DefaultCamelContext.start(DefaultCamelContext.java:246)
at org.apache.camel.spring.SpringCamelContext.start(SpringCamelContext.java:119)
at org.apache.camel.spring.SpringCamelContext.onApplicationEvent(SpringCamelContext.java:151)
at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:176)
at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:169)
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:143)
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:421)
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:378)
at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:938)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:586)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:145)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:754)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:434)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:338)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1343)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1332)
at dk.danskespil.scratchgames.ScratchgamesApplication.main(ScratchgamesApplication.java:22)
Caused by: software.amazon.awssdk.services.s3.model.S3Exception: null (Service: S3, Status Code: 403, Request ID: null, Extended Request ID: FknaUW6/yRkYvJry9d8oIWU2hC4aRk7z8ilAZZxlcDN4s+P4bAoyzWVriJxUYj2bCyzCFFMSGNY=)
Is this operation not possible or what am i missing to do such simple operation ?

Answer 1

I ran into the same issue with Apache Camel component aws2-s3:// which was caused by insufficient rights on AWS S3:
Caused by: software.amazon.awssdk.services.s3.model.S3Exception: null (Service: S3, Status Code: 403, ...)
But I have to mention that S3Client from Amazon SDK works well for reading files with the same rights = the same account.
Explanation: I found that this aws2-s3:// component requires making headBucket API call (and the others) which causes the error because of insufficient rights for making this api call.

Answer 2

Spring doesn't consider configuration beans to be special. It is possible that the route bean is created before the S3 client bean.
I'd try using the #DependsOn({"s3ClientBeanFactory"})-annotation on the route bean. More on it here: Controlling Bean Creation Order with #DependsOn Annotation