I'm trying to use Apache Flink with AWS kinesis. The document says that I have to build the connector on my own.
Therefore, I build the connector and added the jar file for my project and also, I put the dependency on my pom.xml file.
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-connector-kinesis_2.11</artifactId>
<version>1.6.1</version>
</dependency>
However, when I tried to build using mvn clean package I got an error message like this
[INFO] -----------------------< kkaldk:flink-kinesis >-----------------------
[INFO] Building Flink Quickstart Job 0.1
[INFO] --------------------------------[ jar ]---------------------------------
[WARNING] The POM for org.apache.flink:flink-connector-kinesis_2.11:jar:1.6-SNAPSHOT is missing, no dependency information available
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.458 s
[INFO] Finished at: 2018-12-19T17:45:43+09:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project flink-kinesis: Could not resolve dependencies for project kkaldk:flink-kinesis:jar:0.1: Failure to find org.apache.flink:flink-connector-kinesis_2.11:jar:1.6-SNAPSHOT in https://repository.apache.org/content/repositories/snapshots/ was cached in the local repository, resolution will not be reattempted until the update interval of apache.snapshots has elapsed or updates are forced ->
[Help 1]
[ERROR]
Could you help me to fix this?
---------added----------
This is what I've done (which is wrong procedure)
started project following this document
build connector from flink master source (which is 1.8-snapshot)
add dependency with 1.6.1 version.
This is what I've done for solution.
I downloaded 1.6.1 release from here.
unzip the file (the archive will be unzip in .../flink-release-1.6.1/
built it by mvn clean install -Pinclude-kinesis -DskipTests
Then I have flink-connector-kinesis_2.11 version 1.6.1 in my local maven repository
(for me, path was like this): ~/.m2/repository/org/apache/flink/flink-connector-kinesis_2.11
After then, I could use this connector at any project by adding dependency with this code. (I don't have to add the jar file anymore.)
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-connector-kinesis_2.11</artifactId>
<version>1.6.1</version>
</dependency>
Related
According to the aws elasticbeanstalk doc,
"While Elastic Beanstalk deploys your file to your Amazon EC2 instances, you can view the deployment status on the environment's overview."
But I quite cannot understand what exactly is happening when I upload my source files to the elasticbeanstalk(eb).
As I know, if we upload zip file to the elasticbeanstalk(eb), it goes to the s3 first.
But as documentation says, "ec2 deploys those files" with the proper environment.
For example, if I uploaded .zip file which contains simple index.js (simple nodejs server) and package.json (dependencies listed file) to the elasticbeanstalk, it goes to the s3 storage service. And ec2 deploys it with the proper command like npm install, npm start, etc.
The question is that, what is exactly happening between s3 upload and ec2 deploy?
Does elasticbeanstalk makes ec2 instance to access s3 storage and get all the source files from there automatically? otherwise we have to manually connect those two aws resources together?
I've been searched for this all day, but couldn't find anything including official documentation.
Any advice, or any url reference might also really be helpful. Thank you.
I'm sure there are much better informed folks out there, as I am just starting out with AWS and EB in particular, but--since there are no answers to this now 8-month old question--I'll share what I learned so far:
EB CLI tells EC2 to fetch the app artifact and manifest
2022/03/28 23:26:45.063522 [INFO] Downloading EB Application...
2022/03/28 23:26:45.063548 [INFO] Download app version manifest
Then performs some internal magic to ensure the instance is ready to host the app
Then extracts the app artifact and--depending on the platform stack--downloads dependencies and starts the app.
2022/03/28 23:26:47.074138 [INFO] Executing instruction: StageApplication
2022/03/28 23:26:47.223041 [INFO] extracting /opt/elasticbeanstalk/deployment/app_source_bundle to /var/app/staging/
2022/03/28 23:26:52.001284 [INFO] Executing instruction: RunAppDeployPreBuildHooks
2022/03/28 23:26:52.001312 [INFO] Executing platform hooks in .platform/hooks/prebuild/
2022/03/28 23:26:52.049546 [INFO] installing node version 16.14.0
2022/03/28 23:26:52.049555 [INFO] Running command /bin/sh -c uname -m
2022/03/28 23:26:54.962177 [INFO] Executing instruction: Use NPM to install dependencies
2022/03/28 23:26:54.962214 [INFO] checking package.json file
2022/03/28 23:26:54.962224 [INFO] found package.json file, using npm to start application
Then more platform magic follows
....
2022/03/28 23:26:58.109415 [INFO] Executing instruction: RunAppDeployPostDeployHooks
2022/03/28 23:26:58.109428 [INFO] Executing platform hooks in .platform/hooks/postdeploy/
2022/03/28 23:26:58.109448 [INFO] Executing cleanup logic
2022/03/28 23:26:58.109570 [INFO] CommandService Response: {"status":"SUCCESS","api_version":"1.0","results":[{"status":"SUCCESS","msg":"Engine execution has succeeded.","returncode":0,"events":[{"msg":"Instance deployment completed successfully.","timestamp":1648510018,"severity":"INFO"}]}]}
And the app deployment is complete:
2022/03/28 23:26:58.109728 [INFO] Platform Engine finished execution on command: app-deploy
You can see the details of what takes place in eb-engine log.
I am attempting to utilize lein new template build-out behind a corporate proxy or local maven mirror with following failure:
C:\development\clojure> lein new luminus guestbook +h2
Failed to resolve version for luminus:lein-template:jar:RELEASE: Could not find metadata luminus:lein-template/maven-metadata.xml in local (C:\Users\username\.m2\repository)
This could be due to a typo in :dependencies or network issues.
If you are behind a proxy, try setting the 'http_proxy' environment variable.
Could not find template luminus on the classpath.
Placing the following lines in profiles.clj appear to have no effect:
:mirrors {"central" {:name "central"
:url "http://server.company_name.com:8080/artifactory/maven.central/"}}
Environment variables (upper / lower case appears to also have no impact):
HTTP_PROXY=http://username:password#proxy.company_name.com:8080
HTTPS_PROXY=https://username:password#proxy.company_name.com:8080
HTTP_NO_PROXY=*.company_name.com
HTTP_CLIENT=wget --no-check-certificate -O
Note: I have also specified HTTP/S proxy without username and password which results in same failure.
I have also not been able to determine how to produce debug level logging to assist in troubleshooting failure.
Leiningen 2.7.0 on Java 1.8.0_144 Java HotSpot(TM) 64-Bit Server VM
Clojure 1.8.0
I updated the file ~.m2/settings.xml which contains Maven configuration settings as follows:
<settings ...>
...
<profiles>
...
<profile>
<id>alwaysActive</id>
<repositories>
...
<repository>
<id>clojars</id>
<name>Repository for Clojure builds</name>
<snapshots>
<enabled>false</enabled>
</snapshots>
<url>https://clojars.org/repo</url>
<layout>default</layout>
</repository>
</repositories>
</profile>
</profiles>
<activeProfiles>
<activeProfile>alwaysActiveProfile</activeProfile>
</activeProfiles>
</settings>
It appears that this version of Lein adds Clojars repository search path for resolving dependencies but not for project templates. By making this Maven configuration change it causes Maven called by Lein to search the additional Clojars repository automatically. This appears to be inconsistent behavior which will hopefully be resolved in a future version of Lein.
I found the luminus:lein-template:2.9.11.90 artifact at clojars.org and retrieved it manually using Maven into my local M2 repository.
C:\development\clojure> mvn dependency:get -DremoteRepositories=https://clojars.org/repo -Dartifact=luminus:lein-template:2.9.11.90
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Maven Stub Project (No POM) 1
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-dependency-plugin:2.8:get (default-cli) # standalone-pom ---
[INFO] Resolving luminus:lein-template:jar:2.9.11.90 with transitive dependencies
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.160 s
[INFO] Finished at: 2017-09-30T16:58:11-04:00
[INFO] Final Memory: 11M/216M
[INFO] ------------------------------------------------------------------------
Once this artifact was available locally I was able to successfully use the template to create the project:
C:\development\clojure> lein new luminus guestbook +h2
Generating a Luminus project.
After this completed and moving to the new project directory, I was successfully able to download dependencies and run the project using "lein run".
This does not solve the problem of lein using a proxy configuration when creating a project from a template but does provide a workaround.
I am creating the new device from the documentation provided in this link (https://github.com/wso2/carbon-device-mgt-maven-plugin.git).
I performed the following steps
Step 1: Installing the Maven Archetype. Everything went okay! The maven archtype installed was
git clone -b v1.0.0 --single-branch https://github.com/wso2/carbon-device-mgt-maven-plugin.git
In Step 2: Creating a new device type, when I perform the command mvn archetype: generate -DarchetypeCatalog = local. The output does not show me the archetype for me to choose. Look at the output of this command:
C:\Users\eliazar.carvalho\Documents\Tools\WSO2\wso2iot-3.0.0\samples>mvn archetype:generate -DarchetypeCatalog=local
[INFO] Scanning for projects...
[INFO]
[INFO] --------------------------------------------------------------------- ---
[INFO] Building Maven Stub Project (No POM) 1
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] >>> maven-archetype-plugin:3.0.0:generate (default-cli) > generate-sources # standalone-pom >>>
[INFO]
[INFO] <<< maven-archetype-plugin:3.0.0:generate (default-cli) < generate-sources # standalone-pom <<<
[INFO]
[INFO] --- maven-archetype-plugin:3.0.0:generate (default-cli) # standalone-pom ---
[INFO] Generating project in Interactive mode
[INFO] No archetype defined. Using maven-archetype-quickstart (org.apache.maven.archetypes:maven-archetype-quickstart:1.0)
Choose archetype:
Your filter doesn't match any archetype (hint: enter to return to initial list)
Choose a number or apply filter (format: [groupId:]artifactId, case sensitive contains): :
What could be going wrong?
I am using WSO2 IoT Server 3.0
and OS: Ubuntu 14.04 LTS
I also faced the same issue. This is how I fixed it.
mvn archetype:generate -DarchetypeCatalog=local -X
Will give you the exact local catalog file path it is being read. For me it was ~/.m2/repository/archetype-catalog.xml.
But actualy my local repo catalog file is on ~/.m2/archetype-catalog.xml. So I copied archetype-catalog.xml into the correct path with following command.
cp ~/.m2/archetype-catalog.xml ~/.m2/repository/
Now it works fine. It seems we need to update maven-archetype-plugin version in mentioned repository.
WSO2 IoT 3.1.0 is released and it includes 3 ways of introducing a new device type.
Writing Java Extension using maven archetype
Descriptor based model
API based model
Please refer more information here
I have checked out the platform 4.0.3 tag, and have hit an issue with the SecVerifier when running mvn eclipse:eclipse.
I've cleaned the maven cache for SecVerifier:
rm -rf %USERPROFILE%\.m2\repository\org\wso2\carbon\SecVerifier
rm -rf %USERPROFILE%\.m2\repository\org\wso2\carbon\integration
Then tried mvn eclipse:clean eclipse:eclipse:
C:\wso2\src\wso2carbon_platform_tags_403>mvn eclipse:clean eclipse:eclipse
[INFO] Scanning for projects...
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]
[ERROR] The project org.wso2.carbon:SecVerifier:4.0.0 (C:\wso2\src\wso2carbon_platform_tags_403\products\integration\security-verifier\pom.xml) has 1 error
[ERROR] Non-resolvable parent POM: Failure to find org.wso2.carbon:integration:pom:4.0.0 in http://repo.maven.apache.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced and 'parent.relativePath' points at wrong local POM # line 22, column 13 -> [Help 2]
I found this JIRA: link, and applied the fix. Similar error:
[ERROR] The project org.wso2.carbon:SecVerifier:4.0.0 ... has 1 error
...
[ERROR] Non-resolvable parent POM: Could not find artifact
org.wso2.carbon:integration:pom:4.0.0 in central (http://repo.maven.apache.org/maven2) and
'parent.relativePath' points at wrong local POM # line 22, column 13 -> [Help 2]
Any ideas?
According to the error log maven cannot find the parent pom.xml file since it is unavailable on the given location. You can check your checkout code and figure out the actual location and set the relative path correctly.
This need to be work correctly but due to some changes done by users sometimes builds can be break.
Hope this might help you.
Thanks.
I'm trying to build Neo4J , I have downloaded the source from github: neo4j-master.zip
containing the source,
when i build this I hit upon following errors. tried to find out in google but
couldn't relavant details, I'm new to this product and i'm sure I have missed something
related to .git can any one help me if identifing this problem . Thanks in adavance.
Messages from Maven:
[INFO] Neo4j ............................................. SUCCESS [8.300s]
[INFO] Licensing configuration ........................... SUCCESS [3.338s]
[INFO] Neo4j - Graph Database Kernel ..................... FAILURE [2.371s]
.
.
.
.
[ERROR] Failed to execute goal pl.project13.maven:git-commit-id-plugin:2.1.4:rev
ision (generate-git-hash) on project neo4j-kernel: .git directory could not be f
ound! Please specify a valid [dotGitDirectory] in your pom.xml -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e swit
ch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please rea
d the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionE
xception
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
I'm using mvn clean install command for initiating the build.
Regards,
Aslam.