Amazon Kinesis can't find main class in Flink job compiled with Maven - amazon-web-services

Trying to execute a Flink job, compiled with Maven, in Amazon Kinesis Data Analytics and I get the following error:
"org.apache.flink.client.program.ProgramInvocationException: Neither a 'Main-Class', nor a 'program-class' entry was found in the jar file."
I have tried several ways of defining the main class in the pom.xml-file.
Right now it looks like this:
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<manifestEntries>
<Main-Class>com.dataductus.maven.time_aggregator.TimeAggregator</Main-Class>
</manifestEntries>
</transformer>
Here is the pom.xml (the code gets weird when I format it in the comment):
https://textuploader.com/1cq7k
Here is the plugin:
https://textuploader.com/1cq7j
I use "mvn package" to package.
Any ideas?

You don't show the entire plugin configuration, or the command that you use to build the JAR, or the contents of the manifest file (if any), so there are many possible causes.
However, my guess is that you're using an older version of the plugin, so that the documentation here (which you seem to be following) may not apply.
The easiest solution is to follow the documentation here, has an explicit mainClass entry. I can assure you that works; I've been using it for at least ten years.

Thank you I will look at that.
Here is the pom.xml (the code gets weird when I format it in the comment):
https://textuploader.com/1cq7k
Here is the plugin:
https://textuploader.com/1cq7j
I use "mvn package" to package.

Problem solved. No idea what was wrong. I created another identical KDA-app which used the exact same jar and now it works. Strange.

Related

Clojure/leiningen cannot find dependencies

I am new to Clojure and Leiningen. Just started working on an existing project. I pull the repo and did the command Lein run. It complained:
"Could not find artifact arctype:service.jose:jar:0.1.0-SNAPSHOT in clojars (https://repo.clojars.org/)
Could not transfer artifact arctype:service.jose:jar:0.1.0-SNAPSHOT from/to enonic (https://repo.enonic.com/public/): Failed to transfer file https://repo.enonic.com/public/arctype/service.jose/0.1.0-SNAPSHOT/service.jose-0.1.0-SNAPSHOT.jar with status code 409"
What I proceeded to do, under the assumption that the dependencies are bad, is that I went on repo.clojars.org. Manually looked into the dependencies and changed "[arctype/service.jose "0.1.0-SNAPSHOT" :exclusions [arctype/service]]" in project.clj to "[arctype/service "0.1.0-SNAPSHOT":exclusions [arctype/service]]", because that is what it has on the site. But turns out that "Jose" is not inside of arctype/service, and changing the dependencies like I did removed it, so I have to change it back.
What im really confused about, is that, according to the error message, I went on repo.clojars.org to look for the dependencies I want, "arctype.service.jose" is no where to be found. Not even something close.
I saw the second half error message that mentions repo.enonic.com/public , so I try to look for the dependecy there as well. Same result, nothing. In my project.clj file, the repository is repo.enonic.com/public. I am really confused, can somebody please enlighten me! Thanks in advance!
Im not sure if I provided enough, please let me know!
Most open source projects written in Java publish artifacts on Maven Central, and most Clojure projects publish artifacts on Maven Central or clojars.org. The fact that your error message mentions https://repo.enonic.com/public/ might be because the artifact mentioned in your project.clj file was published on that server, not on clojars.org.
If I go to clojars.org (not repo.clojars.org) there is a nice web interface with a search box, and if I enter "arctype" (without the double quotes) and do a search on that, it finds arctype/service, but not arctype/service.jose, probably because arctype/service.jose was not published on the clojars.org site. Or, if it was, someone decided to remove it. Removing published things is fairly uncommon, so my guess is that if arctype/service.jose ever existed, it might have only been published on the repo.enonic.com site.
Is your project open source? Published on Github.com or a similar site where others can get a copy and try it out? If yes, including that in your question would help others see if they get the same error you do.
If the project you are working with is not open source, then hopefully someone more familiar with the particular errors you are seeing than I am will answer.

Show code coverage with a source code in Jenkins wiht Cobertura (run result from other machine)

Background
I have large c++ application with complex directory structure. Structure is so deep that code repository can't be stored in Jenkins workspace, but is some root directory, otherwise build fails since path length limit is busted.
Now since application is tested in different environments, test application is run in diffrent machine. Application and all resources are compressed and copied to test machine where tests are run using OpenCppCoverage and as a result Cobertura xml is produced.
Now since source code is needed to show covarage result xml is copied back to build machine and then feed to Jenkins Cobertura plugin.
Problem
Coverage reports shows only percent results for module or source code. Code content is not show, but this error message is show:
Source
Source code is unavailable. Some possible reasons are:
This is not the most recent build (to save on disk space, this plugin only keeps the most recent build’s source code).
Cobertura found the source code but did not provide enough information to locate the source code.
Cobertura could not find the source code, so this plugin has no hope of finding it.
You do not have sufficient permissions to view this file.
Now I've found this SO answear which is promising:
The output xml file has to be in the same folder as where coverage
is run, so:
coverage xml -o coverage.xml
The reference to the source folder is put into coverage.xml and if
the output file is put into another folder, the reference to the
source folder will be incorrect.
Problem is that:
I've run tests on different machine (this can be overcome by script which modifies paths in xml).
my source code can't be inside a workspace during a build time
placing xml in respective directory of source code is not accepted by Cobertura plugin. It ends with this error:
[Cobertura] Publishing Cobertura coverage report...
FATAL: Unable to find coverage results
java.io.IOException: Expecting Ant GLOB pattern, but saw 'C:/build_coverage/Products/MyMagicProduct/Src/test/*Coverage.xml'. See http://ant.apache.org/manual/Types/fileset.html for syntax
This is part of xml result (before modifications):
<?xml version="1.0" encoding="utf-8"?>
<coverage line-rate="0.63669186741173223" branch-rate="0" complexity="0" branches-covered="0" branches-valid="0" timestamp="0" lines-covered="122029" lines-valid="191661" version="0">
<sources>
<source>c:</source>
<source>C:</source>
</sources>
<packages>
<package name="C:\jenkins\workspace\MMP_coverage\MyMagicProduct\src\x64\Debug\MMPServer.exe" line-rate="0.63040511358728513" branch-rate="0" complexity="0">
<classes>
<class name="AuditHandler.cpp" filename="build_coverage\Products\MyMagicProduct\Src\Common\AuditHandler.cpp" line-rate="0.92682926829268297" branch-rate="0" complexity="0">
<methods/>
<lines>
<line number="18" hits="1"/>
<line number="19" hits="1"/>
<line number="23" hits="1"/>
<line number="25" hits="1"/>
<line number="27" hits="1"/>
....
</lines>
</class>
....
The biggest issue is that I'm not sure if location of xml is actual a problem since plugin doesn't report details of the issues encountered when trying to fetch/find respective source code. Second bullet from Cobertura which may explain problem is totally confusing:
Cobertura found the source code but did not provide enough information to locate the source code.
What else I've tried
I've ensured that anyone can read source code (to avoid problem with access)
I've modified xml so filename contains path relative to: jenkins workspace, path where xml file with coverity report is located
copied my source code to various locations, even containing "cobertura" directory since something like this I've found in plugin source code
I've tried understand the issue by inspecting source code.
I've found some (a bit old) github project which maybe a hint howto fix it - currently I'm trying to understudy what it exactly does (I don't want to import this project to my build structure).
So far no luck.
Update:
Suddenly (I'm not sure what I have done) it works for my account. Problem is that it works only for me all other users have same issue. This clearly indicate that issue must be a security.
I encountered a very similar issue when I had to develop a CI pipeline for a very huge C++ client. I had the best results if I avoided the Cobertura Plugin and instead used the HTML Publisher Plugin. The main issue I had was also finding the source files.
Convert OpenCppCoverage result to HTML
This step is quite easy. You have to add the parameter --export_type=html:<outputPath> (see Commandline-reference) to the OpenCppCoverage call.
mkdir CodeCoverage
OpenCppCoverage.exe --export_type=html:CodeCoverage <GoogleTest.exe>
The commands above should result in a html-file in the directory <jenkins_workspace>/CodeCoverage/index.html
Publish the OpenCppCoverage result
To do this we use the HTML Publisher Plugin as I mentioned above. reportDir is the directory created in step one and which contains our html-file. Its path is relative to the Jenkins workspace.
publishHTML target: [
allowMissing: false,
alwaysLinkToLastBuild: true,
keepAll: true,
reportDir: 'CodeCoverage',
reportFiles: 'index.html',
reportName: 'Code Coverage'
]
and to be sure that everyone can download and check the result locally we archieve the result of OpenCppCoverage:
archiveArtifacts artifacts: 'CodeCoverage/*.*'
You can see the result now in the sidebar of your pipeline under Code Coverage and the result will look like the following:
This is the solution that worked for me.
I hope this helps at least a bit. I can only advice do avoid the Cobertura Plugin. I wasted so much time try to fix it and recognize my sources...
Ok I've found reasons why I had a problems with this plugin.
xml from openCppCoverage is just correct. No changes are needed here to make it work (as far as sources are there where pdb file points to). Sources outside Jenkins workspace are not the problem here. When I copied executable from build machine to test machine, then run tests with openCppCoverage and copied result back to build machine it is just fine.
In job configuration any user which supposed to view code coverage has to have access to Job/workspace in security section. In my case I've enabled this for all logged in users. This covers last bullet point of error message.
Most important thing: build must be successful. I mean form beginning to the end. Doesn't meter if step containing call to cobertura plugin was successful. If any step (even in the future step) fails then cobertura will not show code for this coverage run. In my case build job was failing since one of tests was timing out. This was caused by openCppCoverage overhead which slows down tests by factor 3. My script was detecting timeout and killing one of tests.
I discovered that not successful build was a problem by accident. During experiments I noticed two cases when cobertura has shown source code:
I've rerun job and removed all steps but one responsible for publishing coloratura results
I run whole job such way it run a single test case which passed
Not sowing coverage if build is not successful is reasonable (if test failed then most probably wrong branch of code has been taken), but UI should indicate that in different way.
Conclusion
This is great example how it is important to report errors to user with precise details what went wrong and why. I wasted at least whole weak to figure out what is actually wrong which bullet point of error message is actually my case. In fact error message from plugin doesn't cover all reasons of not showing the code.
I will file report that plugin should give better explanation what went wrong.

Error: Program type already present: com.appsflyer.AFExecutor

I'm struggling to implement AppsFlyer on Android using Java.
I have looked into a couple of posts already such as this, this.
Here is the entire error message: [org.gradle.api.Project] AGPBI: {"kind":"error","text":"Program type already present: com.appsflyer.AFExecutor","sources":[{}],"tool":"D8"}
The version
AppsFlyer SDK: 5.+
Android Studio: 3.5.2
Situation
I have done till 4.1 of this guide so far so good.
On AndroidManifest.xml, the main class name of AF has implemented with android.name attribute.
On AndroidManifest.xml, receiver tag commented out (because in this phase I don't believe I do not need a precise data tracking feature.)
What I have tried.
./gradlew app:dependencies | less To find out AFExecutor in other dependencies
To exclude the program
implementation ('com.appsflyer:af-android-sdk:5.+'){
exclude module: 'com.appsflyer'
}
implementation ('com.appsflyer:af-android-sdk:5.+'){
exclude module: 'AFExecutor'
}
If you have any insights, I'd love to hear that.
Try ./gradlew clean, clean project and invalidate caches and restart. If does not help than delete all build and .idea folders, .iml files.
After exploring a bunch of dependencies, I found the solution. The reason was there was a conflict between com.appsflyer:af-android-sdk:5.+ and AF-Android-SDK.jar which had installed manually. After removing the JAR file and built again, I could make it at last! Thank you so much for sharing your experiences, however, the solution was simple!

Pentaho DI / Kettle / Spoon Configuration with External Jars

I am deveoloping a custom plugin for Pentaho DI / Kettle 4.4 and use custom Validation and XSLT jars as part of it.
Those jars are referenced in plugin.xml along with the plugin's jar itself and deployed in the same location as the plugin's jar.
When I try to load this plugin in Spoon GUI, I can see and run the plugin fine till I reach the reference to the external jars and after that I get ClassNotFOund Exception.
I have tried following so far but has not worked :
Add the expernal Jars to libext folder and expect those to be picked up automatically. - Failed. Jars not loaded.
Add -cp command line option to Spoon.bat and specify the Jar files - Did not work.
Turned on -verbose option and search everywhere in the output but the Jars are ignored.
Deployed the Jars next to the plugin's Jar file in the plugin's folder. - Jars are not recognized.
From Within the plugin, if I dynamically load (using Reflection and URL Class Loader) I am able to force the loading of the Jars and it works that way but I would like a simpler classpath fix.
Any class that I reference in the Jar through XSL externsion (XALAN-J Java Extension) is still not recognized / loaded and XSL terminates at the beginning of Extension call.
Searched Google and this site for something similar but did not find specific answers on why SPoon behaves so differently and avoids external code and content.
A similar issue is reported here though its too old and has not shown final conclusive fix. I tried all that it said. http://forums.pentaho.com/archive/index.php/t-77190.html
Requesting someone to help me identify whats missing. Thanks in advance,
Thanks to this seemingly obscure website and it fixed all pain.
http://www.arulraj.net/2014/09/how-to-access-external-java-libraries-with-in-pentaho.html
Basically, go to the launcher folder and edit the launcher.properties file to include any path, library etc. and it fixes all the issues I reported above. Wish Kettle documentatation had answers like this.

oracle integration repository package

I am new to Oracle and have been having trouble following instructions for importing a package into the integration repository.
The instructions include code for creating a package and body, but don't really tell me what to do with that code. It looks like I should run it in SQL Plus. Is that correct?
It also includes instructions for generating the ILDT file--but only in a generic way. Those instructions just refer me to the "SOA Gateway Implementation Guide". (http://docs.oracle.com/cd/E18727_01/doc.121/e12169/T511175T543269.htm) The command is:
$IAS_ORACLE_HOME/perl/bin/perl $FND_TOP/bin/irep_parser.pl -g -v -username=sysadmin itg:patch/115/sql:fndav.pls:12.0=/tmp/fndav.pls
I'm not sure what to use for the .pls file. Is that generated and then the ILDT file is also created?
Apologies for this newbie question.
I appreciate any help!
Thanks,
Sami
You probably figured this by now, as per oracle conventions .pls file is the package specification and .plb is for package body.