How can I generate incremental Maven build in Jenkins Job DSL? - jenkins-job-dsl

I'm trying to generate what appears in my Jenkins Maven job under Advanced... as
Incremental build - only build changed modules
This is an XML node that's directly within <maven2-moduleset>.
I didn't find it in the API, so I thought I'd use configure, but I can't figure it out. From what I understand, this should work:
mavenJob('foo') {
rootPOM('foo/pom.xml')
goals('clean package')
configure { node ->
node {
incrementalBuild('true')
}
}
}
However, I get an Exception:
groovy.lang.MissingMethodException: No signature of method: groovy.util.Node.call() is applicable for argument types: (Generator$_run_closure1_closure14_closure16) values: [Generator$_run_closure1_closure14_closure16#1f7d8eff]
Possible solutions: wait(), name(), value(), any(), wait(long), any(groovy.lang.Closure)
What am I doing wrong?

In this case you must use the / operator in the configure block to create or update an element, see Transforming XML in the Job DSL wiki.
mavenJob('foo') {
rootPOM('foo/pom.xml')
goals('clean package')
configure { node ->
node / incrementalBuild(true)
}
}

Related

Cannot build Corda (release/os/4.6) on Ubuntu 20.04 due to failing tests (CryptoUtilsTest & NodeH2SecurityTests)

I am trying to build Corda on Ubuntu 20.04. I have the latest sources from the git repo (release/os/4.6) and I run ./gradlew build in the main folder. However the build fails during two tests (see the detail description below). Is there something that I'm missing? Are there some special flags that I should use for building Corda?
First, the test test default SecureRandom uses platformSecureRandom fails at the last assert, i.e.,
// in file net/corda/core/crypto/CryptoUtilsTest.kt
fun `test default SecureRandom uses platformSecureRandom`() {
// [...]
// Remove Corda Provider again and add it as the first Provider entry.
Security.removeProvider(CordaSecurityProvider.PROVIDER_NAME)
Security.insertProviderAt(CordaSecurityProvider(), 1) // This is base-1.
val secureRandomRegisteredFirstCordaProvider = SecureRandom()
assertEquals(PlatformSecureRandomService.algorithm, secureRandomRegisteredFirstCordaProvider.algorithm)
}
The reason for the failed test is Expected <CordaPRNG>, actual <SHA1PRNG>..
For some reason, the test is successful if before inserting the provider, I call the getServices() method, i.e.,
val provider = CordaSecurityProvider()
provider.getServices()
Security.insertProviderAt(provider, 1) // This is base-1.
I also tried to get the service SecureRandom.CordaPRNG directly from the provider and it works, i.e,
println(provider.getService("SecureRandom", "CordaPRNG"))
prints out Corda: SecureRandom.CordaPRNG -> javaClass
Second, the test h2 server on the host IP requires non-default database password fails since it expects a CouldNotCreateDataSourceException, but it gets a NullPointerException instead, i.e.,
// in file net/corda/node/internal/NodeH2SecurityTests.kt
fun `h2 server on the host IP requires non-default database password`() {
// [...]
address = NetworkHostAndPort(InetAddress.getLocalHost().hostAddress, 1080)
val node = MockNode()
val exception = assertFailsWith(CouldNotCreateDataSourceException::class) {
node.startDb()
}
// [...]
}
The problem is that the address is 127.0.1.1:1080, which means that net/corda/node/internal/Node.kt::startDatabase() does not throw CouldNotCreateDataSourceException since the condition to enter the branch
if (!InetAddress.getByName(effectiveH2Settings.address.host).isLoopbackAddress
&& configuration.dataSourceProperties.getProperty("dataSource.password").isBlank()) {
throw CouldNotCreateDataSourceException()
}
is not satisfied. Instead it calls toString() on the parent of the path given by the DB name; the parent is null, and thus, it throws NullPointerException, i.e.,
val databaseName = databaseUrl.removePrefix(h2Prefix).substringBefore(';')
// databaseName=my_file
val baseDir = Paths.get(databaseName).parent.toString()
Unfortunately there't a LOT of reasons that building Corda from source might not work.
Here are my recommendations:
it could be a java issue, there's a docs page on the specific version of java 8 that you need to use, (latest java support is on the roadmap for corda 5 🔥) Here's the docs page on that https://docs.corda.net/docs/corda-os/4.5/getting-set-up.html
Like Alessandro said, you'll want to be aware that 4.6 isn't generally available yet so there may well be bugs and problems in the code until the release. In addition just take another look at the docs page on building corda (here: https://docs.corda.net/docs/corda-os/4.5/building-corda.html#debianubuntu-linux), it's mentioned to use 18.04 but not the latest linux, as there might be some random clib things that could get in the way there.
Good luck to you.

ignoreCommitterStrategy is not working in mutibranch job-dsl generator job

enter code hereI'm trying to implement ignoreCommitterStrategy approach via multibranch generator job (i.e, job-dsl way)
I'm trying to implement ignoreCommitterStrategy approach via multibranch generator job (i.e, job-dsl way)
since we have too many existing multibranch-pipeline jobs and I try to achieve ignorecommitter stragegy inside branchsources of dsl.
After running seed job (i.e., multibranch generator job) I could see ignoreCommiter Strategy updated in existing multibranch pipeline jobs but still ignored author is not added. It means at this moment inside multibranch pipeline jobs -> config I have to manually click Add button and add ignored author list which is bit painful as we have many jobs.
buildStrategies {
ignoreCommitterStrategy {
ignoredAuthors("sathya#xyz.com")
allowBuildIfNotExcludedAuthor(false)
}
}
Note: I even tried with "au.com.versent.jenkins.plugins.ignoreCommitterStrategy"
Expecting IgnoredAuthor to be added into existing multibranch pipeline jobs upon execution of multibranch generator job
Currently the build strategy can not be set by using Dynamic DSL because of JENKINS-26535. But a fix is currently in review and will hopefully be released soon.
The correct syntax would be
multibranchPipelineJob('example') {
branchSources {
branchSource {
buildStrategies {
ignoreCommitterStrategy {
ignoredAuthors('test#acme.org')
allowBuildIfNotExcludedAuthor(true)
}
}
}
}
}
Until the problem has been fixed, you can use a Configure Block to set the necessary options.

Embedding Jetty 9.3 with modular XmlConfiguration

I am migrating from Jetty 8.1.17 to Jetty 9.3.9. Our application embeds Jetty. Previously we had a single XML configuration file jetty.xml which contained everything we needed.
I felt that with Jetty 9.3.9 it would be much nicer to use the modular approach that they suggest, so far I have jetty.xml, jetty-http.xml, jetty-https.xml and jetty-ssl.xml in my $JETTY_HOME/etc; these are pretty much copies of those from the 9.3.9 distribution. This seems to work well when I use start.jar but not through my own code which embeds Jetty.
Ideally I would like to be able to scan for any jetty xml files in the $JETTY_HOME/etc folder and load the configuration. However for embedded mode I have not found a way to do that without explicitly defining the order that those files should be loaded in, due to <ref id="x"/> dependencies between them etc.
My initial attempt is based on How can I programmatically start a jetty server with multiple configuration files? and looks like:
final List<Object> configuredObjects = new ArrayList();
XmlConfiguration last = null;
for(final Path confFile : configFiles) {
logger.info("[loading jetty configuration : {}]", confFile.toString());
try(final InputStream is = Files.newInputStream(confFile)) {
final XmlConfiguration configuration = new XmlConfiguration(is);
if (last != null) {
configuration.getIdMap().putAll(last.getIdMap());
}
configuredObjects.add(configuration.configure());
last = configuration;
}
}
Server server = null;
// For all objects created by XmlConfigurations, start them if they are lifecycles.
for (final Object configuredObject : configuredObjects) {
if(configuredObject instanceof Server) {
server = (Server)configuredObject;
}
if (configuredObject instanceof LifeCycle) {
final LifeCycle lc = (LifeCycle)configuredObject;
if (!lc.isRunning()) {
lc.start();
}
}
}
However, I get Exceptions at startup if jetty-https.xml is loaded before jetty-ssl.xml or if I place a reference in jetty.xml to an object from a sub-configuration jetty-blah.xml which has not been loaded first.
It seems to me like Jetty manages to do this okay itself when you call java -jar start.jar, so what am I missing to get Jetty to not care about what order the config files are parsed in?
Order is extremely important when loading the Jetty XML files.
That's the heart of what the entire start.jar and its module system is about, have an appropriate set of properties, the server classpath is sane, and ensuring proper load order of the XML.
Note: its not possible to have everything in ${jetty.home}/etc loaded at the same time, as you will get conflicts on alternate implementations of common technologies (something start.jar also manages for you)

Jenkins Job DSL always creates a git tag for freestyle jobs using the git plugin

I am creating a freestyle job with the Jenkins Job DSL. It appears to create a git tag every time. The default for scm -> git -> createTag appears to be "false", but this is deprecated. Was this turned ON elsewhere?
My code snippet is as follows (I added the createTag line to try to fix it, but it creates the add tag in "additional behaviors" anyway). Suggestions on how to fix?
scm {
git(buildRepoName, branchName) {
createTag(false)
}
}
If you use the git method with a closure parameter, the "create tag" option is disabled by default. See the API Viewer for details. See also JENKINS-33482.
job('example') {
scm {
git {
remote {
url('https://github.com/jenkinsci/job-dsl-plugin.git')
}
branch('master')
}
}
}

Gradle: How do I publish a zip from a non-java project and consume it in a java project?

I have a multi-project setup. I created a non-java project whose artifact is a zip file that I will unzip in another project. So the idea is as below
my-non-java-project
build.gradle
------------
apply plugin: 'base'
task doZip(type:Zip) { ... }
task build(dependsOn: doZip) << {
}
artifacts {
archives doZip
}
some-java-project
build.gradle
------------
apply plugin: 'war'
configurations {
includeContent // This is a custom configuration
}
dependency {
includeContent project(':my-non-java-project')
}
task expandContent(type:Copy) {
// Here is where I would like get hold of the all the files
// belonging to the 'includeContent' configuration
// But this is always turning out to be empty. Not sure how do I publish
// non-java content to the local repository (as understood by groovy)
}
So, my question is, how do I publish the artifacts of a non-java project to the internal repository of groovy such that I can pick it up at another java-based project?
Not exactly sure what you're after, but here's a quick-and-dirty way to get access to the FileCollection of the :my-non-java-project:doZip task outputs:
project(":my-non-java-project").tasks.getByName("doZip").outputs.files
Note that the archives configuration is added by the Java plugin, not the Base plugin. But you can still define a custom configuration in my-non-java-project and add the artifact to it with the code in your OP:
//in my-non-java-project/build.gradle
configurations {
archives
}
artifacts {
archives doZip
}
Then you can access the task outputs via the configuration, like so (again, quick-and-dirty):
//in some-java-project/build.gradle
project(":my-non-java-project").configurations.archives.artifacts.files
Note that you can expand the content of your zip file using zipTree.
If you need to actually publish the zip produced by my-non-java-project, you can read about that here.