Maven multi-module deploy to repository only after successful unit tests - unit-testing

Question: What is the best solution for executing a 'mvn deploy' such that the deploy part is only run after all unit tests succeed and no processing steps are duplicated?
I was hoping the simple answer was: Execute maven command 'x' (or use a flag) such that the deploy can be run without invoking the prior goals in the default lifecycle.
Sadly this does not appear to have a simple answer. I have included the details on the path I have followed so far below.
We have the following three requirements:
Execute the maven deploy goal to deploy all multi-module artifacts to a remote repository.
Only deploy if ALL unit tests across all projects pass.
Do not repeat any processing.
We started with simply "mvn clean deploy", however we noticed a couple issues:
the build would stop before completing all unit tests :: so we added the --fail-at-end flag
The deploy goal would execute against any modules that were successful.
This results in a "corrupted" state where the remote repository may only has a partial deployment (if there were modules with failures later in the build).
We looked at 3 different solutions:
Staging the artifacts prior to deploying :: this was determined to be too heavy for a fully automated process.
Use a profile to override the default lifecycle such that 'mvn deploy -Pci-deploy' would run without invoking any prior goals :: this worked and was fast, but is obviously an unconventional approach.
Simply running 'mvn clean package' and then only iff successful execute 'mvn deploy' :: this appears to work and seems to only take a minor hit when the goals are invoked (though some of them are smart enough not to reprocess an unchanged workspace)
I pose this question to the community with the background details I have provided to determine if there is a better approach or a strong opinion regarding (potentially) making one of the following requests:
A new deploy goal that can run separate and apart from all other lifecycle goals with the expectation that: all prior steps have already been run and that it will execute the deploy identically to "mvn deploy"
a flag in the deploy goal which would effectively disable the previous goals.
a little more out of the box and definitely against the current convention:
a flag that would tell maven to run the [unit] test goal for all modules prior to proceeding.
Notes:
We are using Jenkins, but for the purposes of this question the CI environment is not the complication.
I tried the 'mvn deploy:deploy' goal, but it had a number of unclear errors.
I have not considered integration tests as part of the requirements.
Update 8/20/2013
I tested the deferred deploy plugin and determined that the tool worked as expected, but took way to long.
For our code base:
mvn clean deploy: for all goals executed in 2:44
mvn clean install 'deferred-deploy-plugin': for all goals executed in 15 min
mvn clean package; mvn deploy -Pci-deploy a custom build profile that disables the earlier goals executed:
for all goals (including deploy): 4:30
deploy only: 1:45
mvn clean package; mvn deploy -Dmaven.test.skip=true on the same workspace executed:
for all goals (including deploy): 4:40
deploy only: 1:54
The clean package followed by deploy skipping the tests runs faster than the deferred deploy and accomplished our desire to delay the deploy until after the tests succeed.
There appears to be a minor time hit for when the deploy lifecycle executes and exits each of the preceding goals (process, compile, test, package, etc). However the only alternative is to hack a non-standard execution, which only saves 10 seconds.

There's a new answer now. Since version 2.8 of the maven deploy plugin there's a way to do this "natively". See the jira issue for details.
Basically you need to force at least v2.8 of the plugin
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-deploy-plugin</artifactId>
<version>2.8</version>
</plugin>
and use the new parameter deployAtEnd. more info here. This setting usually goes along with installAtEnd of the maven-install-plugin

As an alternative, I also found this
http://code.google.com/p/maven-deferred-deploy-plugin/
A maven plugin that iterates through all projects in a reactor and
executes a deploy on each project individually. Can be used to produce
a near-atomic build for a reactor by deferring artifact deployment
until the install phase has completed.
Sounds alot like what you were asking for. I still think my other answer is easier to implement since you use jenkins, just check a checkbox

Two things.
Disabling all the previous phases i don't see it as an option. It is a basic feature of maven, you would be altering the standard lifecycle so i highly doubt anyone would implement something in a plugin to allow this
Since you said you use Jenkins, there is a setting in jenkins specifically for the case of deploying at the end to guarantee that the repo is not in a corrupt/intermediate state
In "Post-build actions"
Deploy artifacts to a Maven repository. In comparison with the
standard mvn deploy, this feature allows you to deploy artifacts after
the entire build is confirmed to be successful.
This prevents a typical problem in Maven, where some modules are deployed before a critical failure is discovered later down the road,
rendering the repository state inconsistent.
Note that regardless of this configuration, you can always manually come back to Jenkins and deploy any of the past artifacts to
any repository of your choice, after the fact.
To use this feature you shouldn't deactivate the automatic artifact archiving.
I have never used this so i can't confirm whether it works, I just know it's there for this particular use-case

Related

When should I execute unit tests and integration tests in a Dockerfile with Flask installed?

I set up a new Flask Python server and I created a Dockerfile with all my codes. I've written some unit tests and I'm executing them locally. When should I execute them if I want to implement a CI/CD?
I also need to write integration tests (to test if I'm querying the database correctly, to understand if the endpoint is exposed correctly, and so on), when should I execute them in a CI/CD?
I was thinking to execute them during the docker build so to put the execution of the tests in the Dockerfile. Is it correct?
Unit tests: Outside of Docker, before you run your docker build. Within your CI pipeline, after checking out the source code and running any setup steps like installing package dependencies.
Integration tests: Launched from outside of Docker; depending on how complex your setup is, either late in your CI pipeline or as part of your CD pipeline.
This assumes a true "unit test" that has no external dependencies; it depends only on the application/library code, and where it needs things like databases, it either mocks out those dependencies or uses something like an embedded SQLite. (Some frameworks are especially bad at this workflow and make it impossible to start up the application at all if the database isn't available. But Rails doesn't run on Python.)
Running unit tests in a Dockerfile will last until it's midnight, you have a production outage, and either your quick fix that will bring the site back up happens to break one obscure unit test, or you can't wait the 5-minute cycle time to run the whole unit-test suite. Since there shouldn't be dependencies on the Docker-or-not environment in your unit tests, I'd just run them outside Docker.
Often you can stand up enough infrastructure to be able to run your application "for real" with a couple of docker run commands or a simple Docker Compose setup. In that case, it makes sense to run an integration test towards the end of your CI pipeline. With a more complex setup (maybe one involving Kubernetes) you might need to actually deploy into a test environment, and if you have separate CI and CD tools, this would turn into "test deploy", "integration test", "pre-production deploy".
As a developer I find having tools not-in-Docker vastly easier to manage than tools that only run in Docker. (I don't subscribe to the "any binary other than /usr/bin/docker is bad" philosophy.) I'd rather just run pytest or curl than remember the 4-line docker run invocation to do some specific task.

Webpack: Should I build bundle on production server or build it locally and then upload?

I am deploying a React app on AWS Elastic Beanstalk. I bundle the app using webpack. However, I'm slightly confused about what best practices are from the production build process. Should I build the app locally (with NODE_ENV=production) using webpack, and then just upload the resultant bundle.js file, along with all node_modules to the Elasticbeanstalk instance? Or, should I upload all the source files, and run webpack on the actual cloud AWS server during deployment?
You should never build for production locally (unless you're the only developer).
Ideally, you have a build process that gets triggered manually or automatically from a git commit which then builds your project for production for you.
By using a centralized build process, you can then be sure that all your builds are built the same way (e.g. same node version, same npm or yarn version).
Both approaches are not really good to be honest. Local building is not a best way to build anything you want to have on production. You might have packages locally that may have inpact on what you're building. Same applies to the OS your doing it on.
And, again, same applies to the building during deployment. As the name of 'deployments' stands for, it's deploying. Just placing your application setup on the server so it may serve as it is supposed to.
That's the point where all CI/CD comes in. Having those kinds of solutions guarantee that each build is done with the same steps and on the same solution stack. No difference between each build is desired, because it allows you to assume that any bug or a change comparing to the 'desing' is because of the code, not environment it was build within.
Assuming that you're the only developer here (because you're asking for such a thing), CI/CD might be definitive overkill here, so just create shell script with steps and use Docker as the environment for build, so it stays the same between each build. That's the closest to the CI/CD option you can get without a hassle.

Fabric task dependencies

I am working on a fabric file to make our code deployment process a little bit easier. Now I would like to have dependencies between certain tasks, similar to what is discussed in one here.
Let's simplify the problem and say I have two task: build and deploy. The build task should build our code and the deploy task will transfer it to a deployment server.
Now, deploy obviously depends on build, but build could also be a standalone task. So someone could just build the code with fab build or deploy the code with fab build deploy. But I also want people to use fab deploy for convenience, but then it should run build first. But build should only be executed once.
So if I include build into the deploy task and then do fab build deploy it will run build twice and then deploy.
I managed to do this with the runs_once decorator and execute function.
The build task is now decorated with runs_once and every task that depends on build, e.g. deploy, will do execute(build) at the beginning. This will execute the build task or silently fail if it was already executed (thanks to the decorator).
This is more like a workaround than a solution but it works in my case. Regardless, thanks to everyone for their input

How to set up TeamCity for Build -> Test -> Deploy flow

I have configured TeamCity with Git to get my ASP.NET MVC project.
My solution contains the web app and the corresponding unit tests:
MY_SOLUTION.sln:
- WebAppProject
- SomeCoreLibrary
- SomeCoreLibraryTests
- OtherProjects...
The steps that I have configured in TeamCity are the following:
Get external packages using NuGet
Build the solution and deploy it
Run Unit Tests
Run Automated Tests (using Selenium)
I want to run the unit tests after building but before deployment and stop deployment if the unit tests failed. Currently the deployment is done after the build using the following Command Line Parameters:
/p:VisualStudioVersion=11.0
/p:DeployOnBuild=true I want this to be done only after SomeCoreLibraryTests.dll unit tests have passed
/p:PublishProfile=MyWebDeploy
/P:AllowUntrustedCertificate=True
/P:UserName=username_here
/P:Password=password_here
Thanks,
Ionut
What I've done in similar cases is to use RoboCopy to just mirror the new website into the deployment path. Doesn't that work for you?
P.S.: if you do get this working, I'd suggest doing a performance improvement change in TeamCity (which would allow you to run the unit tests in parallel to the automated tests):
I assume you are employing a single build configuration for all those steps. If that is the case, what I would recommend instead is using Dependent Build configurations to separate the different concerns. You can see an example here in an open source project of mine:
http://teamcity.codebetter.com/viewLog.html?buildId=112432&buildTypeId=bt1075&tab=dependencies
Log in as Guest and expand the Testeroids :: Publish to NuGet tree node to visualize the build flow.
To achieve this, basically you pass around the result of your build step in the artifacts (e.g. you pass the resulting binaries from Compile into Unit Test). You gain several things by using dependent builds: several independent build steps can run in parallel on different agents, plus if one of your build steps fails because of external factors (e.g. let's say Publish failed because the network went down), you can trigger again the build and it will only rebuild the failed steps.
I am not familiar with the tools that you use. However, I would, in general, use a few build configurations for a project:
build configuration, triggered on change, containing these steps: get the latest source code and packages, build/compile and unit test. Then create an artifact for deployment task.
build configuration to deploy to a development server, triggered by successful completion of and using artifact (via dependency) from (1).
build configuration for long running (eg integration/functional) testing that is scheduled to run less frequently.
An advantage of (2) is that you can, if necessary, re-deploy a build/artifact without having to rebuild the artifact first. Also, if you have multiple agents, (2) and (3) can run independently of each other.
Furthermore, you can also tag build in (2) that have passed development checks and then use its artifact in another build configuration to deploy it to test server, etc.

Can I get TeamCity to DISABLE a build if a different build has failing tests?

I have a TeamCity job that builds my project and runs all the unit tests, and another one that deploys the build to the production server.
Can I disable the "deploy" job so that it's impossible to deploy code if there's currently a failing test in the build project?
Shouldn't the deploy job already be dependent on the build one? Through Artifacts Dependency? You can set up the build trigger for the deploy job to be a successful build trigger on the build job so that the deploy happens when there is a successful build. Also, if a deploy job is triggered, it will take the last successful build. So if unit tests in the build job are failing, that build is not considered.
I wouldn't recommend snapshot dependency though, because it means when you deploy you try to trigger a new build, that is not the logical flow. Of course in the snapshot dependency you can say trigger only if a suitable build is not available ( or something like that ) but still snapshot dependency is not the way to go for this case.
Set up a Snapshot Dependency for Deploy on the most recetly finished Build and make sure the properties to say reject if failed [and make sure Build fails if tests fail in the General settings]