In VS2013 you can run the compiler for native code with the /analyze flag that will generate .xml files holding the output of the analyze. This will be interpreted by the UI and shown to the developer.
Is there a solution on how to integrate this into a Jenkins build or are any tool which can read such .xml files like the vc.nativecodeanalysis.all.xml and display it as a web page?
Jenkins is essentially a dashboard that outsources tasks to other tools to "do their thing".
With .NET builds your only option is a freestyle build with heavy use of Windows Batch command post build steps or MSBuild steps. Jenkins only knows the path to MSBuild that you tell it in "Manage Jenkins > Configure System"
After that it will outsource msbuild Post Build steps to MSBuild with the parameters you pass it.
Jenkins can consume JUnit test results and many other tools are written to convert test results into JUnit for Jenkins to consume. The mountain is brought to Jenkins rather than the other way around.
If MSBuild doesn't produce a graph, Jenkins unfortunately won't be able to.
You might keep an eye on the Static Analysis plugin over time to see if it adds support for this.
Related
So after much hunting I failed to find a continuous testing tool for IntelliJ 14.
I stumbled across a post that references uses eclipse and Ant in order to simulate this. On save, Ant then runs the tests for any tests that were modified.
I've tried to replicate this but, alas! I've never used Ant before and am finding it extremely difficult. I've setup and configured a generic Ant build file in Intellij but simply cannot figure out how to achieve my task.
Any help, pointers in the right direction is very much appreciated. I've searched but only found information that needs to be decrypted first.
Eclipse has the builder feature, you create an AntBuilder for your project, see also https://stackoverflow.com/a/15075732/130683.
IntelliJ has a trigger feature that might serve the purpose.
Also Infinitest , which provides a Continous Testing Plugin for Eclipse and IntelliJ might be helpful.
Ant is a build tool. Although IntelliJ does that for you, you need IntelliJ to do this which means you can't distribute your application without IntelliJ.
Ant uses a dependency matrix for building. This is sometimes difficult for developers to understand, but it basically means that you define the steps, how the steps are dependent upon each other, and let the build tool figure out exactly how to do its job. Ant is for Java like Make is to C and C++ applications.
Ant uses targets which are the steps you specify to do. For example, you might have a target called package that will build your jar or war. That target might depend upon another target called compile to compile the code. That target might depend upon a code generation phases (like if you had WSDL files).
Each target is a set of tasks. For example, the compile target is likely to have the <javac> task in it. It might also need the <mkdir> task to create the work directories where you classfiles are stored.
There are plenty of books on Ant, and there's a tutorial on the Ant Website. You didn't explain the issues you were having, so it's hard to be more specific than this.
Ant can also run your unit tests too. There's a <junit> target which can run the tests, and you specify whether or not you want to run almost all of your <junit> tests via the <batchtest> sub-entity or if you have a program driver you specify via the <test> entity.
Once you get an Ant script that can build and run your tests outside of IntelliJ, you can now get a Continuous Integration tool like Jenkins. A continuous integration tool watches your repository for changes, and if a change occurs, will then build your application. It's a great way to catch errors early on.
What does this have to do with Continuous Testing? Well, if you have your Ant script able to run unit tests, the Continuous Integration engine not only can build your app, but then run the unit tests with each and every change that occurs.
Jenkins is nice because it's very simple to use. You download a jenkins.war and you can launch the Jenkins webpage via the java -jar jenkins.war command. This brings up a web server on port 8080 on your machine. Obviously, Jenkins can be configured to run on different ports and under Tomcat if you so desire. It can integrate with Windows Active Directory, LDAP, and many other user verification systems.
Jenkins will show you charts and graphs of your tests, let you know which tests failed or passed, and will notify you of any problems via email, tweets, IM, Jabber, and even Facebook posts. People have even setup a traffic light in their offices that turns red when builds or tests fail.
Take it one step at a time. Get a good book on Ant. Read the tutorial on the Ant website. Then try to get a working Ant script to just to build your app. If you are having specific issues, you can ask for help.
Once you have the build going, extend the script to run your unit tests. Once that is done, download Jenkins and try to get that up and running.
I am trying to improve our general automation process. We use VS2012 and TFS2012.
Here is what I want to happen upon checkin to our CI branch:
BUILD
Build the selected projects / solutions as configured in the build definition settings.
Generate a deployment package that can be used to deploy the websites (without having to rebuild the entire project again)
Generate a nuget package that can later be published (without having to rebuild the entire project again, i need the dlls to match the symbols created from indexing so we can debug them)
TEST - IF AND ONLY IF BUILD WAS SUCCESSFUL
Run all configured unit tests.
DEPLOY - IF AND ONLY IF ALL UNIT TESTS PASS This is to prevent breaking changes entering our development environment
Take deployment package from (1.2) and publish it to it's intended environment (hopefully configured using Publishing Profiles and transforms)
PUBLISH - IF AND ONLY IF ALL UNIT TESTS PASS
Take nuget package from (1.3) and publish it to our private nuget gallery
I don't need a full tutorial (although that would be awesome) for the entire process, but more how to go about integrating it.
For instance:
Should I use msbuild on a wrapper project?
How do I deal with creating the packages upon build on the TFS build server?
How can I enforce the "IF AND ONLY IF ALL UNIT TESTS PASS" constraints?
What is the best / easiest way to perform the deployment /publishing after as part of the build.
This is the process we want to use and any help is realising this is very much appreciated.
And I'm sure many other people are interested in how to set about integrating this style of process.
Also if it's relevant most solutions have a mix of shared dll projects, websites / apis, and unit tests. One of the reasons I want this process is to be able to split them up and modularise our large dlls into smaller isolated units, which would be to unmanageable ATM without this auto publish mechanism.
Thanks,
Gary.
BUILD Build the selected projects / solutions as configured in the build definition settings. Generate a deployment package that can be
used to deploy the websites (without having to rebuild the entire
project again)
This is out of the box, add deployment profile to your projects, call them 'Release'
Add the following to your MSBuild Arguments
/p:DeployOnBuild=true;PublishProfile=Release
you don't have to use Release, as long as your Publish Profiles match what you put in the MSBuild arguments
This will generate the deployment files as part of your build (MSDEPLOY)
Generate a nuget package that can later be published (without having to rebuild the entire project again, i need the dlls to match the symbols created from indexing so we can debug them)
See Nugetter on code plex http://nugetter.codeplex.com/
TEST - IF AND ONLY IF BUILD WAS SUCCESSFUL Run all configured unit
tests.
Should be out of the box, but you can change the build template to fail the build should compilation be unsucessful, if this suits your needs better.
DEPLOY - IF AND ONLY IF ALL UNIT TESTS PASS This is to prevent
breaking changes entering our development environment Take deployment
package from (1.2) and publish it to it's intended environment
(hopefully configured using Publishing Profiles and transforms)
PUBLISH - IF AND ONLY IF ALL UNIT TESTS PASS Take nuget package from
(1.3) and publish it to our private nuget gallery
See Nugetter on codeplex as listed above
Hopefully I find here someone who has experience with Hudson and its functions.
Now . I have Hudson installed this did not reveal any problems. But now I want to create a new job and that I'm developing in C / C + +.
In addition, I am working on Subversion svn where I run on the first error. Hudson did not find my svn . He says that I need an authentication . As I learned I can at Hudson authenticate but that does not work .
Maybe one of you knows how to create a project.
The things should be done in the job of Hudson.
Hudson is on my computer (local ) delete my project.
Then Hudson to access my SVN and check out the project from there.
The whole is now compiling Hudson . ( The best would be a compiler for C / C + + for Visual Studio 2008 compiler ) . The compiler then creates a * . Exe file.
Now Hudson to start the project on the basis of the *. Exe file and run the program .
Last but not least is to Hudson case of an error or if it was all right, inform the persons working on the project via email.
So that would be it what I 've hoped of Hudson. Otherwise, I take the whole not much. I know that I can do all this via a batch file . But that's not my goal. I want Hudson to automate so that I can start at midnight my builds / tests daily.
Do you think that at Hudson are my requirement too high?
For your help I would be very grateful , as I am stuck for days.
Here is a "basic" Hudson job
Create a new free-style software project job.
Configure that job.
(Optional) Configure triggers, such as "timer", "SCM polling", or others.
(Optional) Under Source Code Management section, select your SCM source and configure your repositories and local workspace
Under Build section, select Add build step and select:
Execute Shell if on *nix
OR
Execute Windows Batch Command if on Windows
OR
Pick whatever build-step plugin you are using.
(If using either of the "execute" build steps) Write your build/make/compile command as you would from command line.
(If using another plugin build step) Configure the plugin options according to your requirements.
(Optional) Archive the artifacts of the build with Archive the artifacts under Post-build Actions
(Optional) Execute other post-build actions
(Optional) Send out an email
Now to address your specific scenario. First things first, your question is too broad, and may get locked. Don't get discouraged if that happens, create separate question for each item individually. I cannot cover in details all these items, but I will give you an overview.
The SCM part
Based on your previous question, No Credentials to try in Hudson, I am now guessing that you are not providing Hudson with an HTTP URL to your SVN server, but trying to give it your local workspace location... Please do the command line check that I asked in that question.
You need to provide it with a proper HTTP server URL. Hudson will check out the project from the SVN URL you provided, under what is called a Workspace. The location of workspace can differ, based on your Hudson configuration, but it is a folder inside Hudson installation that is dedicated to the job. It can be referenced from within the job through %WORKSPACE% environment variable.
There are ways to use a different workspace location, but that is outside the scope of this overview. The whole SCM part is also optional, you can rely on existing file system, but this is not a good approach, and again, out of scope of this overview.
The Build step
After Hudson checked-out/updated the Workspace with your SVN, comes the building step. Hudson can do Execute Windows Batch Command by default. It can also Invoke Ant by default. (It can also do Maven, but that is not applicable to your situation)
To do other types of builds, you need a Build Wrapper plugin. In your particular case, the MSBuild plugin is probably what you want. I've never used MSBuild, so cannot give you details. Again, if you have a specific question on how to use MSBuild plugin, you should probably make a separate question with specific issues.
So, using either Execute Windows Batch Command or MSBuild plugin, configure your building step.
Running the exe???
This is very vague. You want to start the .exe and then what? Will it quit and you need an exit code? Do you want to see it on the screen? Again, this is very broad, and deserves a separate question (or read existing questions). If you just want to make a call to the .exe, you can configure a second Execute Windows Batch Command step, type there call path\to\yourfile.exe. But most likely you will not see that on screen. Read my answer here, Open Excel on Jenkins CI, on details of launching an .exe from Hudson/Jenkins that would be visible on screen.
Email
If you want a simple email, Hudson Post-Build actions has a way to send an email. For better customization options, you would want Email-Ext plugin. Once again, if you need details on how to use the email-ext plugin, create a new question (after searching existing questions first), as this is too much to cover in one question.
Conclusion
Your requirements are not too high, but Hudson is not a magic tool that will do the work for you. You still need to configure every step of it. And unless you have a Maven based project (which integrate very well with Hudson), a lot of actions will need to be done through the Execute Windows Batch Command and scripting of your own.
Is it possible to add memory/performance profiling of unit tests as a team city build step?
I am specifically interested in doing it for our .Net apps (test cases are written in NUnit). Any elaboration on this will be highly appreciated.
Teamcity version is "TeamCity Enterprise 7.1.2 (build 24170)"
Thanks.
As far as I know, TeamCity does not support profiling .Net apps yet. At least not in a direct way.
There is an open feature request TW-20190 planning to integrate dotTrace (a .Net profiler made also by JetBrains) in Teamcity.
But dotTrace can be called from command line, so you can write MSBuild/NAnt script to execute dotTrace. The problem is about how to display the profiling results. In "General Settings" page of your build config, you can point artifacts path to the dotTrace result folder, so that teamcity will publish the profiling results as artifacts for you to download.
I've a server running a proprietary language on which I'm able to run "unit tests" in this language. I cannot install a Hudson slave on this machine, but would like to have these tests results appearing in a job of hudson (to have at least a monitoring of the code quality for this server code).
I'm currently trying to use web services to get the results and store them in Hudson workspace, but I do fear it is not the right solution.
What solutions can you advice me ?
I finally have gotten through the web services path, although it was not easy.
There are some steps in this path
I created a maven mojo with groovy (see GMaven for more infos) which, using groovyws, called a web service that, from tests results, creates the junit report.
Armed with this mojo, I created a maven project that called the web service and stores the junit.xml file in an output folder
Finally, i created in hudson a maven job for this project and called it regularly. Thanks to junit reporting integration in maven builds, my tests results are visible as a graph in Hudson and user can drill down to failing tests.
Not sure if these are possible but...
Maybe one option is when the build job finished execute a second build target or script to scp the test results from the remote server to the local build server so they appear in hudson
Or if the platform allows
Map a directory on the remote machine to the local file system by using something like sshfs etc
karl
Yup, you can scp or whatever the results (in junit xml format) to the current workspace dir using a script task. Then have a "Publish JUnit test result report" post-build task & point it at the copied-in files.
Obviously if it's not in junit-compatible format you'll have to convert it.
Sounds like you're on the right path though