How to fail a TFS build based on Fortify scan results - build

From a PowerShell query, how can I get the count of new critical or high vulnerabilities from a Fortify security scan of .NET code? The query should not include any findings already on the SSC server that were documented as "not an issue" or "suppressed".
We have Team Foundation Server 2017u2. As part of our build and release process we execute HP/Micro Focus Fortify security scans and upload the results to the Fortify SSC server. We are trying to make the build/release fail whenever the scanner detects new critical or high findings in the code. We use the Micro Focus Fortify plugin for TFS to configure the scan step and upload to SSC: (Fortify TFS plugin). We added a PowerShell task afterward to attempt to query for findings and fail the build if needed.
The examples and suggestions we've found use the FPRUtility to query the .fpr file generated from our current scan. However, this current scan does not include any previously entered content from developers documenting false positives or suppressed issues. This results in our builds always failing.
I tried looking through the REST API docs but, while Swagger makes it easy to see the parameters and contracts, I can't find any good documentation stating what all the different controllers are or how I need to orchestrate a series of calls to get the data I want.

I found the answer I needed. After the scan and upload to SSC completes, you call the issues REST API from your PowerShell script in this format:
[host:port]/ssc/api/v1/projectVersions/[versionid]/issues?q=[fortify+priority+order]:high+OR+[fortify+priority+order]:critical&qm=issues
So a simplified PowerShell script to do this in a TFS build step looks like:
$jsonResults = Invoke-RestMethod -Method Get -Uri "https://{host:port}/ssc/api/v1/projectVersions/{projectVersionNumberHere}/issues?q=[fortify+priority+order]:high+OR+[fortify+priority+order]:critical&qm=issues"
$undocumentedFindings = $jsonResults.data | where {$_.primaryTag -eq $null}
if ($undocumentedFindings.Count -gt 0)
{
Write-Error "Fortify detected $undocumentedFindings.Count undocumented critical and high vulnerabilities. These findings must be remediated or documented before the build can continue."
}

Related

Concourse CI - Build results

I've not been able to find any way in Concourse to show a 'build summary page' as you get in Jenkins/TFS etc. In those tools you can see build history (OK/failures), build durations, unit test results, code coverage, various graphs etc - but Concourse just has build history which is simple log files.
There doesn't seem to be any extensions system or other way to achieve this.
I'd prefer to use Concourse for the pipelines and build-in-containers approach, but it's a hard sell to developers who see it as a step backwards.
Thanks
Paul
The containers/workers of your plan/build are spinned up and spinned down. Everything you want to keep you need to PUT to a resource. In the concourse ecosystem there are many resources created.
The results and output logs of your builds/jobs are by default available in concourse UI (ok/not-ok, time). See for example: https://ci.concourse-ci.org/teams/main/pipelines/main/jobs/build-fly/builds/1273 In the job itself you can expand and see the logs, so output log of your unit tests for example.
If you want to keep the sonar report you just install a resource like https://github.com/cathive/concourse-sonarqube-resource and you can simply run this after your unit test by using the PUT (and store it on your Sonar server). So indeed you have no html report you can keep on that specific build, but you can put it everywhere you need it to be by using/creating a plugin. Very simple and nicer because the whole Sonar overview of your project is where it belongs :)

Test Results Analyzer Plugin for GitLab

Jenkins has nice plugin for test results.
Test Results Analyzer Plugin (plugin url).
How to attach the same dashboard (with test results) for GitLab?
Here is my googling result:
Gitlab has "pages" feature but it depends on artifact life and only the last one is shown.
https://docs.gitlab.com/ee/ci/junit_test_reports.html
On the other hand there is an open and long story issue about this requirement for gitlab besides lots of closed ones.
https://gitlab.com/gitlab-org/gitlab-ce/issues/34102
Here the gitlab's info in this comparison page:
https://about.gitlab.com/comparison/gitlab-vs-jenkins.html
"Many languages use frameworks that automatically run tests on your code and create a report: one example is the JUnit format that is common to different tools. GitLab supports browsing artifacts and you can download reports, but we’re still working on a proper way to integrate them directly into the product."
I found a project for a complete solution and fully supported. I just started with this one and for now it is unbelievable. Just upload your reports to it. There are a lot of plug in support.
http://reportportal.io/

Informatica code check-in Hooks from Repository

I am working on a testing automation project and I am able to automate end to end test cases but in my scope I have to enable functionality to run test-case when a workflow is checked-in to repository. For this I need a trigger with information like what workflow and in which folder is checked it's checked in. Unfortunately I am unable to get anything till now and my admins are also not sure if this can be achieved. Any lead will be a great help.
Even if I can get the logs of workflow check-in, I can process them in real-time to get this information.

Coldfusion continuous Integration

let me begin by saying I 'm a coldfusion newbie.
I 'm trying to research if its possible to do the following and what would be the best approach to achieve it.
Whenever a developer checks in code into SVN, I would like to do a get all the new changes/files and do an auto build to check if the code can be deployed successfully to production server. I guess there are two parts to it, one syntax checking and second integration test(if functionality is working as expected). For the later part some unit test tools would have to be used.
Can someone comment on their experience doing something similar for coldfusion.
Sorry for being a bit vague...I know its a very open-ended question but any feedback would be appreciated.
Thanks
There's a project called "Cloudy With A Chance of Tests" that purports to do what you require. In particular it brings together a number of other CFML code analysis projects (VarScope & QueryParam) to check code, as well as unit testing. I am not currently using it myself but did have a look at it some time ago (more than 12 months) and it appeared to be quite good.
https://github.com/mhenke/Cloudy-With-A-Chance-Of-Tests
Personally I run MXUnit tests in Jenkins using the instructions from the MXUnit site - available here:
http://wiki.mxunit.org/display/default/Continuous+Integration+--+Running+tests+with+Jenkins
Essentially this is set up as an ant task in Jenkins, which executes the MXUnit tests and reports back the results.
We're not doing fully continuos integration, but we have a process which automates some of the drudgery of our builds:
replace the site's application.cf(m|c) with one that tells users that the app is being deployed (we had QA staff raising defects that were due to re-deployments)
read a database manifest XML which lists all SQL scripts which make up the current release. We concatenate the scripts into a single upgrade script, suitable for shipping
execute the SQL script against the server's DB, noting any errors. The concatenation process also adds a line of SQL after each imported script that white to a runlog table, so we can see what ran, how long it took and which build it was associated with. If you're looking to replicate this step, take a look at Liquibase
deploy the latest code
make an http call to a ?reset=true type URL to tell the app to re-initialize
execute any tests
The build is requested manually through the build servers we have, but you click a button, make tea and it's done.
We've just extended the above to cope with multiple servers in a cluster and it ticks along nicely. I think the above suggestion of using the Jenkins SVN plugin to automate the process sounds like the way to go.

Continuous Integration: PowerShell vs. CI Server (CC.NET or Hudson)

So, a friend and I have been discussing continuous integration and bat/powershell scripts versus CI servers like CruiseControl.Net or Hudson.
The following powershell pseudo script works to update from SVN, build using msbuild, deploy/copy out, update a build/revision number in the app, and emails on failed builds. The next step would be to add calls to MSTest and email results when not successful.
svn update
msbuild > build_deploy_development_out_msbuild
([xml](svn info --xml)).info.entry.commit.revision + [char]13 + [char]10 + (echo %date% %time%) > build_revision_number.html
$linenumber = Select-String build_deploy_development_out_msbuild -pattern "Build Failed" | Select-Object Linenumber
$smtp = New-Object System.Net.Mail.SMTPClient -ArgumentList localhost | if($linenumber > 0) $smtp.Send("From:Email","To:Email", "build failed", "build failed... some one must die!")
This has lead me to the question of the value of CI servers, when you can write your own shell scripts to accomplish the same goal, using the specific tools of the project (build tool, source control, unit testing) (i.e. msbuild, nant, svn, git, nunit, mstest, etc.)
I have not experienced the maintenance cost as of yet. I wanted to get others opinions on the roll your own shell script versus a CruiseControl.Net or Hudson. Please note, I do not have experience with CI servers, thus the question, so please don't take this as being critical of CI servers; I simply don't know the best answer, and thought I would ask the community.
Best wishes!
Pete Gordon
CI Servers give you several advantages:
Web access, usually with ability to integrate with existing authentication mechanisms (see Hudson's ActiveDirectory/LDAP support)
Tons of existing support for unit testing, zip archive creationg, etc.
Hudson (and others) supports slave build nodes, for doing distributed CI tasks.
No need to maintain it yourself.
Some of these may not be things you need now but are you sure they aren't things you might need in the future?
I've installed Hudson a couple of weeks ago to replace the current CruiseControl server. The greatest advantage I see in Hudson is that pretty much anybody can use it, while launching a parametrized build with CruiseControl (or a batch file) is still scary for a lot of people.
Usually I tend to write all my build scripts with Ant (because it's portable), insert a couple of parameters and I invoke them from Hudson.
Hudson gives your scripts a great visibility (everything can be seen on the front page) and they are self explanatory. Usually with a bash script, you need to write a readme (that nobody reads) and remember where they are located.
... or have it both. Ayende (the creator of Rhino Mocks) has done that recently. He wrote a CI server using PowerShell. Perhaps this provides new insights for your discussion.
For a year I've tried to maintain custom-written python scripts to do basic CI stuff: recieving notifications of commits via e-mail, checking out and building stuff, sending back blames and congrats, then when it came to publishing this for use by everyone else in my team, it turned out raaaather unusable without monitoring, web-access, etc, etc.
Then I've dived into buildbot and found it truly beautiful. I've set up basically the same process in a couple of days. Build script is a true python object, that is customizeable at the master, from where it gets transferred to slaves and executed there. Built upon twisted framework, that is lots of stuff out-of-the-box ;)
Web UI is minimalistic, though sufficient.
Well, this is unpublished too, though I'm close to it this time %)
Below are my thoughts on CI Server over a Powershell scripts
Highly Configurable Plugins are available for all different kinds of version control, notifications and testing.
Logs These are maintained wonderfully. Failure and succesful build logs are at your finger tip.
Scheduling You can set all kinds of scheduling including triggerring based on other succesful build
Security You can set different groups to be able to execute, view only or set to see some projects
Visibility You can use a web dashboard or cctray for different audiences.
Scalability. Easy to scale when needed.
Bottom line if you have to maintain lots of builds for different environment and team projects then CI Server is the way to go. Other than that a simple PowerShell Script is enough for small projects. Once the project grows you can just hook up the existing PowerShell Script to a CI Server.