Are there any issues with running a vNext build agent on the same machine that is running Functional Tests? My plan is to have an agent pool that consists primarily of test machines where each build agent has a one-to-one relationship to running functional tests on each respective machine.
Yes, you have the ability to run some Unit Tests both on a machine different from that one is running the build and on the same machine.
To select build agent in a pool which include multiple agents:
You can add a User Capability to that specific build agent then in the
build definition you put that capability as a demand (General tab)
How to specify a build agent in vNext build
About how to configure build pipeline and deploy test agent, you could refer below tutorials:
Running Automated Test on agent machine using vNext Build
TFS — Setup CI/CD pipeline for automated functional tests
Hope this helps.
Related
We have a UWP app with unit tests that run fine from Visual Studio. However, we are running into issues when trying to run the tests on our build agent. To isolate the issue from our build pipeline, we have tried to run the tests using vstest.console.exe from the command line on our local build machine. The error below is the same regardless of whether we run from the build pipeline or from the command line. We would appreciate any guidance.
TestWindow>vstest.console.exe /Platform:x86 /Framework:FrameworkUap10
/InIsolation /logger:trx;LogFileName=%temp%\SampleUwpAppTest.trx
C:\vsts-agent_work\22\a\AppxPackages\SimpleUwpApp.Tests_1.0.0.0_x86_Test\SimpleUwpApp.Tests_1.0.0.0_x86.appx
Microsoft (R) Test Execution Command Line Tool Version 15.7.2
Copyright (c) Microsoft Corporation. All rights reserved.
Starting test execution, please wait... Attempting to deploy to target
'Local Machine'. Deployment operation succeeded, package moniker:
'7d25c824-2568-4449-a622-ad28662417f9_1.0.0.0_x86__a178p9x51fw8a'.
Attempting to launch app with app user model Id
'7d25c824-2568-4449-a622-ad28662417f9_a178p9x51fw8a!vstest.executionengine.universal.App' and arguments '--port 1428 --endpoint 127.0.0.1:01428 --role client
--parentprocessid 5744 --telemetryoptedin false'. DEP7100: Failed to activate app
'7d25c824-2568-4449-a622-ad28662417f9_a178p9x51fw8a!vstest.executionengine.universal.App' with parameters '--port 1428 --endpoint 127.0.0.1:01428 --role client
--parentprocessid 5744 --telemetryoptedin false'. Access is denied.
Access is denied.
App activation failed.
Microsoft.VisualStudio.TestPlatform.ObjectModel.TestPlatformException:
Failed to initialize client proxy: could not connect to test process.
at
Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.SetupChannel(IEnumerable`1
sources) at
Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyExecutionManager.StartTestRun(TestRunCriteria
testRunCriteria, ITestRunEventsHandler eventHandler) Closing app with
package full name
'7d25c824-2568-4449-a622-ad28662417f9_1.0.0.0_x86__a178p9x51fw8a'.
Test Run Aborted.
Build Information
Our sample solution has two projects: SimpleUwpApp and SimpleUwpApp.Tests. The former's project file has <AppxBundle>Always</AppxBundle> and the latter has <AppxBundle>Never</AppxBundle>. The build agent is a local Windows 10 machine which we have used for UWP builds for a while. Here is a screenshot of our build configuration in Azure DevOps:
You might see this error if developer mode is not enabled on the build agent.
You can enable developer mode in the Settings app, or set the AllowDevelopmentWithoutDevLicense registry key.
I ran across this issue today and eventually traced it to the way in which our Self-Hosted Azure DevOps agent was running - in order to run UWP Unit Tests you have to use Interactive mode for the Agent and not run it as a Windows Service. From the docs:
Visible UI testing using self-hosted Windows agents
Agents that are configured to run as service can run Selenium tests only with headless browsers. If you are not using a headless browser, or if you are running UI tests for desktop apps, Windows agents must be configured to run as an interactive process with auto-logon enabled.
After making this change I found the error went away.
I am very new in TFS, need to implement CICD using TFS 2017 and its build and release feature,
when I tried to run build after creating build definition, I got error like no agent found, I googled and found how to configure agent, but I have logical confusion in my mind as below:
How Agent works with TFS 2017?
Where process of CI will be run on Agent or on TFS server?
Where I need to have msbuild ? where my built code will be placed?
What other dependencies would be there on Agent machine?
all question might be silly but as I have worked with Jenkins and Git, i dont have knowledge of Microsoft technologies, and I can't find well documents for the same.
How Agent works with TFS 2017?
In short to build with TFS, you need to Deploy an agent, in the agent machine you need to install the proper build components/SDKs accordingly based on your project.
Create a build definition. Once a build is triggered , the sources will be downloaded from the TFS repository to the agent machine and then build in the agent machine.
Related documents : Agent pools and queues; Build and Release Agents; Build definition options
Where process of CI will be run on Agent or on TFS server?
You can eanble the CI (turn on the Continuous integration trigger) in build definition. See Configure continuous integration for details. Thus the build will automatically be triggered once changes are checked in.
Related documents: A quick introduction to CI/CD ; Build and release
Where I need to have msbuild ? where my built code will be placed?
For vNext build, it's task based build system. You can define your build definition based on the tasks. See Build and release tasks .
e.g.: You can use MSBuild or Visual Studio Build task, you can specify the MSBuild Arguments as needed.
You can use the utility task: Copy and Publish Build Artifacts and Publish Build Artifacts to specify where the built code will be placed. (Artifact Type : Server/File share path)
What other dependencies would be there on Agent machine?
Refer to the answer for the first question.
I am leveraging Visual Studio Team Services Build Vnext to build ASP.NET application present in my Microsoft GIT Repository.
The Microsoft GIT repository is created under an Agile Team Project of the VSTS instance.
The VSTS build is getting executed successfully without any issues.
But I have a query with respect to the cloning activity of the GIT Repository that gets executed on the VSTS Build Agent.
During the build, the VSTS build Agent is cloning the entire master branch of the GIT repository instead of downloading the required ASP.NET application folder.
Is there a way to download the specific application folder on to the build agent instead of the entire branch.
we have this option for the onpremise TFS build having TFS as the Source control.
As far as I know there is no way to pull only a specific folder on a git repository unlike tfsvc. This is why you don't have such an option for the build tasks.
I am planning to create build definitions for one of our projects for CI automated builds. We have 2 environments - QA and Live and the same application will be running in 3 countries. So we have to have a set of QA and Live build folders for each country.
While publishing the application to each country, we used to manually update the app.config file in the project (have 1 for each country) and deploy to the country specific location. Could this be achieved in the automated builds in some way that for each checkin, a build
gets generated an published to the QA environment of each country by picking the appropriate config file?
Thanks in Advance,
Arun
You can have a different configuration for each country + environment combination (CI-UK, CI-US, QA-UK, QA-US); then in the CI build definition you specify to build all CI configurations for each country (CI-UK and CI-US).
For generating the specific configurations you should use the SlowCheetah Visual Studio add-in.
Through Community TFS Build Extensions, on Codeplex, you can manipulate the config files to deploy and configure a specific environment.
I have configured TeamCity with Git to get my ASP.NET MVC project.
My solution contains the web app and the corresponding unit tests:
MY_SOLUTION.sln:
- WebAppProject
- SomeCoreLibrary
- SomeCoreLibraryTests
- OtherProjects...
The steps that I have configured in TeamCity are the following:
Get external packages using NuGet
Build the solution and deploy it
Run Unit Tests
Run Automated Tests (using Selenium)
I want to run the unit tests after building but before deployment and stop deployment if the unit tests failed. Currently the deployment is done after the build using the following Command Line Parameters:
/p:VisualStudioVersion=11.0
/p:DeployOnBuild=true I want this to be done only after SomeCoreLibraryTests.dll unit tests have passed
/p:PublishProfile=MyWebDeploy
/P:AllowUntrustedCertificate=True
/P:UserName=username_here
/P:Password=password_here
Thanks,
Ionut
What I've done in similar cases is to use RoboCopy to just mirror the new website into the deployment path. Doesn't that work for you?
P.S.: if you do get this working, I'd suggest doing a performance improvement change in TeamCity (which would allow you to run the unit tests in parallel to the automated tests):
I assume you are employing a single build configuration for all those steps. If that is the case, what I would recommend instead is using Dependent Build configurations to separate the different concerns. You can see an example here in an open source project of mine:
http://teamcity.codebetter.com/viewLog.html?buildId=112432&buildTypeId=bt1075&tab=dependencies
Log in as Guest and expand the Testeroids :: Publish to NuGet tree node to visualize the build flow.
To achieve this, basically you pass around the result of your build step in the artifacts (e.g. you pass the resulting binaries from Compile into Unit Test). You gain several things by using dependent builds: several independent build steps can run in parallel on different agents, plus if one of your build steps fails because of external factors (e.g. let's say Publish failed because the network went down), you can trigger again the build and it will only rebuild the failed steps.
I am not familiar with the tools that you use. However, I would, in general, use a few build configurations for a project:
build configuration, triggered on change, containing these steps: get the latest source code and packages, build/compile and unit test. Then create an artifact for deployment task.
build configuration to deploy to a development server, triggered by successful completion of and using artifact (via dependency) from (1).
build configuration for long running (eg integration/functional) testing that is scheduled to run less frequently.
An advantage of (2) is that you can, if necessary, re-deploy a build/artifact without having to rebuild the artifact first. Also, if you have multiple agents, (2) and (3) can run independently of each other.
Furthermore, you can also tag build in (2) that have passed development checks and then use its artifact in another build configuration to deploy it to test server, etc.