I'm new to TeamCity, migrating from a different CI product, and trying to figure out how to configure a working build for a c++ project on version 9.1.6 of TeamCity.
The problem I'm having is that the agent is deleting my dependency directories right before (or during) the component build, and I can't find a record of why this is happening anywhere in the build log.
The build layout for any component in the system looks like so:
<base-dir>
|
+---<to-be-built>
+---<dependency-1>
+---Include
+---Lib
+---<dependency-2>
+---Include
+---Lib
...so, whatever the checkoutDir is for the component, it is assumed that all dependencies will be found in peer directories, named after the dependency, with no version information in the folder name.
For example, if version 3.0.2 of "MyExe" depends on version 1.1.0 of "SomeLibA" and version 2.1.0 of "SomeLibB", the file system should look like so:
MyExe_3.0.2
|
+---MyExe
+---SomeLibA
+---Include
+---Lib
+---SomeLibB
+---Include
+---Lib
So, to create this build layout, the build configuration for version 3.0.2 of "MyExe" has specified a custom checkout directory like so: "MyExe_3.0.2/MyExe".
So far, so good. The dependencies are configured as artifact dependencies and their destination directory is specified as '../'. This also seems straightforward.
When I kick off the build, though, I see the to-be-built component being retrieved, then I see the dependencies arrive, and then the gradle task I've configured for the build runs, and right at that moment, or just before, all of the dependency directories (SomeLibA and SomeLibB) get wiped out, and of course the component can't find any of its dependencies' include files and compilation fails.
I've turned off 'Clean all files in the checkout directory' on the component and 'Clean destination paths before downloading artifacts' on all dependencies, but this has no effect.
I've only found 2 hints possibly related to this behavior, but I'm not sure why either of them would be causing this problem.
The first is a little warning symbol on the Version Control Settings tab for "MyExe" which says "This directory might be cleaned by TeamCity before the build", referring to the custom directory. But, the directory that's getting cleaned out during the build run is not the checkout directory, it's the checkout directory's parent directory.
The only other possible candidate I can find is that the gradle task I've configured isn't the only task specified when the build runs. Instead of seeing "gradlew.bat myGradleTask" in the build log, I'm seeing "gradlew.bat --init-script C:\TeamCity\BuildAgent\plugins\gradle-runner\scripts\init.gradle myGradleTask".
But, I've looked through that init script, and didn't see anything related to directory cleanup.
I'm stumped. Does anyone have an idea what is going on, and how to work around it so this build can complete successfully? Acceptable solutions have to preserve the build layout requirements above.
The problem here was in disabling "Clean all files in the checkout directory before the build".
Disabling this checkbox has the effect of wiping out the contents of the entire path to the checkout directory.
For reference, see documentation here:
https://confluence.jetbrains.com/display/TCD9/Clean+Checkout
...from which the relevant excerpt is:
Automatic Clean Checkout
If clean checkout is not enabled, TeamCity updates the sources in the
checkout directory incrementally to the required state. TeamCity tries
to detect if the sources in the checkout directory are not
corresponding to the expected state and triggers clean checkout in
such cases to ensure sources are appropriate.
This means that under certain circumstances TeamCity can detect clean
checkout is necessary even if it is not enabled in the VCS settings
and not requested by the user from web UI. In such cases all the
content of the checkout directory is deleted and it is re-populated by
the sources from scratch. If any details are available on the
decision, they are added into the build log before checkout-related
logging.
Enabling this checkbox had the effect of leaving the dependency directories in place.
Related
I am using a custom build system (not CMake or qmake) with QtCreator 4.6.0 IDE on Linux, though I have both of these plugins installed.
My project is cross-compiled for an embedded Linux device, which I have set up the kit for. It builds fine, but the trouble is in the deployment step, where the custom build system does not tell it the executable file, so it does not know which files to deploy to the device. I have been referring to this page:
http://doc.qt.io/qtcreator/creator-deployment-embedded-linux.html
In my Run Settings page, I have the Deployment section to "Upload files via SFTP", but the "Files to deploy" list is empty (see the first screenshot in the above link). When I attempt to deploy/run, it confirms no files were deployed, as it outputs:
No deployment action necessary. Skipping.
Deploy step finished.
The documentation link first says to "Edit the qmake INSTALLS variable in the project .pro file to add the missing files.", however, my project has no .pro file at all. I created one for fun, and QtCreator has no idea it exists to be able to look at the INSTALLS line anyway, so that doesn't work.
In the "Deploying CMake Projects to Embedded Linux Devices" section at the bottom, it then says "You must specify all other files in the QtCreatorDeployment.txt file that you create and place in either the root directory of the CMake project or the build directory of the active build configuration." So I created such a file in the main project directory, as well as the build directory:
/home/root
build/test:.
The file makes no difference to the "Files to deploy" list, even after restarting. It seems to suggest this method is for CMake anyway. However, I have not even explicitly turned off the CMake building in the Build&Run > Kits, so I would not even know how to start making it build with CMake (and there is no insight from the documentation page: https://doc.qt.io/qtcreator/creator-project-cmake.html). The one thing I did change on the Build Settings page was in the "Build Steps" to do "Custom Process Step" instead of the usual Make step.
How do I add files to the "Files to deploy" list?
QtCreator 4.9 now supports specifying the QtCreatorDeployment.txt file for generic (non-CMake) projects to achieve this.
https://doc.qt.io/qtcreator/creator-deployment-embedded-linux.html#deploying-cmake-projects-to-embedded-linux-devices
https://blog.qt.io/blog/2019/02/21/qt-creator-4-9-beta-released/
I have a C++ project for which I need to run a custom build tool on some header files to generate code that is required when compiling this project. In general my configuration works. I trigger a build, VS/MSBuild detects whether the output files are up-to-date and runs the custom build tool only if necessary.
However, a problem arises if the build is run in combination with another configuration of the same project. Both configurations depend on the output files of the custom build tool. So if run sequentially only one configuration should trigger the custom build tool to run. For which ever configuration a build is triggered second the output files of the custom build tool are already present and up-to-date. So there is no need to build them again. Unfortunately this is exactly what is happening. Since the custom build tool takes quite some time to run, this increases build times dramatically.
Another interesting aspect is, that once both configuration have run, I can trigger any of them again and the custom build tool is not invoked.
What I would have expected from the documentation is that the custom build tool is triggered:
If any of the files specified as Outputs is missing
If the file for which I specified the custom build tool was modified later than any of the existing files specified as Outputs
If any of the files I specified as Additional Dependencies were modified later than any of the existing files specified as Outputs
But all of this independent from the configuration for which the build was triggered.
Does anyone have an idea on why this might happen? I checked that the settings for the custom build tool are identical for both configurations. The output files are generated into the same folder for both configurations.
The documentation you're referring to is basically correct but it omits to say that everything in there is basically per project configuration/platform because it uses tracker.exe which depends on .tlog files which by default go into the intermediate directory. So as you figured out, making all configurations use the same location for the tlog files should keep the tracker happy and only invoke the custom build tool when needed, independent of configuration/platform. I'm not sure I'd recommend any of this though, sharing temporary object files might cause you problems later.
Another way to deal with this is adding a seperate project with just one configuration, say 'Custom', and do the custom build there. Than make your current project(s) depend on that project and in the solution's Configuration Manager adjust all entries so each configuration you have now builds the 'Custom' configuration for the new project.
Every time I download a project from github I run npm install, which triggers a reindex on the to-be-created node_modules folder. This indexing slows my computer way down. An ugly workaround is to create an empty node_modules folder, exclude it, then run npm install. How can i disable indexing for the node_modules folder in EVERY project by default? Webstorm does this for Meteor projects with .meteor/local, so I assume it's possible.
We usually suggest excluding this folder if it's used for auxiliary purposes (running grunt/gulp/karma, etc.). But we can't exclude it by default, as users developing Node.js applications usually need to have completion/types resolving working for node_modules.
If you need it being excluded from all your projects by default, just add node_modules to 'Ignore files and folders' list in Settings/Editor/File types
Update: since 2016.x, node_modules are auto-excluded by default. Direct dependencies listed in package.json are set up as a JavaScript Library for completion
webstorm treats node_module directory as library root, so there is no mark directory as menu when right click on it. we can change it to a plain directory by delete a config item whose type is project from preference > Language & Framework > Javascript > Libraries, then mark the node_module directory as excluded.
I'd like to setup a TeamCity build that will perform an incremental build.
For this, i want to store the build outputs (.dll files) as artifacts, and reuse them on every subsequent build (copy the latest artifacts to the build agent before starting the build).
This will effectively place the last build's artifacts in the project's output folder, so MSBuild could use those artifacts to determine whether it needs to rebuild anything from sources.
I've tried to do this, but it seems TeamCity doesn't allow configuring artifact dependencies from the same build configuration.
For example, if i have a "Build Plugins" configuration that generates a collection of plugin DLLs, i cannot use these as a dependency for the same build configuration...
Is there any inherent way to overcome this in TeamCity, or to create an easy solution myself?
It appears it is only possible to do this when using templates.
You can create a template for a build. Then you create a build from that template. After that you add this build to the artefact dependencies from the template. This allows for circular dependencies.
I have found no other way.
It looks like you can do this now! It seems to work in 9.0.1, and TW-12984 says it should work as far back as 8.1.
Our company might be moving from CVS to Subversion soon. This has brought about an issue for us, which I am trying to solve.
For CVS and Eclipse, we were able to use team project set files to gather various modules and check them out together (http://vpms.de.csc.com/projectset/). This made it very easy to manage projects, since there was no need to remember each module in the project.
However, project sets do not support SVN. I know there is an 'externals' property for SVN that does approximately (or possible exactly) the same thing. I tried this. Now, for the problem:
When I use the externals property and checkout 2 modules in eclipse, their C/C++ project properties are lost, and so I cannot right click on them to say "build project" or "clean project". They appear to Eclipse to be folders with files in them.
Is there something I am missing here?
EDIT
When I check out each module separately, they check out as projects, so they do have the individual .project/.cproject/settings stuff
You forgot to place Eclipse project metadata into your source control system. Make sure all files starting with '.' in project root make it in along with the entire contents of the .settings directory.
Subversion externals simply allows you to take files from one part of the repository and bring them in under a folder in your local checkout. At my last company, we had a java source directory that called "commonSrc" that was an SVN External for another project's main "src" directory, but in the project it was brought into, it simply acted as another folder (as you are experiencing).
I never really liked that method and wouldn't recommend it unless you have only one/two modules.
In order to do what you are trying to do with SVN, you might have to checkout each project separately, and use "Module Dependencies" in the project's properties to create the proper dependencies in Eclipse. You might be able to commit these project files so that the next person doesn't have to re-link them.
In case anyone needs this, here's what I found:
http://vpms.de.csc.com/projectset/
&
http://www.polarion.org/index.php?page=download&project=subversive
OR
http://www.giniality.com/old/update/projectset/
for Subversion + Project Set integration.
There is no need to break your project set. Once you have the integration plugins installed in Eclipse, all you need to do is change the source from the CVS server to SVN.