I am trying to write a bazel extension that reads some files and generates other files. I am using ctx.actions.declare_file and ctx.actions.declare_directory functions to declare files that I would like to generate, and then declare actions that generate these files. So far so good.
When I try to declare files/directories that are outside the current package though, Bazel complaints. For example, say that I have a source file src = <source file foo/bar.proto>, meanwhile the current rule is being executed when called by a rule //baz/proto:some_target. When I try declaring a generated file as follows
generated_src = ctx.actions.declare_file(src.basename, sibling=src)
Bazel complaints saying Output artifact 'foo/bar.proto' not under package directory 'baz/proto' for target '//baz/proto:some_target'. Looking at the documentation for ctx.actions.declare_file, this is expected.
My question is, is there some other way to declare a file that is not in the current package?
Related
I recently finished the basic tutorial for C++ here and wanted to set up a project based on the proto files from here. I followed a similar directory structure as the tutorial and changed up the CMakeLists.txt file to accommodate the new files.
I'm currently trying to just compile the manager.proto file. I was able to compile the file and get my server/client files, however the files are being outputted within cmake/build/minknow_api rather than the expected cmake/build folder. This meant the make command would return the error:
clang: error: no such file or directory: '/Users/name/Documents/grpc/examples/cpp/minknow_api/cmake/build/manager.grpc.pb.cc'
clang: error: no input files
I read that it was because protoc outputs based on the imports of the proto file, i.e the file comes with import minknow_api/device.proto for instance. I copied the files from within the cmake/build/minknow_api into cmake/build/ and reran make and it seemed to work, however, the C++ file imports as expected are searching within a minknow_api directory, meaning I'd have to manually edit these imports to look within the current directory instead of to then compile successfully.
I've tried experimenting with trying to get rid of the minknow_api from the proto imports, however had no luck and only got more import issues during compilation. It seems some files have the same names for messages etc, which means I had to keep the minknow.somename as the package so I can thus distinguish in the files which imported values I wanted to access. I've also tried moving files into their own directories like instance.proto which has package minknow.instance would go inside of instance directory, but still no luck.
I was wondering if anyone could figure out how to get rid of the minknow_api out of my proto imports properly so that I won't get these import and output directory issues down the track?
I want a run a build step that looks at a .h file, adds some code based on some external params, and hands the resulting file to the preprocessor.
I see the "Custom Build Step" in the project properties. It seems to need an output file. I just want to forward the results to the preprocessor.
It seems like the custom build step wants to do a 1-time process, not per-file or by file type.
The problem is that I don't know how to send my external executable the file currently being processed (eg, "HelloWorld.cpp"). $(InputName) and %(Filename) are blank and docs say it's deprecated. How do I send the filename to my external executable?
But even if I get that working, I don't want to set this per-file. I want all header files to go through this process.
Any ideas?
I've looked at:
https://msdn.microsoft.com/en-us/library/dd293663.aspx?f=255&MSPPError=-2147217396
https://msdn.microsoft.com/en-us/library/hefydhhy(v=vs.90).aspx
https://msdn.microsoft.com/en-us/library/ff770593(v=vs.140).aspx
working on a debug, x64 config on windows.
First of all, No, you cannot modify a file and pass along the results to the next stage (that I could see). I'd need some sort of Program Transformation System.
So I need an intermediate file. That file has to be added to the project, even if it gets overwritten by your code generator. I can associate c++ header files with a custom build tool, and they will all get called one-by-one in the stage of the build specified in the Custom Build Step. The custom build tool will modify the intermediate file(s), and all is well.
The VS 2015 name for the current file being processed is %(Filename). In older versions it has been $(ProjectName) and $(InputName).
I'm working on a C++ project that has some hand coded source files, as well as some source and header files that are generated by a command line tool.
The actual source and header files generated are determined by the contents of a JSON file that the tool reads, and so cannot be hardcoded into the scons script.
I would like to set up scons so that if I clean the project, then make it, it will know to run the command line tool to generate the generated source and header files as the first step, and then after that compile both my hand coded files and the generated source files and link them to make my binary.
Is this possible? I'm at a loss as to how to achieve this, so any help would be much appreciated.
Yes, this is possible. Depending on which tool you're using to create the header/source files, you want to check our ToolIndex at https://bitbucket.org/scons/scons/wiki/ToolsIndex , or read our guide https://bitbucket.org/scons/scons/wiki/ToolsForFools for writing your own Builder.
Based on your description you'll probably have to write your own Emitter, which parses the JSON input file and returns the filenames that will finally result from the call. Then, all you need to do is:
# creates foo.h/cpp and bar.h/cpp
env.YourBuilder('input.json')
env.Program(Glob('*.cpp'))
The Glob will find the created files, even if they don't physically exist on the hard drive yet, and add them to the overall dependencies.
If you have further questions or problems arise, please consider subscribing to our User mailing list at scons-users#scons.org (see also http://scons.org/lists.html ).
Thanks to Dirk Baechle I got this working - for anyone else interested here is the code I used.
import subprocess
env = Environment( MSVC_USE_SCRIPT = "c:\\Program Files (x86)\\Microsoft Visual Studio 11.0\\VC\\bin\\vcvars32.bat")
def modify_targets(target, source, env):
#Call the code generator to generate the list of file names that will be generated.
subprocess.call(["d:/nk/temp/sconstest/codegenerator/CodeGenerator.exe", "-filelist"])
#Read the file name list and add a target for each file.
with open("GeneratedFileList.txt") as f:
content = f.readlines()
content = [x.strip('\n') for x in content]
for newTarget in content:
target.append(newTarget)
return target, source
bld = Builder(action = 'd:/nk/temp/sconstest/codegenerator/CodeGenerator.exe', emitter = modify_targets)
env.Append(BUILDERS = {'GenerateCode' : bld})
env.GenerateCode('input.txt')
# Main.exe depends on all the CPP files in the folder. Note that this
# will include the generated files, even though they may not currently
# exist in the folder.
env.Program('main.exe', Glob('*.cpp'))
There's an example at:
https://github.com/SCons/scons/wiki/UsingCodeGenerators
I'll also echo Dirk's suggestion to join the users mailing list.
I don't quite understand the path in protobuf. My file layout like this:
Top
A
a.proto
B
C
c.proto // import "A/a.proto";
I have written an RPC system based on protobuf and I need generate two kinds of files(client and server code) from c.proto. Client code should be placed in B and Server code still in C.
I can't write a correct command.
Top> protoc -I=. --client_out=./B/ C/c.proto will generate client code in B/C and the #include in code will have a wrong path.
Top/C> protoc -I=../ -I=./ --client_out=./ ./c.proto lead a protobuf_AddDesc_* error.
For every .proto file, protoc tries to determine the file's "canonical name" -- a name which distinguishes it from any other .proto file that may ever find its way into your system. In fact, ideally, the canonical name is different from every other .proto file in the world. The canonical name is the name you use when you import the .proto file from another .proto file. It is also used to decide where to output the generated files and what #includes to generate.
For .proto files specified on the command line, protoc determines the canonical name by trying to figure out what name you would use to import that file. So, it goes through the import paths (specified with -I) and looks for one that is a prefix of the file name. It then removes that prefix to determine the canonical name.
In your case, if you specify -I=. C/c.proto, then the canonical name is C/c.proto. If you specified -I=C C/c.proto, the canonical name would then simply be c.proto.
It is important that any file which attempts to import your .proto file imports it using exactly the canonical name determined when the file itself was compiled. Otherwise, you get the linker error regarding AddDesc.
In general, everything works well if you designate some directory to be the "root" of your source tree, and all of your code lives in a subdirectory of that with a unique name designating your project. Your "root" directory should be the directory you pass to both -I and --client_out. Alternatively, you can have separate directories for source files vs. generated files, but the generated files directory should have an internal structure that mirrors your source directory. You can then specify the generated files directory to --client_out, and when you run the C++ compiler, specify both the source and generated files directories in the include path.
If you have some other setup -- e.g. one where the .proto files live at a different canonical path from the .pb.h files -- then unfortunately you will have some trouble making protoc do what you want. Though, given that you are writing a custom code generator, you could invent whatever rules you want for the way its output files are organized, but straying from the rules the standard code generator follows might lead to lots of little pitfalls.
Is there a way to compile a C++Builder project (a specific build configuration) from the command line?
Something like:
CommandToBuild ProjectNameToBuild BuildConfiguration ...
There are different ways for automating your builds in C++Builder (as of my experience, I'm speaking about old C++Builder versions like 5 and 6).
You can manually call compilers - bcc32.exe (also dcc32.exe, brcc32.exe and tasm32.exe if you have to compile Delphi units, resource files or assembly language lines of code in your sources) and linker - ilink32.exe.
In this case, you will need to manually provide the necessary input files, paths, and keys as arguments for each stage of compilation and linking.
All data necessary for compilation and linking is stored in project files and, hopefully there are special utilities, included in the C++Builder installation, which can automate this dirty work, provide necessary parameters to compilers and linker and run them. Their names are bpr2mak.exe and make.exe.
First you have to run bpr2mak.exe, passing your project *.bpr or *.bpk file as a parameter and then you will get a special *.mak file as output, which you can use to feed on make.exe, which finally will build your project.
Look at this simple cmd script:
#bpr2mak.exe YourProject.bpr
#ren YourProject.mak makefile
#make.exe
You can provide the real name of "YourProject.mak" as a parameter to make.exe, but the most straightforward way is to rename the *.mak file to "makefile", and then make.exe will find it.
To have different build options, you can do the following:
The first way: you can open your project in the IDE, edit options and save it with a different project name in the same folder (usually there are two project files for debug and release compile options). Then you can provide your building script with different *.bpr files. This way, it looks simple, because it doesn't involves scripting, but the user will have to manually maintain coherency of all project files if something changes (forms or units added and so on).
The second way is to make a script which edits the project file or make file. You will have to parse files, find compiler and linker related lines and put in the necessary keys. You can do it even in a cmd script, but surely a specialised scripting language like Python is preferable.
Use:
msbuild project.cbproj /p:config=[build configuration]
More specifics can be found in Building a Project Using an MSBuild Command.
A little detail not mentioned.
Suppose you have external dependencies and that the .dll file does not initially exist in your folder
You will need to include the external dependencies in the ILINK32.CFG file.
This file is usually in the folder
C:\Program Files (x86)\Borland\CBuilder6\Bin\ilink32.cfg
(consider your installation location)
In this file, place the note for your dependencies.
Example: A dependency for TeeChart, would look like this (consider the last parameter):
-L"C:\Program Files (x86)\Borland\CBuilder6\lib";"C:\Program Files (x86)\Borland\CBuilder6\lib\obj";"C:\Program Files (x86)\Borland\CBuilder6\lib\release";"C:\Program Files (x86)\Steema Software\TeeChart 805 for Builder 6\Builder6\Include\";"C:\Program Files (x86)\Steema Software\TeeChart 805 for Builder 6\Builder6\Lib\"
You will also need to include the -f command to compile.
In cmd, do:
//first generate the file.mak
1 - bpr2mak.exe MyProject.bpr
//then compile the .mak
2 - make.exe -f MyProject.mak
You can also generate a temporary mak file with another name, as the answer above says, directly with bpr2mak
bpr2mak.exe MyProject.bpr -oMyTempMak.mak