Why can't scons find my project's source files? - c++

Here's my project on github.
Here's my SConstruct file:
SConscript('main.scons', variant_dir = 'build', duplicate = 0)
Here's my main.scons file:
import sys
import os
import fnmatch
def find_source_files(directory, ext = "cpp"):
matches = []
for root, dirnames, filenames in os.walk(directory):
for filename in fnmatch.filter(filenames, '*.' + ext):
matches.append(os.path.join(root, filename))
return matches
if __name__ == '__main__':
for f in find_source_files('src'):
print f
else:
Program(target = 'main.bin', source = find_source_files('src'))
Here's what I get when I run it:
bitcycle # cypher ~/git/IeiuniumTela $ find $(pwd) -name "*.bin" -or -name "*.o" -exec rm {} \;; scons; find $(pwd) -name "*.bin" -or -name "*.o"
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
scons: building associated VariantDir targets: build
gcc -o build/main.bin
gcc: fatal error: no input files
compilation terminated.
scons: *** [build/main.bin] Error 4
scons: building terminated because of errors.
Here's what happens when I run `python main.scons' to test it:
bitcycle # cypher ~/git/IeiuniumTela $ python main.scons
src/main.cpp
I'm having a hard time understanding why it can't find my source files. Any suggestions or ideas here?
[UPDATE] After getting some good direction from the mailing list, I found that this worked "good enough" for me.
/SConstruct: SConscript('src/main.scons', variant_dir = 'build', duplicate = 0)
/src/main.scons: Program(target = 'main.bin', source = Glob('*.cpp'))
See the github repository for the full source tree. I've also added an empty build directory to the repo for completeness. I find it interesteding that:
a. SCons' version of Glob isn't recursive, in the context of this build tool for discovering source. I would expect that a recursive discovery option is preferred. :(
b. I need to put the scons file in the same directory as the source file (which is annoying).
c. Print statements apparently work, but sys.stdout.write doesn't (from the python module).

Could it be because your main.scons file is already in the src directory, and your find_source_file is actually searching src/src?
It found the source sources for me when I moved the scons files to the top directory.
Update:
Upon investigation, variant_dir sets the working directory to build, so your find_source_files looks for files in build/src and finds nothing. Would be probably better to either call find_source_files from the SConstruct file, or to use VariantDir() inside main.scons instead.

SCons treats relative directory paths differently from Python, so I wouldnt be surprised to see a difference between the test execution and the SCons execution. Typically in SCons everything is relative to the root SConstruct script, or the SConscript script.
Your code appears correct, but how about adding some debugging print statements in find_source_files() to find out exactly what's happening?
Perhaps you plan to use the find_source_files() function more extensively later on, but for the simple case of one source file, it seems like you are over complicating things and you could just use the following in main.scons:
Program(target = 'main.bin', source = 'src/main.cpp')

Related

How to write files to current directory instead of bazel-out

I have the following directory structure:
my_dir
|
--> src
| |
| --> foo.cc
| --> BUILD
|
--> WORKSPACE
|
--> bazel-out/ (symlink)
|
| ...
src/BUILD contains the following code:
cc_binary(
name = "foo",
srcs = ["foo.cc"]
)
The file foo.cc creates a file named bar.txt using the regular way with <fstream> utilities.
However, when I invoke Bazel with bazel run //src:foo the file bar.txt is created and placed in bazel-out/darwin-fastbuild/bin/src/foo.runfiles/foo/bar.txt instead of my_dir/src/bar.txt, where the original source is.
I tried adding an outs field to the foo rule, but Bazel complained that outs is not a recognized attribute for cc_binary.
I also thought of creating a filegroup rule, but there is no deps field where I can declare foo as a dependency for those files.
How can I make sure that the files generated by running the cc_binary rule are placed in my_dir/src/bar.txt instead of bazel-out/...?
Bazel doesn't allow you to modify the state of your workspace, by design.
The short answer is that you don't want the results of the past builds to modify the state of your workspace, hence potentially modifying the results of the future builds. It'll violate reproducibility if running Bazel multiple times on the same workspace results in different outputs.
Given your example: imagine calling bazel run //src:foo which inserts
#define true false
#define false true
at the top of the src/foo.cc. What happens if you call bazel run //src:foo again?
The long answer: https://docs.bazel.build/versions/master/rule-challenges.html#assumption-aim-for-correctness-throughput-ease-of-use-latency
Here's more information on the output directory: https://docs.bazel.build/versions/master/output_directories.html#documentation-of-the-current-bazel-output-directory-layout
There could be a workaround to use genrule. Below is an example that I use genrule to copy a file to the .git folder.
genrule(
name = "precommit",
srcs = glob(["git/**"]),
outs = ["precommit.txt"],
# folder contain this BUILD.bazel file is tool which will be symbol linked, we use cd -P to get to the physical path
cmd = "echo 'setup pre-commit.sh' > $(OUTS) && cd -P tools && ./path/to/your-script.sh",
local = 1, # required
)
If you're passing the name of the output file in when running, you can simply use absolute paths. To make this easier, you can use the realpath utility if you're in linux. If you're on a mac, it is included in brew install coreutils. Then running it looks something like:
bazel run my_app_dir:binary_target -- --output_file=`realpath relative/path/to.output
This has been discussed and explained in a Bazel issue. Recommendation is to use a tool external to Bazel:
As I understand the use-case, this is out-of-scope for building and in the scope of, perhaps, workspace configuration. What I'm sure of is that an external tool would be both easier and safer to write for this purpose, than to introduce such a deep design change to Bazel.
The tool would copy the files from the output tree into the source tree, and update a manifest file (also in the source tree) that lists the path-digest pairs. The sources and the manifest file would all be versioned. A genrule or a sh_test would depend on the file-generating genrules, as well as on this manifest file, and compare the file-generating genrules' outputs' digests (in the output tree) to those in the manifest file, and would fail if there's a mismatch. In that case the user would need to run the external tool, thus update the source tree and the manifest, then rerun the build, which is the same workflow as you described, except you'd run this tool instead of bazel regenerate-autogenerated-sources.

OCamlbuild not resolving subdirectory dependency

I have an OCaml project and I'm running into a weird issue.
The directory structure looks like this:
./tests
test.ml
templatetest.ml
./src
template.ml
...
andsoforth.ml
I am building it like this:
ocamlbuild -Is src,tests tests/test.native
Key point being, tests/test.ml depends on tests/templatetest.ml.
When I build it, I get:
+ /usr/local/bin/ocamlc.opt -c -I tests -I src -o tests/test.cmo tests/test.ml
File "tests/test.ml", line 1, characters 0-17:
Error: Unbound module TemplateTest
Command exited with code 2.
Compilation unsuccessful after building 2 targets (0 cached) in 00:00:00.
I tried adding a _tags file in the top directory containing:
<src> or <tests>: include
That still produced the same error. (I believe ocamlbuild tests/test.native with the _tags file is the exact same thing as the previous build line I used.)
I also created an empty _tags file in both ./tests and ./src, but that didn't change anything either.
Any suggestions? If you need, the entire source is here. Just run make test. Thanks!
It a funny issue: not TemplateTest but Templatetest. For the former name, you module file should be named templateTest.ml

creating SConscript objects in scons

I have my project targets (binaries and libraries) specified in json files.
I can create an environment for the specified target without any problem. Now I'm trying to support a specific build directory.
My knowledge about scons is still very basic but it seems that the right way to do that is using a SConscript together with VariantDir. But I already have my targets specified in json and creating a SConscript file for each target would be redundant (plus a cost in maintenance).
So my question is: is it possible to create a SConscript object dynamically, at run time?
Thanks in advance.
VariantDir doesn't work with SConscturct file (may be i wrong, but i don't found any way to do it).
Just create SConscript file with variant dir and doing what you need.
#SConsruct
env = CreateEnvironment()
SConscript('SConscript', variant_dir = 'mybuilddir', exports = 'env', duplicate = 0)
# Do all work in SConscript
Import('env')
env.Program(...)
env.SharedLibrary(...)
...
Also, you can split your process into 2 states.
State 1 - generated SConscript files.
State 2 - run generated SConscript files.
if 'generate' in COMMAND_LINE_TARGETS:
# your code to generated SConscript from json
Exit(0)
sconscriptFiles = getSconscriptFiles() # some code to get your sconscript, by mask for example
if len(sconscriptFiles) < 1:
print "You need to generate files at first: scons generate"
Exit(1)
for file in sconscriptFiles :
SConscript(file, variant_dir = 'build' + file, duplicate = 0)
You can specify the build directory with the VariantDir() function, or as part of the SConscript() call. All of the different options are discussed here. Considering you dont want to use several SConstruct files, you should just use the VariantDir() function as described in more detail here.
Here is a simple example:
    env = Environment()
# It may be as simple as setting src_dir='.', but set accordingly
# duplicate=0 tells SCons NOT to copy source files to variantDir, set accordingly
# VariantDir() can be called multiple times so as to change dirs per builder call
VariantDir(variant_dir = 'pathToBuildDir', src_dir = 'pathToSource', duplicate=0)
# Now call the builders here
Its still not clear why you want to mix json with SCons. Unless you have some very compelling reasons to do so, I would suggest keeping it all in SCons, which is Python.
EDIT: I just realized you asked about creating a SConscript object, not a file.
I looked through the SCons programming APIs and didnt find anything that allows you to create a SConscript object. Actually, I dont think the concept of a SConscript object exists, since it just treats calls to the SConscript() function as files that need to be opened and processed, and they are almost treated as an extension to the SConstruct.
So, to summarize: You'll either have to create subsidiary SConscript files, or work with calls to VariantDir(). Depending on your project directory structure, it may not be necessary to create SConscript files. You could just do everything from the root SConstruct. The SConscript files arent necessary, they just help organize the build scripts better.

Can we list all the *.[c,h,S] files that are used by "make" command to build the linux kernel?

There are a lot of files in the kernel source that are not used most of the times. I wanted to list out only the files that are compiled when I issue a make command.
I thought that only those files that will be compiled are accessed during a make and hence, I tried the following command :
find . -type f -name *.[chS] -anewer Makefile
But I found that many files that are not a part of the required architecture are also being accessed. Please suggest a method in which I could list those filenames along with their path form the kernel source top directory.
Try make cscope and then check out the file list in the cscope.files plaintext file. You need the cscope tool installed on your system.

CSSTidy is driving me crazy!

I am on OS X + Python 2.6 + django 1.1.
I have tried all possible solution available on the net i.e.
http://thingsilearned.com/2009/01/02/installing-csstidy-and-scons-on-os-x-or-linux/
tried with python 2.5, 2.4 apple-python.. I just can't go past this error after installing scons
$ /usr/local/bin/scons
scons: *** No SConstruct file found.
File "/Library/Python/2.5/site-packages/SCons/Script/Main.py", line 829, in _main
I need csstidy for django-compress (yes I can turn that function off but why?) I mean the maintainer must see some value to have it in default and he knows better.
Unfortunately there is no port for csstidy...
Don't know if you managed to give up or not, or if you solved it.
For anybody who is still having trouble building CSSTidy, here was my solution. If you follow the tutorial in the URL linked above, there is one horrible error in it that I didn't realize until about an hour later.
$ wget http://downloads.sourceforge.net/csstidy/csstidy-source-1.4.zip?modtime=1184828155&big_mirror=0
$ unzip csstidy-source-1.4.zip
$ cd csstidy <<-- THIS IS WRONG. SConstruct isn't here. D:
$ scons
$ sudo cp release/csstidy/csstidy /usr/local/bin/
The errors the author ran into had nothing to do with the --standard-lib flag. When you unzip, the files don't get put into a subdirectory like tar usually does. So the SConstruct file that's magically missing is actually in the directory above csstidy, wherever you ran unzip in the first place.
Once I found that out, I saw the SConstruct file and was able to build it by running scons.
If you just want to build csstidy and don't care about incremental builds, just do this:
g++ *.cpp -o csstidy