GN stands for Generate Ninja. It generates ninja files which build things. The main file is BUILD.GN at the root of the fuchsia source tree
It contains a lot of build_api_module calls:
build_api_module("images") {
testonly = true
data_keys = [ "images" ]
deps = [
# XXX(46415): as the build is specialized by board (bootfs_only)
# for bringup, it is not possible for this to be complete. As this
# is used in the formation of the build API with infrastructure,
# and infrastructure assumes that the board configuration modulates
# the definition of `zircon-a` between bringup/non-bringup, we can
# not in fact have a complete description. See the associated
# conditional at this group also.
"build/images",
# This has the images referred to by $qemu_kernel_label entries.
"//build/zircon/zbi_tests",
]
}
however, it's unclear for me what this does exactly. Looking at its definition on build/config/build_api_module.gn for example:
template("build_api_module") {
if (current_toolchain == default_toolchain) {
generated_file(target_name) {
outputs = [ "$root_build_dir/$target_name.json" ]
forward_variables_from(invoker,
[
"contents",
"data_keys",
"deps",
"metadata",
"testonly",
"visibility",
"walk_keys",
"rebase",
])
output_conversion = "json"
metadata = {
build_api_modules = [ target_name ]
if (defined(invoker.metadata)) {
forward_variables_from(invoker.metadata, "*", [ "build_api_modules" ])
}
}
}
} else {
not_needed([ "target_name" ])
not_needed(invoker, "*")
}
}
it looks like it simply generates a file.
Can someone explain to me how build_api_module("images") ends up building all the zircon kernel images?
The build_api_module() targets generate JSON files that describe something about the current build system configuration. These files are typically consumed by other tools (in some cases dependencies to other build rules) that need to know about the current build.
One example is the tests target which generates the tests.json file. This file is used by fx test to determine which tests are available and match the test name you provide to the component URL to invoke.
Can someone explain to me how build_api_module("images") ends up building all the zircon kernel images?
It doesn't. These targets are descriptive of the current build configuration, they are not prescriptive of what artifacts the build generates. In this specific case, the images.json file is typically used by tools like FEMU and ffx to determine what system images to use on a target device.
Related
I'm porting a TI C6000 DSP build from Makefile to Bazel. This DSP is running on a very complex embedded system, and we are already building the same code for multiple other processors using Bazel, but those other processors use GNU flavored (e.g. gcc-based) compilers.
The cc_rules package seems to make assumptions about the flags, filename extensions etc. I'm hoping I can avoid creating a completely custom toolchain, which would involve porting all the existing BUILD files to use different rules depending on the toolchain.
I can't seem to find any documentation on customizing those attributes, or others. The things that I already know I need to customize:
The flag that is used to specify the output file: -fr=filename and -fs=filename vs -o filename
Implicit dependency generation: someone told me that cc_rules generates .d files under the hood to detect whether you have missing dependencies. I'm not sure if this is true or not, but if so, I need to change the flag and extension used
The extension of the object and library files: as noted above, we build the same files for multiple CPUs, and need to customize the extension of the output for the rules.
There are probably other requirements that I'm not aware of yet as well. It very may well be that I'm doing it all wrong and should take a different approach. We have been using Bazel since the early days (v0.12), and still may have holdovers from then.
We are currently on v1.1.0, which I ported us to from v0.12 six months ago. I'm surprised that the master branch is already on v3.???!!!
Any help is greatly appreciated. Please remind me if I've left out anything important.
EDIT: One thing to note is that the compiler appears to be based on clang and llvm. If there are examples of clang/llvm-based toolchains (I'm pretty sure there are) then I could get started there.
I know that the enscriptem example in the docs is technically a LLVM-based compiler, but that uses a script to do magic to the params, etc. I can do that if that's the right thing to do, but I want to make sure I'm headed down the right path.
This is not a complete answer to all of your questions, but this also goes beyond what could be formatted and posted as comment. To your most recent inquiry. This snippet would redefine option used for output file (instead of -o OUTPUT to -fr=OUTPUT):
compiler_output_flags_feature = feature(
name = "compiler_output_flags",
flag_sets = [
flag_set(
actions = [
ACTION_NAMES.assemble,
ACTION_NAMES.c_compile,
ACTION_NAMES.cpp_compile,
ACTION_NAMES.cpp_header_parsing,
ACTION_NAMES.cpp_module_compile,
ACTION_NAMES.cpp_module_codegen,
],
flag_groups = [
flag_group(
flags = ["-fr=%{output_file}"],
),
],
),
],
)
As for available and used actions, you can check this out. For features, as you've already discovered: disabling legacy features and see what you need is one option. There is also this list in the docs that you've stumbled upon. Beyond that (incl. what variables are available at which point), it's a bit of "use the source, Luke" at least that's where I usually for better or worse ended heading for details. For action a good point would be here.
But I also find checking out other pre-packaged toolchain configs (esp. MSVC for being... different) insightful.
I think that adding your own custom rule that provides CcToolchainConfigInfo, would solve the problem you are having.
def _impl(ctx):
tool_paths = [
tool_path(name = "gcc", path = "/<abs>/<path>/clang"),
tool_path(name = "ld", path = "/<abs>/<path>/ld"),
tool_path(name = "ar", path = "/<abs>/<path>/ar"),
tool_path(name = "cop", path = "/bin/false"),
tool_path(name = "gcov", path = "/bin/false"),
tool_path(name = "nm", path = "/bin/false"),
tool_path(name = "objdump", path = "/bin/false"),
tool_path(name = "strip", path = "/bin/false"),
]
toolchain_compiler_flags = feature(
name = "compiler_flags",
enabled = True,
flag_sets = [
flag_set(
actions = [
ACTION_NAMES.assemble,
ACTION_NAMES.preprocess_assemble,
ACTION_NAMES.linkstamp_compile,
ACTION_NAMES.c_compile,
ACTION_NAMES.cpp_compile,
ACTION_NAMES.cpp_header_parsing,
ACTION_NAMES.cpp_module_compile,
ACTION_NAMES.cpp_module_codegen,
ACTION_NAMES.lto_backend,
ACTION_NAMES.clif_match,
],
flag_groups = [
flag_group(flags = ["<compiler-flags>"]),
],
),
],
)
toolchain_linker_flags = feature(
name = "linker_flags",
enabled = True,
flag_sets = [
flag_set(
actions = [
ACTION_NAMES.linkstamp_compile,
],
flag_groups = [
flag_group(flags = ["<linker-flags>"]),
],
),
],
)
return cc_common.create_cc_toolchain_config_info(
ctx = ctx,
toolchain_identifier = ctx.attr.toolchain_identifier,
host_system_name = ctx.attr.host_system_name,
target_system_name = "<your-system-name>",
target_cpu = "<your-cpu>",
target_libc = "<your-libc>",
compiler = "<your-compiler>,
abi_version = "<your-eabiu>",
abi_libc_version = <your-version>,
tool_paths = tool_paths,
features = [
toolchain_compiler_flags,
toolchain_linker_flags,
<more-custom-features>,
],
)
cc_arm_none_eabi_config = rule(
implementation = _impl,
attrs = {
"toolchain_identifier": attr.string(default = ""),
"host_system_name": attr.string(default = ""),
},
provides = [CcToolchainConfigInfo],
)
I have posted an example about using GCC embedded toolchains with Bazel on Github that you could use as a template. The example works with the arm-none-eabi-gcc compiler, but in principle, it would work just as well with clang.
Im trying to set up a convention to ignore any test that follows a pattern. The filename would look like this if you wanted jest to ignore it during npm test.
DISABLED.MyComponent.test.js
trying to use testPathIgnorePatterns
testPathIgnorePatterns: ['<rootDir>/src/**/DISABLED.{*}.js'],
The error is either it is still being run, or it throws a error that the regex is invalid.
Jest uses glob patterns to match the test files, so prefixing the default entries in the testMatch configuration with !(DISABLED.) should do the job.
package.json
{
// ... rest of the package
"jest": {
"testMatch": [
"**/__tests__/**/!(DISABLED.)*.[jt]s?(x)",
"**/!(DISABLED.)?(*.)+(spec|test).[tj]s?(x)"
]
}
}
Side note: You can also "disable" the test file by renaming the main describe block to xdescribe or describe.skip, which would give visibility that there are skipped tests instead of completely ignore the file
Test Suites: 1 skipped, 11 passed, 11 of 12 total
Tests: 2 skipped, 112 passed, 114 total
The other answers here are helpful but miss the core fact:
testPathIgnorePatterns can be used to ignore your filenames. The problem is that testPathIgnorePatterns takes a RegExp, not a glob as you've specified.
The default setting for testPathIgnorePatterns is "/node_modules/", so it's probably useful to include it as well.
As an example, to ignore end to end tests that have an "e2e-" prefix, you could use:
"testPathIgnorePatterns": [
"/node_modules/",
"/e2e-"
]
In your specific case, this would be something like the following (note how the . is escaped in /DISABLED\\.):
"testPathIgnorePatterns": [
"/node_modules/",
"/DISABLED\\."
]
testPathIgnorePatterns is an array of regex patterns that ignores any path that matches the listed expressions and should contain the default "/node_modules/" as well. While it is not intended you may as well use testPathIgnorePatterns to exclude specific files:
So, your best bet would be to move the ignored test to s separate folder, e.g.
"jest": {
"testPathIgnorePatterns": [
"/node_modules/",
"<rootDir>/ignore/this/path/"
]
}
So #Teneff 's answer nearly worked for me. I had to move the ! to the front of the statement to get it to work. Hope this helps someone else too! I am using Next.JS 12.
testMatch: [
'**/*.test.{ts,tsx}',
'!**/__tests__/**/(DISABLED.)*.[jt]s?(x)',
'!**/(DISABLED.)?(*.)+(spec|test).[tj]s?(x)',
]
I'm using the following spec on my code to generate experiments:
experiment_spec = {
"test_experiment": {
"run": "PPO",
"env": "MultiTradingEnv-v1",
"stop": {
"timesteps_total": 1e6
},
"checkpoint_freq": 100,
"checkpoint_at_end": True,
"local_dir": '~/Documents/experiment/',
"config": {
"lr_schedule": grid_search(LEARNING_RATE_SCHEDULE),
"num_workers": 3,
'observation_filter': 'MeanStdFilter',
'vf_share_layers': True,
"env_config": {
},
}
}
}
ray.init()
run_experiments(experiments=experiment_spec)
Note that I use grid_search to try various learning rates. The problem is "lr_schedule" is defined as:
LEARNING_RATE_SCHEDULE = [
[
[0, 7e-5], # [timestep, lr]
[1e6, 7e-6],
],
[
[0, 6e-5],
[1e6, 6e-6],
]
]
So when the experiment checkpoint is generated it has a lot of [ in it's path name, making the path unreadable to the interpreter. Like this:
~/Documents/experiment/PPO_MultiTradingEnv-v1_0_lr_schedule=[[0, 7e-05], [3500000.0, 7e-06]]_2019-08-14_20-10-100qrtxrjm/checkpoint_40
The logic solution is to manually rename it but I discovered that its name is referenced in other files like experiment_state.json, so the best solution is to set a custom experiment path and name.
I didn't find anything in documentation.
This is my project if it helps
Can someone help?
Thanks in advance
You can set custom trial names - https://ray.readthedocs.io/en/latest/tune-usage.html#custom-trial-names. Let me know if that works for you.
I'm working on app with grunt and typedoc.
I need to prepare documentation using TypeDoc but I have a problem with one scenario.
I have to exclude a few files from documentation which are externals libs.
I can't put those files into exclude section because those files are relating with another. If I tried to exclude it (by put those files into exclude section) I had a errors - something like cannot find to xxx into yyy.ts - where xxx is my excluded element and yyy is related file with xxx. Those
related files are neccessary on this documentation so I can't exclude those too.
I read into TypeDoc documentation about excludeExternals. So I thought that if I set up this boolean as true then I can to define externalPattern to exclude my external files. It's works but only if I put the name of one file - no more.
Do You know how to do it?
It is my typedoc config into gruntfile.js (without excludeExternals options):
typedoc: {
build: {
options: {
module: 'commonjs',
out: '../Documentation/',
name: 'MyApp',
target: 'es5',
exclude: [
'node_modules',
'**/*.d.ts'
],
excludePrivate: true,
mode: 'file'
},
src: [
...some source...
]
}
}
This is list of external files which I have to exclude: A.ts, B.ts, C.ts, D.ts ...
And this is my typedoc config into gruntfile.js (with excludeExternals options):
typedoc: {
build: {
options: {
module: 'commonjs',
out: '../Documentation/',
name: 'MyApp',
target: 'es5',
exclude: [
'node_modules',
'**/*.d.ts'
],
excludeExternals: true,
externalPattern: '**/*/A.ts',
excludePrivate: true,
mode: 'file'
},
src: [
...some source...
]
}
}
This config is working well. I have got a documentation without A.ts file. So now I need to put a few files, so I tried to put on externalPattern something like: **/*/(A|B|C|D).ts but without success (because during recompiling of documentation I had error: Process terminated with code 3. 'B' is not recognized as an internal or external command, operable program or batch file.).
Any ideas?
I found the solution. If I want to exclude externals files using externalPattern I should to write pattern something like that:
externalPattern: "**/*/{A,B,C,D}.ts"
{ } = allows for a comma-separated list of "or" expressions
, = or
Useful for me was this comment from topic about regex in gruntfile.
According to this comment, the right syntax would be **/*/+(A|B|C|D).ts. Also, it looks like you are running into a problem with your shell trying to interpret the pipe characters, so try adding double quotes around the whole thing:
externalPattern: '"**/*/+(A|B|C|D).ts"'
In a traditional Ember app, I have something along the lines of this in my ember-cli-build.js:
//ember-cli-build.js
module.exports = function(defaults) {
var app = new EmberApp(defaults, {
babel: {
includePolyfill: true,
ignore: ['my-ember-ui/models/myFile.js'] // <-- question is here
},
Is there an equivalent to this when using an Ember Engine (or addon)? I couldn't find anything within ember-cli-babel or ember-engines.
I understand that ember-cli-build.js is just for the dummy app when using an engine, so I wouldn't make the change there. I attempted similar to above in the index.js file, but did not have any luck. The file was not ignored by babel. I need a way to ignore a particular file. Thanks!
Well, adding new rules to Cli.build.js is ok depends on what you want to do. However, I may have another solution that you can give it a try.
Babel will look for a .babelrc in the current directory of the file being transpiled. If one does not exist, it will travel up the directory tree until it finds either a .babelrc, or a package.json with a "babel": {} hash within.(.babelrc files are serializable JSON).
{
"plugins": ["transform-react-jsx"],
"ignore": [
"foo.js",
"bar/**/*.js"
]
}
or
{
"name": "my-package",
"version": "1.0.0",
"babel": {
// my babel config here
}
}
There should be another way which seems ok to use. the following does work:
babel src --out-dir build --ignore "**/*.test.js" // or simply add your file
For more information, you can read Babel document