| commit | 4a17a836d84c3946e04ffb94fc21441979367d63 | [log] [tgz] |
|---|---|---|
| author | David 'Digit' Turner <digit@google.com> | Mon Dec 20 15:53:30 2021 +0100 |
| committer | Commit Bot <commit-bot@chromium.org> | Thu Jan 06 17:28:34 2022 +0000 |
| tree | c11df6ece185efa051c3dedd5f4e7b7fe0f62ce9 | |
| parent | 387b368dfe63fec317f8e609d90c634807f2764e [diff] |
Simplify InheritedLibraries
This CL simplifies the implementation of InheritedLibraries
to use a UniqueVector<const Target*> as well as a parallel
vector of booleans (for the public flags). This doesn't
change the semantics of this type.
Combined with the previous CL, measurements show that this
saves about 10% of 'gn gen' time, and 8% of peak RAM usage
for both Fuchsia and Chromium.
Bug: None
--------------------------------------------------------
For Fuchsia
gn-main
Done. Made 184670 targets from 5446 files in 12960ms
Done. Made 184670 targets from 5446 files in 13277ms
Done. Made 184670 targets from 5446 files in 13713ms *
Done. Made 184670 targets from 5446 files in 13882ms
Done. Made 184670 targets from 5446 files in 14259ms
3271324
3275512
3280504 *
3282032
3293832
gn-inherited-libraries
Done. Made 184670 targets from 5446 files in 12248ms
Done. Made 184670 targets from 5446 files in 12267ms
Done. Made 184670 targets from 5446 files in 12326ms *
Done. Made 184670 targets from 5446 files in 12697ms
Done. Made 184670 targets from 5446 files in 12770ms
DIFF = 13713 - 12326 = 1387ms (-10%)
3026696
3027940
3032296 *
3035976
3037732
DIFF = 3280504 - 3032296 = 248208 = 242 MiB (-7.5%)
For Chromium
gn-main
Done. Made 17155 targets from 3025 files in 3289ms
Done. Made 17155 targets from 3025 files in 3314ms
Done. Made 17155 targets from 3025 files in 3328ms *
Done. Made 17155 targets from 3025 files in 3362ms
Done. Made 17155 targets from 3025 files in 3488ms
666764
670480
674792 *
676516
692592
gn-inherited-libraries
Done. Made 17155 targets from 3025 files in 2987ms
Done. Made 17155 targets from 3025 files in 3008ms
Done. Made 17155 targets from 3025 files in 3001ms *
Done. Made 17155 targets from 3025 files in 3021ms
Done. Made 17155 targets from 3025 files in 3106ms
DIFF = 3328 - 3001 = 327ms (-9.8%)
613600
616748
618460 *
621900
626768
DIFF = 674792 - 618460 = 56332 = 55 MiB (-8.3%)
Change-Id: I245367f121cb2b2d542db171146b494209793028
Reviewed-on: https://gn-review.googlesource.com/c/gn/+/12700
Reviewed-by: Brett Wilson <brettw@chromium.org>
Commit-Queue: David Turner <digit@google.com>
GN is a meta-build system that generates build files for Ninja.
Related resources:
GN is currently used as the build system for Chromium, Fuchsia, and related projects. Some strengths of GN are:
It is designed for large projects and large teams. It scales efficiently to many thousands of build files and tens of thousands of source files.
It has a readable, clean syntax. Once a build is set-up, it is generally easy for people with no backround in GN to make basic edits to the build.
It is designed for multi-platform projects. It can cleanly express many complicated build variants across different platforms. A single build invocation can target multiple platforms.
It supports multiple parallel output directories, each with their own configuration. This allows a developer to maintain builds targeting debug, release, or different platforms in parallel without forced rebuilds when switching.
It has a focus on correctness. GN checks for the correct dependencies, inputs, and outputs to the extent possible, and has a number of tools to allow developers to ensure the build evolves as desired (for example, gn check, testonly, assert_no_deps).
It has comprehensive build-in help available from the command-line.
Although small projects successfully use GN, the focus on large projects has some disadvanages:
GN has the goal of being minimally expressive. Although it can be quite flexible, a design goal is to direct members of a large team (who may not have much knowledge about the build) down an easy-to-understand, well-lit path. This isn't necessarily the correct trade-off for smaller projects.
The minimal build configuration is relatively heavyweight. There are several files required and the exact way all compilers are linkers are run must be specified in the configuration (see “Examples” below). There is no default compiler configuration.
It is not easily composable. GN is designed to compile a single large project with relatively uniform settings and rules. Projects like Chromium do bring together multiple repositories from multiple teams, but the projects must agree on some conventions in the build files to allow this to work.
GN is designed with the expectation that the developers building a project want to compile an identical configuration. So while builds can integrate with the user‘s environment like the CXX and CFLAGS variables if they want, this is not the default and most project’s builds do not do this. The result is that many GN projects do not integrate well with other systems like ebuild.
There is no simple release scheme (see “Versioning and distribution” below). Projects are expected to manage the version of GN they require. Getting an appropriate GN binary can be a hurdle for new contributors to a project. Since GN is relatively uncommon, it can be more difficult to find information and examples.
GN can generate Ninja build files for C, C++, Rust, Objective C, and Swift source on most popular platforms. Other languages can be compiled using the general “action” rules which are executed by Python or another scripting language (Google does this to compile Java and Go). But because this is not as clean, generally GN is only used when the bulk of the build is in one of the main built-in languages.
You can download the latest version of GN binary for Linux, macOS and Windows from Google's build infrastructure (see “Versioning and distribution” below for how this is expected to work).
Alternatively, you can build GN from source with a C++17 compiler:
git clone https://gn.googlesource.com/gn cd gn python build/gen.py ninja -C out # To run tests: out/gn_unittests
On Windows, it is expected that cl.exe, link.exe, and lib.exe can be found in PATH, so you'll want to run from a Visual Studio command prompt, or similar.
On Linux, Mac and z/OS, the default compiler is clang++, a recent version is expected to be found in PATH. This can be overridden by setting CC, CXX, and AR.
On z/OS, building GN requires ZOSLIB to be installed, as described at that URL. When building with build/gen.py, use the option --zoslib-dir to specify the path to ZOSLIB:
cd gn python build/gen.py --zoslib-dir /path/to/zoslib
By default, if you don't specify --zoslib-dir, gn/build/gen.py expects to find zoslib directory under gn/third_party/.
There is a simple example in examples/simple_build directory that is a good place to get started with the minimal configuration.
To build and run the simple example with the default gcc compiler:
cd examples/simple_build ../../out/gn gen -C out ninja -C out ./out/hello
For a maximal configuration see the Chromium setup:
and the Fuchsia setup:
If you find a bug, you can see if it is known or report it in the bug database.
GN uses Gerrit for code review hosted at gn-review.googlesource.com. The short version of how to patch is:
Register at https://gn-review.googlesource.com. ... edit code ... ninja -C out && out/gn_unittests
Then, to upload a change for review:
git commit git push origin HEAD:refs/for/main
The first time you do this you'll get an error from the server about a missing change-ID. Follow the directions in the error message to install the change-ID hook and run git commit --amend to apply the hook to the current commit.
When revising a change, use:
git commit --amend git push origin HEAD:refs/for/main
which will add the new changes to the existing code review, rather than creating a new one.
We ask that all contributors sign Google's Contributor License Agreement (either individual or corporate as appropriate, select ‘any other Google project’).
You may ask questions and follow along with GN‘s development on Chromium’s gn-dev@ Google Group.
Most open-source projects are designed to use the developer‘s computer’s current toolchain such as compiler, linker, and build tool. But the large centrally controlled projects that GN is designed for typically want a more hermetic environment. They will ensure that developers are using a specific compatible toolchain that is versioned with the code.
As a result, GN expects that the project choose the appropriate version of GN that will work with each version of the project. There is no “current stable version” of GN that is expected to work for all projects.
As a result, the GN developers do not maintain any packages in any of the various packaging systems (Debian, RedHat, HomeBrew, etc.). Some of these systems to have GN packages, but they are maintained by third parties and you should use them at your own risk. Instead, we recommend you refer your checkout tooling to download binaries for a specific hash from Google's build infrastructure or compile your own.
GN does not guarantee the backwards-compatibility of new versions and has no branches or versioning scheme beyond the sequence of commits to the main git branch (which is expected to be stable).
In practice, however, GN is very backwards-compatible. The core functionality has been stable for many years and there is enough GN code at Google alone to make non-backwards-compatible changes very difficult, even if they were desirable.
There have been discussions about adding a versioning scheme with some guarantees about backwards-compatibility, but nothing has yet been implemented.