Add dep_keys to generated_file() Collecting the outputs of build steps into a generated_file() currently requires the generated_file() to depend on everything that it walks, so that the collected files exist before its consumers run. With walk_deps, the walked targets are no longer dependencies at all, and nothing orders the build steps that produce the collected files before the consumers. This adds a dep_keys variable to generated_file(): metadata keys whose values are labels of the walked target itself or of its direct deps (resolved like walk_keys). The resulting targets become dependencies of the generated_file()'s Ninja build statement, in addition to its deps and data_deps. This allows depending on only the subset of walked targets that produce the collected data. Any dependency cycles created via dep_keys (when combined with walk_deps or validations) are detected by Ninja rather than during "gn gen". No visibility, testonly, or assert_no_deps checks are done for collected targets, since they are found through targets that already depend on them. The motivating use case is Chromium's JNI Zero, whose final registration step re-parses every .java file because it cannot depend on the .jni.pickle outputs of per-library steps without depending on all native libraries and on the APK that depends on it. Bug: 579603584 Change-Id: I51dd90c8d1b7e382dd7d4266f39fa80c849dcd68 Agent-Generated: true Reviewed-on: https://gn-review.googlesource.com/c/gn/+/28722 Reviewed-by: David Turner <digit@google.com> Commit-Queue: Andrew Grieve <agrieve@google.com>
diff --git a/docs/reference.md b/docs/reference.md index 2043388..e55a46c 100644 --- a/docs/reference.md +++ b/docs/reference.md
@@ -128,6 +128,7 @@ * [data_deps: [label list] Non-linked dependencies.](#var_data_deps) * [data_keys: [string list] Keys from which to collect metadata.](#var_data_keys) * [defines: [string list] C preprocessor defines.](#var_defines) + * [dep_keys: [string list] Metadata keys listing targets to depend on.](#var_dep_keys) * [depfile: [string] File name for input dependencies for actions.](#var_depfile) * [deps: [label list] Private linked dependencies.](#var_deps) * [externs: [scope] Set of Rust crate-dependency pairs.](#var_externs) @@ -580,6 +581,7 @@ configs [--tree] (see below) data_keys defines [--blame] + dep_keys depfile deps [--all] [--tree] (see below) framework_dirs @@ -2190,8 +2192,9 @@ but setting `collect_validations_metadata = true` changes this behavior. The metadata collection walk can also start from targets that are not - dependencies of the generated_file() target by listing them in `walk_deps`. - See `gn help walk_deps`. + dependencies of the generated_file() target by listing them in `walk_deps`, + and the targets that produce the collected data can be made dependencies of + it with `dep_keys`. See `gn help walk_deps` and `gn help dep_keys`. ``` #### **Variables** @@ -2204,7 +2207,7 @@ output_extension, output_name, public, sources, testonly, visibility Generated file: contents, data_keys, rebase, walk_keys, output_conversion, - outputs, collect_validations_metadata, walk_deps + outputs, collect_validations_metadata, walk_deps, dep_keys ``` #### **Example (metadata collection)** @@ -6104,6 +6107,52 @@ ``` defines = [ "AWESOME_FEATURE", "LOG_LEVEL=3" ] ``` +### <a name="var_dep_keys"></a>**dep_keys**: Metadata keys listing targets to depend on. [Back to Top](#gn-reference) + +``` + Valid on generated_file() targets that collect metadata. + + During the metadata walk, values of these keys are resolved as labels + relative to each walked target and added as Ninja dependencies of the + generated_file() target. Any resulting dependency cycles are reported by + Ninja. + + Each resolved label must refer to the walked target itself or one of its + direct deps, public_deps, or data_deps (or direct validations when + collect_validations_metadata is true); otherwise GN fails with an error. + All resolved labels are therefore part of the transitive dependencies of the + generated_file() target's deps, public_deps, data_deps, and walk_deps (plus + transitive validations when collect_validations_metadata is true). + + No "visibility", "testonly", or "assert_no_deps" checks are done for + collected targets, since they are found through targets that already depend + on them. They are not reported as dependencies by commands such as "gn desc" + and "gn refs". + + See "gn help generated_file" and "gn help walk_deps". +``` + +#### **Example** + +``` + action("foo_pickle") { + ... + outputs = [ "$target_gen_dir/foo.pickle" ] + metadata = { + pickle_files = [ "$target_gen_dir/foo.pickle" ] + pickle_deps = [ ":foo_pickle" ] + } + } + + # Lists the pickle files of an app, and depends only on the targets that + # create them. + generated_file("app_pickles") { + outputs = [ "$target_gen_dir/app_pickles.txt" ] + data_keys = [ "pickle_files" ] + dep_keys = [ "pickle_deps" ] + walk_deps = [ ":app" ] + } +``` ### <a name="var_depfile"></a>**depfile**: [string] File name for input dependencies for actions. [Back to Top](#gn-reference) ``` @@ -7458,7 +7507,8 @@ data_deps, without making them Ninja dependencies of the generated_file() target (allowing cycles with deps). Like "gen_deps", listed targets are generated when the generated_file() target is generated. Like "deps", they - are subject to "visibility" and "testonly" checks. + are subject to "visibility" and "testonly" checks. Often paired with + "dep_keys". They are used only when writing the generated_file() target itself, and not when the metadata walk of another target passes through it. The metadata of @@ -8592,7 +8642,8 @@ known contents to a file or 'data_keys' to aggregate metadata and write the result to a file. It can also specify 'walk_keys' (to restrict the metadata collection), 'walk_deps' (to collect metadata from targets that are not - dependencies), 'output_conversion', and 'rebase'. + dependencies), 'dep_keys' (to depend on targets listed in metadata), + 'output_conversion', and 'rebase'. ``` #### **Collection and Aggregation**
diff --git a/src/gn/command_desc.cc b/src/gn/command_desc.cc index af2a60b..d853363 100644 --- a/src/gn/command_desc.cc +++ b/src/gn/command_desc.cc
@@ -306,6 +306,7 @@ {variables::kRebase, DefaultHandler}, {variables::kWalkKeys, DefaultHandler}, {variables::kCollectValidationsMetadata, DefaultHandler}, + {variables::kDepKeys, DefaultHandler}, {variables::kWalkDeps, DefaultHandler}, {variables::kWeakFrameworks, DefaultHandler}, {variables::kWeakLibraries, DefaultHandler}, @@ -413,6 +414,7 @@ HandleProperty(variables::kRustflags, handler_map, v, dict); HandleProperty(variables::kWalkKeys, handler_map, v, dict); HandleProperty(variables::kWalkDeps, handler_map, v, dict); + HandleProperty(variables::kDepKeys, handler_map, v, dict); HandleProperty(variables::kWeakFrameworks, handler_map, v, dict); HandleProperty(variables::kWeakLibraries, handler_map, v, dict); HandleProperty(variables::kWriteOutputConversion, handler_map, v, dict); @@ -522,6 +524,7 @@ configs [--tree] (see below) data_keys defines [--blame] + dep_keys depfile deps [--all] [--tree] (see below) framework_dirs
diff --git a/src/gn/desc_builder.cc b/src/gn/desc_builder.cc index 3a3b864..d21bfc6 100644 --- a/src/gn/desc_builder.cc +++ b/src/gn/desc_builder.cc
@@ -62,6 +62,7 @@ // "data_keys" : [ list of target data keys ] // "walk_keys" : [ list of target walk keys ] // "walk_deps" : [ list of metadata walk roots ] +// "dep_keys" : [ list of target dep keys ] // "crate_root" : "root file of a Rust target" // "crate_name" : "name of a Rust target" // "rebase" : true or false @@ -590,6 +591,10 @@ res->SetKey(variables::kCollectValidationsMetadata, base::Value(target_->collect_validations_metadata())); } + if (what(variables::kDepKeys) && !target_->dep_keys().empty()) { + res->SetWithoutPathExpansion(variables::kDepKeys, + RenderValue(target_->dep_keys())); + } if (what(variables::kWalkDeps) && !target_->walk_deps().empty()) { res->SetWithoutPathExpansion(variables::kWalkDeps, RenderLabels(target_->walk_deps()));
diff --git a/src/gn/functions_target.cc b/src/gn/functions_target.cc index e05a36f..ffa6d21 100644 --- a/src/gn/functions_target.cc +++ b/src/gn/functions_target.cc
@@ -990,15 +990,16 @@ but setting `collect_validations_metadata = true` changes this behavior. The metadata collection walk can also start from targets that are not - dependencies of the generated_file() target by listing them in `walk_deps`. - See `gn help walk_deps`. + dependencies of the generated_file() target by listing them in `walk_deps`, + and the targets that produce the collected data can be made dependencies of + it with `dep_keys`. See `gn help walk_deps` and `gn help dep_keys`. Variables )" DEPENDENT_CONFIG_VARS DEPS_VARS GENERAL_TARGET_VARS R"( Generated file: contents, data_keys, rebase, walk_keys, output_conversion, - outputs, collect_validations_metadata, walk_deps + outputs, collect_validations_metadata, walk_deps, dep_keys Example (metadata collection)
diff --git a/src/gn/functions_target_unittest.cc b/src/gn/functions_target_unittest.cc index 760006e..83af8f8 100644 --- a/src/gn/functions_target_unittest.cc +++ b/src/gn/functions_target_unittest.cc
@@ -265,3 +265,39 @@ EXPECT_NE(std::string::npos, err.message().find("won't be used.")) << err.message(); } + +// Checks parsing of dep_keys on generated_file(). +TEST_F(FunctionsTarget, GeneratedFileDepKeys) { + TestWithScope setup; + Scope::ItemVector item_collector; + setup.scope()->set_item_collector(&item_collector); + + TestParseInput input( + R"(generated_file("foo") { + outputs = [ "$root_out_dir/foo.json" ] + data_keys = [ "files" ] + dep_keys = [ "file_deps" ] + })"); + ASSERT_FALSE(input.has_error()); + Err err; + input.parsed()->Execute(setup.scope(), &err); + ASSERT_SUCCESS(err); + + ASSERT_EQ(1u, item_collector.size()); + const Target* target = item_collector[0]->AsTarget(); + ASSERT_TRUE(target); + ASSERT_EQ(1u, target->dep_keys().size()); + EXPECT_EQ("file_deps", target->dep_keys()[0]); + + // dep_keys may not be used with contents. + TestParseInput bad_input( + R"(generated_file("bar") { + outputs = [ "$root_out_dir/bar.json" ] + contents = "bar" + dep_keys = [ "file_deps" ] + })"); + ASSERT_FALSE(bad_input.has_error()); + err = Err(); + bad_input.parsed()->Execute(setup.scope(), &err); + EXPECT_FAILURE(err, "won't be used."); +}
diff --git a/src/gn/generated_file_target_generator.cc b/src/gn/generated_file_target_generator.cc index dea6a10..e227e4e 100644 --- a/src/gn/generated_file_target_generator.cc +++ b/src/gn/generated_file_target_generator.cc
@@ -59,6 +59,8 @@ return; if (!FillWalkDeps()) return; + if (!FillKeys(variables::kDepKeys, &target_->dep_keys())) + return; if (!FillOutputConversion()) return;
diff --git a/src/gn/generated_file_writer.cc b/src/gn/generated_file_writer.cc index ab8e9aa..c86875c 100644 --- a/src/gn/generated_file_writer.cc +++ b/src/gn/generated_file_writer.cc
@@ -15,7 +15,9 @@ #include "gn/trace.h" #include "gn/value.h" -bool WriteGeneratedFileToDisk(const Target* target, Err* err) { +bool WriteGeneratedFileToDisk(const Target* target, + Err* err, + UniqueVector<const Target*>* deps_collected) { const Settings* settings = target->settings(); std::vector<SourceFile> outputs_as_sources; @@ -43,7 +45,8 @@ metadata_walk_trace.SetToolchain(target->settings()->toolchain_label()); if (!target->GetMetadata(target->data_keys(), target->walk_keys(), target->rebase(), /*deps_only = */ true, - &contents.list_value(), &targets_walked, err)) { + &contents.list_value(), &targets_walked, err, + deps_collected)) { return false; } } else {
diff --git a/src/gn/generated_file_writer.h b/src/gn/generated_file_writer.h index 833ed97..b4421ab 100644 --- a/src/gn/generated_file_writer.h +++ b/src/gn/generated_file_writer.h
@@ -5,10 +5,15 @@ #ifndef TOOLS_GN_GENERATED_FILE_WRITER_H_ #define TOOLS_GN_GENERATED_FILE_WRITER_H_ +#include "gn/unique_vector.h" + class Err; class Target; -// Write the content of a given generated_file() target to disk. -bool WriteGeneratedFileToDisk(const Target* target, Err* err); +// Write the content of a given generated_file() target to disk. Targets found +// through its dep_keys are added to |deps_collected|. +bool WriteGeneratedFileToDisk(const Target* target, + Err* err, + UniqueVector<const Target*>* deps_collected); #endif // TOOLS_GN_GENERATED_FILE_WRITER_H_
diff --git a/src/gn/metadata.cc b/src/gn/metadata.cc index a981a61..3cf7ff1 100644 --- a/src/gn/metadata.cc +++ b/src/gn/metadata.cc
@@ -30,7 +30,8 @@ known contents to a file or 'data_keys' to aggregate metadata and write the result to a file. It can also specify 'walk_keys' (to restrict the metadata collection), 'walk_deps' (to collect metadata from targets that are not - dependencies), 'output_conversion', and 'rebase'. + dependencies), 'dep_keys' (to depend on targets listed in metadata), + 'output_conversion', and 'rebase'. Collection and Aggregation
diff --git a/src/gn/ninja_generated_file_target_writer.cc b/src/gn/ninja_generated_file_target_writer.cc index 7314e72..a36977f 100644 --- a/src/gn/ninja_generated_file_target_writer.cc +++ b/src/gn/ninja_generated_file_target_writer.cc
@@ -4,6 +4,8 @@ #include "gn/ninja_generated_file_target_writer.h" +#include <vector> + #include "gn/generated_file_writer.h" #include "gn/output_conversion.h" #include "gn/output_file.h" @@ -24,33 +26,36 @@ void NinjaGeneratedFileTargetWriter::Run() { // Write the file. Targets whose metadata walks require a fully resolved // graph are only written after all targets are resolved (see command_gen.cc). + UniqueVector<const Target*> deps_collected; Err err; - if (!WriteGeneratedFileToDisk(target_, &err)) { + if (!WriteGeneratedFileToDisk(target_, &err, &deps_collected)) { g_scheduler->FailWithError(err); + return; } // A generated_file target should generate a phony target with dependencies - // on each of the deps and data_deps in the target. The actual collection is - // done at gen time, but to have correct input deps in ninja, we add output - // from generated_file targets as deps for the stamp. + // on each of the deps, data_deps, and dep_keys targets. The actual collection + // is done at gen time, but to have correct input deps in ninja, we add output + // from generated_file targets as deps for the stamp. Any dependency cycles + // introduced via dep_keys (when combined with walk_deps or validations) are + // detected by Ninja. std::vector<OutputFile> output_files = target_->computed_outputs(); std::vector<OutputFile> data_output_files; + TargetSet seen; + auto add_deps = [&](const auto& deps, bool data_only) { + for (const Target* dep : deps) { + if (dep->has_dependency_output() && seen.add(dep)) { + if (data_only || dep->IsDataOnly()) + data_output_files.push_back(dep->dependency_output()); + else + output_files.push_back(dep->dependency_output()); + } + } + }; const auto& target_deps = resolved().GetTargetDeps(target_); - for (const Target* dep : target_deps.linked_deps()) { - if (!dep->has_dependency_output()) { - continue; - } - if (dep->IsDataOnly()) { - data_output_files.push_back(dep->dependency_output()); - } else { - output_files.push_back(dep->dependency_output()); - } - } - - for (const Target* data_dep : target_deps.data_deps()) { - if (data_dep->has_dependency_output()) - data_output_files.push_back(data_dep->dependency_output()); - } + add_deps(target_deps.linked_deps(), /*data_only=*/false); + add_deps(deps_collected, /*data_only=*/false); + add_deps(target_deps.data_deps(), /*data_only=*/true); WriteStampOrPhonyForTarget(output_files, data_output_files); }
diff --git a/src/gn/setup_unittest.cc b/src/gn/setup_unittest.cc index e8bde00..3df05a0 100644 --- a/src/gn/setup_unittest.cc +++ b/src/gn/setup_unittest.cc
@@ -1013,6 +1013,14 @@ outputs = [ "$target_gen_dir/foo.pickle" ] metadata = { pickle_files = [ "$target_gen_dir/foo.pickle" ] + pickle_deps = [ ":pickle" ] + } +} +action("direct_pickle") { + script = "//script.py" + outputs = [ "$target_gen_dir/bar.pickle" ] + metadata = { + pickle_deps = [ ":direct_pickle" ] } } group("lib") { @@ -1021,7 +1029,9 @@ generated_file("pickles") { outputs = [ "$target_gen_dir/pickles.txt" ] data_keys = [ "pickle_files" ] + dep_keys = [ "pickle_deps" ] rebase = root_build_dir + deps = [ ":direct_pickle" ] walk_deps = )" + walk_deps + R"( } action("final") { @@ -1045,10 +1055,11 @@ } // namespace -// Tests that walk_deps can form a cycle with deps without becoming -// dependencies of the generated_file(), and that undefined walk_deps targets -// are rejected. -TEST(GenCommandTest, WalkDeps) { +// Tests that walk_deps can form a cycle with deps, that only the targets +// collected through dep_keys become dependencies of the generated_file() +// (once, even if they are also listed in its deps), and that undefined +// walk_deps targets are rejected. +TEST(GenCommandTest, WalkDepsAndDepKeys) { base::ScopedTempDir in_temp_dir; ASSERT_TRUE(in_temp_dir.CreateUniqueTempDir()); base::FilePath in_path = base::MakeAbsoluteFilePath(in_temp_dir.GetPath()); @@ -1061,8 +1072,9 @@ ReadFile(build_path.Append(FILE_PATH_LITERAL("gen/pickles.txt")))); std::string ninja = ReadFile(build_path.Append(FILE_PATH_LITERAL("toolchain.ninja"))); - EXPECT_NE(std::string::npos, - ninja.find("build phony/pickles: phony gen/pickles.txt\n")) + EXPECT_TRUE( + ninja.contains("build phony/pickles: phony gen/pickles.txt " + "phony/direct_pickle phony/pickle\n")) << ninja; // Undefined target in walk_deps fails resolution.
diff --git a/src/gn/target.cc b/src/gn/target.cc index 2bf4f27..d03a092 100644 --- a/src/gn/target.cc +++ b/src/gn/target.cc
@@ -1319,16 +1319,20 @@ MetadataWalker(const KeyList& data_keys, const KeyList& walk_keys, + const KeyList& dep_keys, const SourceDir& rebase_dir, bool collect_validations_metadata, std::vector<Value>* result, + UniqueVector<const Target*>* deps_collected, TargetSet& targets_walked, Err& err) : data_keys_(data_keys), walk_keys_(walk_keys), + dep_keys_(dep_keys), rebase_dir_(rebase_dir), collect_validations_metadata_(collect_validations_metadata), result_(result), + deps_collected_(deps_collected), targets_walked_(targets_walked), err_(err) {} @@ -1355,6 +1359,22 @@ rebase_dir_, &next_walk_labels, ¤t_result, &err_)) { return false; } + if (deps_collected_) { + for (const auto& key : dep_keys_) { + auto iter = target.metadata().contents().find(key); + if (iter == target.metadata().contents().end()) + continue; + for (const Value& value : iter->second.list_value()) { + if (!value.VerifyTypeIs(Value::STRING, &err_)) + return false; + const Target* dep = + ResolveDepLabel(target, value, /*allow_self=*/true); + if (!dep) + return false; + deps_collected_->push_back(dep); + } + } + } } // Gather walk keys and find the appropriate target. Targets identified in @@ -1396,7 +1416,7 @@ } // Otherwise, go down into the specified dep if we haven't walked it yet. - const Target* dep = ResolveDepLabel(target, next); + const Target* dep = ResolveDepLabel(target, next, /*allow_self=*/false); if (!dep || (targets_walked_.add(dep) && !Walk(*dep, false))) return false; } @@ -1408,9 +1428,11 @@ private: // Resolves |value| as a label relative to |target|, and returns the - // dependency of |target| that it refers to. If there is none, sets |err_| and - // returns nullptr. - const Target* ResolveDepLabel(const Target& target, const Value& value) { + // dependency of |target| that it refers to (or |target| itself when + // |allow_self| is true). If there is none, sets |err_| and returns nullptr. + const Target* ResolveDepLabel(const Target& target, + const Value& value, + bool allow_self) { const Settings* settings = target.settings(); const Label& toolchain_label = settings->toolchain_label(); Label label = Label::Resolve(target.label().dir(), @@ -1422,6 +1444,8 @@ return nullptr; } // Match against the label with the toolchain. + if (allow_self && target.label().Matches(label, toolchain_label)) + return ⌖ for (const auto& dep : target.GetDeps(Target::DEPS_ALL)) { if (dep.label.Matches(label, toolchain_label)) return dep.ptr; @@ -1445,9 +1469,11 @@ const KeyList& data_keys_; const KeyList& walk_keys_; + const KeyList& dep_keys_; const SourceDir& rebase_dir_; bool collect_validations_metadata_; std::vector<Value>* result_; + UniqueVector<const Target*>* deps_collected_; TargetSet& targets_walked_; Err& err_; }; @@ -1460,10 +1486,11 @@ bool deps_only, std::vector<Value>* result, TargetSet* targets_walked, - Err* err) const { - MetadataWalker walker(data_keys, walk_keys, rebase_dir, - collect_validations_metadata_, result, *targets_walked, - *err); + Err* err, + UniqueVector<const Target*>* deps_collected) const { + MetadataWalker walker(data_keys, walk_keys, dep_keys(), rebase_dir, + collect_validations_metadata_, result, deps_collected, + *targets_walked, *err); return walker.Walk(*this, deps_only); }
diff --git a/src/gn/target.h b/src/gn/target.h index 3c5ae32..908f3a6 100644 --- a/src/gn/target.h +++ b/src/gn/target.h
@@ -211,14 +211,17 @@ bool has_metadata() const { return metadata_.get(); } // Get metadata from this target and its dependencies. This is intended to - // be called after the target is resolved. + // be called after the target is resolved. If |deps_collected| is not null, + // the values of this target's dep_keys() found in the metadata of walked + // targets are resolved and added to |deps_collected|. bool GetMetadata(const std::vector<std::string>& data_keys, const std::vector<std::string>& walk_keys, const SourceDir& rebase_dir, bool deps_only, std::vector<Value>* result, TargetSet* targets_walked, - Err* err) const; + Err* err, + UniqueVector<const Target*>* deps_collected = nullptr) const; // GeneratedFile-related methods. bool GenerateFile(Err* err) const; @@ -230,6 +233,7 @@ SourceDir rebase_; std::vector<std::string> data_keys_; std::vector<std::string> walk_keys_; + std::vector<std::string> dep_keys_; LabelTargetVector walk_deps_; }; const GeneratedFile& generated_file() const; @@ -255,6 +259,10 @@ return generated_file().walk_keys_; } std::vector<std::string>& walk_keys() { return generated_file().walk_keys_; } + const std::vector<std::string>& dep_keys() const { + return generated_file().dep_keys_; + } + std::vector<std::string>& dep_keys() { return generated_file().dep_keys_; } const LabelTargetVector& walk_deps() const { return generated_file().walk_deps_; }
diff --git a/src/gn/target_unittest.cc b/src/gn/target_unittest.cc index 6521776..5987cf6 100644 --- a/src/gn/target_unittest.cc +++ b/src/gn/target_unittest.cc
@@ -1552,6 +1552,84 @@ EXPECT_EQ(result, std::vector<Value>{Value(nullptr, "foo.pickle")}); } +// Tests that targets listed under dep_keys are resolved relative to the walked +// target, and must be the walked target itself or one of its direct deps (or +// direct validations when collect_validations_metadata is true). +TEST(TargetTest, CollectMetadataWithDepKeys) { + TestWithScope setup; + auto set_file_deps = [](Target& target, const std::string& label) { + Value deps(nullptr, Value::LIST); + deps.list_value().push_back(Value(nullptr, label)); + target.metadata().contents()["file_deps"] = std::move(deps); + }; + + // Lists itself under dep_keys. + TestTarget pickle(setup, "//foo:pickle", Target::ACTION); + set_file_deps(pickle, ":pickle"); + + TestTarget other(setup, "//foo:other", Target::ACTION); + + // Lists its direct dependency under dep_keys. + TestTarget lib(setup, "//foo:lib", Target::SOURCE_SET); + lib.private_deps().push_back(LabelTargetPair(&pickle)); + lib.private_deps().push_back(LabelTargetPair(&other)); + set_file_deps(lib, "//foo:other"); + + TestTarget gen(setup, "//foo:gen", Target::GENERATED_FILE); + gen.private_deps().push_back(LabelTargetPair(&lib)); + gen.dep_keys().push_back("file_deps"); + + Err err; + UniqueVector<const Target*> deps_collected; + auto collect = [&]() { + std::vector<Value> result; + TargetSet targets; + deps_collected.clear(); + err = Err(); + return gen.GetMetadata({"files"}, {""}, SourceDir(), true, &result, + &targets, &err, &deps_collected); + }; + + EXPECT_TRUE(collect()); + EXPECT_SUCCESS(err); + EXPECT_EQ(deps_collected.vector(), + (std::vector<const Target*>{&other, &pickle})); + + TestTarget val(setup, "//foo:val", Target::ACTION); + other.validations().push_back(LabelTargetPair(&val)); + + auto expect_bad_dep = [&](const std::string& bad_label) { + set_file_deps(other, bad_label); + EXPECT_FALSE(collect()); + std::string full_label = + bad_label[0] == ':' ? "//foo" + bad_label : bad_label; + EXPECT_EQ("I was expecting " + full_label + + "(//toolchain:default) to be a dependency of " + "//foo:other(//toolchain:default). Make sure it's included " + "in the deps or data_deps, and that you've specified the " + "appropriate toolchain.", + err.message()); + }; + + // Labels that are neither the walked target itself nor a direct dep fail, + // including targets outside the transitive closure, non-direct targets in + // the transitive closure, and validations when collect_validations_metadata + // is false. + for (const char* bad_label : {"//foo:missing", ":pickle", ":val"}) + expect_bad_dep(bad_label); + + // With collect_validations_metadata = true, validations are accepted, while + // non-deps still fail. + gen.set_collect_validations_metadata(true); + set_file_deps(other, ":val"); + EXPECT_TRUE(collect()); + EXPECT_SUCCESS(err); + EXPECT_EQ(deps_collected.vector(), + (std::vector<const Target*>{&other, &pickle, &val})); + + expect_bad_dep("//foo:missing"); +} + // walk_deps are subject to the same visibility and testonly checks as deps. TEST_F(TargetTest, WalkDepsVisibilityAndTestonly) { TestWithScope setup;
diff --git a/src/gn/variables.cc b/src/gn/variables.cc index e4c9a5e..28ab5cb 100644 --- a/src/gn/variables.cc +++ b/src/gn/variables.cc
@@ -1175,6 +1175,54 @@ defines = [ "AWESOME_FEATURE", "LOG_LEVEL=3" ] )"; +const char kDepKeys[] = "dep_keys"; +const char kDepKeys_HelpShort[] = + "dep_keys: [string list] Metadata keys listing targets to depend on."; +const char kDepKeys_Help[] = + R"(dep_keys: Metadata keys listing targets to depend on. + + Valid on generated_file() targets that collect metadata. + + During the metadata walk, values of these keys are resolved as labels + relative to each walked target and added as Ninja dependencies of the + generated_file() target. Any resulting dependency cycles are reported by + Ninja. + + Each resolved label must refer to the walked target itself or one of its + direct deps, public_deps, or data_deps (or direct validations when + collect_validations_metadata is true); otherwise GN fails with an error. + All resolved labels are therefore part of the transitive dependencies of the + generated_file() target's deps, public_deps, data_deps, and walk_deps (plus + transitive validations when collect_validations_metadata is true). + + No "visibility", "testonly", or "assert_no_deps" checks are done for + collected targets, since they are found through targets that already depend + on them. They are not reported as dependencies by commands such as "gn desc" + and "gn refs". + + See "gn help generated_file" and "gn help walk_deps". + +Example + + action("foo_pickle") { + ... + outputs = [ "$target_gen_dir/foo.pickle" ] + metadata = { + pickle_files = [ "$target_gen_dir/foo.pickle" ] + pickle_deps = [ ":foo_pickle" ] + } + } + + # Lists the pickle files of an app, and depends only on the targets that + # create them. + generated_file("app_pickles") { + outputs = [ "$target_gen_dir/app_pickles.txt" ] + data_keys = [ "pickle_files" ] + dep_keys = [ "pickle_deps" ] + walk_deps = [ ":app" ] + } +)"; + const char kDepfile[] = "depfile"; const char kDepfile_HelpShort[] = "depfile: [string] File name for input dependencies for actions."; @@ -2463,7 +2511,8 @@ data_deps, without making them Ninja dependencies of the generated_file() target (allowing cycles with deps). Like "gen_deps", listed targets are generated when the generated_file() target is generated. Like "deps", they - are subject to "visibility" and "testonly" checks. + are subject to "visibility" and "testonly" checks. Often paired with + "dep_keys". They are used only when writing the generated_file() target itself, and not when the metadata walk of another target passes through it. The metadata of @@ -2676,6 +2725,7 @@ INSERT_VARIABLE(DataDeps) INSERT_VARIABLE(DataKeys) INSERT_VARIABLE(Defines) + INSERT_VARIABLE(DepKeys) INSERT_VARIABLE(Depfile) INSERT_VARIABLE(Deps) INSERT_VARIABLE(Mnemonic)
diff --git a/src/gn/variables.h b/src/gn/variables.h index b2be92c..a3a5a19 100644 --- a/src/gn/variables.h +++ b/src/gn/variables.h
@@ -198,6 +198,10 @@ extern const char kDefines_HelpShort[]; extern const char kDefines_Help[]; +extern const char kDepKeys[]; +extern const char kDepKeys_HelpShort[]; +extern const char kDepKeys_Help[]; + extern const char kDepfile[]; extern const char kDepfile_HelpShort[]; extern const char kDepfile_Help[];