upload gn binary to CAS from CQ
Currently, gn binary available in CIPD and gn binary built from my local
machine has some performance difference (gn in CIPD is faster). And that
makes local performance comparison is bit unreliable.
In some cases, I'd like to compare performance of gn binary built by
existing recipe pipeline instead of replicating them in local machine.
For such purpose, this CL changes recipe to upload gn binary to CAS from
CQ. And we can use uploaded binary from CQ for performance comparison
without submitting CLs.
I added project-gn-try-task-accounts to chromium-swarm-cas-read-write in
https://chrome-infra-auth.appspot.com/auth/change_log?auth_db_rev=44767
for this, and confirmed the recipe behavir with
$ led get-build 8818119718319643841 | led edit-recipe-bundle |\
led launch
https://luci-milo.appspot.com/raw/build/logs.chromium.org/gn/led/tikuta_google.com/9a9e553c22da28cace97964ec59a5a2139451542efac5c00f0d9711b04584198/+/build.proto
Bug: chromium:1062263
Change-Id: I676358f4715128d06027b673ba5144b68fa8212c
Reviewed-on: https://gn-review.googlesource.com/c/gn/+/13401
Reviewed-by: Petr Hosek <phosek@google.com>
Commit-Queue: Takuto Ikuta <tikuta@google.com>
diff --git a/infra/README.recipes.md b/infra/README.recipes.md
index 20e5a6b..5a6ea74 100644
--- a/infra/README.recipes.md
+++ b/infra/README.recipes.md
@@ -95,13 +95,13 @@
### *recipes* / [gn](/infra/recipes/gn.py)
-[DEPS](/infra/recipes/gn.py#10): [macos\_sdk](#recipe_modules-macos_sdk), [target](#recipe_modules-target), [windows\_sdk](#recipe_modules-windows_sdk), [recipe\_engine/buildbucket][recipe_engine/recipe_modules/buildbucket], [recipe\_engine/cipd][recipe_engine/recipe_modules/cipd], [recipe\_engine/context][recipe_engine/recipe_modules/context], [recipe\_engine/file][recipe_engine/recipe_modules/file], [recipe\_engine/json][recipe_engine/recipe_modules/json], [recipe\_engine/path][recipe_engine/recipe_modules/path], [recipe\_engine/platform][recipe_engine/recipe_modules/platform], [recipe\_engine/properties][recipe_engine/recipe_modules/properties], [recipe\_engine/python][recipe_engine/recipe_modules/python], [recipe\_engine/raw\_io][recipe_engine/recipe_modules/raw_io], [recipe\_engine/step][recipe_engine/recipe_modules/step]
+[DEPS](/infra/recipes/gn.py#10): [macos\_sdk](#recipe_modules-macos_sdk), [target](#recipe_modules-target), [windows\_sdk](#recipe_modules-windows_sdk), [recipe\_engine/buildbucket][recipe_engine/recipe_modules/buildbucket], [recipe\_engine/cas][recipe_engine/recipe_modules/cas], [recipe\_engine/cipd][recipe_engine/recipe_modules/cipd], [recipe\_engine/context][recipe_engine/recipe_modules/context], [recipe\_engine/file][recipe_engine/recipe_modules/file], [recipe\_engine/json][recipe_engine/recipe_modules/json], [recipe\_engine/path][recipe_engine/recipe_modules/path], [recipe\_engine/platform][recipe_engine/recipe_modules/platform], [recipe\_engine/properties][recipe_engine/recipe_modules/properties], [recipe\_engine/python][recipe_engine/recipe_modules/python], [recipe\_engine/raw\_io][recipe_engine/recipe_modules/raw_io], [recipe\_engine/step][recipe_engine/recipe_modules/step]
PYTHON_VERSION_COMPATIBILITY: PY3
Recipe for building GN.
-— **def [RunSteps](/infra/recipes/gn.py#104)(api, repository):**
+— **def [RunSteps](/infra/recipes/gn.py#105)(api, repository):**
### *recipes* / [macos\_sdk:examples/full](/infra/recipe_modules/macos_sdk/examples/full.py)
[DEPS](/infra/recipe_modules/macos_sdk/examples/full.py#7): [macos\_sdk](#recipe_modules-macos_sdk), [recipe\_engine/platform][recipe_engine/recipe_modules/platform], [recipe\_engine/properties][recipe_engine/recipe_modules/properties], [recipe\_engine/step][recipe_engine/recipe_modules/step]
@@ -125,6 +125,7 @@
— **def [RunSteps](/infra/recipe_modules/windows_sdk/examples/full.py#15)(api):**
[recipe_engine/recipe_modules/buildbucket]: https://chromium.googlesource.com/infra/luci/recipes-py.git/+/1b1ecd03e0b00399784c43add1465f685b6d1ab9/README.recipes.md#recipe_modules-buildbucket
+[recipe_engine/recipe_modules/cas]: https://chromium.googlesource.com/infra/luci/recipes-py.git/+/1b1ecd03e0b00399784c43add1465f685b6d1ab9/README.recipes.md#recipe_modules-cas
[recipe_engine/recipe_modules/cipd]: https://chromium.googlesource.com/infra/luci/recipes-py.git/+/1b1ecd03e0b00399784c43add1465f685b6d1ab9/README.recipes.md#recipe_modules-cipd
[recipe_engine/recipe_modules/context]: https://chromium.googlesource.com/infra/luci/recipes-py.git/+/1b1ecd03e0b00399784c43add1465f685b6d1ab9/README.recipes.md#recipe_modules-context
[recipe_engine/recipe_modules/file]: https://chromium.googlesource.com/infra/luci/recipes-py.git/+/1b1ecd03e0b00399784c43add1465f685b6d1ab9/README.recipes.md#recipe_modules-file
diff --git a/infra/recipes/gn.expected/cq_linux.json b/infra/recipes/gn.expected/cq_linux.json
index d19691e..574d790 100644
--- a/infra/recipes/gn.expected/cq_linux.json
+++ b/infra/recipes/gn.expected/cq_linux.json
@@ -703,6 +703,185 @@
},
{
"cmd": [],
+ "name": "release.linux-amd64.upload",
+ "~followup_annotations": [
+ "@@@STEP_NEST_LEVEL@2@@@"
+ ]
+ },
+ {
+ "cmd": [
+ "vpython",
+ "-u",
+ "RECIPE_MODULE[recipe_engine::file]/resources/fileutil.py",
+ "--json-output",
+ "/path/to/tmp/json",
+ "copy",
+ "RECIPE_MODULE[recipe_engine::cas]/resources/infra.sha1",
+ "/path/to/tmp/"
+ ],
+ "cwd": "[START_DIR]/gn",
+ "env": {
+ "AR": "[START_DIR]/cipd/bin/llvm-ar",
+ "CC": "[START_DIR]/cipd/bin/clang",
+ "CFLAGS": "--target=x86_64-linux-gnu --sysroot=[START_DIR]/cipd/sysroot",
+ "CXX": "[START_DIR]/cipd/bin/clang++",
+ "LDFLAGS": "--target=x86_64-linux-gnu --sysroot=[START_DIR]/cipd/sysroot -static-libstdc++"
+ },
+ "infra_step": true,
+ "luci_context": {
+ "realm": {
+ "name": "gn:try"
+ },
+ "resultdb": {
+ "current_invocation": {
+ "name": "invocations/build:8945511751514863184",
+ "update_token": "token"
+ },
+ "hostname": "rdbhost"
+ }
+ },
+ "name": "release.linux-amd64.upload.read infra revision",
+ "~followup_annotations": [
+ "@@@STEP_NEST_LEVEL@3@@@",
+ "@@@STEP_LOG_LINE@infra.sha1@git_revision:mock_infra_git_revision@@@",
+ "@@@STEP_LOG_END@infra.sha1@@@"
+ ]
+ },
+ {
+ "cmd": [],
+ "name": "release.linux-amd64.upload.install infra/tools/luci/cas",
+ "~followup_annotations": [
+ "@@@STEP_NEST_LEVEL@3@@@"
+ ]
+ },
+ {
+ "cmd": [
+ "vpython",
+ "-u",
+ "RECIPE_MODULE[recipe_engine::file]/resources/fileutil.py",
+ "--json-output",
+ "/path/to/tmp/json",
+ "ensure-directory",
+ "--mode",
+ "0777",
+ "[START_DIR]/cipd_tool/infra/tools/luci/cas/git_revision%3Amock_infra_git_revision"
+ ],
+ "cwd": "[START_DIR]/gn",
+ "env": {
+ "AR": "[START_DIR]/cipd/bin/llvm-ar",
+ "CC": "[START_DIR]/cipd/bin/clang",
+ "CFLAGS": "--target=x86_64-linux-gnu --sysroot=[START_DIR]/cipd/sysroot",
+ "CXX": "[START_DIR]/cipd/bin/clang++",
+ "LDFLAGS": "--target=x86_64-linux-gnu --sysroot=[START_DIR]/cipd/sysroot -static-libstdc++"
+ },
+ "infra_step": true,
+ "luci_context": {
+ "realm": {
+ "name": "gn:try"
+ },
+ "resultdb": {
+ "current_invocation": {
+ "name": "invocations/build:8945511751514863184",
+ "update_token": "token"
+ },
+ "hostname": "rdbhost"
+ }
+ },
+ "name": "release.linux-amd64.upload.install infra/tools/luci/cas.ensure package directory",
+ "~followup_annotations": [
+ "@@@STEP_NEST_LEVEL@4@@@"
+ ]
+ },
+ {
+ "cmd": [
+ "cipd",
+ "ensure",
+ "-root",
+ "[START_DIR]/cipd_tool/infra/tools/luci/cas/git_revision%3Amock_infra_git_revision",
+ "-ensure-file",
+ "infra/tools/luci/cas/${platform} git_revision:mock_infra_git_revision",
+ "-max-threads",
+ "0",
+ "-json-output",
+ "/path/to/tmp/json"
+ ],
+ "cwd": "[START_DIR]/gn",
+ "env": {
+ "AR": "[START_DIR]/cipd/bin/llvm-ar",
+ "CC": "[START_DIR]/cipd/bin/clang",
+ "CFLAGS": "--target=x86_64-linux-gnu --sysroot=[START_DIR]/cipd/sysroot",
+ "CXX": "[START_DIR]/cipd/bin/clang++",
+ "LDFLAGS": "--target=x86_64-linux-gnu --sysroot=[START_DIR]/cipd/sysroot -static-libstdc++"
+ },
+ "infra_step": true,
+ "luci_context": {
+ "realm": {
+ "name": "gn:try"
+ },
+ "resultdb": {
+ "current_invocation": {
+ "name": "invocations/build:8945511751514863184",
+ "update_token": "token"
+ },
+ "hostname": "rdbhost"
+ }
+ },
+ "name": "release.linux-amd64.upload.install infra/tools/luci/cas.ensure_installed",
+ "~followup_annotations": [
+ "@@@STEP_NEST_LEVEL@4@@@",
+ "@@@STEP_LOG_LINE@json.output@{@@@",
+ "@@@STEP_LOG_LINE@json.output@ \"result\": {@@@",
+ "@@@STEP_LOG_LINE@json.output@ \"\": [@@@",
+ "@@@STEP_LOG_LINE@json.output@ {@@@",
+ "@@@STEP_LOG_LINE@json.output@ \"instance_id\": \"resolved-instance_id-of-git_revision:moc\", @@@",
+ "@@@STEP_LOG_LINE@json.output@ \"package\": \"infra/tools/luci/cas/resolved-platform\"@@@",
+ "@@@STEP_LOG_LINE@json.output@ }@@@",
+ "@@@STEP_LOG_LINE@json.output@ ]@@@",
+ "@@@STEP_LOG_LINE@json.output@ }@@@",
+ "@@@STEP_LOG_LINE@json.output@}@@@",
+ "@@@STEP_LOG_END@json.output@@@"
+ ]
+ },
+ {
+ "cmd": [
+ "[START_DIR]/cipd_tool/infra/tools/luci/cas/git_revision%3Amock_infra_git_revision/cas",
+ "archive",
+ "-cas-instance",
+ "projects/example-cas-server/instances/default_instance",
+ "-dump-digest",
+ "/path/to/tmp/",
+ "-paths",
+ "[START_DIR]/gn/out:gn"
+ ],
+ "cwd": "[START_DIR]/gn",
+ "env": {
+ "AR": "[START_DIR]/cipd/bin/llvm-ar",
+ "CC": "[START_DIR]/cipd/bin/clang",
+ "CFLAGS": "--target=x86_64-linux-gnu --sysroot=[START_DIR]/cipd/sysroot",
+ "CXX": "[START_DIR]/cipd/bin/clang++",
+ "LDFLAGS": "--target=x86_64-linux-gnu --sysroot=[START_DIR]/cipd/sysroot -static-libstdc++"
+ },
+ "infra_step": true,
+ "luci_context": {
+ "realm": {
+ "name": "gn:try"
+ },
+ "resultdb": {
+ "current_invocation": {
+ "name": "invocations/build:8945511751514863184",
+ "update_token": "token"
+ },
+ "hostname": "rdbhost"
+ }
+ },
+ "name": "release.linux-amd64.upload.upload binary to CAS",
+ "~followup_annotations": [
+ "@@@STEP_NEST_LEVEL@3@@@",
+ "@@@STEP_LINK@CAS UI@https://cas-viewer.appspot.com/projects/example-cas-server/instances/default_instance/blobs/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/0/tree@@@"
+ ]
+ },
+ {
+ "cmd": [],
"name": "release.linux-arm64",
"~followup_annotations": [
"@@@STEP_NEST_LEVEL@1@@@"
@@ -774,6 +953,51 @@
]
},
{
+ "cmd": [],
+ "name": "release.linux-arm64.upload",
+ "~followup_annotations": [
+ "@@@STEP_NEST_LEVEL@2@@@"
+ ]
+ },
+ {
+ "cmd": [
+ "[START_DIR]/cipd_tool/infra/tools/luci/cas/git_revision%3Amock_infra_git_revision/cas",
+ "archive",
+ "-cas-instance",
+ "projects/example-cas-server/instances/default_instance",
+ "-dump-digest",
+ "/path/to/tmp/",
+ "-paths",
+ "[START_DIR]/gn/out:gn"
+ ],
+ "cwd": "[START_DIR]/gn",
+ "env": {
+ "AR": "[START_DIR]/cipd/bin/llvm-ar",
+ "CC": "[START_DIR]/cipd/bin/clang",
+ "CFLAGS": "--target=aarch64-linux-gnu --sysroot=[START_DIR]/cipd/sysroot",
+ "CXX": "[START_DIR]/cipd/bin/clang++",
+ "LDFLAGS": "--target=aarch64-linux-gnu --sysroot=[START_DIR]/cipd/sysroot -static-libstdc++"
+ },
+ "infra_step": true,
+ "luci_context": {
+ "realm": {
+ "name": "gn:try"
+ },
+ "resultdb": {
+ "current_invocation": {
+ "name": "invocations/build:8945511751514863184",
+ "update_token": "token"
+ },
+ "hostname": "rdbhost"
+ }
+ },
+ "name": "release.linux-arm64.upload.upload binary to CAS",
+ "~followup_annotations": [
+ "@@@STEP_NEST_LEVEL@3@@@",
+ "@@@STEP_LINK@CAS UI@https://cas-viewer.appspot.com/projects/example-cas-server/instances/default_instance/blobs/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/0/tree@@@"
+ ]
+ },
+ {
"name": "$result"
}
]
\ No newline at end of file
diff --git a/infra/recipes/gn.expected/cq_mac.json b/infra/recipes/gn.expected/cq_mac.json
index 4017d0b..f9d745e 100644
--- a/infra/recipes/gn.expected/cq_mac.json
+++ b/infra/recipes/gn.expected/cq_mac.json
@@ -605,6 +605,185 @@
]
},
{
+ "cmd": [],
+ "name": "release.mac-amd64.upload",
+ "~followup_annotations": [
+ "@@@STEP_NEST_LEVEL@2@@@"
+ ]
+ },
+ {
+ "cmd": [
+ "vpython",
+ "-u",
+ "RECIPE_MODULE[recipe_engine::file]/resources/fileutil.py",
+ "--json-output",
+ "/path/to/tmp/json",
+ "copy",
+ "RECIPE_MODULE[recipe_engine::cas]/resources/infra.sha1",
+ "/path/to/tmp/"
+ ],
+ "cwd": "[START_DIR]/gn",
+ "env": {
+ "AR": "[START_DIR]/cipd/bin/llvm-ar",
+ "CC": "[START_DIR]/cipd/bin/clang",
+ "CFLAGS": "--target=x86_64-apple-darwin --sysroot=/some/xcode/path -nostdinc++ -cxx-isystem [CACHE]/macos_sdk/XCode.app/include/c++/v1",
+ "CXX": "[START_DIR]/cipd/bin/clang++",
+ "LDFLAGS": "--target=x86_64-apple-darwin --sysroot=/some/xcode/path"
+ },
+ "infra_step": true,
+ "luci_context": {
+ "realm": {
+ "name": "gn:try"
+ },
+ "resultdb": {
+ "current_invocation": {
+ "name": "invocations/build:8945511751514863184",
+ "update_token": "token"
+ },
+ "hostname": "rdbhost"
+ }
+ },
+ "name": "release.mac-amd64.upload.read infra revision",
+ "~followup_annotations": [
+ "@@@STEP_NEST_LEVEL@3@@@",
+ "@@@STEP_LOG_LINE@infra.sha1@git_revision:mock_infra_git_revision@@@",
+ "@@@STEP_LOG_END@infra.sha1@@@"
+ ]
+ },
+ {
+ "cmd": [],
+ "name": "release.mac-amd64.upload.install infra/tools/luci/cas",
+ "~followup_annotations": [
+ "@@@STEP_NEST_LEVEL@3@@@"
+ ]
+ },
+ {
+ "cmd": [
+ "vpython",
+ "-u",
+ "RECIPE_MODULE[recipe_engine::file]/resources/fileutil.py",
+ "--json-output",
+ "/path/to/tmp/json",
+ "ensure-directory",
+ "--mode",
+ "0777",
+ "[START_DIR]/cipd_tool/infra/tools/luci/cas/git_revision%3Amock_infra_git_revision"
+ ],
+ "cwd": "[START_DIR]/gn",
+ "env": {
+ "AR": "[START_DIR]/cipd/bin/llvm-ar",
+ "CC": "[START_DIR]/cipd/bin/clang",
+ "CFLAGS": "--target=x86_64-apple-darwin --sysroot=/some/xcode/path -nostdinc++ -cxx-isystem [CACHE]/macos_sdk/XCode.app/include/c++/v1",
+ "CXX": "[START_DIR]/cipd/bin/clang++",
+ "LDFLAGS": "--target=x86_64-apple-darwin --sysroot=/some/xcode/path"
+ },
+ "infra_step": true,
+ "luci_context": {
+ "realm": {
+ "name": "gn:try"
+ },
+ "resultdb": {
+ "current_invocation": {
+ "name": "invocations/build:8945511751514863184",
+ "update_token": "token"
+ },
+ "hostname": "rdbhost"
+ }
+ },
+ "name": "release.mac-amd64.upload.install infra/tools/luci/cas.ensure package directory",
+ "~followup_annotations": [
+ "@@@STEP_NEST_LEVEL@4@@@"
+ ]
+ },
+ {
+ "cmd": [
+ "cipd",
+ "ensure",
+ "-root",
+ "[START_DIR]/cipd_tool/infra/tools/luci/cas/git_revision%3Amock_infra_git_revision",
+ "-ensure-file",
+ "infra/tools/luci/cas/${platform} git_revision:mock_infra_git_revision",
+ "-max-threads",
+ "0",
+ "-json-output",
+ "/path/to/tmp/json"
+ ],
+ "cwd": "[START_DIR]/gn",
+ "env": {
+ "AR": "[START_DIR]/cipd/bin/llvm-ar",
+ "CC": "[START_DIR]/cipd/bin/clang",
+ "CFLAGS": "--target=x86_64-apple-darwin --sysroot=/some/xcode/path -nostdinc++ -cxx-isystem [CACHE]/macos_sdk/XCode.app/include/c++/v1",
+ "CXX": "[START_DIR]/cipd/bin/clang++",
+ "LDFLAGS": "--target=x86_64-apple-darwin --sysroot=/some/xcode/path"
+ },
+ "infra_step": true,
+ "luci_context": {
+ "realm": {
+ "name": "gn:try"
+ },
+ "resultdb": {
+ "current_invocation": {
+ "name": "invocations/build:8945511751514863184",
+ "update_token": "token"
+ },
+ "hostname": "rdbhost"
+ }
+ },
+ "name": "release.mac-amd64.upload.install infra/tools/luci/cas.ensure_installed",
+ "~followup_annotations": [
+ "@@@STEP_NEST_LEVEL@4@@@",
+ "@@@STEP_LOG_LINE@json.output@{@@@",
+ "@@@STEP_LOG_LINE@json.output@ \"result\": {@@@",
+ "@@@STEP_LOG_LINE@json.output@ \"\": [@@@",
+ "@@@STEP_LOG_LINE@json.output@ {@@@",
+ "@@@STEP_LOG_LINE@json.output@ \"instance_id\": \"resolved-instance_id-of-git_revision:moc\", @@@",
+ "@@@STEP_LOG_LINE@json.output@ \"package\": \"infra/tools/luci/cas/resolved-platform\"@@@",
+ "@@@STEP_LOG_LINE@json.output@ }@@@",
+ "@@@STEP_LOG_LINE@json.output@ ]@@@",
+ "@@@STEP_LOG_LINE@json.output@ }@@@",
+ "@@@STEP_LOG_LINE@json.output@}@@@",
+ "@@@STEP_LOG_END@json.output@@@"
+ ]
+ },
+ {
+ "cmd": [
+ "[START_DIR]/cipd_tool/infra/tools/luci/cas/git_revision%3Amock_infra_git_revision/cas",
+ "archive",
+ "-cas-instance",
+ "projects/example-cas-server/instances/default_instance",
+ "-dump-digest",
+ "/path/to/tmp/",
+ "-paths",
+ "[START_DIR]/gn/out:gn"
+ ],
+ "cwd": "[START_DIR]/gn",
+ "env": {
+ "AR": "[START_DIR]/cipd/bin/llvm-ar",
+ "CC": "[START_DIR]/cipd/bin/clang",
+ "CFLAGS": "--target=x86_64-apple-darwin --sysroot=/some/xcode/path -nostdinc++ -cxx-isystem [CACHE]/macos_sdk/XCode.app/include/c++/v1",
+ "CXX": "[START_DIR]/cipd/bin/clang++",
+ "LDFLAGS": "--target=x86_64-apple-darwin --sysroot=/some/xcode/path"
+ },
+ "infra_step": true,
+ "luci_context": {
+ "realm": {
+ "name": "gn:try"
+ },
+ "resultdb": {
+ "current_invocation": {
+ "name": "invocations/build:8945511751514863184",
+ "update_token": "token"
+ },
+ "hostname": "rdbhost"
+ }
+ },
+ "name": "release.mac-amd64.upload.upload binary to CAS",
+ "~followup_annotations": [
+ "@@@STEP_NEST_LEVEL@3@@@",
+ "@@@STEP_LINK@CAS UI@https://cas-viewer.appspot.com/projects/example-cas-server/instances/default_instance/blobs/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/0/tree@@@"
+ ]
+ },
+ {
"cmd": [
"xcrun",
"--show-sdk-path"
@@ -730,6 +909,51 @@
]
},
{
+ "cmd": [],
+ "name": "release.mac-arm64.upload",
+ "~followup_annotations": [
+ "@@@STEP_NEST_LEVEL@2@@@"
+ ]
+ },
+ {
+ "cmd": [
+ "[START_DIR]/cipd_tool/infra/tools/luci/cas/git_revision%3Amock_infra_git_revision/cas",
+ "archive",
+ "-cas-instance",
+ "projects/example-cas-server/instances/default_instance",
+ "-dump-digest",
+ "/path/to/tmp/",
+ "-paths",
+ "[START_DIR]/gn/out:gn"
+ ],
+ "cwd": "[START_DIR]/gn",
+ "env": {
+ "AR": "[START_DIR]/cipd/bin/llvm-ar",
+ "CC": "[START_DIR]/cipd/bin/clang",
+ "CFLAGS": "--target=arm64-apple-darwin --sysroot=/some/xcode/path -nostdinc++ -cxx-isystem [CACHE]/macos_sdk/XCode.app/include/c++/v1",
+ "CXX": "[START_DIR]/cipd/bin/clang++",
+ "LDFLAGS": "--target=arm64-apple-darwin --sysroot=/some/xcode/path"
+ },
+ "infra_step": true,
+ "luci_context": {
+ "realm": {
+ "name": "gn:try"
+ },
+ "resultdb": {
+ "current_invocation": {
+ "name": "invocations/build:8945511751514863184",
+ "update_token": "token"
+ },
+ "hostname": "rdbhost"
+ }
+ },
+ "name": "release.mac-arm64.upload.upload binary to CAS",
+ "~followup_annotations": [
+ "@@@STEP_NEST_LEVEL@3@@@",
+ "@@@STEP_LINK@CAS UI@https://cas-viewer.appspot.com/projects/example-cas-server/instances/default_instance/blobs/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/0/tree@@@"
+ ]
+ },
+ {
"cmd": [
"sudo",
"xcode-select",
diff --git a/infra/recipes/gn.expected/cq_win.json b/infra/recipes/gn.expected/cq_win.json
index df7c728..e1f742b 100644
--- a/infra/recipes/gn.expected/cq_win.json
+++ b/infra/recipes/gn.expected/cq_win.json
@@ -495,6 +495,189 @@
]
},
{
+ "cmd": [],
+ "name": "release.windows-amd64.upload",
+ "~followup_annotations": [
+ "@@@STEP_NEST_LEVEL@2@@@"
+ ]
+ },
+ {
+ "cmd": [
+ "vpython",
+ "-u",
+ "RECIPE_MODULE[recipe_engine::file]\\resources\\fileutil.py",
+ "--json-output",
+ "/path/to/tmp/json",
+ "copy",
+ "RECIPE_MODULE[recipe_engine::cas]\\resources\\infra.sha1",
+ "/path/to/tmp/"
+ ],
+ "cwd": "[START_DIR]\\gn",
+ "env": {
+ "VSINSTALLDIR": "[CACHE]\\windows_sdk"
+ },
+ "env_prefixes": {
+ "PATH": [
+ "[CACHE]\\windows_sdk\\win_sdk\\bin\\x64"
+ ]
+ },
+ "infra_step": true,
+ "luci_context": {
+ "realm": {
+ "name": "gn:try"
+ },
+ "resultdb": {
+ "current_invocation": {
+ "name": "invocations/build:8945511751514863184",
+ "update_token": "token"
+ },
+ "hostname": "rdbhost"
+ }
+ },
+ "name": "release.windows-amd64.upload.read infra revision",
+ "~followup_annotations": [
+ "@@@STEP_NEST_LEVEL@3@@@",
+ "@@@STEP_LOG_LINE@infra.sha1@git_revision:mock_infra_git_revision@@@",
+ "@@@STEP_LOG_END@infra.sha1@@@"
+ ]
+ },
+ {
+ "cmd": [],
+ "name": "release.windows-amd64.upload.install infra/tools/luci/cas",
+ "~followup_annotations": [
+ "@@@STEP_NEST_LEVEL@3@@@"
+ ]
+ },
+ {
+ "cmd": [
+ "vpython",
+ "-u",
+ "RECIPE_MODULE[recipe_engine::file]\\resources\\fileutil.py",
+ "--json-output",
+ "/path/to/tmp/json",
+ "ensure-directory",
+ "--mode",
+ "0777",
+ "[START_DIR]\\cipd_tool\\infra\\tools\\luci\\cas\\git_revision%3Amock_infra_git_revision"
+ ],
+ "cwd": "[START_DIR]\\gn",
+ "env": {
+ "VSINSTALLDIR": "[CACHE]\\windows_sdk"
+ },
+ "env_prefixes": {
+ "PATH": [
+ "[CACHE]\\windows_sdk\\win_sdk\\bin\\x64"
+ ]
+ },
+ "infra_step": true,
+ "luci_context": {
+ "realm": {
+ "name": "gn:try"
+ },
+ "resultdb": {
+ "current_invocation": {
+ "name": "invocations/build:8945511751514863184",
+ "update_token": "token"
+ },
+ "hostname": "rdbhost"
+ }
+ },
+ "name": "release.windows-amd64.upload.install infra/tools/luci/cas.ensure package directory",
+ "~followup_annotations": [
+ "@@@STEP_NEST_LEVEL@4@@@"
+ ]
+ },
+ {
+ "cmd": [
+ "cipd.bat",
+ "ensure",
+ "-root",
+ "[START_DIR]\\cipd_tool\\infra\\tools\\luci\\cas\\git_revision%3Amock_infra_git_revision",
+ "-ensure-file",
+ "infra/tools/luci/cas/${platform} git_revision:mock_infra_git_revision",
+ "-max-threads",
+ "0",
+ "-json-output",
+ "/path/to/tmp/json"
+ ],
+ "cwd": "[START_DIR]\\gn",
+ "env": {
+ "VSINSTALLDIR": "[CACHE]\\windows_sdk"
+ },
+ "env_prefixes": {
+ "PATH": [
+ "[CACHE]\\windows_sdk\\win_sdk\\bin\\x64"
+ ]
+ },
+ "infra_step": true,
+ "luci_context": {
+ "realm": {
+ "name": "gn:try"
+ },
+ "resultdb": {
+ "current_invocation": {
+ "name": "invocations/build:8945511751514863184",
+ "update_token": "token"
+ },
+ "hostname": "rdbhost"
+ }
+ },
+ "name": "release.windows-amd64.upload.install infra/tools/luci/cas.ensure_installed",
+ "~followup_annotations": [
+ "@@@STEP_NEST_LEVEL@4@@@",
+ "@@@STEP_LOG_LINE@json.output@{@@@",
+ "@@@STEP_LOG_LINE@json.output@ \"result\": {@@@",
+ "@@@STEP_LOG_LINE@json.output@ \"\": [@@@",
+ "@@@STEP_LOG_LINE@json.output@ {@@@",
+ "@@@STEP_LOG_LINE@json.output@ \"instance_id\": \"resolved-instance_id-of-git_revision:moc\", @@@",
+ "@@@STEP_LOG_LINE@json.output@ \"package\": \"infra/tools/luci/cas/resolved-platform\"@@@",
+ "@@@STEP_LOG_LINE@json.output@ }@@@",
+ "@@@STEP_LOG_LINE@json.output@ ]@@@",
+ "@@@STEP_LOG_LINE@json.output@ }@@@",
+ "@@@STEP_LOG_LINE@json.output@}@@@",
+ "@@@STEP_LOG_END@json.output@@@"
+ ]
+ },
+ {
+ "cmd": [
+ "[START_DIR]\\cipd_tool\\infra\\tools\\luci\\cas\\git_revision%3Amock_infra_git_revision\\cas",
+ "archive",
+ "-cas-instance",
+ "projects/example-cas-server/instances/default_instance",
+ "-dump-digest",
+ "/path/to/tmp/",
+ "-paths",
+ "[START_DIR]\\gn\\out:gn.exe"
+ ],
+ "cwd": "[START_DIR]\\gn",
+ "env": {
+ "VSINSTALLDIR": "[CACHE]\\windows_sdk"
+ },
+ "env_prefixes": {
+ "PATH": [
+ "[CACHE]\\windows_sdk\\win_sdk\\bin\\x64"
+ ]
+ },
+ "infra_step": true,
+ "luci_context": {
+ "realm": {
+ "name": "gn:try"
+ },
+ "resultdb": {
+ "current_invocation": {
+ "name": "invocations/build:8945511751514863184",
+ "update_token": "token"
+ },
+ "hostname": "rdbhost"
+ }
+ },
+ "name": "release.windows-amd64.upload.upload binary to CAS",
+ "~followup_annotations": [
+ "@@@STEP_NEST_LEVEL@3@@@",
+ "@@@STEP_LINK@CAS UI@https://cas-viewer.appspot.com/projects/example-cas-server/instances/default_instance/blobs/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/0/tree@@@"
+ ]
+ },
+ {
"cmd": [
"taskkill.exe",
"/f",
diff --git a/infra/recipes/gn.py b/infra/recipes/gn.py
index bfbc4f7..0f8d470 100644
--- a/infra/recipes/gn.py
+++ b/infra/recipes/gn.py
@@ -9,6 +9,7 @@
DEPS = [
'recipe_engine/buildbucket',
+ 'recipe_engine/cas',
'recipe_engine/cipd',
'recipe_engine/context',
'recipe_engine/file',
@@ -260,16 +261,20 @@
if target.is_host:
api.step('test', [src_dir.join('out', 'gn_unittests')])
- if build_input.gerrit_changes:
- continue
-
if config['name'] != 'release':
continue
with api.step.nest('upload'):
- cipd_pkg_name = 'gn/gn/%s' % target.platform
gn = 'gn' + ('.exe' if target.is_win else '')
+ if build_input.gerrit_changes:
+ # Upload to CAS from CQ.
+ api.cas.archive('upload binary to CAS', src_dir.join('out'),
+ src_dir.join('out', gn))
+ continue
+
+ cipd_pkg_name = 'gn/gn/%s' % target.platform
+
pkg_def = api.cipd.PackageDefinition(
package_name=cipd_pkg_name,
package_root=src_dir.join('out'),
