Creates and runs a matrix of tests according to the given specifications. Unsupported environments will be returned in the state UNSUPPORTED. A test matrix is limited to use at most 2000 devices in parallel. The returned matrix will not yet contain the executions that will be created for this matrix. Execution creation happens later on and will require a call to GetTestMatrix. May return any of the following canonical error codes: - PERMISSION_DENIED - if the user is not authorized to write to project - INVALID_ARGUMENT - if the request is malformed or if the matrix tries to use too many simultaneous devices.

Scopes

You will need authorization for the https://www.googleapis.com/auth/cloud-platform scope to make a valid call.

If unset, the scope for this method defaults to https://www.googleapis.com/auth/cloud-platform. You can set the scope for this method like this: testing1 --scope <scope> projects test-matrices-create ...

Required Scalar Argument

<project-id> (string)
- The GCE project under which this job will run.

Required Request Value

The request value is a data-structure with various fields. Each field may be a simple scalar or another data-structure. In the latter case it is advised to set the field-cursor to the data-structure's field to specify values more concisely.

For example, a structure like this:

TestMatrix:
  client-info:
    name: string
  environment-matrix:
    android-matrix:
      android-model-ids: [string]
      android-version-ids: [string]
      locales: [string]
      orientations: [string]
  fail-fast: boolean
  flaky-test-attempts: integer
  invalid-matrix-details: string
  outcome-summary: string
  project-id: string
  result-storage:
    google-cloud-storage:
      gcs-path: string
    results-url: string
    tool-results-execution:
      execution-id: string
      history-id: string
      project-id: string
    tool-results-history:
      history-id: string
      project-id: string
  state: string
  test-matrix-id: string
  test-specification:
    android-instrumentation-test:
      app-apk:
        gcs-path: string
      app-bundle:
        bundle-location:
          gcs-path: string
      app-package-id: string
      orchestrator-option: string
      sharding-option:
        smart-sharding:
          targeted-shard-duration: string
        uniform-sharding:
          num-shards: integer
      test-apk:
        gcs-path: string
      test-package-id: string
      test-runner-class: string
      test-targets: [string]
    android-robo-test:
      app-apk:
        gcs-path: string
      app-bundle:
        bundle-location:
          gcs-path: string
      app-initial-activity: string
      app-package-id: string
      max-depth: integer
      max-steps: integer
      robo-mode: string
      robo-script:
        gcs-path: string
    android-test-loop:
      app-apk:
        gcs-path: string
      app-bundle:
        bundle-location:
          gcs-path: string
      app-package-id: string
      scenario-labels: [string]
      scenarios: [integer]
    disable-performance-metrics: boolean
    disable-video-recording: boolean
    ios-robo-test:
      app-bundle-id: string
      app-ipa:
        gcs-path: string
      robo-script:
        gcs-path: string
    ios-test-loop:
      app-bundle-id: string
      app-ipa:
        gcs-path: string
      scenarios: [integer]
    ios-test-setup:
      network-profile: string
    ios-xc-test:
      app-bundle-id: string
      test-special-entitlements: boolean
      tests-zip:
        gcs-path: string
      xcode-version: string
      xctestrun:
        gcs-path: string
    test-setup:
      directories-to-pull: [string]
      dont-autogrant-permissions: boolean
      network-profile: string
      systrace:
        duration-seconds: integer
    test-timeout: string
  timestamp: string

can be set completely with the following arguments which are assumed to be executed in the given order. Note how the cursor position is adjusted to the respective structures, allowing simple field names to be used most of the time.

-r .client-info name=ut
- Required. Client name, such as gcloud.
..environment-matrix.android-matrix android-model-ids=gubergren
- Required. The ids of the set of Android device to be used. Use the TestEnvironmentDiscoveryService to get supported options.
- Each invocation of this argument appends the given value to the array.
android-version-ids=rebum.
- Required. The ids of the set of Android OS version to be used. Use the TestEnvironmentDiscoveryService to get supported options.
- Each invocation of this argument appends the given value to the array.
locales=est
- Required. The set of locales the test device will enable for testing. Use the TestEnvironmentDiscoveryService to get supported options.
- Each invocation of this argument appends the given value to the array.
orientations=ipsum
- Required. The set of orientations to test with. Use the TestEnvironmentDiscoveryService to get supported options.
- Each invocation of this argument appends the given value to the array.
... fail-fast=true
- If true, only a single attempt at most will be made to run each execution/shard in the matrix. Flaky test attempts are not affected. Normally, 2 or more attempts are made if a potential infrastructure issue is detected. This feature is for latency sensitive workloads. The incidence of execution failures may be significantly greater for fail-fast matrices and support is more limited because of that expectation.
flaky-test-attempts=94
- The number of times a TestExecution should be re-attempted if one or more of its test cases fail for any reason. The maximum number of reruns allowed is 10. Default is 0, which implies no reruns.
invalid-matrix-details=gubergren
- Output only. Describes why the matrix is considered invalid. Only useful for matrices in the INVALID state.
outcome-summary=ea
- Output Only. The overall outcome of the test. Only set when the test matrix state is FINISHED.
project-id=dolor
- The cloud project that owns the test matrix.
result-storage.google-cloud-storage gcs-path=lorem
- Required. The path to a directory in GCS that will eventually contain the results for this test. The requesting user must have write access on the bucket in the supplied path.
.. results-url=eos
- Output only. URL to the results in the Firebase Web Console.
tool-results-execution execution-id=labore
- Output only. A tool results execution ID.
history-id=sed
- Output only. A tool results history ID.
project-id=duo
- Output only. The cloud project that owns the tool results execution.
..tool-results-history history-id=sed
- Required. A tool results history ID.
project-id=no
- Required. The cloud project that owns the tool results history.
... state=stet
- Output only. Indicates the current progress of the test matrix.
test-matrix-id=kasd
- Output only. Unique id set by the service.
test-specification.android-instrumentation-test.app-apk gcs-path=et
- A path to a file in Google Cloud Storage. Example: gs://build-app-1414623860166/app%40debug-unaligned.apk These paths are expected to be url encoded (percent encoding)
..app-bundle.bundle-location gcs-path=sed
- A path to a file in Google Cloud Storage. Example: gs://build-app-1414623860166/app%40debug-unaligned.apk These paths are expected to be url encoded (percent encoding)
... app-package-id=et
- The java package for the application under test. The default value is determined by examining the application's manifest.
orchestrator-option=et
- The option of whether running each test within its own invocation of instrumentation with Android Test Orchestrator or not. Orchestrator is only compatible with AndroidJUnitRunner version 1.1 or higher! Orchestrator offers the following benefits: - No shared state - Crashes are isolated - Logs are scoped per test See for more information about Android Test Orchestrator. If not set, the test will be run without the orchestrator.
sharding-option.smart-sharding targeted-shard-duration=vero
- The amount of time tests within a shard should take. Default: 300 seconds (5 minutes). The minimum allowed: 120 seconds (2 minutes). The shard count is dynamically set based on time, up to the maximum shard limit (described below). To guarantee at least one test case for each shard, the number of shards will not exceed the number of test cases. Shard duration will be exceeded if: - The maximum shard limit is reached and there is more calculated test time remaining to allocate into shards. - Any individual test is estimated to be longer than the targeted shard duration. Shard duration is not guaranteed because smart sharding uses test case history and default durations which may not be accurate. The rules for finding the test case timing records are: - If the service has processed a test case in the last 30 days, the record of the latest successful test case will be used. - For new test cases, the average duration of other known test cases will be used. - If there are no previous test case timing records available, the default test case duration is 15 seconds. Because the actual shard duration can exceed the targeted shard duration, we recommend that you set the targeted value at least 5 minutes less than the maximum allowed test timeout (45 minutes for physical devices and 60 minutes for virtual), or that you use the custom test timeout value that you set. This approach avoids cancelling the shard before all tests can finish. Note that there is a limit for maximum number of shards. When you select one or more physical devices, the number of shards must be <= 50. When you select one or more ARM virtual devices, it must be <= 200. When you select only x86 virtual devices, it must be <= 500. To guarantee at least one test case for per shard, the number of shards will not exceed the number of test cases. Each shard created counts toward daily test quota.
..uniform-sharding num-shards=70
- Required. The total number of shards to create. This must always be a positive number that is no greater than the total number of test cases. When you select one or more physical devices, the number of shards must be <= 50. When you select one or more ARM virtual devices, it must be <= 200. When you select only x86 virtual devices, it must be <= 500.
...test-apk gcs-path=sed
- A path to a file in Google Cloud Storage. Example: gs://build-app-1414623860166/app%40debug-unaligned.apk These paths are expected to be url encoded (percent encoding)
.. test-package-id=duo
- The java package for the test to be executed. The default value is determined by examining the application's manifest.
test-runner-class=dolore
- The InstrumentationTestRunner class. The default value is determined by examining the application's manifest.
test-targets=et
- Each target must be fully qualified with the package name or class name, in one of these formats: - "package package_name" - "class package_name.class_name" - "class package_name.class_name#method_name" If empty, all targets in the module will be run.
- Each invocation of this argument appends the given value to the array.
..android-robo-test.app-apk gcs-path=voluptua.
- A path to a file in Google Cloud Storage. Example: gs://build-app-1414623860166/app%40debug-unaligned.apk These paths are expected to be url encoded (percent encoding)
..app-bundle.bundle-location gcs-path=amet.
- A path to a file in Google Cloud Storage. Example: gs://build-app-1414623860166/app%40debug-unaligned.apk These paths are expected to be url encoded (percent encoding)
... app-initial-activity=consetetur
- The initial activity that should be used to start the app.
app-package-id=diam
- The java package for the application under test. The default value is determined by examining the application's manifest.
max-depth=52
- The max depth of the traversal stack Robo can explore. Needs to be at least 2 to make Robo explore the app beyond the first activity. Default is 50.
max-steps=83
- The max number of steps Robo can execute. Default is no limit.
robo-mode=et
- The mode in which Robo should run. Most clients should allow the server to populate this field automatically.
robo-script gcs-path=sadipscing
- A path to a file in Google Cloud Storage. Example: gs://build-app-1414623860166/app%40debug-unaligned.apk These paths are expected to be url encoded (percent encoding)
...android-test-loop.app-apk gcs-path=stet
- A path to a file in Google Cloud Storage. Example: gs://build-app-1414623860166/app%40debug-unaligned.apk These paths are expected to be url encoded (percent encoding)
..app-bundle.bundle-location gcs-path=dolor
- A path to a file in Google Cloud Storage. Example: gs://build-app-1414623860166/app%40debug-unaligned.apk These paths are expected to be url encoded (percent encoding)
... app-package-id=duo
- The java package for the application under test. The default is determined by examining the application's manifest.
scenario-labels=vero
- The list of scenario labels that should be run during the test. The scenario labels should map to labels defined in the application's manifest. For example, player_experience and com.google.test.loops.player_experience add all of the loops labeled in the manifest with the com.google.test.loops.player_experience name to the execution. Scenarios can also be specified in the scenarios field.
- Each invocation of this argument appends the given value to the array.
scenarios=25
- The list of scenarios that should be run during the test. The default is all test loops, derived from the application's manifest.
- Each invocation of this argument appends the given value to the array.
.. disable-performance-metrics=false
- Disables performance metrics recording. May reduce test latency.
disable-video-recording=true
- Disables video recording. May reduce test latency.
ios-robo-test app-bundle-id=vero
- The bundle ID for the app-under-test. This is determined by examining the application's "Info.plist" file.
app-ipa gcs-path=elitr
- A path to a file in Google Cloud Storage. Example: gs://build-app-1414623860166/app%40debug-unaligned.apk These paths are expected to be url encoded (percent encoding)
..robo-script gcs-path=lorem
- A path to a file in Google Cloud Storage. Example: gs://build-app-1414623860166/app%40debug-unaligned.apk These paths are expected to be url encoded (percent encoding)
...ios-test-loop app-bundle-id=diam
- Output only. The bundle id for the application under test.
app-ipa gcs-path=no
- A path to a file in Google Cloud Storage. Example: gs://build-app-1414623860166/app%40debug-unaligned.apk These paths are expected to be url encoded (percent encoding)
.. scenarios=1
- The list of scenarios that should be run during the test. Defaults to the single scenario 0 if unspecified.
- Each invocation of this argument appends the given value to the array.
..ios-test-setup network-profile=accusam
- The network traffic profile used for running the test. Available network profiles can be queried by using the NETWORK_CONFIGURATION environment type when calling TestEnvironmentDiscoveryService.GetTestEnvironmentCatalog.
..ios-xc-test app-bundle-id=takimata
- Output only. The bundle id for the application under test.
test-special-entitlements=true
- The option to test special app entitlements. Setting this would re-sign the app having special entitlements with an explicit application-identifier. Currently supports testing aps-environment entitlement.
tests-zip gcs-path=voluptua.
- A path to a file in Google Cloud Storage. Example: gs://build-app-1414623860166/app%40debug-unaligned.apk These paths are expected to be url encoded (percent encoding)
.. xcode-version=et
- The Xcode version that should be used for the test. Use the TestEnvironmentDiscoveryService to get supported options. Defaults to the latest Xcode version Firebase Test Lab supports.
xctestrun gcs-path=erat
- A path to a file in Google Cloud Storage. Example: gs://build-app-1414623860166/app%40debug-unaligned.apk These paths are expected to be url encoded (percent encoding)
...test-setup directories-to-pull=consetetur
- List of directories on the device to upload to GCS at the end of the test; they must be absolute paths under /sdcard, /storage or /data/local/tmp. Path names are restricted to characters a-z A-Z 0-9 _ - . + and / Note: The paths /sdcard and /data will be made available and treated as implicit path substitutions. E.g. if /sdcard on a particular device does not map to external storage, the system will replace it with the external storage path prefix for that device.
- Each invocation of this argument appends the given value to the array.
dont-autogrant-permissions=true
- Whether to prevent all runtime permissions to be granted at app install
network-profile=et
- The network traffic profile used for running the test. Available network profiles can be queried by using the NETWORK_CONFIGURATION environment type when calling TestEnvironmentDiscoveryService.GetTestEnvironmentCatalog.
systrace duration-seconds=78
- Systrace duration in seconds. Should be between 1 and 30 seconds. 0 disables systrace.
... test-timeout=voluptua.
- Max time a test execution is allowed to run before it is automatically cancelled. The default value is 5 min.
.. timestamp=dolore
- Output only. The time this test matrix was initially created.

About Cursors

The cursor position is key to comfortably set complex nested structures. The following rules apply:

The cursor position is always set relative to the current one, unless the field name starts with the . character. Fields can be nested such as in -r f.s.o .
The cursor position is set relative to the top-level structure if it starts with ., e.g. -r .s.s
You can also set nested fields without setting the cursor explicitly. For example, to set a value relative to the current cursor position, you would specify -r struct.sub_struct=bar.
You can move the cursor one level up by using ... Each additional . moves it up one additional level. E.g. ... would go three levels up.

Optional Output Flags

The method's return value a JSON encoded structure, which will be written to standard output by default.

-o out
- out specifies the destination to which to write the server's result to. It will be a JSON-encoded structure. The destination may be - to indicate standard output, or a filepath that is to contain the received bytes. If unset, it defaults to standard output.

Optional Method Properties

You may set the following properties to further configure the call. Please note that -p is followed by one or more key-value-pairs, and is called like this -p k1=v1 k2=v2 even though the listing below repeats the -p for completeness.

-p request-id=string
- A string id used to detect duplicated requests. Ids are automatically scoped to a project, so users should ensure the ID is unique per-project. A UUID is recommended. Optional, but strongly recommended.

Optional General Properties

The following properties can configure any call, and are not specific to this method.

-p $-xgafv=string
- V1 error format.
-p access-token=string
- OAuth access token.
-p alt=string
- Data format for response.
-p callback=string
- JSONP
-p fields=string
- Selector specifying which fields to include in a partial response.
-p key=string
- API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
-p oauth-token=string
- OAuth 2.0 token for the current user.
-p pretty-print=boolean
- Returns response with indentations and line breaks.
-p quota-user=string
- Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
-p upload-type=string
- Legacy upload protocol for media (e.g. "media", "multipart").
-p upload-protocol=string
- Upload protocol for media (e.g. "raw", "multipart").