This repository was archived by the owner on Nov 11, 2022. It is now read-only.
Repository navigation
This repository was archived by the owner on Nov 11, 2022. It is now read-only.
Using Service Account with dataflow #554
Description
I am trying to use a dataflow job with a service account.
I am using the GcpOptions flags --serviceAccountKeyfile="dataflow-service-account.p12" --serviceAccountName="dataflow"
I am getting following error:
Unable to verify that GCS bucket exists.com.google.cloud.dataflow.sdk.util.DataflowPathValidator.verifyPathIsAccessible(DataflowPathValidator.java:84)
com.google.cloud.dataflow.sdk.util.DataflowPathValidator.validateOutputFilePrefixSupported(DataflowPathValidator.java:63)
com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner.fromOptions(DataflowPipelineRunner.java:274)
I would like to use service acoount credentials against application default credentials.
Activity
seehans commentedon Mar 1, 2017
It works if i give following value to serviceAccountName flag:
--serviceAccountName="dataflow@my-project.gserviceaccount.com"
Documentation is misleading, we are actually giving "Service account ID" value to serviceAccountName flag and not "Service account name" .
davorbonaci commentedon Mar 1, 2017
I'll reopen to clarify the documentation.
Thanks @seehans! Also, if you are interested, feel free to contribute a documentation fix to the Apache Beam codebase: https://github.com/apache/beam.