Workload Identity Federation: Call Vertex AI from AWS Without Service Account Keys

TL;DR Workload identity federation lets a workload on AWS prove its identity to Google Cloud with its IAM role and receive a short-lived GCP token, so no service account key file is stored anywhere. Bifrost v2.3.0 adds aws_workload_identity to Vertex AI keys: it signs an STS GetCallerIdentity request with the workload's AWS credentials and exchanges it at Google's Security Token Service. On the GCP side you create a workload identity pool with an AWS provider, restrict it to the Bifrost role, and grant Vertex AI access directly or through an impersonated service account. On AWS you attach the role with IRSA, EKS Pod Identity, an ECS task role, or an EC2 instance profile, and keep static AWS_ACCESS_KEY_ID values out of the pod. Workload identity federation is a Google Cloud IAM feature that lets workloads running outside Google Cloud exchange their own identity, such as an AWS IAM role, for short-lived Google credentials instead of using a service account key. Bifrost, the open-source AI gateway built in Go, now supports this exchange natively for Vertex AI: a gateway running on EKS, ECS, or EC2 can call Gemini and Claude models on Vertex using only the IAM role attached to it. This tutorial walks through the GCP setup, the AWS role, the Bifrost key, and the checks that confirm which identity reached Vertex. What workload identity federation is A service account key is a long-lived secret: a JSON file that grants whoever holds it the account's permissions until someone rotates or deletes it. Workload identity federation removes that file. Google Cloud trusts an external identity provider (AWS, Azure, or an OIDC or SAML provider), and the external workload presents proof of its identity to Google's Security Token Service, which returns a short-lived access token. For AWS, the proof is a signed GetCallerIdentity request. The workload signs the request with its AWS credentials but does not send it to AWS; Google verifies the signature and reads the caller's ARN from it. Google documents the model in its guide to federation with other clouds. Service account key Workload identity federation Secret stored on AWS JSON key file in a secret store or pod None Credential lifetime Until rotated or deleted Short-lived tokens, refreshed automatically Rotation Manual, per key Not needed Who can use it Anyone holding the file Only workloads with the trusted AWS role Audit identity The service account The AWS role, or the service account with the role recorded as delegate For a gateway that holds credentials for every model provider, removing one long-lived secret matters. The AI security resource page covers why credential sprawl is a common risk in AI stacks, and the non-human identity guide explains how machine identities like this one should be managed. Why use it for Vertex AI calls from AWS Many teams run their applications on AWS but want models that Vertex AI serves, such as Gemini or Claude through Google's regions and billing. Before this release, a Bifrost gateway on AWS reached Vertex with a service account JSON key. That key had to be created, stored in AWS Secrets Manager or a Kubernetes secret, mounted into the pod, and rotated on a schedule. With aws_workload_identity, the Bifrost pod uses the IAM role it already has. Nothing Google-issued is stored in AWS, and revoking access is a matter of changing the GCP IAM binding or the pool's attribute condition. The rest of the gateway works as before: virtual keys, routing rules, fallbacks, and request logs apply to Vertex traffic like any other provider. If you also use Bedrock from the same cluster, the AWS Bedrock resource page covers that side. The guide to Bedrock, Vertex, Gemini, and Anthropic models through Bifrost shows how Vertex fits beside other providers in one LLM gateway. How Bifrost exchanges an AWS identity for a GCP token When a request needs a Vertex AI token, Bifrost resolves AWS credentials through the AWS SDK default chain, signs an STS GetCallerIdentity request with them, and exchanges that signature at Google's Security Token Service. If you set service_account_email, the federated token is then swapped for a service account token; otherwise the federated principal calls Vertex AI directly. Figure 1: No key file exists on either side: the AWS role proves identity, and Google STS turns that proof into a short-lived token. The aws_workload_identity block sits inside vertex_key_config and takes precedence over auth_credentials; the two must not be combined. Its fields: Field Required Purpose audience Yes Full resource name of the AWS provider: //iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL_ID/providers/PROVIDER_ID service_account_email No Service account to impersonate; omit to give the federated principal Vertex access directly token_lifetime_seconds No Lifetime of the impersonated token, 600 to 43200 (default 3600); only used with service_account_email aws_region No Region for the STS signature; defaults to AWS_REGION or AWS_DEFAULT_REGION, then instance metadata aws_role_arn No A role to assume before the exchange; usually unnecessary Every string field accepts env.VAR_NAME references, and AWS GovCloud and China partitions work because the STS host is derived from the region. The full field reference is on the Vertex AI provider page. Step 1: Enable APIs and create the pool and AWS provider Run these commands in the Google Cloud project that will be billed for Vertex AI. They follow the GCP prerequisites for AWS in the Bifrost docs. First enable the APIs. sts.googleapis.com performs the token exchange, and iamcredentials.googleapis.com is needed only if you impersonate a service account: gcloud services enable aiplatform.googleapis.com sts.googleapis.com iamcredentials.googleapis.com \ --project=PROJECT_ID Next, create a workload identity pool and an AWS provider that trusts your AWS account. The attribute mapping exposes the assumed role name as attribute.aws_role, and the attribute condition limits the pool to the Bifrost role, so no other role in the account can use it: gcloud iam workload-identity-pools create POOL_ID \ --project=PROJECT_ID \ --location=global \ --display-name="Bifrost on AWS" gcloud iam workload-identity-pools providers create-aws PROVIDER_ID \ --project=PROJECT_ID \ --location=global \ --workload-identity-pool=POOL_ID \ --account-id=AWS_ACCOUNT_ID \ --attribute-mapping="google.subject=assertion.arn,attribute.aws_role=assertion.arn.extract('assumed-role/{role}/')" \ --attribute-condition="attribute.aws_role == 'BIFROST_ROLE_NAME'" Print the provider's full name, which becomes the audience in the Bifrost key: gcloud iam workload-identity-pools providers describe PROVIDER_ID \ --project=PROJECT_ID --location=global --workload-identity-pool=POOL_ID \ --format="value(name)" If the output starts with projects/, prefix it with //iam.googleapis.com/. The audience uses the numeric project number, not the project ID; gcloud projects describe PROJECT_ID --format="value(projectNumber)" prints it. Step 2: Grant Vertex AI access The principal you grant access to is the pool-scoped identity of your AWS role: principalSet://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL_ID/attribute.aws_role/BIFROST_ROLE_NAME You can give that principal Vertex access directly, or let it impersonate a service account that has access. Either way, keep the grant as narrow as the AI agent governance guide recommends for machine identities: one role, one project. Figure 2: The attribute condition limits the pool to the Bifrost role, and the IAM binding decides between direct access and impersonation. Option A: direct access. Leave service_account_email empty in the Bifrost key and bind the Vertex AI user role to the principal: gcloud projects add-iam-policy-binding PROJECT_ID \ --member="principalSet://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL_ID/attribute.aws_role/BIFROST_ROLE_NAME" \ --role="roles/aiplatform.user" Option B: impersonate a service account. Create a service account, give it Vertex access, and allow the federated principal to impersonate it: gcloud iam service-accounts create IAM_SA_NAME \ --display-name="Bifrost Vertex AI" --project=PROJECT_ID gcloud projects add-iam-policy-binding PROJECT_ID \ --member="serviceAccount:IAM_SA_NAME@PROJECT_ID.iam.gserviceaccount.com" \ --role="roles/aiplatform.user" gcloud iam service-accounts add-iam-policy-binding \ IAM_SA_NAME@PROJECT_ID.iam.gserviceaccount.com \ --role="roles/iam.workloadIdentityUser" \ --member="principalSet://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL_ID/attribute.aws_role/BIFROST_ROLE_NAME" Option A: direct access Option B: impersonation Extra GCP objects None A service account and an impersonation binding service_account_email in the key Empty Set Identity in Cloud Audit Logs The federated AWS principal The service account, with the AWS identity as delegate Good fit Simple setups where the AWS role is the identity you want to audit Organizations that grant permissions to service accounts by policy Whichever option you pick, budgets and model access for the applications behind the gateway are still enforced by Bifrost, as the governance resource page describes. Option B also lets you set token_lifetime_seconds. Values above 3600 require the organization policy constraint constraints/iam.allowServiceAccountCredentialLifetimeExtension to allow the service account; otherwise token generation fails. Step 3: Attach an IAM role to the Bifrost workload The IAM role needs no AWS permissions for this flow, because sts:GetCallerIdentity is always allowed. It only has to reach the Bifrost process. On EKS, use IRSA or EKS Pod Identity. With IRSA, annotate the Kubernetes service account that the Bifrost pod runs as: kubectl annotate serviceaccount KSA_NAME \ --namespace NAMESPACE \ eks.amazonaws.com/role-arn=arn:aws:iam::AWS_ACCOUNT_ID:role/BIFROST_ROLE_NAME With EKS Pod Identity, install the Pod Identity Agent add-on, make sure the role's trust policy allows the pods.eks.amazonaws.com service principal to call sts:AssumeRole and sts:TagSession, and create the association: aws eks create-addon --cluster-name CLUSTER_NAME --addon-name eks-pod-identity-agent aws eks create-pod-identity-association \ --cluster-name CLUSTER_NAME \ --namespace NAMESPACE \ --service-account KSA_NAME \ --role-arn arn:aws:iam::AWS_ACCOUNT_ID:role/BIFROST_ROLE_NAME An IRSA-style OIDC trust policy does not work with Pod Identity. On ECS, attach the role as the task role; on EC2, attach it through the instance profile. If you deploy with the Bifrost Helm chart, set the IRSA annotation through serviceAccount.annotations, as shown in the Helm provider configuration. Figure 3: Static AWS keys in the pod win over the attached role, so remove them when the role is the identity you mean to federate. Two details cause most failed setups. First, the AWS SDK default chain checks static credentials before the attached role, so if AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY are set in the pod, Bifrost federates as that key. Remove them. Second, a region is not guaranteed: IRSA usually injects AWS_REGION, but a self-managed webhook may not, and Pod Identity does not set one. Set aws_region in the key or AWS_REGION in the pod if neither the environment nor instance metadata provides one. The in-VPC deployment docs cover running Bifrost inside your own VPC on AWS. Step 4: Add a Vertex key with aws_workload_identity In the dashboard, go to Model Providers → Configurations → Google Vertex, add a key, and choose Workload Identity (AWS) as the authentication method. Set the project ID, the region (for example us-central1), and the Workload Identity Provider Audience. The service account, AWS region, and role ARN fields are optional. The same key in config.json, using Option B: { "providers": { "vertex": { "keys": [ { "name": "vertex-eks-key", "value": "", "models": ["*"], "weight": 1.0, "vertex_key_config": { "project_id": "env.VERTEX_PROJECT_ID", "region": "us-central1", "aws_workload_identity": { "audience": "//iam.googleapis.com/projects/123456789012/locations/global/workloadIdentityPools/eks-pool/providers/aws", "service_account_email": "bifrost-vertex@my-gcp-project.iam.gserviceaccount.com" } } } ] } } } For Option A, drop service_account_email. To create the key on a running gateway, POST the same key object to /api/providers/vertex/keys after creating the provider with POST /api/providers and {"provider": "vertex"}. The Vertex AI setup guide covers regions and model names, and key management explains how weights spread traffic when you add more than one Vertex key. Applications then call Vertex models through Bifrost with the usual model prefix, for example vertex/gemini-2.5-flash, using a virtual key for access control and budgets. Step 5: Verify the identity end to end Confirm the AWS side from inside the Bifrost pod: kubectl exec -n NAMESPACE POD_NAME -- aws sts get-caller-identity # "Arn": "arn:aws:sts::AWS_ACCOUNT_ID:assumed-role/BIFROST_ROLE_NAME/..." If the ARN shows an IAM user or a node role instead of BIFROST_ROLE_NAME, fix that before going further. Then send a request through the gateway: curl http://localhost:8080/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer sk-bf-your-virtual-key" \ -d '{"model": "vertex/gemini-2.5-flash", "messages": [{"role": "user", "content": "Reply with OK"}]}' Finally, check Google's side. The Cloud Audit Log entry for the Vertex call identifies the caller under protoPayload.authenticationInfo. With Option A and the mapping above, principalSubject names the pool subject with the AWS role's assumed-role ARN. With Option B, principalEmail is the service account and the AWS identity appears as principalSubject inside serviceAccountDelegationInfo[]. The principalSet:// form from Step 2 is the binding target and does not appear as the logged identity. Bifrost records the request on its own side as well: the built-in request logs show the provider, model, key name, and latency, and the AI governance page explains how those logs tie each call back to a virtual key, team, or customer. Troubleshooting workload identity federation for Vertex AI Symptom Likely cause Fix no EC2 IMDS role found or failed to refresh cached credentials No IAM role reaches the pod Check the IRSA annotation or Pod Identity association; aws sts get-caller-identity in the pod must succeed aws region could not be determined No AWS_REGION and no instance metadata Set aws_region in the key or AWS_REGION in the pod invalid_grant or The audience in the token does not match Wrong audience Copy the provider's full resource name, starting with //iam.googleapis.com/ The given credential is rejected by the attribute condition Role name does not match the condition Align --attribute-condition with the role attached to the pod 403 Permission 'iam.serviceAccounts.getAccessToken' denied Missing impersonation grant Add roles/iam.workloadIdentityUser on the service account (Option B) 403 Forbidden from the Vertex API Principal or service account lacks Vertex access Grant roles/aiplatform.user Requests succeed as the node role An older Bifrost resolved an external_account JSON through Google's library Upgrade Bifrost, or switch the key to aws_workload_identity When a Vertex key fails for reasons tied to the key, such as a rejected credential, Bifrost classifies the error and can rotate to another key or fall back to another provider, as described in retries and fallbacks. The routing and fallback guide shows how to keep traffic flowing while you fix the binding. Frequently asked questions What is workload identity federation? Workload identity federation is a Google Cloud IAM feature that lets workloads outside Google Cloud, such as applications on AWS or Azure, exchange their own identity for short-lived Google credentials. It replaces long-lived service account keys with a trust relationship between Google Cloud and the external identity provider. Do I still need a service account? No. With Option A, the federated AWS principal gets roles/aiplatform.user directly and no service account exists. Use a service account (Option B) when your organization grants permissions to service accounts by policy or wants the service account recorded as the caller. Both are supported by the Bifrost Vertex AI provider. Does the AWS role need any IAM permissions? No. The flow only signs an STS GetCallerIdentity request, which every AWS identity is allowed to call. The role needs a trust relationship for IRSA, Pod Identity, ECS, or EC2 so the workload can assume it, and nothing else. Can I use workload identity federation on ECS or EC2 instead of EKS? Yes. Bifrost resolves AWS credentials through the AWS SDK default chain, which covers ECS task roles and EC2 instance profiles as well as IRSA and EKS Pod Identity. The GCP-side setup is the same; only how the role reaches the process changes. Can I combine aws_workload_identity with a service account JSON key? No. aws_workload_identity takes precedence over auth_credentials, and the two must not be set on the same key. To migrate gradually, add a new Vertex key that uses workload identity alongside the existing key, shift weight to it, and then remove the JSON key. The Bifrost enterprise page covers rolling changes like this across clustered gateways. Which Vertex AI models work with this setup? Authentication does not change model support, so the Gemini and Anthropic models that Bifrost serves through Vertex AI work the same way with a federated key. The supported providers overview lists what each provider supports. Run Vertex AI from AWS without key files Workload identity federation removes the last long-lived Google secret from an AWS deployment: the Bifrost pod's IAM role is the credential, Google issues short-lived tokens, and access is controlled by IAM bindings you can audit. To see Vertex AI, Bedrock, and other providers governed through one gateway in your own VPC, book a demo with the Bifrost team, or read the Bifrost enterprise deployment guide to plan the rollout.