A service account key leaks, or a laptop with valid gcloud credentials gets stolen, and suddenly the attacker doesn't need to break anything — they just ask BigQuery or Cloud Storage nicely, using permissions that were legitimately granted for a different job. IAM roles answer "who can do what." They never answer "where can the data physically go." That second question is what VPC Service Controls (VPC SC) is built to answer, and it's the control most teams skip until after an incident review tells them to add it.
This post walks through building a real security perimeter around BigQuery, Cloud Storage (GCS), and Cloud Run, so that even a valid, over-permissioned credential can't exfiltrate data outside the boundary you define.
1. What VPC Service Controls actually does
IAM controls identity: does this principal have the roles/bigquery.dataViewer role on this dataset. VPC Service Controls controls context: from where, and to where, can API calls to that dataset happen at all, regardless of identity.
A service perimeter wraps a list of Google Cloud projects (or a VPC network) and a list of supported APIs — BigQuery, GCS, Cloud Run, Pub/Sub, and others. Once a project is inside a perimeter, calls to those APIs from outside the perimeter's trusted boundary are denied at the network layer, before IAM is even evaluated for cross-perimeter access. A compromised OAuth token or a leaked service account key that is valid from an untrusted network location simply gets a 403 PERMISSION_DENIED with a VPC SC-specific error, not a data leak.
This matters most against two realistic failure modes: an insider or compromised CI pipeline trying to copy a BigQuery table to a personal Google account via bq extract, and malware on a developer's laptop trying to pull objects out of a GCS bucket using stolen application-default credentials. Both have valid IAM permissions. Neither should be able to move data across the perimeter boundary.
2. Prerequisites: Access Context Manager and an access policy
VPC SC lives under the Access Context Manager API, which operates at the Google Cloud organization level, not the project level. You need org-level IAM permissions (roles/accesscontextmanager.policyAdmin or equivalent) to set this up.
First, enable the API and check whether an access policy already exists for your organization — most orgs only get one:
gcloud services enable accesscontextmanager.googleapis.com
gcloud access-context-manager policies list \
--organization=123456789012
# If none exists, create one:
gcloud access-context-manager policies create \
--organization=123456789012 \
--title="org-access-policy"Grab the policy ID from the list output — you'll reference it as ACCESS_POLICY in every command below.
3. Creating the perimeter
A perimeter needs: a list of projects inside it, a list of restricted services, and (optionally) access levels that define trusted exceptions. Start with the basic perimeter around the three services this post cares about:
gcloud access-context-manager perimeters create data_perimeter \
--title="Data Perimeter" \
--resources=projects/111111111111,projects/222222222222 \
--restricted-services=bigquery.googleapis.com,storage.googleapis.com,run.googleapis.com \
--policy=ACCESS_POLICY \
--perimeter-type=regularProjects are referenced by numeric project number, not project ID — a common first mistake. Get it with:
gcloud projects describe my-project-id --format="value(projectNumber)"Once created, any call to BigQuery, GCS, or Cloud Run APIs that originates from outside the perimeter — a different project, a personal account's console session, an on-prem machine without a VPN path explicitly allowed — is blocked, even with valid IAM roles on the resource.
4. Dry-run mode: test before you break production
Enforcing a perimeter on day one is how outages happen: some existing pipeline that pulls BigQuery results from a Cloud Function in another project suddenly gets denied. VPC SC supports a dry-run perimeter that logs what would be denied, without denying it:
gcloud access-context-manager perimeters update data_perimeter \
--policy=ACCESS_POLICY \
--set-dry-run-resources=projects/111111111111,projects/222222222222 \
--set-dry-run-restricted-services=bigquery.googleapis.com,storage.googleapis.com,run.googleapis.comDry-run violations show up in Cloud Logging under the log name cloudaudit.googleapis.com%2Fpolicy, with violationReason fields like NO_MATCHING_ACCESS_LEVEL. Run this for at least a week against real traffic, review every violation, and either fix the calling pattern or add an explicit access level for it. Only then promote dry-run to enforced:
gcloud access-context-manager perimeters update data_perimeter \
--policy=ACCESS_POLICY \
--enforce-dry-run-resources \
--enforce-dry-run-restricted-services5. Access levels: letting legitimate traffic back in
A pure perimeter with no exceptions is often too strict — your data scientist's laptop, or a CI runner in a management project, legitimately needs to query BigQuery inside the perimeter. Access levels define conditions — IP ranges, device policy, identity — under which outside callers are still allowed in.
A basic IP-range access level for a corporate network range:
title: "corp_network"
basic:
conditions:
- ipSubnetworks:
- "203.0.113.0/24"gcloud access-context-manager levels create corp_network \
--policy=ACCESS_POLICY \
--title="Corp Network" \
--basic-level-spec=corp_network.yaml
gcloud access-context-manager perimeters update data_perimeter \
--policy=ACCESS_POLICY \
--add-access-levels=corp_networkFor project-to-project access — say, a BI project outside the perimeter that legitimately needs read access to BigQuery datasets inside it — don't widen the perimeter itself. Instead use an ingress policy, which allows specific identities from specific outside sources to reach specific resources inside, without opening the whole perimeter:
- ingressFrom:
sources:
- resource: "projects/333333333333"
identityType: ANY_IDENTITY
ingressTo:
operations:
- serviceName: "bigquery.googleapis.com"
methodSelectors:
- method: "google.cloud.bigquery.v2.JobService.Query"
resources:
- "projects/111111111111"gcloud access-context-manager perimeters update data_perimeter \
--policy=ACCESS_POLICY \
--set-ingress-policies=ingress.yaml6. Verifying the perimeter actually holds
Test from a VM or Cloud Shell session outside the perimeter's trusted projects, using a service account that genuinely has roles/bigquery.dataViewer on a dataset inside the perimeter:
bq query --project_id=my-project-id 'SELECT COUNT(*) FROM `my-project-id.dataset.table`'Expect something like:
BigQuery error in query operation: Error processing job: Request is prohibited by organization's policy. vpcServiceControlsUniqueIdentifier: ...That error, with a vpcServiceControlsUniqueIdentifier, is the perimeter working as designed — valid IAM role, denied by context. Hand that identifier to Google support if you ever need to dispute or debug a specific denial.
7. Securing Cloud Run inside the perimeter
Cloud Run services inside the perimeter need VPC egress configured through a Serverless VPC Access connector so their outbound calls to BigQuery or GCS stay inside Google's private network path rather than routing over the public internet, where perimeter rules for client context get evaluated differently:
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
name: data-api
annotations:
run.googleapis.com/vpc-access-connector: projects/my-project/locations/us-central1/connectors/perimeter-connector
run.googleapis.com/vpc-access-egress: all-traffic
spec:
template:
spec:
containers:
- image: gcr.io/my-project/data-api:latestvpc-access-egress: all-traffic forces all outbound traffic, not just RFC 1918 private ranges, through the connector. Without it, calls to bigquery.googleapis.com can still exit over the public internet path and get evaluated as coming from outside the perimeter's private boundary, producing confusing denials for a service that should clearly be "inside."
Common pitfalls
- Using project IDs instead of project numbers in
--resources. The API silently rejects or misinterprets this; always resolve to the numeric project number first. - Enforcing before dry-running. Teams that skip the dry-run step reliably break a BI tool, a scheduled export, or a cross-project pipeline on day one.
- Forgetting Cloud Run's VPC egress setting. A Cloud Run service inside the perimeter, without
all-trafficegress, can still get treated as an outside caller for its own outbound BigQuery/GCS calls. - Assuming VPC SC replaces IAM. It doesn't. A denied-by-perimeter call never reached IAM evaluation to begin with. You still need tight IAM; VPC SC is a second, independent layer.
- One perimeter for everything. A single perimeter containing every project in the org makes any legitimate cross-project workflow painful. Group projects by actual data-sensitivity boundary instead.
Troubleshooting
If a request is denied and you're not sure why, check Cloud Logging for the audit log entry under cloudaudit.googleapis.com/policy. The violationReason field tells you the specific cause:
NO_MATCHING_ACCESS_LEVEL— the caller is outside the perimeter and doesn't match any configured access level. Add one, or confirm this denial is actually wanted.RESOURCE_NOT_IN_SAME_SERVICE_PERIMETER— a cross-project call where the source and destination projects are in different perimeters with no ingress/egress rule bridging them.SERVICE_NOT_RESTRICTEDappearing unexpectedly — you added a new API-consuming resource (a new Cloud Function, a new BigQuery connection) and forgot to add its service to--restricted-services.
If dry-run mode shows zero violations after a week, double-check that dry-run logging is actually enabled and that you're querying the right log name — a silent perimeter with no logged traffic at all usually means the audit log sink or filter is misconfigured, not that there's genuinely no cross-perimeter traffic.
Where to go next
Once the perimeter is enforced, layer VPC SC context-aware access combined with Identity-Aware Proxy (IAP) for human access, and look at organization policy constraints (like restricting resource sharing across perimeters) to prevent someone from just creating a new project outside the boundary and granting themselves cross-project access. Google's own reference architecture for "secure data perimeter" is worth reading end to end before you roll this out beyond one pilot project.
Experienced DevOps Engineer proficient in Python, Docker, Kubernetes, CI/CD, and cloud technologies. Specialized in automating processes and building scalable infrastructure for seamless development and deployment.
