BigQuery
catalog.dataset.tableWhat you need
| Field | Notes |
|---|---|
| Project id | The project holding the data |
| Dataset | Pin this. See below |
| Service account key | JSON, pasted once |
| Billing project | Optional, if queries should bill elsewhere |
| Read-only | Optional |
Only the service-account email is kept in the clear, so you can see which identity a catalog runs as; the key itself goes to the secrets manager.
Always pin the dataset
An unpinned catalog enumerates every dataset and table in the project the first time a query is planned. Measured against one real project: 62 seconds and 1.27 GiB to plan, versus 0.9 seconds and 164 MiB with the dataset set. Because planning happens in a fresh process per submission, that cost is paid on every query — it reads to users as a hang.
One catalog per dataset is the rule.
IAM the identity actually needs
Reads go through the BigQuery Storage Read API, which is not covered by
dataViewer alone:
| Role | Scope | Why |
|---|---|---|
roles/bigquery.dataViewer | dataset | read the tables |
roles/bigquery.jobUser | project | run a query at all |
roles/bigquery.readSessionUser | project | create a read session |
Missing the last one produces the user does not have 'bigquery.readsessions.create' permission, and it blocks metadata collection as
well as queries. readsessions.create cannot be granted at dataset scope.
To write, add roles/bigquery.dataEditor on the dataset and leave the catalog
writable.