GCP VPC Flow Logs¶
Pulls Google Cloud VPC flow log records from a Pub/Sub subscription, fed by a Cloud Logging sink, and summarizes them into network conversations with the same flow engine as the on-premises NetFlow/IPFIX collector and AWS VPC Flow Logs.
| Category | Cloud Security |
| Connector Name | signal-client.gcp-vpc-flow-pull |
| Auth Method | Service Account JSON |
| Polling Interval | 2 min |
| Multi-Instance | Yes (up to 5) |
| Vendor Docs | About VPC Flow Logs records |
Prerequisites¶
Vendor Requirements
A Google Cloud project with VPC Flow Logs, and IAM permission to create a Cloud Logging sink, a Pub/Sub topic and subscription, and a service account.
- VPC Flow Logs enabled on the subnets you want visibility into
- Permission to create a Cloud Logging sink, a Pub/Sub topic and a pull subscription
- Permission to create a service account and a key for it
- Network: Conductor can reach the Google Cloud APIs (
*.googleapis.com) on port 443
Step 1: Create API Credentials¶
Route Flow Logs to Pub/Sub¶
- Enable VPC Flow Logs on the subnets you want to collect
-
Create a Cloud Logging sink to a Pub/Sub topic with this filter:
-
Create a pull subscription on that topic
Create a Service Account¶
- Create a service account for the connector
- Grant it
roles/pubsub.subscriberon the subscription only. The connector only pulls and acknowledges messages, so it needs nothing more - Create a JSON key for the service account and download it
Step 2: Configure in Conductor¶
- Open the Conductor UI at
https://<conductor-ip>/admin/settings/integrations - From the Add Integration dropdown, select GCP VPC Flow Logs
- Enter a unique name for this instance
-
Fill in the settings form:
Field Value Description Project ID my-gcp-projectThe project that owns the Pub/Sub subscription Pub/Sub Subscription vpc-flows-subThe subscription ID, or the full projects/PROJECT/subscriptions/NAMEpath the console showsService Account Key (JSON) (paste key file) The whole downloaded key file. It is stored with the integration's settings and never written to a log -
Toggle Enabled to on
- Click Save
Step 3: Validate Data Flow¶
After saving, verify the integration is working:
- Check connection status — The integration tile should show a green status indicator within 1–2 polling cycles
-
Check Signal Client logs:
-
Check artifacts in Analytics — Navigate to WitFoo Analytics Signals → Search and search for network flow artifacts from this source
No TCP flags
The VPC Flow Logs record format carries byte and packet counts but no TCP flag field, so the SYN, RST and FIN counters are zero for this source. A zero there means "not reported by GCP", not "none observed".
Troubleshooting¶
Authentication Failed (401 or 403)¶
- Paste the whole service account key file, including the braces
- Check that the service account has
roles/pubsub.subscriberon the subscription - Check that Project ID is the project that owns the subscription
No Data Appearing¶
- Confirm the integration shows Enabled in the Conductor UI
- Confirm the Cloud Logging sink uses the filter above and writes to the topic the subscription is attached to
- Confirm the subscription is a pull subscription
- Check Signal Client logs:
docker logs signal-client-svc --tail=100
Pulls Pause for Ten Minutes¶
- One failed hand-off of the connector's output to the pipeline pauses its pulls for ten minutes. This is a known limit of this release; collection resumes on its own
See also: Integration Catalog · Integration Management · Signal Client · Common Troubleshooting