Monitor Google Cloud Platform (GCP) 🔗
You can easily monitor Google Cloud Platform (GCP) in SignalFx using Google StackDriver metrics. If you haven’t already done so, follow the instructions below to connect SignalFx to GCP.
Connect to Google Cloud Platform 🔗
To connect SignalFx to Google Cloud Platform (GCP), you’ll create a new service account key for SignalFx to use and specify services to monitor in SignalFx. You must be an administrator of your SignalFx account to connect SignalFx to GCP.
Part 1 - Select a role for the GCP service account 🔗
The recommended method to connect to GCP is to use a Project Viewer role for the service account you create. Choosing this role ensures that any future functionality updates implemented in SignalFx will not require any changes to your GCP setup. If you plan to use this role, you can skip to Part 2 - Configure GCP.
If you want to create a role with more restrictive permissions than those available to the Project Viewer role, you can create a new role to use for the service account you create.
The following table specifies the permissions required for this role.
Permission | Required? |
---|---|
monitoring.metricDescriptors.get |
yes |
monitoring.metricDescriptors.list |
yes |
monitoring.timeSeries.list |
yes |
resourcemanager.projects.get |
if you want to sync project metadata, such as labels |
compute.instances.list |
if the Compute Engine service is enabled |
compute.machineTypes.list |
if the Compute Engine service is enabled |
spanner.instances.list |
if the Spanner service is enabled |
storage.buckets.list |
if the Spanner service is enabled |
Part 2 - Configure GCP 🔗
In a new tab, log into your GCP account.
Open the GCP web console and select a project you want to monitor.
From the sidebar, select
.Click CREATE SERVICE ACCOUNT at the top of the screen and enter
SignalFx
as the service account name. The service account ID autofills; enter a Service account description. Click CREATE.Select a role to grant this Service account access to the selected project (optional). Click CONTINUE.
On the right side panel, enable Key type JSON, and then click CREATE. This will cause a new service account key file (.json) to be downloaded to your computer in the Download folder.
In a separate window or tab, go to https://console.cloud.google.com/apis/library/cloudresourcemanager.googleapis.com/ and confirm the Cloud Resource Manager API is enabled for this project. SignalFx uses this API to validate permissions on the service account keys.
For each project you want to monitor using this integration, repeat the above steps so that you have downloaded a service account key and enabled the Cloud Resource Manager API for each project.
Part 3 - Complete the configuration in SignalFx 🔗
Open SignalFx and click Integrations to open the Integrations page.
Click the Google Cloud Platform tile. Display the Setup tab, then click Create New Integration to display the configuration options.
Enter a name for this integration.
Click Add Project, then click Import Service Account Key. Select one or more of the JSON keys that you downloaded from GCP in the steps above. Select all the keys you want to add, then click Open. Project IDs corresponding to the service account keys you imported will be displayed.
By default, all available services are monitored, and any new services that are added later will also be monitored. If you want to import metrics from only some of the available services, click All Services to display a list of the services you can monitor. Select the services you want to monitor and click Apply.
You can set the poll rate to 5 minutes or 1 minute.
If you selected Compute Engine as one of the services to monitor, you can enter a comma-separated list of Compute Engine Instance metadata keys to send as properties. As described in Monitor Google Cloud Platform (GCP), these are sent as properties named
gcp_metadata_<metadata-key>
.Click Save.
When you have completed the GCP and SignalFx configuration steps, SignalFx will start receiving metrics from GCP for the specified project(s).
Your GCP integration is now complete.
If you installed this integration while going through the Quick Start guide, continue by installing the Smart Agent, which monitors host infrastructure metrics.
StackDriver metrics 🔗
Once you have completed the steps in Connect to Google Cloud Platform, metrics from StackDriver will be synced into SignalFx. Specifically the metrics under GCP metrics in the StackDriver metric list will be synced. We don’t sync any AWS or agent metrics from Stackdriver, as we recommend using our AWS integration or our collectd agent for monitoring those metrics.
These metrics will contain dimensions that correspond to the Labels described in the GCP metrics reference and the StackDriver Monitored Resource Types reference. To determine which monitored resource a metric corresponds to, use the monitored_resource
dimension.
Uniquely identifying Google Cloud Platform resources 🔗
All of the metrics that our StackDriver integration sends contain a dimension called gcp_id
. The value of this dimension will always start with the project ID that contains the resource followed by _
and then other properties specific to that resource. If you install collectd on a Compute Engine instance using the standard install script this dimension will automatically be added.
If you wish to manually send metrics with this dimension, perhaps the simplest way to discover the unique ID value is to find a timeseries that contains this dimension using the Metadata Catalog in SignalFx. The timeseries should contain other dimensions that give a more friendly identification to the underlying Google Cloud Platform resource.
Dimensions 🔗
The metric time series associated with Google Cloud Platform metrics have the following generic dimensions. These dimensions are common to all services.
Dimension name | Description |
---|---|
gcp_id |
unique identifier for GCP objects |
project_id |
project ID of the monitored resource |
monitored_resource |
name of the monitored resource |
service |
service to which the metric belongs |
Apart from the above dimensions, each service also has a dimension that identifies the resource to which the metric belongs. For example, Compute instances have an instance_id
dimension to identify an instance, and Storage buckets have a bucket_name
dimension to identify a bucket.
Resource Metadata 🔗
Our Google Cloud Platform integration also queries the GCP API for metadata about the resources it is monitoring, so you can filter and group metrics by this metadata in charts and in the Infrastructure Navigator.
- Metadata that is common to all services within a project (project-level metadata) are put on properties of
project_id
dimension. - Metadata that are service-specific (service-level metadata) are put on properties of the
gcp_id
dimension.
Project-level metadata 🔗
Here is the metadata that is currently synced at a project level:
GCP name | Custom property | Description |
---|---|---|
creationTimestamp |
gcp_project_creation_time |
time project was created (e.g. Thu Oct 19 18:16:25 UTC 2017 ) |
Labels * | gcp_project_label_<name-of-label> (if user has labels) |
all project-wide labels except for signalfx-id |
name |
gcp_project_name |
human readable project name |
project_number |
gcp_project_number |
project_number given by GCP |
status |
gcp_project_status |
project status (e.g. ACTIVE , DELETE_IN_PROGRESS , DELETE_REQUESTED ) |
* This property is a list of key value pairs in GCP. For example, if GCP has [key1:label01
, key2:label02
] as the labels property, we will have two properties: gcp_project_label_key1
and gcp_project_label_key2
.)
Service-level metadata 🔗
Here is the metadata that is synced at a service level for the services listed below.
Compute Engine instance 🔗
For Google Cloud Platform Compute Engine instances, SignalFx gets a subset of metadata about the instance, as well as custom metadata specified by the user on an instance level. For detailed information on properties, see Google’s documentation.
GCP name | Custom property | Description |
---|---|---|
scheduling.automaticRestart |
gcp_auto_restart |
whether the instance should be automatically restarted if it is terminated by Compute Engine (not terminated by a user) |
scheduling.onHostMaintenance |
gcp_behavior_on_maintenance |
maintenance behavior for the instance |
cpuPlatform |
gcp_cpu_platform |
CPU platform used by this instance |
CPU |
gcp_cpus |
number of virtual CPUs that are available to the instance |
creationTimestamp |
gcp_creation_time |
time when the instance was created, (e.g. Thu Oct 19 18:16:25 UTC 2017 ) |
description |
gcp_description |
description of this instance |
disks[].licenses[] * |
gcp_image_license |
license corresponding to the disks used by the instance |
canIpForward |
gcp_ip_forward |
whether to allow this instance to send and receive packets with non-matching destination or source IPs |
machineType |
gcp_machine_type |
type of gcp machine this instance corresponds to |
memory |
gcp_memory |
amount of physical memory available to the instance, defined in MB |
metadata ** |
gcp_metadata_<metadata-key> |
custom metadata key for the instance (generated based on whitelisted properties specified when completing the integration in SignalFx) |
scheduling.preemptible |
gcp_preemptibility |
true if the instance is preemptible, otherwise false |
status |
gcp_status |
status of the instance( e.g. RUNNING , STAGING , PROVISIONING ) |
zone |
gcp_zone |
name of the zone this instance belongs to |
region |
gcp_region |
name of the region this instance belongs to |
* There is not a one-to-one mapping between our gcp_image_license property to one in GCP because this property’s value is composed from the licenses of the disks, corresponding to the disks associated with the compute instance.
** This property is a list of key value pairs in GCP. For example, if GCP has [key1:val1
, key2:val2
] as the metadata property, we will have two properties: gcp_metadata_key1
and gcp_metadata_key2
.)
Cloud Spanner Instance 🔗
For Spanner instances, we currently sync the following properties.
GCP name | Custom property | Description |
---|---|---|
state |
gcp_state |
state of the spanner instance (e.g. CREATING , READY ) |
Labels * | gcp_label_<name-of-label> (if user has labels) |
user‑specified labels |
* This property is a list of key value pairs in GCP. For example, if GCP has [key1:label01
, key2:label02
] as the labels property, we will have two properties: gcp_label_key1
and gcp_label_key2
.)
Cloud Storage Bucket 🔗
For Storage buckets, we currently sync the following properties.
GCP name | Custom property | Description |
---|---|---|
creationTimestamp |
gcp_creation_time |
time at which the bucket was created, (e.g. Thu Oct 19 18:16:25 UTC 2017 ) |
Labels * | gcp_label_<name-of-label> (if user has labels) |
user‑specified labels |
Storage class | gcp_storage_class |
bucket’s storage class, such as coldline |
* This property is a list of key value pairs in GCP. For example, if GCP has [key1:label01
, key2:label02
] as the labels property, we will have two properties: gcp_label_key1
and gcp_label_key2
.)