Docs » Integrations Guide » Monitor Google Cloud Platform (GCP)

Monitor Google Cloud Platform (GCP)

You can easily monitor Google Cloud Platform (GCP) in SignalFx using Google StackDriver metrics. (For instructions on setting up this integration, see Connect to Google Cloud Platform.) SignalFx provides a robust integration with StackDriver, has a StackDriver-powered mode for the Infrastructure Navigator, and includes many built-in-dashboards to help you get started monitoring Google Cloud Platform services.

You can also monitor Google Cloud Platform’s Compute Engine instances and the services running on them by using the SignalFx Smart Agent. The SignalFx collectd agent offers a much higher degree of customization than is possible with StackDriver, and may be preferable for instances where you want to see metrics at a finer resolution, or where detailed control over the metrics sent matters.

Note

The option to use collectd applies only to the cases where you have direct control over the software installed on an instance, as you do with Container Engine. In many cases, you must use StackDriver to monitor Google Cloud Platform services. As a result, it is a common practice to use both the StackDriver integration and the SignalFx collectd agent.

In addition, you can monitor Google Cloud Platform’s Google Container Engine (GKE) instances and the services running on them by using our Kubernetes agent.

Irrespective of the mechanism by which you collect and send your metrics, you can also take advantage of SignalFx’s importing of Google Cloud Platform metadata, which applies not only to the relevant GCP services, but can also be used with metrics collected via the SignalFx collectd agent. The metadata allows you to slice and dice by custom tags, zones, host names, and other properties or dimensions.

StackDriver metrics

Once you have completed the steps in Connect to Google Cloud Platform, metrics from StackDriver will be synced into SignalFx. Specifically the metrics under GCP metrics in the StackDriver metric list will be synced. We don’t sync any AWS or agent metrics from Stackdriver, as we recommend using our AWS integration or our collectd agent for monitoring those metrics.

These metrics will contain dimensions that correspond to the Labels described in the GCP metrics reference and the StackDriver Monitored Resource Types reference. To determine which monitored resource a metric corresponds to, use the monitored_resource dimension.

Uniquely identifying Google Cloud Platform resources

All of the metrics that our StackDriver integration sends contain a dimension called gcp_id. The value of this dimension will always start with the project ID that contains the resource followed by _ and then other properties specific to that resource. If you install collectd on a Compute Engine instance using the standard install script this dimension will automatically be added.

If you wish to manually send metrics with this dimension, perhaps the simplest way to discover the unique ID value is to find a timeseries that contains this dimension using the Metrics Catalog in SignalFx. The timeseries should contain other dimensions that give a more friendly identification to the underlying Google Cloud Platform resource.

Dimensions

The metric time series associated with Google Cloud Platform metrics have the following generic dimensions. These dimensions are common to all services.

Dimension name Description
gcp_id unique identifier for GCP objects
project_id project ID of the monitored resource
monitored_resource name of the monitored resource
service service to which the metric belongs

Apart from the above dimensions, each service also has a dimension that identifies the resource to which the metric belongs. For example, Compute instances have an instance_id dimension to identify an instance, and Storage buckets have a bucket_name dimension to identify a bucket.

Resource Metadata

Our Google Cloud Platform integration also queries the GCP API for metadata about the resources it is monitoring, so you can filter and group metrics by this metadata in charts and in the Infrastructure Navigator.

  • Metadata that is common to all services within a project (project-level metadata) are put on properties of project_id dimension.
  • Metadata that are service-specific (service-level metadata) are put on properties of the gcp_id dimension.

Project-level metadata

Here is the metadata that is currently synced at a project level:

GCP name Custom property Description
creationTimestamp gcp_project_creation_time time project was created (e.g. Thu Oct 19 18:16:25 UTC 2017)
Labels * gcp_project_label_<name-of-label> (if user has labels) all project-wide labels except for signalfx-id
name gcp_project_name human readable project name
project_number gcp_project_number project_number given by GCP
status gcp_project_status project status (e.g. ACTIVE, DELETE_IN_PROGRESS, DELETE_REQUESTED)

* This property is a list of key value pairs in GCP. For example, if GCP has [key1:label01, key2:label02] as the labels property, we will have two properties: gcp_project_label_key1 and gcp_project_label_key2.)

Service-level metadata

Here is the metadata that is synced at a service level for the services listed below.

Compute Engine instance

For Google Cloud Platform Compute Engine instances, SignalFx gets a subset of metadata about the instance, as well as custom metadata specified by the user on an instance level. For detailed information on properties, see Google’s documentation.

GCP name Custom property Description
scheduling.automaticRestart gcp_auto_restart whether the instance should be automatically restarted if it is terminated by Compute Engine (not terminated by a user)
scheduling.onHostMaintenance gcp_behavior_on_maintenance maintenance behavior for the instance
cpuPlatform gcp_cpu_platform CPU platform used by this instance
CPU gcp_cpus number of virtual CPUs that are available to the instance
creationTimestamp gcp_creation_time time when the instance was created, (e.g. Thu Oct 19 18:16:25 UTC 2017)
description gcp_description description of this instance
disks[].licenses[] * gcp_image_license license corresponding to the disks used by the instance
canIpForward gcp_ip_forward whether to allow this instance to send and receive packets with non-matching destination or source IPs
machineType gcp_machine_type type of gcp machine this instance corresponds to
memory gcp_memory amount of physical memory available to the instance, defined in MB
metadata ** gcp_metadata_<metadata-key> custom metadata key for the instance (generated based on whitelisted properties specified when completing the integration in SignalFx)
scheduling.preemptible gcp_preemptibility true if the instance is preemptible, otherwise false
status gcp_status status of the instance( e.g. RUNNING, STAGING, PROVISIONING)
zone gcp_zone name of the zone this instance belongs to
region gcp_region name of the region this instance belongs to

* There is not a one-to-one mapping between our gcp_image_license property to one in GCP because this property’s value is composed from the licenses of the disks, corresponding to the disks associated with the compute instance.

** This property is a list of key value pairs in GCP. For example, if GCP has [key1:val1, key2:val2] as the metadata property, we will have two properties: gcp_metadata_key1 and gcp_metadata_key2.)

Cloud Spanner Instance

For Spanner instances, we currently sync the following properties.

GCP name Custom property Description
state gcp_state state of the spanner instance (e.g. CREATING, READY)
Labels * gcp_label_<name-of-label> (if user has labels) user‑specified labels

* This property is a list of key value pairs in GCP. For example, if GCP has [key1:label01, key2:label02] as the labels property, we will have two properties: gcp_label_key1 and gcp_label_key2.)

Cloud Storage Bucket

For Storage buckets, we currently sync the following properties.

GCP name Custom property Description
creationTimestamp gcp_creation_time time at which the bucket was created, (e.g. Thu Oct 19 18:16:25 UTC 2017)
Labels * gcp_label_<name-of-label> (if user has labels) user‑specified labels
Storage class gcp_storage_class bucket’s storage class, such as coldline

* This property is a list of key value pairs in GCP. For example, if GCP has [key1:label01, key2:label02] as the labels property, we will have two properties: gcp_label_key1 and gcp_label_key2.)