Docs » Integrations Guide » Monitor Google Cloud Platform (GCP)

Monitor Google Cloud Platform (GCP) 🔗

You can easily monitor Google Cloud Platform (GCP) in SignalFx using Google StackDriver metrics. If you haven’t already done so, follow the instructions below to connect SignalFx to GCP.

Connect to Google Cloud Platform 🔗

To connect SignalFx to Google Cloud Platform (GCP), you’ll create a new service account key for SignalFx to use and specify services to monitor in SignalFx. You must be an administrator of your SignalFx account to connect SignalFx to GCP.

Part 1 - Select a role for the GCP service account 🔗

The recommended method to connect to GCP is to use a Project Viewer role for the service account you create. Choosing this role ensures that any future functionality updates implemented in SignalFx will not require any changes to your GCP setup. If you plan to use this role, you can skip to Part 2 - Configure GCP.

If you want to create a role with more restrictive permissions than those available to the Project Viewer role, you can create a new role to use for the service account you create.

About permissions

If you don’t specify sufficient permissions for the role associated with the service account, you will see an error message when you try to connect the project in SignalFx. If you see this error, review the permissions assigned to the role and add any permissions that have not been enabled or, alternately, change the role for the service account to Project Viewer.

The following table specifies the permissions required for this role.

Permission Required?
monitoring.metricDescriptors.get yes
monitoring.metricDescriptors.list yes
monitoring.timeSeries.list yes
resourcemanager.projects.get if you want to sync project metadata, such as labels
compute.instances.list if the Compute Engine service is enabled
compute.machineTypes.list if the Compute Engine service is enabled
spanner.instances.list if the Spanner service is enabled
storage.buckets.list if the Spanner service is enabled

Part 2 - Configure GCP 🔗

  1. In a new tab, log into your GCP account.

  2. Open the GCP web console and select a project you want to monitor.

  3. From the sidebar, select IAM & admin ‣ Service Accounts.

    ../_images/gcp-iam.png


  4. Click CREATE SERVICE ACCOUNT at the top of the screen and enter SignalFx as the service account name. The service account ID autofills; enter a Service account description. Click CREATE.

    ../_images/gcp-create-service-account.png


  5. Select a role to grant this Service account access to the selected project (optional). Click CONTINUE.

  6. On the right side panel, enable Key type JSON, and then click CREATE. This will cause a new service account key file (.json) to be downloaded to your computer in the Download folder.

    ../_images/gcp-private-key-saved.png


  7. In a separate window or tab, go to https://console.cloud.google.com/apis/library/cloudresourcemanager.googleapis.com/ and confirm the Cloud Resource Manager API is enabled for this project. SignalFx uses this API to validate permissions on the service account keys.

  8. For each project you want to monitor using this integration, repeat the above steps so that you have downloaded a service account key and enabled the Cloud Resource Manager API for each project.

Part 3 - Complete the configuration in SignalFx 🔗

  1. Open SignalFx and click Integrations to open the Integrations page.

  2. Click the Google Cloud Platform tile. Display the Setup tab, then click Create New Integration to display the configuration options.

    ../_images/gcp-create-new-integration.png


  3. Enter a name for this integration.

  4. Click Add Project, then click Import Service Account Key. Select one or more of the JSON keys that you downloaded from GCP in the steps above. Select all the keys you want to add, then click Open. Project IDs corresponding to the service account keys you imported will be displayed.

    ../_images/gcp-added-project.png

  5. By default, all available services are monitored, and any new services that are added later will also be monitored. If you want to import metrics from only some of the available services, click All Services to display a list of the services you can monitor. Select the services you want to monitor and click Apply.

  6. You can set the poll rate to 5 minutes or 1 minute.

  7. If you selected Compute Engine as one of the services to monitor, you can enter a comma-separated list of Compute Engine Instance metadata keys to send as properties. As described in Monitor Google Cloud Platform (GCP), these are sent as properties named gcp_metadata_<metadata-key>.

  8. Click Save.

    When you have completed the GCP and SignalFx configuration steps, SignalFx will start receiving metrics from GCP for the specified project(s).

Your GCP integration is now complete.

If you installed this integration while going through the Quick Start guide, continue by installing the Smart Agent, which monitors host infrastructure metrics.

Google Cloud Platform integration overview 🔗

SignalFx provides a robust integration with StackDriver, has a StackDriver-powered mode for the Infrastructure Navigator, and includes many built-in-dashboards to help you get started monitoring Google Cloud Platform services.

You can also monitor Google Cloud Platform’s Compute Engine instances and the services running on them by using the SignalFx Smart Agent. The SignalFx collectd agent offers a much higher degree of customization than is possible with StackDriver, and may be preferable for instances where you want to see metrics at a finer resolution, or where detailed control over the metrics sent matters.

Note

The option to use collectd applies only to the cases where you have direct control over the software installed on an instance, as you do with Container Engine. In many cases, you must use StackDriver to monitor Google Cloud Platform services. As a result, it is a common practice to use both the StackDriver integration and the SignalFx collectd agent.

In addition, you can monitor Google Cloud Platform’s Google Container Engine (GKE) instances and the services running on them by using our Kubernetes agent.

Irrespective of the mechanism by which you collect and send your metrics, you can also take advantage of SignalFx’s importing of Google Cloud Platform metadata, which applies not only to the relevant GCP services, but can also be used with metrics collected via the SignalFx collectd agent. The metadata allows you to slice and dice by custom tags, zones, host names, and other properties or dimensions.

StackDriver metrics 🔗

Once you have completed the steps in Connect to Google Cloud Platform, metrics from StackDriver will be synced into SignalFx. Specifically the metrics under GCP metrics in the StackDriver metric list will be synced. We don’t sync any AWS or agent metrics from Stackdriver, as we recommend using our AWS integration or our collectd agent for monitoring those metrics.

These metrics will contain dimensions that correspond to the Labels described in the GCP metrics reference and the StackDriver Monitored Resource Types reference. To determine which monitored resource a metric corresponds to, use the monitored_resource dimension.

Uniquely identifying Google Cloud Platform resources 🔗

All of the metrics that our StackDriver integration sends contain a dimension called gcp_id. The value of this dimension will always start with the project ID that contains the resource followed by _ and then other properties specific to that resource. If you install collectd on a Compute Engine instance using the standard install script this dimension will automatically be added.

If you wish to manually send metrics with this dimension, perhaps the simplest way to discover the unique ID value is to find a timeseries that contains this dimension using the Metadata Catalog in SignalFx. The timeseries should contain other dimensions that give a more friendly identification to the underlying Google Cloud Platform resource.

Dimensions 🔗

The metric time series associated with Google Cloud Platform metrics have the following generic dimensions. These dimensions are common to all services.

Dimension name Description
gcp_id unique identifier for GCP objects
project_id project ID of the monitored resource
monitored_resource name of the monitored resource
service service to which the metric belongs

Apart from the above dimensions, each service also has a dimension that identifies the resource to which the metric belongs. For example, Compute instances have an instance_id dimension to identify an instance, and Storage buckets have a bucket_name dimension to identify a bucket.

Resource Metadata 🔗

Our Google Cloud Platform integration also queries the GCP API for metadata about the resources it is monitoring, so you can filter and group metrics by this metadata in charts and in the Infrastructure Navigator.

  • Metadata that is common to all services within a project (project-level metadata) are put on properties of project_id dimension.
  • Metadata that are service-specific (service-level metadata) are put on properties of the gcp_id dimension.

Project-level metadata 🔗

Here is the metadata that is currently synced at a project level:

GCP name Custom property Description
creationTimestamp gcp_project_creation_time time project was created (e.g. Thu Oct 19 18:16:25 UTC 2017)
Labels * gcp_project_label_<name-of-label> (if user has labels) all project-wide labels except for signalfx-id
name gcp_project_name human readable project name
project_number gcp_project_number project_number given by GCP
status gcp_project_status project status (e.g. ACTIVE, DELETE_IN_PROGRESS, DELETE_REQUESTED)

* This property is a list of key value pairs in GCP. For example, if GCP has [key1:label01, key2:label02] as the labels property, we will have two properties: gcp_project_label_key1 and gcp_project_label_key2.)

Service-level metadata 🔗

Here is the metadata that is synced at a service level for the services listed below.

Compute Engine instance 🔗

For Google Cloud Platform Compute Engine instances, SignalFx gets a subset of metadata about the instance, as well as custom metadata specified by the user on an instance level. For detailed information on properties, see Google’s documentation.

GCP name Custom property Description
scheduling.automaticRestart gcp_auto_restart whether the instance should be automatically restarted if it is terminated by Compute Engine (not terminated by a user)
scheduling.onHostMaintenance gcp_behavior_on_maintenance maintenance behavior for the instance
cpuPlatform gcp_cpu_platform CPU platform used by this instance
CPU gcp_cpus number of virtual CPUs that are available to the instance
creationTimestamp gcp_creation_time time when the instance was created, (e.g. Thu Oct 19 18:16:25 UTC 2017)
description gcp_description description of this instance
disks[].licenses[] * gcp_image_license license corresponding to the disks used by the instance
canIpForward gcp_ip_forward whether to allow this instance to send and receive packets with non-matching destination or source IPs
machineType gcp_machine_type type of gcp machine this instance corresponds to
memory gcp_memory amount of physical memory available to the instance, defined in MB
metadata ** gcp_metadata_<metadata-key> custom metadata key for the instance (generated based on whitelisted properties specified when completing the integration in SignalFx)
scheduling.preemptible gcp_preemptibility true if the instance is preemptible, otherwise false
status gcp_status status of the instance( e.g. RUNNING, STAGING, PROVISIONING)
zone gcp_zone name of the zone this instance belongs to
region gcp_region name of the region this instance belongs to

* There is not a one-to-one mapping between our gcp_image_license property to one in GCP because this property’s value is composed from the licenses of the disks, corresponding to the disks associated with the compute instance.

** This property is a list of key value pairs in GCP. For example, if GCP has [key1:val1, key2:val2] as the metadata property, we will have two properties: gcp_metadata_key1 and gcp_metadata_key2.)

Cloud Spanner Instance 🔗

For Spanner instances, we currently sync the following properties.

GCP name Custom property Description
state gcp_state state of the spanner instance (e.g. CREATING, READY)
Labels * gcp_label_<name-of-label> (if user has labels) user‑specified labels

* This property is a list of key value pairs in GCP. For example, if GCP has [key1:label01, key2:label02] as the labels property, we will have two properties: gcp_label_key1 and gcp_label_key2.)

Cloud Storage Bucket 🔗

For Storage buckets, we currently sync the following properties.

GCP name Custom property Description
creationTimestamp gcp_creation_time time at which the bucket was created, (e.g. Thu Oct 19 18:16:25 UTC 2017)
Labels * gcp_label_<name-of-label> (if user has labels) user‑specified labels
Storage class gcp_storage_class bucket’s storage class, such as coldline

* This property is a list of key value pairs in GCP. For example, if GCP has [key1:label01, key2:label02] as the labels property, we will have two properties: gcp_label_key1 and gcp_label_key2.)