Sending Data to SignalFx¶

For the quickest ways to get started sending data in to SignalFx, see Install the SignalFx Smart Agent, Connect to AWS CloudWatch, Connect to Google Cloud Platform, or Connect to Microsoft Azure.

SignalFx also lets you send in data for many other infrastructures or applications, such as Kubernetes, Docker, Elasticsearch, and the Windows Perfcounter Reporter. Click Integrations in the navigation bar to open the Integrations page, which lists the available ways you can connect to SignalFx to send data in and receive notifications out.

For more detailed information about how to use the data that is sent in, and for detailed information on monitoring a Kubernetes environment, refer to our Integrations Guide.

Install the SignalFx Smart Agent¶

The SignalFx Smart Agent is a metric agent written in Go for monitoring infrastructure and application services in a variety of different environments.

To install the Smart Agent, open the Integrations page and click the SignalFx Smart Agent tile. Display the Install tab and follow the instructions for installing the agent using your preferred installation method. For more information about the Smart Agent, see Use the SignalFx Smart Agent.

Connect to AWS CloudWatch¶

If you’re an Amazon Web Services customer, SignalFx can automatically import metrics and metadata about your services from AWS CloudWatch. To connect SignalFx to CloudWatch, you’ll create a new policy and IAM role in AWS for SignalFx to use, provide information from SignalFx to the new role, then provide SignalFx with the role’s ARN. You must be an administrator of your SignalFx account to connect SignalFx to AWS CloudWatch.

Part 1 - Create a policy in AWS¶

You need to perform these steps only when you first install an Amazon Web Services integration; additional integrations will use the same policy. If you have already installed at least one AWS integration and created a policy, you can skip to Part 2 - Create an Amazon Web Services role and connect it to SignalFx.

To get started, log into your Amazon Web Services account.
From the Services list, select IAM to open Identity & Access Management.
In the sidebar, click Policies, then click Create Policy.
Click the JSON tab. You may see a message saying the policy is not valid; dismiss the message. Then delete the existing text in the policy.

Copy the following code and paste it into the policy:

{
 "Version": "2012-10-17",
 "Statement": [
  {
   "Effect": "Allow",
   "Action": [
    "dynamodb:ListTables",
    "dynamodb:DescribeTable",
    "dynamodb:ListTagsOfResource",
    "ec2:DescribeInstances",
    "ec2:DescribeInstanceStatus",
    "ec2:DescribeVolumes",
    "ec2:DescribeReservedInstances",
    "ec2:DescribeReservedInstancesModifications",
    "ec2:DescribeTags",
    "organizations:DescribeOrganization",
    "cloudwatch:ListMetrics",
    "cloudwatch:GetMetricData",
    "cloudwatch:GetMetricStatistics",
    "cloudwatch:DescribeAlarms",
    "sqs:ListQueues",
    "sqs:GetQueueAttributes",
    "sqs:ListQueueTags",
    "elasticmapreduce:ListClusters",
    "elasticmapreduce:DescribeCluster",
    "kinesis:ListShards",
    "kinesis:ListStreams",
    "kinesis:DescribeStream",
    "kinesis:ListTagsForStream",
    "rds:DescribeDBInstances",
    "rds:ListTagsForResource",
    "elasticloadbalancing:DescribeLoadBalancers",
    "elasticloadbalancing:DescribeTags",
    "elasticache:describeCacheClusters",
    "elasticache:ListTagsForResource",
    "redshift:DescribeClusters",
    "lambda:GetAlias",
    "lambda:ListFunctions",
    "lambda:ListTags",
    "autoscaling:DescribeAutoScalingGroups"
   ],
   "Resource": "*"
  }
 ]
}

If the invalid policy message appears again, confirm you have pasted the entire text correctly.

Click Review policy.
Give the policy a name and optional description, then click Create policy.

Part 2 - Create an Amazon Web Services role and connect it to SignalFx¶

In the Amazon Web Services console, if the Identity & Access Management page is not open, display the Services list and select IAM.
Select Roles in the sidebar, then click Create role. For type of trusted entity, select Another AWS account, then select Require external ID to display the External ID field.
Next, you’ll provide the role with information from SignalFx. In a new tab, open SignalFx and click Integrations to open the Integrations page. Click the Amazon Web Services tile. Display the Install tab, then click New Integration.
Give the integration a name, and then, from the fields that appear, copy and paste the SignalFx Account ID and External ID from the SignalFx tab (first illustration below) into the Account ID and External ID fields in the AWS tab (second illustration below). (You will return to the SignalFx tab in a later step, so leave it open). Then click Next: Permissions.
Find the policy that you created earlier in Part 1 - Create a policy in AWS, select it (click the checkbox to its left) and then click Next: Tags.
Add tags to the role, if desired. You can use tags to specify the resources for which data will be imported or excluded. Click Next: Review.
Give the role a name and optional description, review the information that you’ve entered, then click Create Role. You have now created a role that SignalFx can use to connect to AWS CloudWatch.

Part 3 - Complete the integration in SignalFx¶

Again select Roles in the sidebar. Click on the role you just created, and copy the Role ARN.
Switch back to the SignalFx tab, and paste the ARN in the text field labelled Role ARN.
By default, SignalFx will import all available data for every CloudWatch-enabled region in your Amazon Web Services account, excluding GovCloud and China regions. To import only a subset, or to import from those regions, click All Regions to display options for choosing specific regions.

Each AWS integration can import data from either GovCloud regions, China regions, or other regions. For example, if you want to import data from US East regions and from GovCloud (US‑East), you will have to add two AWS integrations.
Select the data that you wish SignalFx to import from Amazon Web Services. Except for the first option below, you can change this configuration later.
- “Synchronize metadata”: SignalFx imports the properties of your AWS hosts and services, including region, account ID, and more. You can use this information to search, filter and aggregate your metrics.
- “Import data for AWS Optimizer”: With this option selected, SignalFx will import cost and usage data from AWS and will also generate metrics you can use to visualize EC2 usage and approximated costs, as well as get alerted in real-time on unexpected changes in cost or usage.
Notes on using AWS Optimizer
- AWS Optimizer is available in the SignalFx Enterprise Edition.
- The imported data does not include Amazon Web Services billing data.
- Data is not imported and metrics are not generated for EC2 Spot Instances.
- If you have multiple AWS Accounts, you must add an Amazon Web Services integration for each account, and each integration must have “Import data for SignalFx AWS Optimizer” selected. If this is not the case, the generated metrics may not reflect accurate values.
- “Import CloudWatch”: With this option selected, SignalFx will import CloudWatch metrics.
  - By default, SignalFx imports all metrics from all resources in all built-in namespaces. You can limit what data is imported on a per-namespace basis, and you can also import data from custom namespaces. For more information, see Specifying Namespaces, Metrics, and Tags below.
  - See Importing S3 metrics and Importing metrics via the Cloudwatch agent for specifics on importing data from those sources.
  - You can set the poll rate for importing CloudWatch metrics to 5 minutes or 1 minute; the latter is appropriate if you are using Detailed Monitoring on an Amazon Web Services namespace.
Your integration is created in a disabled state. You can either save your integration now by clicking Save, and come back later to enable it. Or you can click Save and Enable to enable it now. When you first enable your integration, SignalFx will validate it.

When you see the Validated! message, SignalFx will start receiving metrics from Amazon Web Services. You can view data from AWS in Built-In Dashboards and Charts and in the Infrastructure Navigator. For more information about monitoring Amazon Web Services in SignalFx, see Monitor Amazon Web Services.

Tips

To create additional integrations (for example, to monitor different regions), repeat Part 2 - Create an Amazon Web Services role and connect it to SignalFx and Part 3 - Complete the integration in SignalFx for each integration.

You can use the Actions menu for an integration to disable and enable it, or to delete it.

Specifying Namespaces, Metrics, and Tags¶

By default, SignalFx imports all metrics from all resources in all built-in namespaces. If you want to limit the amount of data AWS sends to SignalFx, you can specify the built-in namespaces from which you want to receive data. For each namespace, you can then filter the data you import based on tags on the resource and/or by metric name. You can also import data from custom namespaces.

To specify built-in namespaces, click Select namespaces, then choose the namespace(s) for which you want data.
To specify custom namespaces, click Add custom namespaces, type the name of a custom namespace, then press Enter. You can specify multiple custom namespaces.

Specifying a filter on data to import has no effect on tag syncing. SignalFx will continue to sync all tags for all resources in a specified namespace.

In the example below, we are importing data only from Amazon ElasticSearch Service and EC2. By default, Import All is selected, which means SignalFx will import all metrics from all resources in both namespaces.

We are going to limit the data coming in from EC2. Click the dropdown arrow to see options for including or excluding data.

Use Import only if you want to specify a filter for the data to import (whitelist); no other data will be imported.
Use Don’t import if you want to specify a filter for the data to exclude (blacklist); all other data will be imported.

Tags are used to specify the resources for which data will be imported or excluded. In the example below, we are excluding one tag, which means SignalFx will import all metrics from all resources that do not have the tag version:canary.

SignalFx adds the prefix aws_tag_ to the names of tags after we sync them from AWS, to indicate their origin. For example, an AWS tag version:canary would appear in SignalFx as aws_tag_version:canary. When you filter your CloudWatch integration by tag, enter the name of the tag just as it appears in AWS.

You can also choose specific metrics to import or exclude. For example, consider the following conditions.

Only metricA and metricB will be imported, and only for resources specified by the tags. In other words:

For a resource that has the tag env:prod or env:beta, metricA and metricB will be imported.
For a resource that does not have either the tag env:prod or env:beta, no metrics will be imported.
No other metrics will be imported.

Wildcards are supported. For example, if you want to import a metric, but only for a resource that has specified tags with any value, you could use a condition like the following.

In this case, MetricA and MetricB will be imported for resources that have the env tag set to any value. No other metrics will be imported.

You can use the Actions menu to the right of a specified namespace to copy or paste filters from one namespace to another, to clear the filters (for example, if you want to specify different metrics or tags), or to remove the namespace from the list. Removing a namespace means that SignalFx will no longer import metrics from that namespace.

When you have finished specifying the namespaces, metrics, and tags to include or exclude, click Save.

Note

It is possible to specify more complex filtering options for a namespace through the use of the SignalFx API. In this case, instead of seeing a visual representation of the specified metrics or tags, you will see a message indicating that the filter was defined programmatically. You can click View filter code to see which metrics and tags are being included or excluded for that namespace.

Connect to Google Cloud Platform¶

To connect SignalFx to Google Cloud Platform (GCP), you’ll create a new service account key for SignalFx to use and specify services to monitor in SignalFx. You must be an administrator of your SignalFx account to connect SignalFx to GCP.

Part 1 - Select a role for the GCP service account¶

The recommended method to connect to GCP is use a Project Viewer role for the service account you create. Choosing this role assures that any future functionality updates implemented in SignalFx will not require any changes to your GCP setup. If you plan to use this role, you can skip to Part 2 - Configure GCP.

If you want to create a role with more restrictive permissions than those available to the Project Viewer role, you can create a new role to use for the service account you create.

About permissions

If you don’t specify sufficient permissions for the role associated with the service account, you will see an error message when you try to connect the project in SignalFx. If you see this error, review the permissions assigned to the role and add any permissions that have not been enabled or, alternately, change the role for the service account to Project Viewer.

The following table specifies the permissions required for this role.

Permission	Required?
`monitoring.metricDescriptors.get`	yes
`monitoring.metricDescriptors.list`	yes
`monitoring.timeSeries.list`	yes
`resourcemanager.projects.get`	if you want to sync project metadata, such as labels
`compute.instances.list`	if the Compute Engine service is enabled
`compute.machineTypes.list`	if the Compute Engine service is enabled
`spanner.instances.list`	if the Spanner service is enabled
`storage.buckets.list`	if the Spanner service is enabled

Part 2 - Configure GCP¶

In a new tab, log into your GCP account.
Open the GCP web console and select a project you want to monitor.
From the sidebar, select IAM & admin ‣ Service Accounts.
Click CREATE SERVICE ACCOUNT at the top of the screen and enter SignalFx as the service account name. Set the role as either Project ‣ Viewer or a role you created in Part 1.
Select Furnish a new private key and specify a key type of JSON.
Click CREATE. This will cause a new service account key file to be downloaded to your computer. Make sure you know where it is downloaded to, because you will need it later.
In a separate window or tab, go to https://console.cloud.google.com/apis/library/cloudresourcemanager.googleapis.com/ and confirm the Resource Manager API is enabled for this project. SignalFx uses this API to validate permissions on the service account keys.
For each project you want to monitor using this integration, repeat the above steps so that you have downloaded a service account key and enabled the Resource Manager API for each project.

Part 3 - Complete the configuration in SignalFx¶

Open SignalFx and click Integrations to open the Integrations page. Look for the tile named Google Cloud Platform. You can search for it by name, or find it in either the Essential Services or Services Monitored sections.
Click the Google Cloud Platform tile. Display the Install tab, then click New Integration to display the configuration options.
Enter a name for this integration.
Click Add Project, then click Import Service Account Key. Select one or more of the JSON keys that you downloaded from GCP in the steps above. Select all the keys you want to add, then click Open. Project IDs corresponding to the service account keys you imported will be displayed.
By default, all available services are monitored, and any new services that are added later will also be monitored. If you want to import metrics from only some of the available services, click the All Services link to display a list of the services you can monitor. Select the services you want to monitor and click Apply.
You can set the poll rate to 5 minutes or 1 minute.
If you selected Compute Engine as one of the services to monitor, you can enter a comma-separated list of Compute Engine Instance metadata keys to send as properties. As described in Monitor Google Cloud Platform (GCP), these are sent as properties named gcp_metadata_<metadata-key>.
Click Save to complete the process of implementing this integration.

Tip

When you have completed the GCP and SignalFx configuration steps, SignalFx will start receiving metrics from GCP for the specified project(s). For information on GCP metrics and metadata that are available in SignalFx, see Monitor Google Cloud Platform (GCP).

Connect to Microsoft Azure¶

To connect SignalFx to Azure, you start in your Azure portal, where you create an Azure application (Part 1 - Create an Azure Active Directory application) and specify subscriptions to monitor (Part 2 - Specifying and setting permissions for subscriptions). Then, in SignalFx, you set up the integration and specify services to monitor (Part 3 - Complete the configuration in SignalFx).

You must be an administrator of your SignalFx account to connect SignalFx to Azure.

Part 1 - Create an Azure Active Directory application¶

Important

Before continuing, make sure you have sufficient permissions to create an Azure Active Directory application, as described here. The following steps won’t work correctly if required permissions are not set.

In a new tab, log into your Azure portal.
Navigate to Azure Active Directory and select App registrations. Then click New application registration.
Select Web app / API, enter a name and sign-on URL, and then click Create. (SignalFx does not use this information, but it is required to create an app on Azure). Remember the name you entered, as you will need it later.
You’ll be taken to a new blade with information about the application. Copy the Application ID and save it for later use.
Click Settings and then click Keys in the Settings sidebar.
Create a secret key by providing a description and value, and set the duration to Never expires.
Click Save. The secret key will be displayed. Copy the secret key and save it for later use.
Navigate to Azure Active Directory and select Properties. Copy the Directory ID and save it for later use.

Part 2 - Specifying and setting permissions for subscriptions¶

In the Azure portal, navigate to All services and then click Subscriptions.
Find a subscription you want to monitor, then click on the subscription name.
Navigate to Access control (IAM) and click Add.
Set the Role to Monitoring Reader, enter or select the name of the App you just created in the Select field, then click Save.

Repeat the above steps for each subscription you want to monitor.

Part 3 - Complete the configuration in SignalFx¶

After you have created an Active Directory app and specified the subsciptions you want to monitor, you are ready to integrate Azure in SignalFx.

Open SignalFx and click Integrations to open the Integrations page. Look for the tile named Microsoft Azure. You can search for it by name, or find it in either the Essential Services or Services Monitored sections.
Click the Azure tile.
Display the Install tab, then click New Integration.
- Enter a name for this integration.
- Enter the Tenant ID (Directory ID), App ID, and Secret Key you saved from Part 1 - Create an Azure Active Directory application.
- Select the services and subscriptions you want to monitor.
- You can set the poll rate to 5 minutes or 1 minute.
Click Save to complete the process of implementing this integration.

Tip

When you see the Validated! message, SignalFx will start receiving metrics from Azure for the specified subscription(s) and service(s). For information on Microsoft Azure metrics and metadata that are available in SignalFx, see Monitor Microsoft Azure.