Docs » Connect to your cloud service provider » Connect to AWS and send data to Splunk Observability Cloud

Connect to AWS and send data to Splunk Observability Cloud 🔗

To leverage the benefits of data monitoring across your infrastructure, connect Splunk Observability Cloud to AWS following these steps:

  1. Verify the prerequisites.

  2. Plan your integration.

  3. Choose among our connection options.

  4. Check our recommended next steps.

You can also set the following configuration options to complete the integration:

  • Select Amazon Web Services (AWS) regions to collect data from.

  • Enable the ingestion of metrics through polling or streaming.

  • Decide whether to process information about application logs.

Following configuration, you can use Amazon CloudWatch to import metrics and logs from supported AWS services into Splunk Observability Cloud, and analyze your data using Observability Cloud tools.

1. AWS integration prerequisites

To connect AWS to Observability Cloud and integrate those platforms, you must meet the following prerequisites:

  • Administrator privileges in Observability Cloud and your AWS accounts

  • One of the following authentication methods: - An AWS IAM role and an external ID from Observability Cloud - A secure token, which combines an access key ID and a secret access key

The AWS GovCloud and China regions require a secure token for access.

2. Plan your integration

Regardless of the connection option you choose, you can configure your system more efficiently if you decide beforehand what data types and sources you want.

To determine the best connection method and configuration settings, answer the following questions before you connect AWS to Splunk Observability Cloud:

  • Do I want to collect metrics through API polling at specified intervals, or through CloudWatch Metric Streams?

  • Do I want to collect logs in addition to metrics? If yes, then include logs while configuring through the API or when given that option while performing a guided setup.

3. AWS connection options

You can connect Observability Cloud to AWS in several different ways. Choose the connection method that best matches your needs:

Connection option

Reason for using this method

Connect to AWS using the guided setup in Splunk Observability Cloud

Guides you step-by-step to set up an AWS connection and default configuration in Observability Cloud. Guided setup includes links to Amazon CloudFormation templates that you can select to create needed AWS IAM roles.

Connect to AWS using the Splunk Observability Cloud API

Requires knowledge of POST and PUT call syntax, but includes options and automation that are not part of the guided setup. Choose this method if you want to configure many integrations at once.

Connect to AWS using Splunk Terraform

Can be used if you already manage your infrastructure as code by deploying through Terraform.

Note

Splunk is not responsible for data availability, and it can take up to several minutes (or longer, depending on your configuration) from the time you connect until you start seeing valid data from your account.

By default, Splunk Observability Cloud will bring in data from all supported AWS services associated with your account. To limit the amount of data to import, see Specifying data and metadata to import.

If you can’t connect AWS to Splunk Observability Cloud, see Troubleshoot your AWS connection.

Use Metric Streams to forward data to Splunk Observability Cloud

Rather than polling for metrics data at specified intervals, CloudWatch Metric Streams sends metrics to a Kinesis Data Firehose stream, reducing latency. See Low Latency Observability Into AWS Services With Splunk in the DevOps blog for more information.

You can enable Metric Streams both with our guided setup, or the Splunk Observability Cloud API.

Although Metric Streams are more efficient than API polling, consider the constraints below.

Collection interval

CloudWatch Metric Streams continually stream Amazon CloudWatch metrics as soon as they are published. In most cases, the metrics are published once per minute.

For customers currently collecting Amazon CloudWatch metrics at the default polling rate of 300 seconds (5 minutes), this difference in intervals typically increases Amazon CloudWatch usage costs.

Customers already polling at 1-minute intervals generally see a slight decrease in Amazon CloudWatch usage costs compared to Metric Streams.

High data volume warning

After an AWS integration is created, Observability Cloud checks if more than 100,000 metrics are fetched from CloudWatch. If this is the case, the integration gets automatically disabled, and a warning email is sent.

This check runs just once per integration. If you enable the integration afterwards, it will work correctly.

You can disable this check by setting the enableCheckLargeVolume field in the AWS integration to false using the API.

Tag filtering

CloudWatch Metric Streams do not support filtering based on resource tags. Configuration applies to individual services, and all resources that report metrics from a configured service stream those metrics.

If you filter data based on tags, your costs for Amazon CloudWatch and Splunk Infrastructure Monitoring might increase.

4. Next steps

After you’re all set, we recommend the following: