Connect to AWS and send data to Splunk Observability Cloud 🔗
To leverage the benefits of data monitoring across your infrastructure, connect Splunk Observability Cloud to AWS following these steps:
Verify the prerequisites.
Choose among our connection options.
Check our recommended next steps.
You can also set the following configuration options to complete the integration:
Select Amazon Web Services (AWS) regions to collect data from.
Enable the ingestion of metrics through polling or streaming.
Decide whether to process information about application logs.
Following configuration, you can use Amazon CloudWatch to import metrics and logs from supported AWS services into Splunk Observability Cloud, and analyze your data using Observability Cloud tools.
To connect AWS to Observability Cloud and integrate those platforms, you must meet the following prerequisites:
Administrator privileges in Observability Cloud and your AWS accounts
One of the following authentication methods: - An AWS IAM role and an external ID from Observability Cloud - A secure token, which combines an access key ID and a secret access key
The AWS GovCloud and China regions require a secure token for access.
Regardless of the connection option you choose, you can configure your system more efficiently if you decide beforehand what data types and sources you want.
To determine the best connection method and configuration settings, answer the following questions before you connect AWS to Splunk Observability Cloud:
Do I want to collect metrics through API polling at specified intervals, or through CloudWatch Metric Streams?
Do I want to collect logs in addition to metrics? If yes, then include logs while configuring through the API or when given that option while performing a guided setup.
You can connect Observability Cloud to AWS in several different ways. Choose the connection method that best matches your needs:
Reason for using this method
Connect to AWS using the guided setup in Splunk Observability Cloud
Guides you step-by-step to set up an AWS connection and default configuration in Observability Cloud. Guided setup includes links to Amazon CloudFormation templates that you can select to create needed AWS IAM roles.
Connect to AWS using the Splunk Observability Cloud API
Requires knowledge of POST and PUT call syntax, but includes options and automation that are not part of the guided setup. Choose this method if you want to configure many integrations at once.
Connect to AWS using Splunk Terraform
Can be used if you already manage your infrastructure as code by deploying through Terraform.
Splunk is not responsible for data availability, and it can take up to several minutes (or longer, depending on your configuration) from the time you connect until you start seeing valid data from your account.
By default, Splunk Observability Cloud will bring in data from all supported AWS services associated with your account. To limit the amount of data to import, see Specifying data and metadata to import.
If you can’t connect AWS to Splunk Observability Cloud, see Troubleshoot your AWS connection.
Rather than polling for metrics data at specified intervals, CloudWatch Metric Streams sends metrics to a Kinesis Data Firehose stream, reducing latency. See Low Latency Observability Into AWS Services With Splunk in the DevOps blog for more information.
Although Metric Streams are more efficient than API polling, consider the constraints below.
CloudWatch Metric Streams continually stream Amazon CloudWatch metrics as soon as they are published. In most cases, the metrics are published once per minute.
For customers currently collecting Amazon CloudWatch metrics at the default polling rate of 300 seconds (5 minutes), this difference in intervals typically increases Amazon CloudWatch usage costs.
Customers already polling at 1-minute intervals generally see a slight decrease in Amazon CloudWatch usage costs compared to Metric Streams.
After an AWS integration is created, Observability Cloud checks if more than 100,000 metrics are fetched from CloudWatch. If this is the case, the integration gets automatically disabled, and a warning email is sent.
This check runs just once per integration. If you enable the integration afterwards, it will work correctly.
You can disable this check by setting the
enableCheckLargeVolume field in the AWS integration to
false using the API.
CloudWatch Metric Streams do not support filtering based on resource tags. Configuration applies to individual services, and all resources that report metrics from a configured service stream those metrics.
If you filter data based on tags, your costs for Amazon CloudWatch and Splunk Infrastructure Monitoring might increase.
After you’re all set, we recommend the following:
See Leverage data from integration with AWS for an overview of what you can do after you connect Observability Cloud to AWS.
Learn about our AWS Infrastructure Monitoring options. You’ll find instructions on how to import AWS metrics and metadata, or AWS tag and log information using namespaces and filters.
Refer to the AWS official documentation for a list of the available AWS metrics and other data, or read about the metadatada we provide.
To collect traces and metrics of your AWS Lambda functions for Splunk APM, see Instrument AWS Lambda functions for Splunk Observability Cloud.