Overview
The Dynatrace ETL Service provides a semi-continuous export of custom-formatted collections of metrics and properties for Dynatrace entities and sessions. This solution pushes data on a scheduled bases to enterprise messaging infrastructures, databases, files or APIs.
This is intended for users, who:
- Want to extract metrics from Dynatrace for consumption elsewhere.
- Want to perform advanced queries of the SmartScape model without burdening the cluster.
This enables you to:
- Stream metrics to any endpoint.
- Extract service flow metrics.
- Extract related entities.
- Query entities by any property or tag.
- Synchronize the SmartScape with a CMDB.
Compatibility Requirements
The ESA ETL Service requires NodeJS. It leverages an embedded database, but it can use MariaDB or MySQL as well.
Installation
It is recommended to use the Dynatrace ESA Solution Server to install this application.
Details
The ESA ETL Service provides a web UI in which the user can define jobs that specify which collections of entities or sessions need to be retrieved, the required metrics for them, and the frequency of their retrieval.
The UI also reports on key performance and health characteristics to help users make informed decisions with respect to the stress their selections impose on the Dynatrace APIs.
The ETL Service contains a StreamerService that dynamically distributes the work and automatically scales the collection of worker processes to maximize throughput. Worker processes individually optimize and limit their usage of the designated API endpoints.
The ETL Service also contains a LibraryService that synchronizes and indexes all Dynatrace entities and relationships in the SmartScape, which enables it to very efficiently execute queries.
There are limits to (1) the volume of data that can be retrieved from the Dynatrace APIs within a given timeframe and (2) the number of messages that can be posted to a data target in the allocated timeframes. The ETL Service therefore automatically adapts its scaling and parallelization strategies to maximize throughput while not exceeding any rate limits placed upon it.