Datadog 리소스 쉽게 관리하기(datadog-sync)

Datadog의 리소스를 쉽게 관리할 수 있는 datadog-sync tool에 대해 가이드 한다.

---

1. 설치

• git clone을 통해 datadog-sync-cli 저장소를 복제해 온 후 pip로 install을 수행한다.(python3.9이상)

root@test:~# git clone https://github.com/DataDog/datadog-sync-cli.git
Cloning into 'datadog-sync-cli'...
remote: Enumerating objects: 6300, done.
remote: Counting objects: 100% (2083/2083), done.
remote: Compressing objects: 100% (602/602), done.
remote: Total 6300 (delta 1462), reused 1705 (delta 1331), pack-reused 4217
Receiving objects: 100% (6300/6300), 1.97 MiB | 24.56 MiB/s, done.
Resolving deltas: 100% (4424/4424), done.

root@test:~/datadog-sync-cli# pip install .

Processing /root/datadog-sync-cli

  Preparing metadata (setup.py) ... done

....

      Successfully uninstalled datadog-sync-cli-0.0.0

Successfully installed charset-normalizer-3.1.0 click-8.1.3 configobj-5.0.8 datadog-sync-cli-0.3.1 deepdiff-6.2.3 orjson-3.9.1 requests-2.28.2

 

2. datadog-sync cli 사용법

Usage: datadog-sync COMMAND [OPTIONS]

  Initialize cli

Options:
  --source-api-key TEXT                       Datadog source organization API key. [required for import]
  --source-app-key TEXT                       Datadog source organization APP key. [required for import]
  --source-api-url TEXT                       Datadog source organization API url.
  --destination-api-key TEXT                  Datadog destination organization API key. [required for sync/diffs]
  --destination-app-key TEXT                  Datadog destination organization APP key. [required for sync/diffs]
  --destination-api-url TEXT                  Datadog destination organization API url.
  --validate BOOLEAN                          Enables validation of the provided API
                                              during client initialization. On import,
                                              only source api key is validated. On
                                              sync/diffs, only destination api key is validated. [default: True]
  --http-client-timeout INTEGER               The HTTP request timeout period. Defaults to `30s`.
  --http-client-retry-timeout INTEGER         The HTTP request retry timeout period. Defaults to `60s`.
  --resources TEXT                            Optional comma separated list of resource to
                                              import. All supported resources are imported
                                              by default. See [Filtering] section for more details.
  --cleanup [True|False|Force]                Cleanup resources from destination org. [default: False]
  -v, --verbose                               Enable verbose logging.
  --filter TEXT                               Filter imported resources. See [Filtering] section for more details.
  --filter-operator TEXT                      Filter operator when multiple filters are passed. Supports `AND` or `OR`.
  --config FILE                               Read configuration from FILE. See [Config] section for more details.
  --max-workers INTEGER                       Max number of workers when running
                                              operations in multi-threads. Defaults to the number of processors on the machine, multiplied by 5.
  --skip-failed-resource-connections BOOLEAN  Skip resource if resource connection fails. [default: True]  [sync + import only]
  --force-missing-dependencies                Force importing and syncing resources that
                                              could be potential dependencies to the
                                              requested resources. [sync only]
  --help                                      Show this message and exit.

Commands:
  diffs   Log resource diffs.
  import  Import Datadog resources.
  sync    Sync Datadog resources to destination.

• 추가 내용 확인은 아래 URL을 참고하면 된다.
• https://github.com/DataDog/datadog-sync-cli

GitHub - DataDog/datadog-sync-cli: Datadog cli tool to sync resources across organizations.

 

3. 사용 예시

• A ORG에서 B ORG로 monitor항목을 복사해 온다.
• datadog-sync config파일 생성(source, destination, filter, resource 항목등일 들어감)

destination_api_url=https://app.datadoghq.com
destination_api_key="58208798cb20f****"
destination_app_key="0fbbe9378d53416568****"
source_api_key="bb6ad45d147100ef5fb****"
source_app_key="2a291d7dc242b839dbd8****"
source_api_url=https://us5.datadoghq.com
filter=["Type=Monitors;Name=tags;Value=env:prd"]
resources="monitors"

• source와 destination의 api_key, app_key, api_url 등을 입력한다.
  (app_key는 Datadog > Organization Settings > Application Keys 에서 직접 생성)
• resources : sync를 맞출 리소스 type 입력을 입력한다. 지원하는 type은 아래를 참고한다.
• filter : sync를 맞출 대상을 세부적으로 선정할 때 활용한다.
  위의 예시는 monitors 항목 중 env:prd tag가 있는 리소스만 sync 대상으로 지정한다.

roles
users
synthetics_private_locations
synthetics_tests
synthetics_global_variables
monitors
downtimes
service_level_objectives
slo_corrections
spans_metrics
dashboards
dashboard_lists
logs_custom_pipelines
notebooks
host_tags
logs_indexes
logs_metrics
logs_restriction_queries
metric_tag_configurations

 

• 명령어를 통해 source ORG의 monitor항목 중 env:prd tag를 가진 리소스를 local로 가져온다.

root@test:~/datadog-sync# datadog-sync import --config config
2023-06-30 11:08:25,572 - INFO - clients validated successfully
2023-06-30 11:08:25,575 - INFO - Starting import...
2023-06-30 11:08:25,575 - INFO - Importing monitors
2023-06-30 11:08:25,859 - INFO - Finished importing monitors: 3 successes, 0 errors
2023-06-30 11:08:25,859 - INFO - Finished import

• filtering된 3개의 항목을 가져왔다는 메시지가 보이며 datadog-sync설치경로/resources/source/monitors.json 파일이 생성된 것을 확인 가능하다.

• 저장된 monitors.json을 기준으로 destination ORG에 리소스를 생성한다.

root@test:~/datadog-sync# datadog-sync sync --config config
2023-06-30 11:08:34,328 - INFO - clients validated successfully
2023-06-30 11:08:34,331 - INFO - Starting sync...
2023-06-30 11:08:34,333 - INFO - Running create for monitors with 218026
2023-06-30 11:08:34,334 - INFO - Running create for monitors with 218027
2023-06-30 11:08:34,334 - INFO - Running create for monitors with 218028
2023-06-30 11:08:34,694 - INFO - finished create for monitors with 218027
2023-06-30 11:08:35,515 - INFO - finished create for monitors with 218028
2023-06-30 11:08:35,635 - INFO - finished create for monitors with 218026
2023-06-30 11:08:35,651 - INFO - Finished sync: 3 successes, 0 errors

• 마찬가지로 3개의 항목이 sync된 것을 확인 가능하며 실제 ORG에 접속해도 확인 가능하다.

---

표준 set을 정의해 두고 새로운 ORG가 생성될 때 마다 sync를 통해 쉽게 리소스 관리가 가능하다.