Skip to content

Conversation

@justinmae
Copy link

Description

Add support for the --monitoring-configuration parameter to the emr create-cluster command, enabling users to configure CloudWatch logging for EMR clusters through the CLI.

Motivation

This change exposes an existing EMR API feature (MonitoringConfiguration) through the AWS CLI, allowing users to configure cluster-level CloudWatch logging without needing to use the AWS SDK or Console.

Changes

  • Added MONITORING_CONFIGURATION_SCHEMA to argumentschema.py
  • Added MONITORING_CONFIGURATION help text to helptext.py
  • Integrated --monitoring-configuration parameter in createcluster.py
  • Added unit test test_create_cluster_with_monitoring_configuration

Testing

  • Unit tests pass locally
  • Test validates parameter parsing and API call structure
  • No changes to botocore or service models required

Example Usage

aws emr create-cluster \
  --release-label emr-7.11.0 \
  --instance-groups InstanceGroupType=MASTER,InstanceType=m5.xlarge,InstanceCount=1 \
  --monitoring-configuration '{
    "CloudWatchLogConfiguration": {
        "Enabled": true,
        "LogGroupName": "MyLogGroup",
        "LogStreamNamePrefix": "MyPrefix",
        "EncryptionKeyArn": "arn:aws:kms:us-east-1:123456789012:key/12345678-1234-1234-1234-123456789012",
        "LogTypes": {
            "STEP_LOGS": ["STDOUT", "STDERR"],
            "SPARK_DRIVER": ["STDOUT"],
            "SPARK_EXECUTOR": ["STDERR"]
        }
    }
}'

Add support for the --monitoring-configuration parameter to the
emr create-cluster command. This enables users to configure CloudWatch
logging for EMR clusters through the CLI.

The monitoring configuration allows specification of:
* CloudWatch log group name and log stream prefix
* Encryption key ARN for log encryption
* Log types (STEP_LOGS, SPARK_DRIVER, SPARK_EXECUTOR) mapped to
  output streams (STDOUT, STDERR)

Implementation includes:
* MONITORING_CONFIGURATION_SCHEMA in argumentschema.py
* MONITORING_CONFIGURATION help text in helptext.py
* Parameter integration in createcluster.py
* Unit test test_create_cluster_with_monitoring_configuration

This change adds CLI support for an existing EMR API feature,
requiring no changes to botocore or service models.
@hssyoo hssyoo self-assigned this Dec 3, 2025
@codecov-commenter
Copy link

codecov-commenter commented Dec 3, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.35%. Comparing base (68039fa) to head (d97ab09).
⚠️ Report is 430 commits behind head on develop.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #9891      +/-   ##
===========================================
- Coverage    93.40%   93.35%   -0.06%     
===========================================
  Files          211      209       -2     
  Lines        17012    16831     -181     
===========================================
- Hits         15890    15712     -178     
+ Misses        1122     1119       -3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Contributor

@hssyoo hssyoo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks @justinmae!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants