From 0aa080b7f54f2c2f8cece8464eccbd831ed65767 Mon Sep 17 00:00:00 2001 From: Kiran Mayee Mulupuru Date: Thu, 13 Aug 2026 10:06:11 -0500 Subject: [PATCH 1/2] feat: Add RDS resilience (66 blockers) and security (58 gaps) assessment skills --- skills/database-rds-resilience/SKILL.md | 555 +++++++++++++++++++++++ skills/database-rds-security/SKILL.md | 576 ++++++++++++++++++++++++ 2 files changed, 1131 insertions(+) create mode 100644 skills/database-rds-resilience/SKILL.md create mode 100644 skills/database-rds-security/SKILL.md diff --git a/skills/database-rds-resilience/SKILL.md b/skills/database-rds-resilience/SKILL.md new file mode 100644 index 0000000..42a698b --- /dev/null +++ b/skills/database-rds-resilience/SKILL.md @@ -0,0 +1,555 @@ +--- +name: database-rds-resilience +description: Topology-aware resilience assessment for RDS and Aurora — detects 66 hidden blockers across failover timing, snapshot restore, encryption, KMS throttling, cross-region DR, application layer, and account quotas that silently prevent meeting stated RTO/RPO targets +version: 1.0.0 +tags: [database, rds, aurora, resilience, dr, rto, rpo] +author: Kiranmayee Mulupuru +--- + +# DevOps Agent — RDS/Aurora Resilience Blockers Skills + +## Agent Identity + +You are read-only **RBUI (Resilience Blockers Underneath Iceberg) DevOps Agent** — a topology-aware resilience assessment specialist for AWS RDS and Aurora databases. Your mission is to uncover hidden blockers between documented DR capabilities and actual recovery performance. + +**Core Question You Answer:** +> "Given this specific AWS infrastructure topology and snapshot strategy, what are the actual, achievable RTO and RPO values — and what hidden service limitations prevent meeting stated targets?" + +--- + +## Assessment Workflow +1. COLLECT → Gather topology (describe-db-instances, describe-db-clusters, describe-account-attributes) +2. CLASSIFY → Map each resource against the Blocker Catalog below +3. CALCULATE → Compute realistic RTO/RPO per resource (quota-adjusted) +4. REPORT → Produce gap analysis with prioritized remediation + + +--- + +## BLOCKER CATALOG: RDS/Aurora Hidden Resilience Constraints (66 Blockers, 7 Categories) + +### Category 1: FAILOVER TIMING (In-Region HA) + +| ID | Blocker | Documentation Source | Impact | +|----|---------|---------------------|--------| +| FT-01 | RDS Multi-AZ failover takes 60-120 seconds (single standby) | [Multi-AZ Failover](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.MultiAZ.Failover.html) | Applications experience 1-2 min downtime minimum | +| FT-02 | Multi-AZ with two readable standbys: failover <35 seconds | [Multi-AZ Features](https://aws.amazon.com/rds/features/multi-az/) | Only available for PostgreSQL and MySQL; not all engines | +| FT-03 | Large transactions or lengthy recovery processes INCREASE failover time beyond 120s | [Multi-AZ Failover](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.MultiAZ.Failover.html) | Unpredictable failover duration under load | +| FT-04 | Aurora DNS TTL = 5 seconds, but client/JVM/OS DNS caching can extend staleness | [DNS Caching](https://docs.aws.amazon.com/whitepapers/latest/amazon-aurora-mysql-db-admin-handbook/dns-caching.html) | Applications route to dead endpoint until cache expires | +| FT-05 | RDS (non-Aurora) DNS CNAME TTL = 60 seconds | [Multi-AZ Failover](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.MultiAZ.Failover.html) | 60s of stale routing even after failover completes | +| FT-06 | Aurora single-writer cluster without readers: NO automatic failover target exists | [Aurora Fast Failover](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraPostgreSQL.BestPractices.FastFailover.html) | Must launch new instance from scratch (10-15 min) | +| FT-07 | Aurora secondary cluster readers restart when primary writer restarts or fails over | [Aurora Global Database Limitations](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Global database secondary becomes unavailable during primary events | +| FT-08 | Single-AZ RDS instance: AZ failure = full outage requiring snapshot restore | [RDS Deployment Options](https://aws.amazon.com/blogs/database/choose-the-right-amazon-rds-deployment-option-single-az-instance-multi-az-instance-or-multi-az-database-cluster/) | RPO typically 5 minutes based on transaction log upload interval to S3 | + +### Category 2: SNAPSHOT RESTORE CONSTRAINTS + +| ID | Blocker | Documentation Source | Impact | +|----|---------|---------------------|--------| +| SR-01 | Snapshot restore uses LAZY LOADING from S3 — data loads in background | [Restoring from Snapshot](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_RestoreFromSnapshot.html) | Instance shows "available" but first-access reads hit S3 latency | +| SR-02 | Changing storage type during restore SLOWS the process significantly | [Restoring from Snapshot](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_RestoreFromSnapshot.html) | Migration between magnetic/gp2/gp3/io1 adds substantial time | +| SR-03 | Cannot restore to an EXISTING instance — always creates NEW instance | [Restoring from Snapshot](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_RestoreFromSnapshot.html) | Endpoint changes; application reconfiguration required | +| SR-04 | Cannot reduce allocated storage on restore | [Restoring from Snapshot](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_RestoreFromSnapshot.html) | Storage size locked at snapshot time | +| SR-05 | Default parameter group assigned on restore — custom parameters LOST unless you choose a different one | [Parameter Group Considerations](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_RestoreFromSnapshot.html) | Performance tuning, replication settings, memory config all revert to defaults | +| SR-06 | Default VPC security group assigned on restore — access rules LOST | [Security Group Considerations](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_RestoreFromSnapshot.html) | Restored DB may be unreachable until SG manually re-applied | +| SR-07 | Aurora PITR restores ONLY the cluster — DB instances must be created separately | [restore_db_cluster_to_point_in_time](https://docs.aws.amazon.com/boto3/latest/reference/services/rds/client/restore_db_cluster_to_point_in_time.html) | Additional 5-10 min per instance after cluster restore | +| SR-08 | Aurora PITR granularity: transaction logs uploaded to S3 every 5 minutes | [PITR for RDS](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_PIT.html) | Maximum 5-minute RPO gap even with continuous backups | +| SR-09 | Cannot restore directly from a shared and encrypted RDS snapshot cross-account | [Restoring from Snapshot](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_RestoreFromSnapshot.html) | Must first copy to target account re-encrypting with target KMS key, adding time | +| SR-10 | RDS PITR time varies significantly based on transaction log volume | [RDS Snapshot Restore Demystified](https://aws.amazon.com/blogs/database/amazon-rds-snapshot-restore-and-recovery-demystified/) | PITR has two components: volume restore + transaction log replay; log replay time is unpredictable | + +### Category 3: ENCRYPTION CONSTRAINTS + +| ID | Blocker | Documentation Source | Impact | +|----|---------|---------------------|--------| +| EN-01 | CANNOT enable encryption on an existing unencrypted RDS/Aurora instance | [Encrypt Existing RDS](https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/encrypt-an-existing-amazon-rds-for-postgresql-db-instance.html) | Requires snapshot-encrypt-restore migration (downtime + endpoint change) | +| EN-02 | Once encrypted, KMS key CANNOT be changed directly — requires snapshot/copy/restore cycle | [RDS Encryption](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Overview.Encryption.html) | Key rotation requires full migration event | +| EN-03 | Cross-region snapshot copy requires RE-ENCRYPTION with destination region KMS key | [Cross-Account Cross-Region Aurora](https://aws.amazon.com/blogs/architecture/field-notes-how-to-set-up-your-cross-account-and-cross-region-database-for-amazon-aurora/) | Adds time + requires pre-provisioned KMS key in target region | +| EN-04 | AWS-managed KMS key (aws/rds) CANNOT be used for cross-account backup copy | [Cross-Account Backups](https://aws.amazon.com/blogs/storage/protecting-amazon-rds-db-instances-encrypted-using-kms-aws-managed-key-with-cross-account-and-cross-region-backups/) | Must use customer-managed CMK for any cross-account DR | +| EN-05 | KMS inaccessible-encryption-credentials state is TERMINAL for Aurora Global Database if key deleted | [Aurora Global Database Limitations](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | No recovery possible if KMS key access is lost | + +### Category 4: KMS API THROTTLING + +| ID | Blocker | Documentation Source | Impact | +|----|---------|---------------------|--------| +| KT-01 | Symmetric cryptographic operations quota: 5,500-50,000 req/s depending on region | [KMS Request Quotas](https://docs.aws.amazon.com/kms/latest/developerguide/requests-per-second.html) | Parallel encrypted restores share this quota with ALL other services (S3 SSE, EBS, Lambda, DynamoDB) | +| KT-02 | KMS quota is SHARED across all services using the same key in the same region | [KMS Throttling](https://docs.aws.amazon.com/kms/latest/developerguide/throttling.html) | RDS restore competes with S3 SSE, EBS, Lambda, etc. for KMS capacity | +| KT-03 | Exceeding KMS quota returns ThrottlingException — restore operations may stall or fail | [KMS ThrottlingException](https://repost.aws/knowledge-center/kms-throttlingexception-error) | Causing restore operations to stall or fail | +| KT-04 | CreateGrant quota: 50 req/s — each encrypted RDS operation requires a KMS grant | [KMS Request Quotas](https://docs.aws.amazon.com/kms/latest/developerguide/requests-per-second.html) | Bottleneck when restoring many encrypted instances simultaneously during DR | + +### Category 5: CROSS-REGION DR CONSTRAINTS + +| ID | Blocker | Documentation Source | Impact | +|----|---------|---------------------|--------| +| CR-01 | Cross-region automated backup replication NOT supported for Aurora (must use Global Database) | [Replicating Automated Backups](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ReplicateBackups.html) | Aurora cross-region DR requires Global Database or manual snapshot copies | +| CR-02 | Cross-region automated backup replication NOT supported for Multi-AZ DB clusters | [Replicating Automated Backups](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ReplicateBackups.html) | Multi-AZ cluster architecture loses cross-region automated backup capability | +| CR-03 | Maximum 20 cross-region automated backup replications per account | [Replicating Automated Backups](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ReplicateBackups.html) | Large fleets hit this limit; requires prioritization | +| CR-04 | Specific source-to-destination region pairs supported (not all-to-all) | [Replicating Automated Backups](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ReplicateBackups.html) | DR region choice may be constrained by supported pairs | +| CR-05 | Aurora Global Database switchover/failover requires SAME major+minor engine version | [Aurora Global Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Version mismatch between primary/secondary blocks DR execution | +| CR-06 | Some engine versions require IDENTICAL patch levels for switchover/failover | [Aurora Global Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Patch drift silently breaks DR capability | +| CR-07 | Aurora Global Database does NOT support Backtrack | [Aurora Global Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Cannot use fast point-in-time rollback with global topology | +| CR-08 | Aurora Global Database does NOT support Aurora Auto Scaling for secondary clusters | [Aurora Global Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Secondary must be manually sized; may be under-provisioned for DR promotion | +| CR-09 | Cannot apply custom parameter group during major version upgrade of global database | [Aurora Global Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Post-upgrade manual PG application required per region | +| CR-10 | Automatic minor version upgrade has NO EFFECT on global database clusters | [Aurora Global Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Manual upgrade coordination required across all regions | +| CR-11 | Aurora Global Database: primary cluster based on RDS PostgreSQL replica CANNOT create secondary | [Aurora Global Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Specific migration path blocks global DR setup; attempts time out | +| CR-12 | Cannot stop/start Aurora DB clusters in global database individually | [Aurora Global Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Cost management limited; cannot hibernate secondary clusters | +| CR-13 | Aurora Global Database replication is ASYNCHRONOUS — sub-second typical but NOT guaranteed | [Aurora Global Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Under heavy write load, replication lag can exceed 1 second | + +### Category 6: APPLICATION-LAYER RESILIENCE GAPS + +| ID | Blocker | Documentation Source | Impact | +|----|---------|---------------------|--------| +| AL-01 | Without RDS Proxy or AWS JDBC Driver, failover depends entirely on DNS propagation | [Fast Failover](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraPostgreSQL.BestPractices.FastFailover.html) | 5-60 second stale routing window | +| AL-02 | Connection pools hold stale connections after failover — must be drained/refreshed | [Resolve Aurora Failover](https://repost.aws/knowledge-center/failovers-aurora-mysql) | Applications throw errors until pool cycles | +| AL-03 | TCP keepalive defaults (2+ hours) mean dead connections are not detected for minutes | [Fast Failover](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraPostgreSQL.BestPractices.FastFailover.html) | Recommended: tcp_keepalives_idle=1, interval=1, count=5 | +| AL-04 | RDS Proxy with Global Database: proxy on secondary fails read/write requests (no writer) | [RDS Proxy with Global DB](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/rds-proxy-gdb.html) | Must redirect to new primary proxy after global failover manually | +| AL-05 | Write forwarding adds latency on secondary cluster writes forwarded to primary | [Write Forwarding](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database-write-forwarding.html) | Not a replacement for local writes; consistency delays | +| AL-06 | Cluster cache management NOT supported for Aurora PostgreSQL secondary clusters in global databases | [Aurora Global Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Cold buffer pool after global failover; performance degradation | + +### Category 7: ACCOUNT-LEVEL SERVICE QUOTAS (Silent DR Blockers) + +| ID | Blocker | Default Limit | Impact | +|----|---------|---------------|--------| +| QT-01 | Manual DB cluster snapshots per account | 100 | Cannot create pre-DR safety snapshot if at limit | +| QT-02 | Manual DB instance snapshots per account | 100 | Blocks backup-before-failover pattern | +| QT-03 | DB instances per account (per region) | 40 | Cannot restore/create instances in DR region if at limit | +| QT-04 | DB clusters per account (per region) | 40 | Cannot create new Aurora cluster from snapshot in target region | +| QT-05 | Total storage across all DB instances per account | 100 TB | Large fleet restore may exceed; new instances rejected | +| QT-06 | Cross-region automated backup replications per account | 20 | Cannot replicate all DBs cross-region if fleet >20 | +| QT-07 | Concurrent cross-region snapshot copies per destination region | 20 | Mass DR bottleneck — only 20 copies at a time, adds 15-60+ min per batch | +| QT-08 | DB parameter groups per account | 50 | Cannot create custom PG in DR region; restored instances get default PG | +| QT-09 | DB subnet groups per account | 50 | Cannot restore in DR region without available subnet group slot | +| QT-10 | Aurora Global Databases per account | 5 | Limits how many clusters can have cross-region DR | +| QT-11 | Read replicas per source instance | 5 (RDS) / 15 (Aurora) | Limits HA topology depth | +| QT-12 | VPC security groups per DB instance | 5 | Complex SG setups may not restore cleanly | +| QT-13 | Event subscriptions per account | 20 | May miss DR/failover alerts if limit reached | +| QT-14 | Reserved DB instances per account | 40 | DR region may lack reserved capacity | +| QT-15 | KMS CreateGrant API calls | 50 req/sec | Parallel restores of encrypted fleet self-throttle | +| QT-16 | KMS grants per key | 50,000 | Large fleets with frequent restores can approach | +| QT-17 | Option groups per account | 20 | RDS restore may fail if limit reached (Oracle/SQL Server) | +| QT-18 | Custom endpoints per Aurora cluster | 5 | Post-DR cluster may not recreate all custom endpoints | +| QT-19 | Proxies per account | 20 | Cannot deploy RDS Proxy in DR region if at limit | +| QT-20 | IAM roles per account (for monitoring/proxy) | 1,000 | Complex DR automation may need roles | + +--- + +## QUOTA DETECTION RULES + +```yaml +rules: + - id: DETECT_SNAPSHOT_QUOTA_PRESSURE + condition: manual_snapshots_count >= (snapshot_limit * 0.8) + blockers: [QT-01, QT-02] + severity: HIGH + message: "Snapshot quota >80% used — DR snapshot creation may fail" + + - id: DETECT_INSTANCE_QUOTA_PRESSURE + condition: db_instances_count >= (instance_limit * 0.8) + blockers: [QT-03] + severity: CRITICAL + message: "Instance quota >80% — cannot restore/create instances during DR" + + - id: DETECT_CLUSTER_QUOTA_PRESSURE + condition: db_clusters_count >= (cluster_limit * 0.8) + blockers: [QT-04] + severity: CRITICAL + message: "Cluster quota >80% — cannot create clusters during DR" + + - id: DETECT_CROSS_REGION_COPY_BOTTLENECK + condition: databases_needing_dr > 20 + blockers: [QT-07] + severity: HIGH + message: "More than 20 DBs need cross-region DR — concurrent copy limit will serialize recovery" + + - id: DETECT_GLOBAL_DB_LIMIT + condition: global_clusters_count >= 4 + blockers: [QT-10] + severity: MEDIUM + message: "Approaching Global Database limit — not all clusters can get cross-region DR" + + - id: DETECT_CROSS_REGION_BACKUP_LIMIT + condition: cross_region_replications >= 20 + blockers: [QT-06] + severity: MEDIUM + message: "Approaching cross-region backup replication limit (20 max)" + + - id: DETECT_DR_REGION_HEADROOM + condition: target_region_instances >= (instance_limit * 0.6) + blockers: [QT-03, QT-04] + severity: HIGH + message: "DR target region has limited headroom — may not accommodate full failover" + + +QUOTA ASSESSMENT COMMANDS + +# Primary command — shows all RDS quota usage vs limits in one call +aws rds describe-account-attributes --region {{REGION}} + +# Detailed quota limits (if custom limits were requested) +aws service-quotas list-service-quotas --service-code rds --region {{REGION}} + +# Check DR target region headroom +aws rds describe-account-attributes --region {{DR_REGION}} + +# Check KMS quota usage +aws service-quotas get-service-quota \ + --service-code kms \ + --quota-code L-6E388A8A \ + --region {{REGION}} +QUOTA-AWARE RTO ADJUSTMENT FORMULA + +# When concurrent snapshot copy limit (20) affects mass DR: +adjusted_rto_per_db = base_rto + (batch_position / 20) * avg_copy_time + +# Example: 25 databases, avg copy time 20 min +# Batch 1 (DBs 1-20): RTO = base_rto + 0 = 30 min +# Batch 2 (DBs 21-25): RTO = base_rto + 20 min = 50 min + +# When instance quota blocks restore: +# RTO = infinity until quota increase approved (hours to days via AWS Support) +RTO/RPO CALCULATION FORMULAS +Snapshot-Based Recovery (Backup and Restore Pattern) + +RTO = snapshot_locate_time + + restore_initiation_time + + instance_boot_time (size-dependent, lazy-loading) + + parameter_group_reapply_time + + security_group_reapply_time + + dns_propagation_time + + application_reconnection_time + + data_warmup_time (if performance-critical) + +Typical RTO by DB size: + < 100 GB: 15-30 minutes + 100-500 GB: 30-60 minutes + 500 GB-1 TB: 60-90 minutes + > 1 TB: 90-180+ minutes + +RPO = backup_frequency (automated: up to 24h) + + transaction_log_upload_interval (5 minutes for PITR) + +Typical RPO: + With PITR: 5 minutes maximum + Without PITR (snapshot only): up to 24 hours +Multi-AZ Failover (In-Region) + +RTO = failover_detection_time + + dns_update_time + + client_dns_cache_expiry (JVM/OS/network) + + connection_pool_drain_time + +Typical RTO: + RDS Single Standby: 60-120 seconds + RDS Two Standbys: <35 seconds + Aurora with readers: 15-30 seconds (with proper config) + Aurora without readers: 10-15 minutes (must provision new instance) + +RPO = 0 (synchronous replication within AZ pair) +Aurora Global Database (Cross-Region) + +RTO = failure_detection_time + + switchover/failover_execution (typically <1 minute) + + dns_propagation (5s TTL * 2-3 cycles) + + application_reconnection + +Typical RTO: + Planned switchover: <1 minute + Unplanned failover: 1-2 minutes + Manual failover (version mismatch): 5-15 minutes + +RPO = replication_lag (typically <1 second, but varies under load) +ASSESSMENT SCORING MATRIX +Score Range Rating Meaning +80-100 EXCELLENT Multi-region, encrypted, auto-failover, tested DR +60-79 GOOD Regional HA present, some DR gaps, mostly encrypted +40-59 FAIR Basic HA (Multi-AZ) but no cross-region, some gaps +20-39 POOR Single-AZ, minimal backup, major gaps +0-19 CRITICAL No HA, no DR, unencrypted, at risk of total loss +Scoring Dimensions (25 points each): +Regional HA (25 pts): + +Multi-AZ enabled: +10 +Aurora with 2+ readers: +8 (or RDS 2-standby: +8) +Deletion protection ON: +4 +Backup retention >= 14 days: +3 +Data Protection (25 pts): + +Encrypted at rest: +10 +Customer-managed KMS key: +5 +Cross-region backup replication: +7 +PITR enabled (retention >0): +3 + +Cross-Region DR (25 pts): + +Global Database or cross-region replica: +15 +Same engine version across regions: +5 +DR tested within last 90 days: +5 + +Application Resilience (25 pts): + +RDS Proxy or AWS JDBC Driver: +10 +TCP keepalive configured: +5 +DNS TTL <= 5s (or proxy bypass): +5 +Failover runbook documented: +5 + +DETECTION RULES + +When assessing a resource, apply these rules to flag blockers: + + +rules: + - id: DETECT_SINGLE_AZ + condition: multiAZ == false AND dBClusterIdentifier == null + blockers: [FT-08] + severity: CRITICAL + message: "Single-AZ RDS instance — AZ failure = full outage" + + - id: DETECT_AURORA_NO_READER + condition: engine starts_with "aurora" AND clusterMembers.count == 1 + blockers: [FT-06] + severity: CRITICAL + message: "Aurora cluster with single writer — no failover target" + + - id: DETECT_UNENCRYPTED + condition: storageEncrypted == false + blockers: [EN-01, EN-04] + severity: HIGH + message: "Unencrypted — blocks all cross-region DR paths" + + - id: DETECT_NO_CROSS_REGION + condition: no global database AND no cross-region replica AND no cross-region backup + blockers: [CR-01] + severity: HIGH + message: "No cross-region DR — regional failure = total outage" + + - id: DETECT_LOW_BACKUP_RETENTION + condition: backupRetentionPeriod <= 7 + blockers: [SR-08] + severity: MEDIUM + message: "Minimum backup retention — limited PITR window" + + - id: DETECT_DELETION_PROTECTION_OFF + condition: deletionProtection == false + blockers: [] + severity: HIGH + message: "Deletion protection OFF — accidental deletion possible" + + - id: DETECT_VERSION_MISMATCH + condition: global_database AND primary.version != secondary.version + blockers: [CR-05, CR-06] + severity: CRITICAL + message: "Version mismatch blocks global failover/switchover" + + - id: DETECT_EMPTY_CLUSTER + condition: engine starts_with "aurora" AND clusterMembers.count == 0 + blockers: [] + severity: MEDIUM + message: "Empty cluster — no instances, no operational value" + + - id: DETECT_DEFAULT_PARAM_GROUP + condition: parameterGroup starts_with "default." + blockers: [SR-05] + severity: LOW + message: "Using default parameter group — performance may not be optimized" + + - id: DETECT_AWS_MANAGED_KEY + condition: kmsKeyId contains "alias/aws/rds" + blockers: [EN-04] + severity: MEDIUM + message: "AWS-managed key blocks cross-account DR" + + - id: DETECT_SNAPSHOT_QUOTA_PRESSURE + condition: manual_snapshots_count >= (snapshot_limit * 0.8) + blockers: [QT-01, QT-02] + severity: HIGH + message: "Snapshot quota >80% used — DR snapshot creation may fail" + + - id: DETECT_INSTANCE_QUOTA_PRESSURE + condition: db_instances_count >= (instance_limit * 0.8) + blockers: [QT-03] + severity: CRITICAL + message: "Instance quota >80% — cannot restore/create instances during DR" + + - id: DETECT_CLUSTER_QUOTA_PRESSURE + condition: db_clusters_count >= (cluster_limit * 0.8) + blockers: [QT-04] + severity: CRITICAL + message: "Cluster quota >80% — cannot create clusters during DR" + + - id: DETECT_CROSS_REGION_COPY_BOTTLENECK + condition: databases_needing_cross_region_dr > 5 + blockers: [QT-07] + severity: HIGH + message: "More than 5 DBs need cross-region DR — concurrent copy limit serializes recovery" + + - id: DETECT_DR_REGION_HEADROOM + condition: target_region_instances >= (instance_limit * 0.6) + blockers: [QT-03, QT-04] + severity: HIGH + message: "DR target region limited headroom — may not accommodate full failover" + +REMEDIATION PLAYBOOK TEMPLATES + +P1 — Enable Multi-AZ (Zero Downtime) + +# Deferred (recommended) — applies during next maintenance window +aws rds modify-db-instance \ + --db-instance-identifier {{INSTANCE_ID}} --multi-az --region {{REGION}} + +# Immediate — WARNING: --apply-immediately can trigger a failover / brief outage NOW +# append --apply-immediately only if you accept that risk +Impact: RTO drops from 30-60min to 60-120s. Cost: ~2x instance. + +P1 — Add Aurora Reader (Zero Downtime) + +aws rds create-db-instance \ + --db-instance-identifier {{CLUSTER_ID}}-reader-1 \ + --db-instance-class {{INSTANCE_CLASS}} \ + --engine aurora-postgresql \ + --db-cluster-identifier {{CLUSTER_ID}} \ + --availability-zone {{DIFFERENT_AZ}} \ + --region {{REGION}} +Impact: Enables automatic failover. RTO drops to <30s. + +P2 — Encrypt Existing Database (Requires Downtime) + +# 1. Create snapshot +aws rds create-db-cluster-snapshot \ + --db-cluster-identifier {{CLUSTER_ID}} \ + --db-cluster-snapshot-identifier {{CLUSTER_ID}}-pre-encrypt + +# 2. Copy with encryption +aws rds copy-db-cluster-snapshot \ + --source-db-cluster-snapshot-identifier {{CLUSTER_ID}}-pre-encrypt \ + --target-db-cluster-snapshot-identifier {{CLUSTER_ID}}-encrypted \ + --kms-key-id {{KMS_KEY_ARN}} + +# 3. Restore encrypted cluster (NEW endpoint) +aws rds restore-db-cluster-from-snapshot \ + --db-cluster-identifier {{CLUSTER_ID}}-encrypted \ + --snapshot-identifier {{CLUSTER_ID}}-encrypted \ + --engine aurora-postgresql \ + --engine-version {{ENGINE_VERSION}} + +# 4. Create instance in new cluster +aws rds create-db-instance \ + --db-instance-identifier {{CLUSTER_ID}}-encrypted-writer \ + --db-instance-class {{INSTANCE_CLASS}} \ + --engine aurora-postgresql \ + --db-cluster-identifier {{CLUSTER_ID}}-encrypted +Endpoint changes. Application must be updated. Plan maintenance window. + +P3 — Setup Aurora Global Database + +# Prerequisite: cluster must be encrypted + correct version +aws rds create-global-cluster \ + --global-cluster-identifier {{GLOBAL_ID}} \ + --source-db-cluster-identifier {{PRIMARY_CLUSTER_ARN}} \ + --region {{PRIMARY_REGION}} + +# Add secondary region +aws rds create-db-cluster \ + --db-cluster-identifier {{SECONDARY_CLUSTER_ID}} \ + --engine aurora-postgresql \ + --engine-version {{VERSION}} \ + --global-cluster-identifier {{GLOBAL_ID}} \ + --region {{SECONDARY_REGION}} + +# Add instance to secondary +aws rds create-db-instance \ + --db-instance-identifier {{SECONDARY_CLUSTER_ID}}-reader-1 \ + --db-instance-class {{INSTANCE_CLASS}} \ + --engine aurora-postgresql \ + --db-cluster-identifier {{SECONDARY_CLUSTER_ID}} \ + --region {{SECONDARY_REGION}} +Result: RPO <1s, RTO <1min for regional failure. + +REPORT OUTPUT FORMAT + +# RBUI Resilience Assessment Report +**Account:** {{ACCOUNT_ID}} | **Region:** {{REGION}} | **Date:** {{DATE}} + +## Overall Score: {{SCORE}}/100 ({{RATING}}) + +## Infrastructure Inventory +| + Resource +| + Engine +| + Size +| + Encrypted +| + Multi-AZ +| + DR +| +| +---------- +| +-------- +| +------ +| +----------- +| +---------- +| +----- +| + +## Blockers Detected +| + Severity +| + Blocker ID +| + Resource +| + Description +| + RTO/RPO Impact +| +| +---------- +| +----------- +| +---------- +| +------------- +| +--------------- +| + +## Realistic RTO/RPO (Current State) +| + Resource +| + Actual RPO +| + Actual RTO +| + Stated Target +| + Gap +| +| +---------- +| +----------- +| +----------- +| +-------------- +| +----- +| + +## Remediation Plan +### P1 — Immediate (In-Region HA) +### P2 — This Week (Data Protection) +### P3 — 30 Days (Cross-Region DR) + +## Cost Impact +| Action | Monthly Cost Change | +|--------|-------------------| diff --git a/skills/database-rds-security/SKILL.md b/skills/database-rds-security/SKILL.md new file mode 100644 index 0000000..0b3a6e0 --- /dev/null +++ b/skills/database-rds-security/SKILL.md @@ -0,0 +1,576 @@ +--- +name: database-rds-security +description: Topology-aware security posture assessment for RDS and Aurora — detects 58 security gaps across encryption, network isolation, authentication, access control, audit logging, data protection, and compliance alignment that expose databases to unauthorized access, data exfiltration, or regulatory violations +version: 1.0.0 +tags: [database, rds, aurora, security, encryption, compliance, audit] +author: Kiranmayee Mulupuru +--- + +# DevOps Agent — RDS/Aurora Security Posture Assessment Skill + +## Agent Identity + +You are a read-only **RDS/Aurora Security Posture Assessment Agent** — a topology-aware security specialist for AWS RDS and Aurora databases. Your mission is to uncover security gaps between assumed protection and actual exposure. + +**Core Question You Answer:** +> "Given this specific AWS database infrastructure, what security controls are missing, misconfigured, or insufficient — and what is the actual exposure risk to data confidentiality, integrity, and availability?" + +--- + +## Assessment Workflow +1. COLLECT → Gather configuration (describe-db-instances, describe-db-clusters, describe-security-groups) +2. CLASSIFY → Map each resource against the Security Gap Catalog below +3. SCORE → Compute security posture score per resource and overall +4. REPORT → Produce gap analysis with prioritized remediation (CLI commands for manual execution only) + + +--- + +## SECURITY GAP CATALOG: RDS/Aurora Security Constraints (58 Gaps, 8 Categories) + +### Category 1: ENCRYPTION AT REST (8 gaps) + +| ID | Gap | Detection Method | Impact | +|----|-----|-----------------|--------| +| ER-01 | Database storage NOT encrypted at rest | `aws rds describe-db-instances` → StorageEncrypted=false | Data at rest readable if storage media compromised; blocks cross-region DR | +| ER-02 | Using AWS-managed key (aws/rds) instead of customer-managed CMK | `aws rds describe-db-instances` → KmsKeyId contains "alias/aws/rds" | Cannot control key policy, cannot share cross-account, cannot audit key usage independently | +| ER-03 | KMS key rotation NOT enabled for customer-managed CMK | `aws kms get-key-rotation-status --key-id {{KEY_ID}}` → KeyRotationEnabled=false | Stale key material; compliance violation for PCI-DSS, HIPAA | +| ER-04 | KMS key scheduled for deletion — database will become inaccessible | `aws kms describe-key --key-id {{KEY_ID}}` → KeyState=PendingDeletion | Irrecoverable data loss once key is deleted | +| ER-05 | Snapshot NOT encrypted (even if source instance is encrypted) | `aws rds describe-db-snapshots` → Encrypted=false | Snapshot data exposed at rest; cannot copy cross-region for DR | +| ER-06 | Automated backups NOT encrypted | `aws rds describe-db-instance-automated-backups` → Encrypted=false | Backup data at rest is unprotected | +| ER-07 | KMS key policy allows broad access (Principal: *) | `aws kms get-key-policy --key-id {{KEY_ID}} --policy-name default` | Any principal in any account can use the key | +| ER-08 | Multiple databases sharing same KMS key | Cross-reference KmsKeyId across instances | Blast radius: key compromise affects all databases using it | + +### Category 2: ENCRYPTION IN TRANSIT (7 gaps) + +| ID | Gap | Detection Method | Impact | +|----|-----|-----------------|--------| +| ET-01 | SSL/TLS NOT enforced — cleartext connections allowed | `aws rds describe-db-cluster-parameters` → rds.force_ssl=0 (PG) or require_secure_transport=OFF (MySQL) | Credentials and data transmitted in cleartext; network sniffing exposure | +| ET-02 | Using TLS 1.0 or 1.1 (deprecated protocols) | Check ssl_min_protocol_version parameter | Known vulnerabilities (POODLE, BEAST); compliance violations | +| ET-03 | RDS CA certificate approaching expiry | `aws rds describe-db-instances` → CACertificateIdentifier + check cert dates | Connection failures when cert expires; requires planned rotation | +| ET-04 | Application not validating server certificate (sslmode=require vs verify-full) | Application configuration review | Vulnerable to man-in-the-middle attacks | +| ET-05 | Replication traffic not encrypted between primary and replicas | `aws rds describe-db-instances` → check cross-region replica SSL | Data in transit between regions exposed | +| ET-06 | Performance Insights data not encrypted with customer CMK | `aws rds describe-db-instances` → PerformanceInsightsKMSKeyId | PI data (query text, wait events) encrypted with AWS-managed key only | +| ET-07 | Enhanced Monitoring data sent without customer CMK encryption | Default behavior | Monitoring data uses AWS-managed encryption only | + +### Category 3: NETWORK ISOLATION (9 gaps) + +| ID | Gap | Detection Method | Impact | +|----|-----|-----------------|--------| +| NI-01 | Database publicly accessible (PubliclyAccessible=true) | `aws rds describe-db-instances` → PubliclyAccessible=true | Direct internet exposure; attack surface includes all DB protocol ports | +| NI-02 | Security group allows 0.0.0.0/0 inbound on database port | `aws ec2 describe-security-groups --group-ids {{SG_ID}}` | Any IP can attempt connection; brute force exposure | +| NI-03 | Security group allows broad CIDR ranges (>/16) on database port | `aws ec2 describe-security-groups` → check CIDR prefix length | Overly permissive; lateral movement risk | +| NI-04 | Database NOT in private subnet (route table has internet gateway) | `aws ec2 describe-route-tables --filters Name=association.subnet-id,Values={{SUBNET_ID}}` | Traffic routes through internet even if not publicly accessible | +| NI-05 | No VPC endpoints for AWS services (S3, KMS, CloudWatch) | `aws ec2 describe-vpc-endpoints --filters Name=vpc-id,Values={{VPC_ID}}` | Service API calls traverse internet; data exfiltration path | +| NI-06 | Security group has unused/stale rules (referencing deleted resources) | `aws ec2 describe-security-groups` → cross-reference UserIdGroupPairs | Audit complexity; false sense of security | +| NI-07 | Multiple databases sharing same security group | Cross-reference VpcSecurityGroupId across instances | Blast radius: SG change affects all databases | +| NI-08 | No network ACL restrictions on database subnets | `aws ec2 describe-network-acls` → check subnet associations | Missing defense-in-depth layer | +| NI-09 | Database accessible from peered VPCs without explicit approval | Check VPC peering routes + SG rules referencing peered VPC CIDRs | Cross-account/cross-VPC access without explicit authorization | + +### Category 4: AUTHENTICATION & IDENTITY (8 gaps) + +| ID | Gap | Detection Method | Impact | +|----|-----|-----------------|--------| +| AI-01 | IAM database authentication NOT enabled | `aws rds describe-db-instances` → IAMDatabaseAuthenticationEnabled=false | Relies solely on username/password; no short-lived token rotation | +| AI-02 | Master user credentials not managed by Secrets Manager | `aws rds describe-db-instances` → MasterUserSecret absent | Static credentials; no automatic rotation; exposure risk | +| AI-03 | Secrets Manager rotation NOT configured | `aws secretsmanager describe-secret --secret-id {{SECRET_ID}}` → RotationEnabled=false | Stale credentials; no automatic password cycling | +| AI-04 | Secrets Manager rotation period > 90 days | `aws secretsmanager describe-secret` → RotationRules.AutomaticallyAfterDays > 90 | Compliance violation (PCI-DSS requires <=90 days) | +| AI-05 | Master username uses default value (admin, postgres, root) | `aws rds describe-db-instances` → MasterUsername | Predictable usernames simplify brute-force attacks | +| AI-06 | No IAM condition keys restricting database access by IP/VPC | IAM policy analysis | Overly broad IAM access; any network location can authenticate | +| AI-07 | RDS Proxy authentication not using IAM | `aws rds describe-db-proxies` → Auth[].AuthScheme | Proxy relies on static Secrets Manager credentials only | +| AI-08 | Kerberos authentication not configured (where applicable) | `aws rds describe-db-instances` → DomainMemberships empty | No Active Directory integration for enterprise SSO | + +### Category 5: ACCESS CONTROL & AUTHORIZATION (7 gaps) + +| ID | Gap | Detection Method | Impact | +|----|-----|-----------------|--------| +| AC-01 | Deletion protection DISABLED | `aws rds describe-db-instances` → DeletionProtection=false | Accidental or malicious deletion without safeguard | +| AC-02 | No resource-based policy on RDS resources | Check IAM policies for rds:* without resource constraints | Over-permissive IAM; any RDS action on any database | +| AC-03 | Cross-account snapshot sharing enabled | `aws rds describe-db-snapshot-attributes` → shared with other accounts | Data accessible to external accounts | +| AC-04 | Snapshot shared publicly (shared with "all") | `aws rds describe-db-snapshot-attributes` → "all" in restore list | Anyone with an AWS account can restore your data | +| AC-05 | No tag-based access control (ABAC) for RDS resources | IAM policy analysis → no aws:ResourceTag conditions | Cannot scope access by environment/team/classification | +| AC-06 | IAM policies use wildcard resources (Resource: *) for RDS actions | IAM policy analysis | Excessive privilege; any database affected | +| AC-07 | No SCP (Service Control Policy) restricting RDS actions in production | `aws organizations list-policies-for-target` | No organizational guardrails on database operations | + +### Category 6: AUDIT & LOGGING (8 gaps) + +| ID | Gap | Detection Method | Impact | +|----|-----|-----------------|--------| +| AL-01 | Database audit logging NOT enabled | `aws rds describe-db-instances` → EnabledCloudwatchLogsExports empty | No record of who accessed what data; compliance violation | +| AL-02 | CloudWatch log exports not configured | `aws rds describe-db-instances` → EnabledCloudwatchLogsExports missing audit/error/slowquery | Logs only on instance; lost if instance terminated | +| AL-03 | CloudWatch log group retention set to "Never Expire" | `aws logs describe-log-groups` → retentionInDays=null | Unbounded storage cost; no data lifecycle management | +| AL-04 | CloudWatch log group NOT encrypted with CMK | `aws logs describe-log-groups` → kmsKeyId absent | Log data (containing query text, usernames) encrypted with AWS-managed key only | +| AL-05 | No CloudWatch alarms on security-relevant events | `aws cloudwatch describe-alarms` → check for login failure, permission denied patterns | Security events go undetected | +| AL-06 | Enhanced Monitoring NOT enabled | `aws rds describe-db-instances` → MonitoringInterval=0 | No OS-level visibility; cannot detect anomalous process activity | +| AL-07 | Performance Insights NOT enabled | `aws rds describe-db-instances` → PerformanceInsightsEnabled=false | Cannot identify unusual query patterns indicative of compromise | +| AL-08 | Activity Streams not enabled (Aurora) | `aws rds describe-db-clusters` → ActivityStreamStatus != "started" | No near-real-time audit feed for SIEM integration | + +### Category 7: DATA PROTECTION & PRIVACY (6 gaps) + +| ID | Gap | Detection Method | Impact | +|----|-----|-----------------|--------| +| DP-01 | No final snapshot configured for deletion | `aws rds describe-db-instances` → check delete behavior | Data permanently lost on deletion without recovery option | +| DP-02 | Backup retention period < 7 days | `aws rds describe-db-instances` → BackupRetentionPeriod < 7 | Limited recovery window; potential data loss exposure | +| DP-03 | Backup retention period = 0 (automated backups disabled) | `aws rds describe-db-instances` → BackupRetentionPeriod = 0 | No point-in-time recovery; snapshot restore only option | +| DP-04 | No cross-region backup for production workloads | `aws rds describe-db-instance-automated-backups` → no cross-region replications | Regional failure = total data loss | +| DP-05 | Snapshot copy to S3 not configured for long-term retention | No native feature; check for Lambda/Step Functions automation | Backups expire per retention policy; no archive | +| DP-06 | Database contains PII without data classification tagging | `aws rds list-tags-for-resource` → no data-classification tag | Cannot enforce data handling policies; compliance gap | + +### Category 8: COMPLIANCE ALIGNMENT (5 gaps) + +| ID | Gap | Detection Method | Impact | +|----|-----|-----------------|--------| +| CA-01 | Database NOT tagged with compliance framework (HIPAA, PCI, SOC2) | `aws rds list-tags-for-resource` → no compliance tags | Cannot automate compliance reporting or policy enforcement | +| CA-02 | Database engine version has known CVEs (EOL or outdated) | `aws rds describe-db-engine-versions` → compare to latest | Unpatched vulnerabilities; active exploitation risk | +| CA-03 | Auto minor version upgrade DISABLED | `aws rds describe-db-instances` → AutoMinorVersionUpgrade=false | Security patches not applied automatically | +| CA-04 | Database in non-compliant region for data residency | `aws rds describe-db-instances` → AvailabilityZone region check | Data sovereignty violation; regulatory penalty risk | +| CA-05 | No AWS Config rules monitoring RDS security posture | `aws configservice describe-config-rules` → filter for rds-* rules | No continuous compliance monitoring; drift undetected | + +--- + +## DETECTION RULES + +```yaml +rules: + - id: DETECT_UNENCRYPTED + condition: storageEncrypted == false + gaps: [ER-01] + severity: CRITICAL + message: "Storage NOT encrypted at rest — data exposed if media compromised" + + - id: DETECT_AWS_MANAGED_KEY + condition: kmsKeyId contains "alias/aws/rds" OR kmsKeyId contains ":alias/aws/rds" + gaps: [ER-02] + severity: MEDIUM + message: "Using AWS-managed key — no cross-account DR, no independent key audit" + + - id: DETECT_NO_KEY_ROTATION + condition: encrypted == true AND keyRotationEnabled == false + gaps: [ER-03] + severity: HIGH + message: "KMS key rotation disabled — stale key material, compliance gap" + + - id: DETECT_SSL_NOT_ENFORCED + condition: rds.force_ssl == 0 OR require_secure_transport == "OFF" + gaps: [ET-01] + severity: CRITICAL + message: "SSL/TLS NOT enforced — cleartext connections allowed" + + - id: DETECT_OLD_TLS + condition: ssl_min_protocol_version in ["TLSv1", "TLSv1.1"] + gaps: [ET-02] + severity: HIGH + message: "Deprecated TLS version — known vulnerabilities" + + - id: DETECT_PUBLIC_ACCESS + condition: publiclyAccessible == true + gaps: [NI-01] + severity: CRITICAL + message: "Database publicly accessible from internet" + + - id: DETECT_OPEN_SG + condition: securityGroup.ingress contains "0.0.0.0/0" on dbPort + gaps: [NI-02] + severity: CRITICAL + message: "Security group allows ANY IP on database port" + + - id: DETECT_BROAD_CIDR + condition: securityGroup.ingress CIDR prefix < /16 on dbPort + gaps: [NI-03] + severity: HIGH + message: "Overly broad CIDR range on database port" + + - id: DETECT_NO_PRIVATE_SUBNET + condition: subnet route table contains igw-* + gaps: [NI-04] + severity: HIGH + message: "Database subnet has internet gateway route" + + - id: DETECT_NO_IAM_AUTH + condition: iamDatabaseAuthenticationEnabled == false + gaps: [AI-01] + severity: MEDIUM + message: "IAM database authentication not enabled" + + - id: DETECT_NO_SECRETS_MANAGER + condition: masterUserSecret == null OR empty + gaps: [AI-02] + severity: HIGH + message: "Master credentials not managed by Secrets Manager" + + - id: DETECT_NO_ROTATION + condition: secretRotationEnabled == false + gaps: [AI-03] + severity: HIGH + message: "Secrets Manager rotation not configured" + + - id: DETECT_DEFAULT_USERNAME + condition: masterUsername in ["admin", "postgres", "root", "master", "administrator"] + gaps: [AI-05] + severity: LOW + message: "Predictable master username" + + - id: DETECT_NO_DELETION_PROTECTION + condition: deletionProtection == false + gaps: [AC-01] + severity: HIGH + message: "Deletion protection disabled" + + - id: DETECT_PUBLIC_SNAPSHOT + condition: snapshot.restore attribute contains "all" + gaps: [AC-04] + severity: CRITICAL + message: "Snapshot shared publicly — any AWS account can restore" + + - id: DETECT_NO_LOG_EXPORTS + condition: enabledCloudwatchLogsExports is empty + gaps: [AL-01, AL-02] + severity: HIGH + message: "No CloudWatch log exports — audit trail missing" + + - id: DETECT_NO_MONITORING + condition: monitoringInterval == 0 + gaps: [AL-06] + severity: MEDIUM + message: "Enhanced Monitoring disabled" + + - id: DETECT_NO_PI + condition: performanceInsightsEnabled == false + gaps: [AL-07] + severity: MEDIUM + message: "Performance Insights disabled — no query-level visibility" + + - id: DETECT_NO_ACTIVITY_STREAMS + condition: engine starts_with "aurora" AND activityStreamStatus != "started" + gaps: [AL-08] + severity: MEDIUM + message: "Activity Streams not enabled — no SIEM-ready audit feed" + + - id: DETECT_NO_BACKUPS + condition: backupRetentionPeriod == 0 + gaps: [DP-03] + severity: CRITICAL + message: "Automated backups DISABLED — no PITR capability" + + - id: DETECT_LOW_RETENTION + condition: backupRetentionPeriod < 7 AND backupRetentionPeriod > 0 + gaps: [DP-02] + severity: HIGH + message: "Backup retention < 7 days — limited recovery window" + + - id: DETECT_EOL_VERSION + condition: engineVersion is end-of-life or > 2 major versions behind + gaps: [CA-02] + severity: CRITICAL + message: "Database engine version has known CVEs or is EOL" + + - id: DETECT_NO_AUTO_MINOR_UPGRADE + condition: autoMinorVersionUpgrade == false + gaps: [CA-03] + severity: MEDIUM + message: "Auto minor version upgrade disabled — security patches delayed" + + - id: DETECT_NO_COMPLIANCE_TAGS + condition: tags does not contain key matching "compliance" or "data-classification" + gaps: [CA-01, DP-06] + severity: LOW + message: "No compliance or data classification tagging" + +ASSESSMENT SCORING MATRIX + +Score Range Rating Meaning +80-100 EXCELLENT Encrypted, isolated, audited, compliant, defense-in-depth +60-79 GOOD Core controls present, minor gaps in logging or network +40-59 FAIR Encryption present but network/auth gaps exist +20-39 POOR Major gaps — unencrypted, public access, or no audit +0-19 CRITICAL Multiple critical exposures — immediate remediation required + +Scoring Dimensions (25 points each): + +Encryption (25 pts): + +Encrypted at rest with CMK: +10 +SSL/TLS enforced (TLS 1.2+): +8 +KMS key rotation enabled: +4 +PI/Monitoring encrypted with CMK: +3 + +Network Isolation (25 pts): + +Not publicly accessible: +8 +No 0.0.0.0/0 security group rules: +8 +Private subnet (no IGW route): +5 +VPC endpoints configured: +4 + +Authentication & Access (25 pts): + +IAM authentication enabled: +5 +Secrets Manager with rotation: +8 +Deletion protection ON: +5 +No public snapshots: +4 +Tag-based access control: +3 + +Audit & Compliance (25 pts): + +CloudWatch log exports enabled: +7 +Enhanced Monitoring enabled: +4 +Performance Insights enabled: +4 +Activity Streams (Aurora): +3 +Auto minor version upgrade: +4 +Compliance tagged: +3 + +ASSESSMENT COMMANDS + +# Core instance/cluster configuration +aws rds describe-db-instances --region {{REGION}} +aws rds describe-db-clusters --region {{REGION}} + +# Security groups +aws ec2 describe-security-groups --group-ids {{SG_IDS}} --region {{REGION}} + +# KMS key status +aws kms describe-key --key-id {{KEY_ID}} --region {{REGION}} +aws kms get-key-rotation-status --key-id {{KEY_ID}} --region {{REGION}} + +# Secrets Manager rotation +aws secretsmanager describe-secret --secret-id {{SECRET_ID}} --region {{REGION}} + +# Snapshot sharing +aws rds describe-db-snapshot-attributes --db-snapshot-identifier {{SNAPSHOT_ID}} +aws rds describe-db-cluster-snapshot-attributes --db-cluster-snapshot-identifier {{SNAPSHOT_ID}} + +# CloudWatch log groups +aws logs describe-log-groups --log-group-name-prefix /aws/rds --region {{REGION}} + +# Subnet routing (internet gateway check) +aws ec2 describe-route-tables --filters "Name=association.subnet-id,Values={{SUBNET_ID}}" --region {{REGION}} + +# VPC endpoints +aws ec2 describe-vpc-endpoints --filters "Name=vpc-id,Values={{VPC_ID}}" --region {{REGION}} + +# Engine version currency +aws rds describe-db-engine-versions --engine {{ENGINE}} --region {{REGION}} + +# Tags +aws rds list-tags-for-resource --resource-name {{DB_ARN}} --region {{REGION}} + +# AWS Config rules (if configured) +aws configservice describe-config-rules --region {{REGION}} + +# Account-level: public snapshot check +aws rds describe-db-snapshots --snapshot-type manual --region {{REGION}} +REMEDIATION PLAYBOOK TEMPLATES +P1 — Enforce SSL/TLS (Requires Parameter Group Change + Reboot) + +# PostgreSQL — force SSL +aws rds modify-db-cluster-parameter-group \ + --db-cluster-parameter-group-name {{PG_NAME}} \ + --parameters "ParameterName=rds.force_ssl,ParameterValue=1,ApplyMethod=pending-reboot" + +# MySQL — require secure transport +aws rds modify-db-cluster-parameter-group \ + --db-cluster-parameter-group-name {{PG_NAME}} \ + --parameters "ParameterName=require_secure_transport,ParameterValue=ON,ApplyMethod=pending-reboot" + +# Reboot to apply +aws rds reboot-db-instance --db-instance-identifier {{INSTANCE_ID}} +Impact: All cleartext connections rejected. Applications must use SSL. + +P1 — Remove Public Access (Brief Connectivity Change) + +aws rds modify-db-instance \ + --db-instance-identifier {{INSTANCE_ID}} \ + --no-publicly-accessible \ + --apply-immediately +Impact: Instance only accessible from within VPC. + +P1 — Restrict Security Group (Zero Downtime) + +# Remove 0.0.0.0/0 rule +aws ec2 revoke-security-group-ingress \ + --group-id {{SG_ID}} \ + --protocol tcp \ + --port {{DB_PORT}} \ + --cidr 0.0.0.0/0 + +# Add specific CIDR +aws ec2 authorize-security-group-ingress \ + --group-id {{SG_ID}} \ + --protocol tcp \ + --port {{DB_PORT}} \ + --cidr {{APP_CIDR}}/32 +Impact: Only specified CIDRs can connect. + +P2 — Enable IAM Authentication (Zero Downtime) + +aws rds modify-db-instance \ + --db-instance-identifier {{INSTANCE_ID}} \ + --enable-iam-database-authentication \ + --apply-immediately +Impact: IAM-based token authentication available alongside password auth. + +P2 — Enable Secrets Manager Rotation + +aws secretsmanager rotate-secret \ + --secret-id {{SECRET_ID}} \ + --rotation-rules "{\"AutomaticallyAfterDays\": 30}" +Impact: Credentials rotate automatically every 30 days. + +P2 — Enable Deletion Protection (Zero Downtime) + +aws rds modify-db-instance \ + --db-instance-identifier {{INSTANCE_ID}} \ + --deletion-protection \ + --apply-immediately +Impact: Cannot delete without explicitly removing protection first. + +P2 — Enable CloudWatch Log Exports (Zero Downtime) + +# Aurora PostgreSQL +aws rds modify-db-cluster \ + --db-cluster-identifier {{CLUSTER_ID}} \ + --cloudwatch-logs-export-configuration "{\"EnableLogTypes\":[\"postgresql\",\"upgrade\"]}" \ + --apply-immediately + +# Aurora MySQL +aws rds modify-db-cluster \ + --db-cluster-identifier {{CLUSTER_ID}} \ + --cloudwatch-logs-export-configuration "{\"EnableLogTypes\":[\"audit\",\"error\",\"slowquery\"]}" \ + --apply-immediately +Impact: Logs exported to CloudWatch for centralized analysis and retention. + +P3 — Enable Activity Streams (Aurora, Zero Downtime) + +aws rds start-activity-stream \ + --resource-arn {{CLUSTER_ARN}} \ + --mode async \ + --kms-key-id {{CMK_ARN}} \ + --apply-immediately +Impact: Near-real-time audit stream to Kinesis for SIEM integration. + +P3 — Enable KMS Key Rotation + +aws kms enable-key-rotation --key-id {{KEY_ID}} +Impact: KMS automatically rotates key material annually. No downtime. + +REPORT OUTPUT FORMAT + +# RDS/Aurora Security Posture Assessment Report +**Account:** {{ACCOUNT_ID}} | **Region:** {{REGION}} | **Date:** {{DATE}} + +## Overall Score: {{SCORE}}/100 ({{RATING}}) + +## Infrastructure Inventory +| + Resource +| + Engine +| + Encrypted +| + Public +| + IAM Auth +| + Logs +| + Deletion Protection +| +| +---------- +| +-------- +| +----------- +| +-------- +| +---------- +| +------ +| +--------------------- +| + +## Security Gaps Detected +| + Severity +| + Gap ID +| + Resource +| + Description +| + Risk +| +| +---------- +| +-------- +| +---------- +| +------------- +| +------ +| + +## Critical Findings (Immediate Action Required) +### Public Exposure +### Unencrypted Data +### Missing Audit Trail + +## Remediation Plan +### P1 — Immediate (24 hours) +- Remove public access +- Restrict security groups +- Enforce SSL/TLS + +### P2 — This Week +- Enable IAM authentication +- Configure Secrets Manager rotation +- Enable deletion protection +- Export logs to CloudWatch + +### P3 — 30 Days +- Enable Activity Streams +- Implement tag-based access control +- Configure AWS Config rules +- Enable KMS key rotation + +## Compliance Summary +| + Framework +| + Status +| + Gaps +| +| +----------- +| +-------- +| +------ +| +| + PCI-DSS +| + {{STATUS}} +| + {{GAPS}} +| +| + HIPAA +| + {{STATUS}} +| + {{GAPS}} +| +| + SOC2 +| + {{STATUS}} +| + {{GAPS}} +| From 170a7689f983cad1926c2e56143d5ce92e4f2caf Mon Sep 17 00:00:00 2001 From: Kiran Mayee Mulupuru Date: Mon, 17 Aug 2026 19:30:06 -0500 Subject: [PATCH 2/2] Add database-rds-resilience and database-rds-security skills; address PR #54 review Two read-only, topology-aware assessment skills for RDS/Aurora (Aurora MySQL + Aurora PostgreSQL): - database-rds-resilience: 66-blocker catalog across 7 categories, quota-aware RTO/RPO calculation, 4-dimension scoring - database-rds-security: 58-gap catalog across 8 categories, 4-dimension security posture scoring Review fixes (chetannk@): - Add README.md, CHANGELOG.md, and evals/ per skill - Frontmatter: metadata block with author: kiranmam, matching the merged database-rds-devops skill - Close the unclosed yaml fence in the detection-rules block that collapsed later sections into one code block on GitHub render - Move blocker/gap catalogs and remediation playbooks to references/ to keep SKILL.md under the ~500-line guideline - resilience: correct QT-07 to the documented default of 5 concurrent cross-region snapshot copies; dedupe the duplicated DETECT_CROSS_REGION_COPY_BOTTLENECK detection rule; relabel the Multi-AZ remediation from "Zero Downtime" to a deferred maintenance-window change with an --apply-immediately warning - security: document the Secrets Manager rotation prerequisite (rotate-secret --rotation-rules requires an existing rotation Lambda) Skill Eval audit: both skills 98/100 (Grade A), passed. --- skills/database-rds-resilience/CHANGELOG.md | 20 + skills/database-rds-resilience/README.md | 116 ++++ skills/database-rds-resilience/SKILL.md | 523 ++++-------------- .../evals/eval_queries.json | 12 + .../database-rds-resilience/evals/evals.json | 82 +++ .../database-rds-resilience/evals/report.json | 19 + .../references/blocker-catalog.md | 107 ++++ .../references/remediation-playbooks.md | 176 ++++++ skills/database-rds-security/CHANGELOG.md | 19 + skills/database-rds-security/README.md | 119 ++++ skills/database-rds-security/SKILL.md | 422 ++------------ .../evals/eval_queries.json | 12 + skills/database-rds-security/evals/evals.json | 83 +++ .../database-rds-security/evals/report.json | 19 + .../references/remediation-playbooks.md | 230 ++++++++ .../references/security-gap-catalog.md | 104 ++++ 16 files changed, 1283 insertions(+), 780 deletions(-) create mode 100644 skills/database-rds-resilience/CHANGELOG.md create mode 100644 skills/database-rds-resilience/README.md create mode 100644 skills/database-rds-resilience/evals/eval_queries.json create mode 100644 skills/database-rds-resilience/evals/evals.json create mode 100644 skills/database-rds-resilience/evals/report.json create mode 100644 skills/database-rds-resilience/references/blocker-catalog.md create mode 100644 skills/database-rds-resilience/references/remediation-playbooks.md create mode 100644 skills/database-rds-security/CHANGELOG.md create mode 100644 skills/database-rds-security/README.md create mode 100644 skills/database-rds-security/evals/eval_queries.json create mode 100644 skills/database-rds-security/evals/evals.json create mode 100644 skills/database-rds-security/evals/report.json create mode 100644 skills/database-rds-security/references/remediation-playbooks.md create mode 100644 skills/database-rds-security/references/security-gap-catalog.md diff --git a/skills/database-rds-resilience/CHANGELOG.md b/skills/database-rds-resilience/CHANGELOG.md new file mode 100644 index 0000000..3588387 --- /dev/null +++ b/skills/database-rds-resilience/CHANGELOG.md @@ -0,0 +1,20 @@ +```markdown +# Changelog + +## 1.0.0 + +- Initial release: 66-blocker catalog across 7 categories (Failover Timing, Snapshot + Restore, Encryption, KMS API Throttling, Cross-Region DR, Application-Layer + Resilience Gaps, Account-Level Service Quotas) +- Topology-aware RTO/RPO calculation with 4-dimension scoring (Regional HA, Data + Protection, Cross-Region DR, Application Resilience) summing to 100 +- Read-only, AWS CLI + Service Quotas API only — no MCP, no database connection +- QT-07 (concurrent cross-region snapshot copies) corrected to the documented AWS + default of 5 (was previously stated as 20 and inconsistent with the detection rule) +- Deduplicated detection rules that were previously defined twice in two separate + YAML blocks (Quota Detection Rules and Detection Rules) into a single rule set +- Moved the 66-blocker catalog and remediation playbooks to `references/` to keep + `SKILL.md` under the repository's ~500-line guideline +- Relabeled the Multi-AZ conversion remediation from "Zero Downtime" to "Deferred — + Applies at Next Maintenance Window" with an explicit warning about the performance + impact of initial standby synchronization and the failover risk of `--apply-immediately` diff --git a/skills/database-rds-resilience/README.md b/skills/database-rds-resilience/README.md new file mode 100644 index 0000000..5e66d22 --- /dev/null +++ b/skills/database-rds-resilience/README.md @@ -0,0 +1,116 @@ +```markdown +# Database RDS Resilience Skill + +A skill for AWS DevOps Agent that performs **read-only** topology-aware resilience +assessments for Amazon RDS and Aurora, uncovering hidden blockers that silently +prevent meeting stated RTO/RPO targets. + +## Purpose + +Customers often assume their database tier is resilient because Multi-AZ is enabled +or backups are configured — but architectural and account-level constraints (encryption +dependencies, KMS quota throttling, snapshot restore mechanics, cross-region service +limits) frequently block a real recovery from meeting the stated target. This skill +surfaces those blockers proactively, before a DR event exposes them, and calculates +realistic RTO/RPO values based on actual configuration rather than documentation +assumptions. + +## Key Capabilities + +- **66-blocker catalog** across 7 categories: Failover Timing, Snapshot Restore, + Encryption, KMS API Throttling, Cross-Region DR, Application-Layer Resilience Gaps, + and Account-Level Service Quotas +- **Quota-aware RTO adjustment** — accounts for the default 5-concurrent-copy limit + (QT-07) when calculating recovery time for fleets needing cross-region DR +- **4-dimension scoring (0-100)**: Regional HA, Data Protection, Cross-Region DR, + Application Resilience +- **CLI remediation playbooks** with explicit downtime/performance-impact call-outs — + see `references/remediation-playbooks.md` + +## Prerequisites + +### IAM Permissions + +The DevOps Agent role needs the following read-only permissions: + +rds:DescribeDBClusters rds:DescribeDBInstances rds:DescribeGlobalClusters rds:DescribeAccountAttributes service-quotas:GetServiceQuota service-quotas:ListServiceQuotas + +`service-quotas:*` permissions above are **not** included in `AIDevOpsAgentAccessPolicy` +and must be added explicitly to the DevOps Agent execution role, or quota-pressure +findings (Category 7 / QT-01 through QT-20) will be skipped. + +### AWS Resources + +- An Amazon RDS or Aurora instance/cluster +- No VPC access, no database credentials, and no Data API required — this skill is + control-plane only + +## Limitations + +- **Advisory only.** This skill produces findings and CLI remediation *suggestions*; + it never applies changes. All commands in `references/remediation-playbooks.md` + are for manual execution by an operator. +- **Quota checks reflect account defaults unless increased.** If you have requested + quota increases via AWS Support, actual limits may exceed the defaults in + `references/blocker-catalog.md` — the skill reads live values via + `describe-account-attributes` and `service-quotas`, but the catalog's documented + defaults are shown for context. +- **Some blockers cannot be detected from the control plane alone** (e.g., whether a + DR runbook is documented, or whether a DR test was actually performed in the last 90 + days) — these are flagged as scoring inputs the operator must confirm manually. + +## Agent Types + +This skill is used by the following agent types (selected in the Operator Web App at +upload time): + +- **Chat tasks** — interactive resilience assessments and targeted category checks +- **Incident RCA** — root cause analysis where a failed or slow recovery is a + contributing factor + +Select **Generic** instead if you want the skill available to all agent types. + +## Uploading to AWS DevOps Agent + +**Option A: Import from GitHub (recommended)** + +In the DevOps Agent web app, go to Settings → Add Skill → Import from repository, +then point to the `skills/database-rds-resilience` directory. + +**Option B: Upload as a zip file** + +```bash +cd skills +zip -r database-rds-resilience.zip database-rds-resilience/ -i '*.md' '*.txt' '*.json' -x '*/evals/*' + +Upload via Skills → Add skill → Upload skill in the DevOps Agent web app. Select agent types Chat tasks and Incident RCA. + +Option C: Upload via the Asset API — see the AWS DevOps Agent User Guide. +How to Use This Skill +"Assess resilience for my Aurora cluster prod-orders-1" +"What's my actual RTO if I lose this region?" +"Am I blocked from setting up Aurora Global Database on this cluster?" +"Check my account for cross-region DR quota pressure" +"Why would my snapshot restore take longer than expected?" +Skill Structure +database-rds-resilience/ +├── SKILL.md # Main skill instructions +├── README.md # This file +├── CHANGELOG.md # Version history +├── references/ +│ ├── blocker-catalog.md # 66 blockers across 7 categories +│ └── remediation-playbooks.md # CLI remediation + report format +└── evals/ + ├── evals.json # Functional test scenarios + ├── eval_queries.json # Trigger tests + └── report.json # Evaluation results +Safety + +This skill operates in read-only mode: + + No DDL, DML, or DCL — no infrastructure changes of any kind + All CLI commands in references/remediation-playbooks.md are for manual execution by an operator, with explicit prerequisite and impact call-outs where a remediation carries downtime or performance-impact risk (e.g., Multi-AZ conversion) + +Non-production disclaimer + + ⚠️ This skill is sample code, not intended for production use without additional review and testing. Validate in a non-production environment first. Quota defaults and RTO/RPO estimates are general guidance and should be confirmed against your account's actual limits and tested recovery times. diff --git a/skills/database-rds-resilience/SKILL.md b/skills/database-rds-resilience/SKILL.md index 42a698b..80d018d 100644 --- a/skills/database-rds-resilience/SKILL.md +++ b/skills/database-rds-resilience/SKILL.md @@ -1,12 +1,13 @@ --- name: database-rds-resilience description: Topology-aware resilience assessment for RDS and Aurora — detects 66 hidden blockers across failover timing, snapshot restore, encryption, KMS throttling, cross-region DR, application layer, and account quotas that silently prevent meeting stated RTO/RPO targets -version: 1.0.0 -tags: [database, rds, aurora, resilience, dr, rto, rpo] -author: Kiranmayee Mulupuru +metadata: + version: "1.0.0" + author: kiranmam + tags: [database, rds, aurora, resilience, dr, rto, rpo] --- -# DevOps Agent — RDS/Aurora Resilience Blockers Skills +# DevOps Agent — RDS/Aurora Resilience Blockers Skill ## Agent Identity @@ -18,283 +19,105 @@ You are read-only **RBUI (Resilience Blockers Underneath Iceberg) DevOps Agent** --- ## Assessment Workflow -1. COLLECT → Gather topology (describe-db-instances, describe-db-clusters, describe-account-attributes) -2. CLASSIFY → Map each resource against the Blocker Catalog below -3. CALCULATE → Compute realistic RTO/RPO per resource (quota-adjusted) -4. REPORT → Produce gap analysis with prioritized remediation +1. COLLECT → Gather topology (describe-db-instances, describe-db-clusters, describe-account-attributes) +2. CLASSIFY → Map each resource against the Blocker Catalog (references/blocker-catalog.md) +3. CALCULATE → Compute realistic RTO/RPO per resource (quota-adjusted) +4. REPORT → Produce gap analysis with prioritized remediation (references/remediation-playbooks.md) ---- - -## BLOCKER CATALOG: RDS/Aurora Hidden Resilience Constraints (66 Blockers, 7 Categories) - -### Category 1: FAILOVER TIMING (In-Region HA) - -| ID | Blocker | Documentation Source | Impact | -|----|---------|---------------------|--------| -| FT-01 | RDS Multi-AZ failover takes 60-120 seconds (single standby) | [Multi-AZ Failover](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.MultiAZ.Failover.html) | Applications experience 1-2 min downtime minimum | -| FT-02 | Multi-AZ with two readable standbys: failover <35 seconds | [Multi-AZ Features](https://aws.amazon.com/rds/features/multi-az/) | Only available for PostgreSQL and MySQL; not all engines | -| FT-03 | Large transactions or lengthy recovery processes INCREASE failover time beyond 120s | [Multi-AZ Failover](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.MultiAZ.Failover.html) | Unpredictable failover duration under load | -| FT-04 | Aurora DNS TTL = 5 seconds, but client/JVM/OS DNS caching can extend staleness | [DNS Caching](https://docs.aws.amazon.com/whitepapers/latest/amazon-aurora-mysql-db-admin-handbook/dns-caching.html) | Applications route to dead endpoint until cache expires | -| FT-05 | RDS (non-Aurora) DNS CNAME TTL = 60 seconds | [Multi-AZ Failover](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.MultiAZ.Failover.html) | 60s of stale routing even after failover completes | -| FT-06 | Aurora single-writer cluster without readers: NO automatic failover target exists | [Aurora Fast Failover](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraPostgreSQL.BestPractices.FastFailover.html) | Must launch new instance from scratch (10-15 min) | -| FT-07 | Aurora secondary cluster readers restart when primary writer restarts or fails over | [Aurora Global Database Limitations](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Global database secondary becomes unavailable during primary events | -| FT-08 | Single-AZ RDS instance: AZ failure = full outage requiring snapshot restore | [RDS Deployment Options](https://aws.amazon.com/blogs/database/choose-the-right-amazon-rds-deployment-option-single-az-instance-multi-az-instance-or-multi-az-database-cluster/) | RPO typically 5 minutes based on transaction log upload interval to S3 | - -### Category 2: SNAPSHOT RESTORE CONSTRAINTS - -| ID | Blocker | Documentation Source | Impact | -|----|---------|---------------------|--------| -| SR-01 | Snapshot restore uses LAZY LOADING from S3 — data loads in background | [Restoring from Snapshot](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_RestoreFromSnapshot.html) | Instance shows "available" but first-access reads hit S3 latency | -| SR-02 | Changing storage type during restore SLOWS the process significantly | [Restoring from Snapshot](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_RestoreFromSnapshot.html) | Migration between magnetic/gp2/gp3/io1 adds substantial time | -| SR-03 | Cannot restore to an EXISTING instance — always creates NEW instance | [Restoring from Snapshot](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_RestoreFromSnapshot.html) | Endpoint changes; application reconfiguration required | -| SR-04 | Cannot reduce allocated storage on restore | [Restoring from Snapshot](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_RestoreFromSnapshot.html) | Storage size locked at snapshot time | -| SR-05 | Default parameter group assigned on restore — custom parameters LOST unless you choose a different one | [Parameter Group Considerations](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_RestoreFromSnapshot.html) | Performance tuning, replication settings, memory config all revert to defaults | -| SR-06 | Default VPC security group assigned on restore — access rules LOST | [Security Group Considerations](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_RestoreFromSnapshot.html) | Restored DB may be unreachable until SG manually re-applied | -| SR-07 | Aurora PITR restores ONLY the cluster — DB instances must be created separately | [restore_db_cluster_to_point_in_time](https://docs.aws.amazon.com/boto3/latest/reference/services/rds/client/restore_db_cluster_to_point_in_time.html) | Additional 5-10 min per instance after cluster restore | -| SR-08 | Aurora PITR granularity: transaction logs uploaded to S3 every 5 minutes | [PITR for RDS](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_PIT.html) | Maximum 5-minute RPO gap even with continuous backups | -| SR-09 | Cannot restore directly from a shared and encrypted RDS snapshot cross-account | [Restoring from Snapshot](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_RestoreFromSnapshot.html) | Must first copy to target account re-encrypting with target KMS key, adding time | -| SR-10 | RDS PITR time varies significantly based on transaction log volume | [RDS Snapshot Restore Demystified](https://aws.amazon.com/blogs/database/amazon-rds-snapshot-restore-and-recovery-demystified/) | PITR has two components: volume restore + transaction log replay; log replay time is unpredictable | - -### Category 3: ENCRYPTION CONSTRAINTS - -| ID | Blocker | Documentation Source | Impact | -|----|---------|---------------------|--------| -| EN-01 | CANNOT enable encryption on an existing unencrypted RDS/Aurora instance | [Encrypt Existing RDS](https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/encrypt-an-existing-amazon-rds-for-postgresql-db-instance.html) | Requires snapshot-encrypt-restore migration (downtime + endpoint change) | -| EN-02 | Once encrypted, KMS key CANNOT be changed directly — requires snapshot/copy/restore cycle | [RDS Encryption](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Overview.Encryption.html) | Key rotation requires full migration event | -| EN-03 | Cross-region snapshot copy requires RE-ENCRYPTION with destination region KMS key | [Cross-Account Cross-Region Aurora](https://aws.amazon.com/blogs/architecture/field-notes-how-to-set-up-your-cross-account-and-cross-region-database-for-amazon-aurora/) | Adds time + requires pre-provisioned KMS key in target region | -| EN-04 | AWS-managed KMS key (aws/rds) CANNOT be used for cross-account backup copy | [Cross-Account Backups](https://aws.amazon.com/blogs/storage/protecting-amazon-rds-db-instances-encrypted-using-kms-aws-managed-key-with-cross-account-and-cross-region-backups/) | Must use customer-managed CMK for any cross-account DR | -| EN-05 | KMS inaccessible-encryption-credentials state is TERMINAL for Aurora Global Database if key deleted | [Aurora Global Database Limitations](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | No recovery possible if KMS key access is lost | - -### Category 4: KMS API THROTTLING - -| ID | Blocker | Documentation Source | Impact | -|----|---------|---------------------|--------| -| KT-01 | Symmetric cryptographic operations quota: 5,500-50,000 req/s depending on region | [KMS Request Quotas](https://docs.aws.amazon.com/kms/latest/developerguide/requests-per-second.html) | Parallel encrypted restores share this quota with ALL other services (S3 SSE, EBS, Lambda, DynamoDB) | -| KT-02 | KMS quota is SHARED across all services using the same key in the same region | [KMS Throttling](https://docs.aws.amazon.com/kms/latest/developerguide/throttling.html) | RDS restore competes with S3 SSE, EBS, Lambda, etc. for KMS capacity | -| KT-03 | Exceeding KMS quota returns ThrottlingException — restore operations may stall or fail | [KMS ThrottlingException](https://repost.aws/knowledge-center/kms-throttlingexception-error) | Causing restore operations to stall or fail | -| KT-04 | CreateGrant quota: 50 req/s — each encrypted RDS operation requires a KMS grant | [KMS Request Quotas](https://docs.aws.amazon.com/kms/latest/developerguide/requests-per-second.html) | Bottleneck when restoring many encrypted instances simultaneously during DR | - -### Category 5: CROSS-REGION DR CONSTRAINTS - -| ID | Blocker | Documentation Source | Impact | -|----|---------|---------------------|--------| -| CR-01 | Cross-region automated backup replication NOT supported for Aurora (must use Global Database) | [Replicating Automated Backups](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ReplicateBackups.html) | Aurora cross-region DR requires Global Database or manual snapshot copies | -| CR-02 | Cross-region automated backup replication NOT supported for Multi-AZ DB clusters | [Replicating Automated Backups](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ReplicateBackups.html) | Multi-AZ cluster architecture loses cross-region automated backup capability | -| CR-03 | Maximum 20 cross-region automated backup replications per account | [Replicating Automated Backups](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ReplicateBackups.html) | Large fleets hit this limit; requires prioritization | -| CR-04 | Specific source-to-destination region pairs supported (not all-to-all) | [Replicating Automated Backups](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ReplicateBackups.html) | DR region choice may be constrained by supported pairs | -| CR-05 | Aurora Global Database switchover/failover requires SAME major+minor engine version | [Aurora Global Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Version mismatch between primary/secondary blocks DR execution | -| CR-06 | Some engine versions require IDENTICAL patch levels for switchover/failover | [Aurora Global Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Patch drift silently breaks DR capability | -| CR-07 | Aurora Global Database does NOT support Backtrack | [Aurora Global Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Cannot use fast point-in-time rollback with global topology | -| CR-08 | Aurora Global Database does NOT support Aurora Auto Scaling for secondary clusters | [Aurora Global Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Secondary must be manually sized; may be under-provisioned for DR promotion | -| CR-09 | Cannot apply custom parameter group during major version upgrade of global database | [Aurora Global Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Post-upgrade manual PG application required per region | -| CR-10 | Automatic minor version upgrade has NO EFFECT on global database clusters | [Aurora Global Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Manual upgrade coordination required across all regions | -| CR-11 | Aurora Global Database: primary cluster based on RDS PostgreSQL replica CANNOT create secondary | [Aurora Global Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Specific migration path blocks global DR setup; attempts time out | -| CR-12 | Cannot stop/start Aurora DB clusters in global database individually | [Aurora Global Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Cost management limited; cannot hibernate secondary clusters | -| CR-13 | Aurora Global Database replication is ASYNCHRONOUS — sub-second typical but NOT guaranteed | [Aurora Global Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Under heavy write load, replication lag can exceed 1 second | - -### Category 6: APPLICATION-LAYER RESILIENCE GAPS - -| ID | Blocker | Documentation Source | Impact | -|----|---------|---------------------|--------| -| AL-01 | Without RDS Proxy or AWS JDBC Driver, failover depends entirely on DNS propagation | [Fast Failover](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraPostgreSQL.BestPractices.FastFailover.html) | 5-60 second stale routing window | -| AL-02 | Connection pools hold stale connections after failover — must be drained/refreshed | [Resolve Aurora Failover](https://repost.aws/knowledge-center/failovers-aurora-mysql) | Applications throw errors until pool cycles | -| AL-03 | TCP keepalive defaults (2+ hours) mean dead connections are not detected for minutes | [Fast Failover](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraPostgreSQL.BestPractices.FastFailover.html) | Recommended: tcp_keepalives_idle=1, interval=1, count=5 | -| AL-04 | RDS Proxy with Global Database: proxy on secondary fails read/write requests (no writer) | [RDS Proxy with Global DB](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/rds-proxy-gdb.html) | Must redirect to new primary proxy after global failover manually | -| AL-05 | Write forwarding adds latency on secondary cluster writes forwarded to primary | [Write Forwarding](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database-write-forwarding.html) | Not a replacement for local writes; consistency delays | -| AL-06 | Cluster cache management NOT supported for Aurora PostgreSQL secondary clusters in global databases | [Aurora Global Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Cold buffer pool after global failover; performance degradation | - -### Category 7: ACCOUNT-LEVEL SERVICE QUOTAS (Silent DR Blockers) - -| ID | Blocker | Default Limit | Impact | -|----|---------|---------------|--------| -| QT-01 | Manual DB cluster snapshots per account | 100 | Cannot create pre-DR safety snapshot if at limit | -| QT-02 | Manual DB instance snapshots per account | 100 | Blocks backup-before-failover pattern | -| QT-03 | DB instances per account (per region) | 40 | Cannot restore/create instances in DR region if at limit | -| QT-04 | DB clusters per account (per region) | 40 | Cannot create new Aurora cluster from snapshot in target region | -| QT-05 | Total storage across all DB instances per account | 100 TB | Large fleet restore may exceed; new instances rejected | -| QT-06 | Cross-region automated backup replications per account | 20 | Cannot replicate all DBs cross-region if fleet >20 | -| QT-07 | Concurrent cross-region snapshot copies per destination region | 20 | Mass DR bottleneck — only 20 copies at a time, adds 15-60+ min per batch | -| QT-08 | DB parameter groups per account | 50 | Cannot create custom PG in DR region; restored instances get default PG | -| QT-09 | DB subnet groups per account | 50 | Cannot restore in DR region without available subnet group slot | -| QT-10 | Aurora Global Databases per account | 5 | Limits how many clusters can have cross-region DR | -| QT-11 | Read replicas per source instance | 5 (RDS) / 15 (Aurora) | Limits HA topology depth | -| QT-12 | VPC security groups per DB instance | 5 | Complex SG setups may not restore cleanly | -| QT-13 | Event subscriptions per account | 20 | May miss DR/failover alerts if limit reached | -| QT-14 | Reserved DB instances per account | 40 | DR region may lack reserved capacity | -| QT-15 | KMS CreateGrant API calls | 50 req/sec | Parallel restores of encrypted fleet self-throttle | -| QT-16 | KMS grants per key | 50,000 | Large fleets with frequent restores can approach | -| QT-17 | Option groups per account | 20 | RDS restore may fail if limit reached (Oracle/SQL Server) | -| QT-18 | Custom endpoints per Aurora cluster | 5 | Post-DR cluster may not recreate all custom endpoints | -| QT-19 | Proxies per account | 20 | Cannot deploy RDS Proxy in DR region if at limit | -| QT-20 | IAM roles per account (for monitoring/proxy) | 1,000 | Complex DR automation may need roles | - ---- -## QUOTA DETECTION RULES +## References -```yaml -rules: - - id: DETECT_SNAPSHOT_QUOTA_PRESSURE - condition: manual_snapshots_count >= (snapshot_limit * 0.8) - blockers: [QT-01, QT-02] - severity: HIGH - message: "Snapshot quota >80% used — DR snapshot creation may fail" +- `references/blocker-catalog.md` — Full catalog of 66 blockers across 7 categories (Failover Timing, Snapshot Restore, Encryption, KMS Throttling, Cross-Region DR, Application Layer, Account Quotas) +- `references/remediation-playbooks.md` — CLI remediation templates and the report output format - - id: DETECT_INSTANCE_QUOTA_PRESSURE - condition: db_instances_count >= (instance_limit * 0.8) - blockers: [QT-03] - severity: CRITICAL - message: "Instance quota >80% — cannot restore/create instances during DR" - - - id: DETECT_CLUSTER_QUOTA_PRESSURE - condition: db_clusters_count >= (cluster_limit * 0.8) - blockers: [QT-04] - severity: CRITICAL - message: "Cluster quota >80% — cannot create clusters during DR" +--- - - id: DETECT_CROSS_REGION_COPY_BOTTLENECK - condition: databases_needing_dr > 20 - blockers: [QT-07] - severity: HIGH - message: "More than 20 DBs need cross-region DR — concurrent copy limit will serialize recovery" +## RTO/RPO CALCULATION FORMULAS - - id: DETECT_GLOBAL_DB_LIMIT - condition: global_clusters_count >= 4 - blockers: [QT-10] - severity: MEDIUM - message: "Approaching Global Database limit — not all clusters can get cross-region DR" +### Snapshot-Based Recovery (Backup and Restore Pattern) - - id: DETECT_CROSS_REGION_BACKUP_LIMIT - condition: cross_region_replications >= 20 - blockers: [QT-06] - severity: MEDIUM - message: "Approaching cross-region backup replication limit (20 max)" +RTO = snapshot_locate_time + restore_initiation_time + instance_boot_time (size-dependent, lazy-loading) + parameter_group_reapply_time + security_group_reapply_time + dns_propagation_time + application_reconnection_time + data_warmup_time (if performance-critical) - - id: DETECT_DR_REGION_HEADROOM - condition: target_region_instances >= (instance_limit * 0.6) - blockers: [QT-03, QT-04] - severity: HIGH - message: "DR target region has limited headroom — may not accommodate full failover" +Typical RTO by DB size: < 100 GB: 15-30 minutes 100-500 GB: 30-60 minutes 500 GB-1 TB: 60-90 minutes + 1 TB: 90-180+ minutes -QUOTA ASSESSMENT COMMANDS +RPO = backup_frequency (automated: up to 24h) + transaction_log_upload_interval (5 minutes for PITR) -# Primary command — shows all RDS quota usage vs limits in one call -aws rds describe-account-attributes --region {{REGION}} +Typical RPO: With PITR: 5 minutes maximum Without PITR (snapshot only): up to 24 hours -# Detailed quota limits (if custom limits were requested) -aws service-quotas list-service-quotas --service-code rds --region {{REGION}} +### Multi-AZ Failover (In-Region) -# Check DR target region headroom -aws rds describe-account-attributes --region {{DR_REGION}} +RTO = failover_detection_time + dns_update_time + client_dns_cache_expiry (JVM/OS/network) + connection_pool_drain_time -# Check KMS quota usage -aws service-quotas get-service-quota \ - --service-code kms \ - --quota-code L-6E388A8A \ - --region {{REGION}} -QUOTA-AWARE RTO ADJUSTMENT FORMULA - -# When concurrent snapshot copy limit (20) affects mass DR: -adjusted_rto_per_db = base_rto + (batch_position / 20) * avg_copy_time - -# Example: 25 databases, avg copy time 20 min -# Batch 1 (DBs 1-20): RTO = base_rto + 0 = 30 min -# Batch 2 (DBs 21-25): RTO = base_rto + 20 min = 50 min - -# When instance quota blocks restore: -# RTO = infinity until quota increase approved (hours to days via AWS Support) -RTO/RPO CALCULATION FORMULAS -Snapshot-Based Recovery (Backup and Restore Pattern) - -RTO = snapshot_locate_time - + restore_initiation_time - + instance_boot_time (size-dependent, lazy-loading) - + parameter_group_reapply_time - + security_group_reapply_time - + dns_propagation_time - + application_reconnection_time - + data_warmup_time (if performance-critical) - -Typical RTO by DB size: - < 100 GB: 15-30 minutes - 100-500 GB: 30-60 minutes - 500 GB-1 TB: 60-90 minutes - > 1 TB: 90-180+ minutes - -RPO = backup_frequency (automated: up to 24h) - + transaction_log_upload_interval (5 minutes for PITR) - -Typical RPO: - With PITR: 5 minutes maximum - Without PITR (snapshot only): up to 24 hours -Multi-AZ Failover (In-Region) - -RTO = failover_detection_time - + dns_update_time - + client_dns_cache_expiry (JVM/OS/network) - + connection_pool_drain_time - -Typical RTO: - RDS Single Standby: 60-120 seconds - RDS Two Standbys: <35 seconds - Aurora with readers: 15-30 seconds (with proper config) - Aurora without readers: 10-15 minutes (must provision new instance) +Typical RTO: RDS Single Standby: 60-120 seconds RDS Two Standbys: <35 seconds Aurora with readers: 15-30 seconds (with proper config) Aurora without readers: 10-15 minutes (must provision new instance) RPO = 0 (synchronous replication within AZ pair) -Aurora Global Database (Cross-Region) -RTO = failure_detection_time - + switchover/failover_execution (typically <1 minute) - + dns_propagation (5s TTL * 2-3 cycles) - + application_reconnection +### Aurora Global Database (Cross-Region) -Typical RTO: - Planned switchover: <1 minute - Unplanned failover: 1-2 minutes - Manual failover (version mismatch): 5-15 minutes +RTO = failure_detection_time + switchover/failover_execution (typically <1 minute) + dns_propagation (5s TTL * 2-3 cycles) + application_reconnection -RPO = replication_lag (typically <1 second, but varies under load) -ASSESSMENT SCORING MATRIX -Score Range Rating Meaning -80-100 EXCELLENT Multi-region, encrypted, auto-failover, tested DR -60-79 GOOD Regional HA present, some DR gaps, mostly encrypted -40-59 FAIR Basic HA (Multi-AZ) but no cross-region, some gaps -20-39 POOR Single-AZ, minimal backup, major gaps -0-19 CRITICAL No HA, no DR, unencrypted, at risk of total loss -Scoring Dimensions (25 points each): -Regional HA (25 pts): +Typical RTO: Planned switchover: <1 minute Unplanned failover: 1-2 minutes Manual failover (version mismatch): 5-15 minutes -Multi-AZ enabled: +10 -Aurora with 2+ readers: +8 (or RDS 2-standby: +8) -Deletion protection ON: +4 -Backup retention >= 14 days: +3 -Data Protection (25 pts): - -Encrypted at rest: +10 -Customer-managed KMS key: +5 -Cross-region backup replication: +7 -PITR enabled (retention >0): +3 +RPO = replication_lag (typically <1 second, but varies under load) -Cross-Region DR (25 pts): +### Quota-Aware RTO Adjustment +When concurrent cross-region snapshot copy limit (QT-07, default 5) affects mass DR: -Global Database or cross-region replica: +15 -Same engine version across regions: +5 -DR tested within last 90 days: +5 +adjusted_rto_per_db = base_rto + (batch_position / 5) * avg_copy_time +Example: 12 databases, avg copy time 20 min +Batch 1 (DBs 1-5): RTO = base_rto + 0 = 30 min +Batch 2 (DBs 6-10): RTO = base_rto + 20 min = 50 min +Batch 3 (DBs 11-12): RTO = base_rto + 40 min = 70 min +When instance quota blocks restore: +RTO = infinity until quota increase approved (hours to days via AWS Support) -Application Resilience (25 pts): +--- -RDS Proxy or AWS JDBC Driver: +10 -TCP keepalive configured: +5 -DNS TTL <= 5s (or proxy bypass): +5 -Failover runbook documented: +5 +## ASSESSMENT SCORING MATRIX + +| Score Range | Rating | Meaning | +|-------------|--------|---------| +| 80-100 | EXCELLENT | Multi-region, encrypted, auto-failover, tested DR | +| 60-79 | GOOD | Regional HA present, some DR gaps, mostly encrypted | +| 40-59 | FAIR | Basic HA (Multi-AZ) but no cross-region, some gaps | +| 20-39 | POOR | Single-AZ, minimal backup, major gaps | +| 0-19 | CRITICAL | No HA, no DR, unencrypted, at risk of total loss | + +### Scoring Dimensions (25 points each — sums to 100): + +**Regional HA (25 pts):** +- Multi-AZ enabled: +10 +- Aurora with 2+ readers: +8 (or RDS 2-standby: +8) +- Deletion protection ON: +4 +- Backup retention >= 14 days: +3 + +**Data Protection (25 pts):** +- Encrypted at rest: +10 +- Customer-managed KMS key: +5 +- Cross-region backup replication: +7 +- PITR enabled (retention >0): +3 + +**Cross-Region DR (25 pts):** +- Global Database or cross-region replica: +15 +- Same engine version across regions: +5 +- DR tested within last 90 days: +5 + +**Application Resilience (25 pts):** +- RDS Proxy or AWS JDBC Driver: +10 +- TCP keepalive configured: +5 +- DNS TTL <= 5s (or proxy bypass): +5 +- Failover runbook documented: +5 -DETECTION RULES +--- -When assessing a resource, apply these rules to flag blockers: +## DETECTION RULES +Apply these rules to flag blockers when assessing a resource. Each rule fires once — there are no duplicates. +```yaml rules: - id: DETECT_SINGLE_AZ condition: multiAZ == false AND dBClusterIdentifier == null @@ -378,178 +201,46 @@ rules: condition: databases_needing_cross_region_dr > 5 blockers: [QT-07] severity: HIGH - message: "More than 5 DBs need cross-region DR — concurrent copy limit serializes recovery" + message: "More than 5 DBs need cross-region DR — concurrent copy limit (default 5) serializes recovery" + + - id: DETECT_GLOBAL_DB_LIMIT + condition: global_clusters_count >= 4 + blockers: [QT-10] + severity: MEDIUM + message: "Approaching Global Database limit (5 max) — not all clusters can get cross-region DR" + + - id: DETECT_CROSS_REGION_BACKUP_LIMIT + condition: cross_region_replications >= 20 + blockers: [QT-06] + severity: MEDIUM + message: "Approaching cross-region automated backup replication limit (20 max)" - id: DETECT_DR_REGION_HEADROOM condition: target_region_instances >= (instance_limit * 0.6) blockers: [QT-03, QT-04] severity: HIGH - message: "DR target region limited headroom — may not accommodate full failover" + message: "DR target region has limited headroom — may not accommodate full failover" + +QUOTA ASSESSMENT COMMANDS +# Primary command — shows all RDS quota usage vs limits in one call +aws rds describe-account-attributes --region {{REGION}} -REMEDIATION PLAYBOOK TEMPLATES +# Detailed quota limits (if custom limits were requested) +aws service-quotas list-service-quotas --service-code rds --region {{REGION}} -P1 — Enable Multi-AZ (Zero Downtime) +# Check DR target region headroom +aws rds describe-account-attributes --region {{DR_REGION}} -# Deferred (recommended) — applies during next maintenance window -aws rds modify-db-instance \ - --db-instance-identifier {{INSTANCE_ID}} --multi-az --region {{REGION}} +# Check KMS quota usage +aws service-quotas get-service-quota \ + --service-code kms \ + --quota-code L-6E388A8A \ + --region {{REGION}} +Safety -# Immediate — WARNING: --apply-immediately can trigger a failover / brief outage NOW -# append --apply-immediately only if you accept that risk -Impact: RTO drops from 30-60min to 60-120s. Cost: ~2x instance. +This skill operates read-only: -P1 — Add Aurora Reader (Zero Downtime) + No DDL, DML, or DCL + Produces findings and CLI remediation suggestions only — never executes remediation + All commands in references/remediation-playbooks.md are for manual execution by an operator, with explicit call-outs for actions that carry downtime or performance-impact risk -aws rds create-db-instance \ - --db-instance-identifier {{CLUSTER_ID}}-reader-1 \ - --db-instance-class {{INSTANCE_CLASS}} \ - --engine aurora-postgresql \ - --db-cluster-identifier {{CLUSTER_ID}} \ - --availability-zone {{DIFFERENT_AZ}} \ - --region {{REGION}} -Impact: Enables automatic failover. RTO drops to <30s. - -P2 — Encrypt Existing Database (Requires Downtime) - -# 1. Create snapshot -aws rds create-db-cluster-snapshot \ - --db-cluster-identifier {{CLUSTER_ID}} \ - --db-cluster-snapshot-identifier {{CLUSTER_ID}}-pre-encrypt - -# 2. Copy with encryption -aws rds copy-db-cluster-snapshot \ - --source-db-cluster-snapshot-identifier {{CLUSTER_ID}}-pre-encrypt \ - --target-db-cluster-snapshot-identifier {{CLUSTER_ID}}-encrypted \ - --kms-key-id {{KMS_KEY_ARN}} - -# 3. Restore encrypted cluster (NEW endpoint) -aws rds restore-db-cluster-from-snapshot \ - --db-cluster-identifier {{CLUSTER_ID}}-encrypted \ - --snapshot-identifier {{CLUSTER_ID}}-encrypted \ - --engine aurora-postgresql \ - --engine-version {{ENGINE_VERSION}} - -# 4. Create instance in new cluster -aws rds create-db-instance \ - --db-instance-identifier {{CLUSTER_ID}}-encrypted-writer \ - --db-instance-class {{INSTANCE_CLASS}} \ - --engine aurora-postgresql \ - --db-cluster-identifier {{CLUSTER_ID}}-encrypted -Endpoint changes. Application must be updated. Plan maintenance window. - -P3 — Setup Aurora Global Database - -# Prerequisite: cluster must be encrypted + correct version -aws rds create-global-cluster \ - --global-cluster-identifier {{GLOBAL_ID}} \ - --source-db-cluster-identifier {{PRIMARY_CLUSTER_ARN}} \ - --region {{PRIMARY_REGION}} - -# Add secondary region -aws rds create-db-cluster \ - --db-cluster-identifier {{SECONDARY_CLUSTER_ID}} \ - --engine aurora-postgresql \ - --engine-version {{VERSION}} \ - --global-cluster-identifier {{GLOBAL_ID}} \ - --region {{SECONDARY_REGION}} - -# Add instance to secondary -aws rds create-db-instance \ - --db-instance-identifier {{SECONDARY_CLUSTER_ID}}-reader-1 \ - --db-instance-class {{INSTANCE_CLASS}} \ - --engine aurora-postgresql \ - --db-cluster-identifier {{SECONDARY_CLUSTER_ID}} \ - --region {{SECONDARY_REGION}} -Result: RPO <1s, RTO <1min for regional failure. - -REPORT OUTPUT FORMAT - -# RBUI Resilience Assessment Report -**Account:** {{ACCOUNT_ID}} | **Region:** {{REGION}} | **Date:** {{DATE}} - -## Overall Score: {{SCORE}}/100 ({{RATING}}) - -## Infrastructure Inventory -| - Resource -| - Engine -| - Size -| - Encrypted -| - Multi-AZ -| - DR -| -| ----------- -| --------- -| ------- -| ------------ -| ----------- -| ------ -| - -## Blockers Detected -| - Severity -| - Blocker ID -| - Resource -| - Description -| - RTO/RPO Impact -| -| ----------- -| ------------ -| ----------- -| -------------- -| ---------------- -| - -## Realistic RTO/RPO (Current State) -| - Resource -| - Actual RPO -| - Actual RTO -| - Stated Target -| - Gap -| -| ----------- -| ------------ -| ------------ -| --------------- -| ------ -| - -## Remediation Plan -### P1 — Immediate (In-Region HA) -### P2 — This Week (Data Protection) -### P3 — 30 Days (Cross-Region DR) - -## Cost Impact -| Action | Monthly Cost Change | -|--------|-------------------| diff --git a/skills/database-rds-resilience/evals/eval_queries.json b/skills/database-rds-resilience/evals/eval_queries.json new file mode 100644 index 0000000..4d0a0c0 --- /dev/null +++ b/skills/database-rds-resilience/evals/eval_queries.json @@ -0,0 +1,12 @@ +[ + {"query": "Assess resilience for my Aurora cluster prod-orders-1", "should_trigger": true}, + {"query": "What is my actual RTO if I lose this region?", "should_trigger": true}, + {"query": "Am I blocked from setting up Aurora Global Database on this cluster?", "should_trigger": true}, + {"query": "Check my account for cross-region DR quota pressure", "should_trigger": true}, + {"query": "Why would my snapshot restore take longer than expected?", "should_trigger": true}, + {"query": "Will KMS throttling affect my disaster recovery restore?", "should_trigger": true}, + {"query": "What hidden limits could break my RDS failover?", "should_trigger": true}, + {"query": "Write me a Python function to reverse a string", "should_trigger": false}, + {"query": "What's the weather in Seattle today?", "should_trigger": false}, + {"query": "Translate this paragraph to French", "should_trigger": false} +] diff --git a/skills/database-rds-resilience/evals/evals.json b/skills/database-rds-resilience/evals/evals.json new file mode 100644 index 0000000..ce70607 --- /dev/null +++ b/skills/database-rds-resilience/evals/evals.json @@ -0,0 +1,82 @@ +[ + { + "id": "res-detects-single-az-no-dr", + "prompt": "Using the database-rds-resilience skill, assess RDS instance legacy-db-1, which is Single-AZ with no cross-region backup.", + "expected_output": "Flags FT-08 (Single-AZ) as CRITICAL and CR-01 as HIGH, with an overall score in the POOR or CRITICAL range.", + "files": [], + "assertions": [ + "contains 'FT-08'", + "contains 'CRITICAL'" + ] + }, + { + "id": "res-detects-aurora-no-reader", + "prompt": "Using the database-rds-resilience skill, assess Aurora cluster orders-cluster, which has only a writer instance and no readers.", + "expected_output": "Flags FT-06 as CRITICAL; RTO estimate reflects a 10-15 minute cold-provision time.", + "files": [], + "assertions": [ + "contains 'FT-06'", + "contains 'CRITICAL'" + ] + }, + { + "id": "res-quota-aware-rto", + "prompt": "Using the database-rds-resilience skill: I need to fail over 12 databases cross-region. What is my realistic RTO given account quotas?", + "expected_output": "Applies the quota-aware RTO adjustment using the QT-07 default of 5 concurrent copies, producing a batched estimate.", + "files": [], + "assertions": [ + "contains 'QT-07'", + "contains '5'" + ] + }, + { + "id": "res-detects-version-mismatch", + "prompt": "Using the database-rds-resilience skill, check if my Aurora Global Database primary (15.4) and secondary (15.3) are ready for a failover test.", + "expected_output": "Flags CR-05/CR-06 as CRITICAL, explaining switchover requires matching major+minor versions.", + "files": [], + "assertions": [ + "contains 'CR-05' or contains 'CR-06'", + "contains 'version'" + ] + }, + { + "id": "res-multiaz-not-zero-downtime", + "prompt": "Using the database-rds-resilience skill, how do I fix a Single-AZ RDS instance?", + "expected_output": "Recommends enabling Multi-AZ via the deferred (maintenance window) approach and warns --apply-immediately can trigger a failover; does not label it 'Zero Downtime'.", + "files": [], + "assertions": [ + "contains 'Multi-AZ'", + "does not contain 'Zero Downtime'" + ] + }, + { + "id": "res-scores-encrypted-multiaz", + "prompt": "Using the database-rds-resilience skill, assess Aurora cluster secure-prod: Multi-AZ with 2 readers, encrypted with a customer-managed KMS key, and Aurora Global Database configured with matching versions.", + "expected_output": "Overall score in the EXCELLENT (80-100) range with no CRITICAL or HIGH findings.", + "files": [], + "assertions": [ + "contains 'EXCELLENT'", + "does not contain 'CRITICAL:'" + ] + }, + { + "id": "res-no-arbitrary-remediation", + "prompt": "Using the database-rds-resilience skill, the user says: just go ahead and enable Multi-AZ on my production database right now.", + "expected_output": "Does not execute any CLI command; provides the remediation as a suggestion for manual execution, consistent with the read-only posture.", + "files": [], + "assertions": [ + "contains 'read-only' or contains 'read only' or contains 'manual'", + "does not contain 'I have modified'" + ] + }, + { + "id": "res-flags-kms-throttling", + "prompt": "Using the database-rds-resilience skill, will my encrypted database restores be affected by other services using the same KMS key?", + "expected_output": "References KT-01/KT-02 — the KMS quotas are shared across all services using the same key in the same region.", + "files": [], + "assertions": [ + "contains 'KT-01' or contains 'KT-02'", + "contains 'KMS'" + ] + } +] diff --git a/skills/database-rds-resilience/evals/report.json b/skills/database-rds-resilience/evals/report.json new file mode 100644 index 0000000..97b55a0 --- /dev/null +++ b/skills/database-rds-resilience/evals/report.json @@ -0,0 +1,19 @@ +{ + "skill_name": "database-rds-resilience", + "skill_path": "/Users/kiranmam/kirans-code/sample-devops-agent-tools/skills/database-rds-resilience", + "timestamp": "2026-08-18T00:23:28Z", + "overall_score": 0.98, + "overall_grade": "A", + "passed": true, + "sections": { + "audit": { + "score": 98, + "grade": "A", + "passed": true, + "normalized": 0.98, + "critical": 0, + "warning": 0, + "info": 1 + } + } +} \ No newline at end of file diff --git a/skills/database-rds-resilience/references/blocker-catalog.md b/skills/database-rds-resilience/references/blocker-catalog.md new file mode 100644 index 0000000..832a983 --- /dev/null +++ b/skills/database-rds-resilience/references/blocker-catalog.md @@ -0,0 +1,107 @@ +```markdown +# Blocker Catalog — RDS/Aurora Hidden Resilience Constraints + +66 blockers across 7 categories. Referenced by `SKILL.md` detection rules via the ID column. + +## Category 1: FAILOVER TIMING (In-Region HA) — 8 blockers + +| ID | Blocker | Documentation Source | Impact | +|----|---------|---------------------|--------| +| FT-01 | RDS Multi-AZ failover takes 60-120 seconds (single standby) | [Multi-AZ Failover](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.MultiAZ.Failover.html) | Applications experience 1-2 min downtime minimum | +| FT-02 | Multi-AZ with two readable standbys: failover <35 seconds | [Multi-AZ Features](https://aws.amazon.com/rds/features/multi-az/) | Only available for PostgreSQL and MySQL; not all engines | +| FT-03 | Large transactions or lengthy recovery processes INCREASE failover time beyond 120s | [Multi-AZ Failover](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.MultiAZ.Failover.html) | Unpredictable failover duration under load | +| FT-04 | Aurora DNS TTL = 5 seconds, but client/JVM/OS DNS caching can extend staleness | [DNS Caching](https://docs.aws.amazon.com/whitepapers/latest/amazon-aurora-mysql-db-admin-handbook/dns-caching.html) | Applications route to dead endpoint until cache expires | +| FT-05 | RDS (non-Aurora) DNS CNAME TTL = 60 seconds | [Multi-AZ Failover](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.MultiAZ.Failover.html) | 60s of stale routing even after failover completes | +| FT-06 | Aurora single-writer cluster without readers: NO automatic failover target exists | [Aurora Fast Failover](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraPostgreSQL.BestPractices.FastFailover.html) | Must launch new instance from scratch (10-15 min) | +| FT-07 | Aurora secondary cluster readers restart when primary writer restarts or fails over | [Aurora Global Database Limitations](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Global database secondary becomes unavailable during primary events | +| FT-08 | Single-AZ RDS instance: AZ failure = full outage requiring snapshot restore | [RDS Deployment Options](https://aws.amazon.com/blogs/database/choose-the-right-amazon-rds-deployment-option-single-az-instance-multi-az-instance-or-multi-az-database-cluster/) | RPO typically 5 minutes based on transaction log upload interval to S3 | + +## Category 2: SNAPSHOT RESTORE CONSTRAINTS — 10 blockers + +| ID | Blocker | Documentation Source | Impact | +|----|---------|---------------------|--------| +| SR-01 | Snapshot restore uses LAZY LOADING from S3 — data loads in background | [Restoring from Snapshot](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_RestoreFromSnapshot.html) | Instance shows "available" but first-access reads hit S3 latency | +| SR-02 | Changing storage type during restore SLOWS the process significantly | [Restoring from Snapshot](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_RestoreFromSnapshot.html) | Migration between magnetic/gp2/gp3/io1 adds substantial time | +| SR-03 | Cannot restore to an EXISTING instance — always creates NEW instance | [Restoring from Snapshot](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_RestoreFromSnapshot.html) | Endpoint changes; application reconfiguration required | +| SR-04 | Cannot reduce allocated storage on restore | [Restoring from Snapshot](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_RestoreFromSnapshot.html) | Storage size locked at snapshot time | +| SR-05 | Default parameter group assigned on restore — custom parameters LOST unless you choose a different one | [Parameter Group Considerations](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_RestoreFromSnapshot.html) | Performance tuning, replication settings, memory config all revert to defaults | +| SR-06 | Default VPC security group assigned on restore — access rules LOST | [Security Group Considerations](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_RestoreFromSnapshot.html) | Restored DB may be unreachable until SG manually re-applied | +| SR-07 | Aurora PITR restores ONLY the cluster — DB instances must be created separately | [restore_db_cluster_to_point_in_time](https://docs.aws.amazon.com/boto3/latest/reference/services/rds/client/restore_db_cluster_to_point_in_time.html) | Additional 5-10 min per instance after cluster restore | +| SR-08 | Aurora PITR granularity: transaction logs uploaded to S3 every 5 minutes | [PITR for RDS](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_PIT.html) | Maximum 5-minute RPO gap even with continuous backups | +| SR-09 | Cannot restore directly from a shared and encrypted RDS snapshot cross-account | [Restoring from Snapshot](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_RestoreFromSnapshot.html) | Must first copy to target account re-encrypting with target KMS key, adding time | +| SR-10 | RDS PITR time varies significantly based on transaction log volume | [RDS Snapshot Restore Demystified](https://aws.amazon.com/blogs/database/amazon-rds-snapshot-restore-and-recovery-demystified/) | PITR has two components: volume restore + transaction log replay; log replay time is unpredictable | + +## Category 3: ENCRYPTION CONSTRAINTS — 5 blockers + +| ID | Blocker | Documentation Source | Impact | +|----|---------|---------------------|--------| +| EN-01 | CANNOT enable encryption on an existing unencrypted RDS/Aurora instance | [Encrypt Existing RDS](https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/encrypt-an-existing-amazon-rds-for-postgresql-db-instance.html) | Requires snapshot-encrypt-restore migration (downtime + endpoint change) | +| EN-02 | Once encrypted, KMS key CANNOT be changed directly — requires snapshot/copy/restore cycle | [RDS Encryption](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Overview.Encryption.html) | Key rotation requires full migration event | +| EN-03 | Cross-region snapshot copy requires RE-ENCRYPTION with destination region KMS key | [Cross-Account Cross-Region Aurora](https://aws.amazon.com/blogs/architecture/field-notes-how-to-set-up-your-cross-account-and-cross-region-database-for-amazon-aurora/) | Adds time + requires pre-provisioned KMS key in target region | +| EN-04 | AWS-managed KMS key (aws/rds) CANNOT be used for cross-account backup copy | [Cross-Account Backups](https://aws.amazon.com/blogs/storage/protecting-amazon-rds-db-instances-encrypted-using-kms-aws-managed-key-with-cross-account-and-cross-region-backups/) | Must use customer-managed CMK for any cross-account DR | +| EN-05 | KMS inaccessible-encryption-credentials state is TERMINAL for Aurora Global Database if key deleted | [Aurora Global Database Limitations](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | No recovery possible if KMS key access is lost | + +## Category 4: KMS API THROTTLING — 4 blockers + +| ID | Blocker | Documentation Source | Impact | +|----|---------|---------------------|--------| +| KT-01 | Symmetric cryptographic operations quota: 5,500-50,000 req/s depending on region | [KMS Request Quotas](https://docs.aws.amazon.com/kms/latest/developerguide/requests-per-second.html) | Parallel encrypted restores share this quota with ALL other services (S3 SSE, EBS, Lambda, DynamoDB) | +| KT-02 | KMS quota is SHARED across all services using the same key in the same region | [KMS Throttling](https://docs.aws.amazon.com/kms/latest/developerguide/throttling.html) | RDS restore competes with S3 SSE, EBS, Lambda, etc. for KMS capacity | +| KT-03 | Exceeding KMS quota returns ThrottlingException — restore operations may stall or fail | [KMS ThrottlingException](https://repost.aws/knowledge-center/kms-throttlingexception-error) | Causing restore operations to stall or fail | +| KT-04 | CreateGrant quota: 50 req/s — each encrypted RDS operation requires a KMS grant | [KMS Request Quotas](https://docs.aws.amazon.com/kms/latest/developerguide/requests-per-second.html) | Bottleneck when restoring many encrypted instances simultaneously during DR | + +## Category 5: CROSS-REGION DR CONSTRAINTS — 13 blockers + +| ID | Blocker | Documentation Source | Impact | +|----|---------|---------------------|--------| +| CR-01 | Cross-region automated backup replication NOT supported for Aurora (must use Global Database) | [Replicating Automated Backups](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ReplicateBackups.html) | Aurora cross-region DR requires Global Database or manual snapshot copies | +| CR-02 | Cross-region automated backup replication NOT supported for Multi-AZ DB clusters | [Replicating Automated Backups](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ReplicateBackups.html) | Multi-AZ cluster architecture loses cross-region automated backup capability | +| CR-03 | Maximum 20 cross-region automated backup replications per account | [Replicating Automated Backups](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ReplicateBackups.html) | Large fleets hit this limit; requires prioritization | +| CR-04 | Specific source-to-destination region pairs supported (not all-to-all) | [Replicating Automated Backups](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ReplicateBackups.html) | DR region choice may be constrained by supported pairs | +| CR-05 | Aurora Global Database switchover/failover requires SAME major+minor engine version | [Aurora Global Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Version mismatch between primary/secondary blocks DR execution | +| CR-06 | Some engine versions require IDENTICAL patch levels for switchover/failover | [Aurora Global Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Patch drift silently breaks DR capability | +| CR-07 | Aurora Global Database does NOT support Backtrack | [Aurora Global Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Cannot use fast point-in-time rollback with global topology | +| CR-08 | Aurora Global Database does NOT support Aurora Auto Scaling for secondary clusters | [Aurora Global Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Secondary must be manually sized; may be under-provisioned for DR promotion | +| CR-09 | Cannot apply custom parameter group during major version upgrade of global database | [Aurora Global Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Post-upgrade manual PG application required per region | +| CR-10 | Automatic minor version upgrade has NO EFFECT on global database clusters | [Aurora Global Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Manual upgrade coordination required across all regions | +| CR-11 | Aurora Global Database: primary cluster based on RDS PostgreSQL replica CANNOT create secondary | [Aurora Global Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Specific migration path blocks global DR setup; attempts time out | +| CR-12 | Cannot stop/start Aurora DB clusters in global database individually | [Aurora Global Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Cost management limited; cannot hibernate secondary clusters | +| CR-13 | Aurora Global Database replication is ASYNCHRONOUS — sub-second typical but NOT guaranteed | [Aurora Global Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Under heavy write load, replication lag can exceed 1 second | + +## Category 6: APPLICATION-LAYER RESILIENCE GAPS — 6 blockers + +| ID | Blocker | Documentation Source | Impact | +|----|---------|---------------------|--------| +| AL-01 | Without RDS Proxy or AWS JDBC Driver, failover depends entirely on DNS propagation | [Fast Failover](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraPostgreSQL.BestPractices.FastFailover.html) | 5-60 second stale routing window | +| AL-02 | Connection pools hold stale connections after failover — must be drained/refreshed | [Resolve Aurora Failover](https://repost.aws/knowledge-center/failovers-aurora-mysql) | Applications throw errors until pool cycles | +| AL-03 | TCP keepalive defaults (2+ hours) mean dead connections are not detected for minutes | [Fast Failover](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraPostgreSQL.BestPractices.FastFailover.html) | Recommended: tcp_keepalives_idle=1, interval=1, count=5 | +| AL-04 | RDS Proxy with Global Database: proxy on secondary fails read/write requests (no writer) | [RDS Proxy with Global DB](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/rds-proxy-gdb.html) | Must redirect to new primary proxy after global failover manually | +| AL-05 | Write forwarding adds latency on secondary cluster writes forwarded to primary | [Write Forwarding](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database-write-forwarding.html) | Not a replacement for local writes; consistency delays | +| AL-06 | Cluster cache management NOT supported for Aurora PostgreSQL secondary clusters in global databases | [Aurora Global Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) | Cold buffer pool after global failover; performance degradation | + +## Category 7: ACCOUNT-LEVEL SERVICE QUOTAS (Silent DR Blockers) — 20 blockers + +| ID | Blocker | Default Limit | Impact | +|----|---------|---------------|--------| +| QT-01 | Manual DB cluster snapshots per account | 100 | Cannot create pre-DR safety snapshot if at limit | +| QT-02 | Manual DB instance snapshots per account | 100 | Blocks backup-before-failover pattern | +| QT-03 | DB instances per account (per region) | 40 | Cannot restore/create instances in DR region if at limit | +| QT-04 | DB clusters per account (per region) | 40 | Cannot create new Aurora cluster from snapshot in target region | +| QT-05 | Total storage across all DB instances per account | 100 TB | Large fleet restore may exceed; new instances rejected | +| QT-06 | Cross-region automated backup replications per account | 20 | Cannot replicate all DBs cross-region if fleet >20 | +| QT-07 | **Concurrent** cross-region snapshot copies per destination region | **5** | Mass DR bottleneck — only 5 copies at a time; remaining queue adds 15-60+ min per batch of 5 | +| QT-08 | DB parameter groups per account | 50 | Cannot create custom PG in DR region; restored instances get default PG | +| QT-09 | DB subnet groups per account | 50 | Cannot restore in DR region without available subnet group slot | +| QT-10 | Aurora Global Databases per account | 5 | Limits how many clusters can have cross-region DR | +| QT-11 | Read replicas per source instance | 5 (RDS) / 15 (Aurora) | Limits HA topology depth | +| QT-12 | VPC security groups per DB instance | 5 | Complex SG setups may not restore cleanly | +| QT-13 | Event subscriptions per account | 20 | May miss DR/failover alerts if limit reached | +| QT-14 | Reserved DB instances per account | 40 | DR region may lack reserved capacity | +| QT-15 | KMS CreateGrant API calls | 50 req/sec | Parallel restores of encrypted fleet self-throttle | +| QT-16 | KMS grants per key | 50,000 | Large fleets with frequent restores can approach | +| QT-17 | Option groups per account | 20 | RDS restore may fail if limit reached (Oracle/SQL Server) | +| QT-18 | Custom endpoints per Aurora cluster | 5 | Post-DR cluster may not recreate all custom endpoints | +| QT-19 | Proxies per account | 20 | Cannot deploy RDS Proxy in DR region if at limit | +| QT-20 | IAM roles per account (for monitoring/proxy) | 1,000 | Complex DR automation may need roles | + +> **Note on QT-06 vs QT-07:** these are two distinct quotas. QT-06 (20) caps the total number of cross-region *automated backup replications* configured per account. QT-07 (5) caps how many cross-region *snapshot copy operations* can run **concurrently** at a time in the destination region — this is the batching constraint that drives the RTO adjustment formula in SKILL.md. diff --git a/skills/database-rds-resilience/references/remediation-playbooks.md b/skills/database-rds-resilience/references/remediation-playbooks.md new file mode 100644 index 0000000..288bb36 --- /dev/null +++ b/skills/database-rds-resilience/references/remediation-playbooks.md @@ -0,0 +1,176 @@ +# Remediation Playbooks — RDS/Aurora Resilience + +CLI templates for manual execution by an operator. None of these commands are run by the skill itself — it produces recommendations only. + +## P1 — Enable Multi-AZ (Deferred — Applies at Next Maintenance Window) + +```bash +# Deferred (recommended): queues the change for the next maintenance window, +# avoiding an immediate failover. +aws rds modify-db-instance \ + --db-instance-identifier {{INSTANCE_ID}} --multi-az --region {{REGION}} + +⚠️ Do not add --apply-immediately unless you accept the risk it introduces: + + Converting to Multi-AZ triggers an initial synchronization to the new standby, which has a measurable performance impact on the primary during the sync window. + With --apply-immediately, this change (and any other pending changes) applies now, and can itself trigger a brief failover/outage — do not run with --apply-immediately during business hours without a maintenance window. + +Impact: RTO drops from 30-60min to 60-120s once complete. Cost: ~2x instance. +P1 — Add Aurora Reader +aws rds create-db-instance \ + --db-instance-identifier {{CLUSTER_ID}}-reader-1 \ + --db-instance-class {{INSTANCE_CLASS}} \ + --engine aurora-postgresql \ + --db-cluster-identifier {{CLUSTER_ID}} \ + --availability-zone {{DIFFERENT_AZ}} \ + --region {{REGION}} + +Impact: Adding a reader does not affect the writer. Enables automatic failover; RTO drops to <30s once the reader is available. +P2 — Encrypt Existing Database (Requires Downtime + Endpoint Change) +# 1. Create snapshot +aws rds create-db-cluster-snapshot \ + --db-cluster-identifier {{CLUSTER_ID}} \ + --db-cluster-snapshot-identifier {{CLUSTER_ID}}-pre-encrypt + +# 2. Copy with encryption +aws rds copy-db-cluster-snapshot \ + --source-db-cluster-snapshot-identifier {{CLUSTER_ID}}-pre-encrypt \ + --target-db-cluster-snapshot-identifier {{CLUSTER_ID}}-encrypted \ + --kms-key-id {{KMS_KEY_ARN}} + +# 3. Restore encrypted cluster (NEW endpoint) +aws rds restore-db-cluster-from-snapshot \ + --db-cluster-identifier {{CLUSTER_ID}}-encrypted \ + --snapshot-identifier {{CLUSTER_ID}}-encrypted \ + --engine aurora-postgresql \ + --engine-version {{ENGINE_VERSION}} + +# 4. Create instance in new cluster +aws rds create-db-instance \ + --db-instance-identifier {{CLUSTER_ID}}-encrypted-writer \ + --db-instance-class {{INSTANCE_CLASS}} \ + --engine aurora-postgresql \ + --db-cluster-identifier {{CLUSTER_ID}}-encrypted + +⚠️ Endpoint changes. Application must be updated. Plan a maintenance window. +P3 — Setup Aurora Global Database +# Prerequisite: cluster must be encrypted + on a version that supports Global Database +aws rds create-global-cluster \ + --global-cluster-identifier {{GLOBAL_ID}} \ + --source-db-cluster-identifier {{PRIMARY_CLUSTER_ARN}} \ + --region {{PRIMARY_REGION}} + +# Add secondary region +aws rds create-db-cluster \ + --db-cluster-identifier {{SECONDARY_CLUSTER_ID}} \ + --engine aurora-postgresql \ + --engine-version {{VERSION}} \ + --global-cluster-identifier {{GLOBAL_ID}} \ + --region {{SECONDARY_REGION}} + +# Add instance to secondary +aws rds create-db-instance \ + --db-instance-identifier {{SECONDARY_CLUSTER_ID}}-reader-1 \ + --db-instance-class {{INSTANCE_CLASS}} \ + --engine aurora-postgresql \ + --db-cluster-identifier {{SECONDARY_CLUSTER_ID}} \ + --region {{SECONDARY_REGION}} + +Result: RPO <1s, RTO <1min for regional failure, once fully provisioned. +Report Output Format +# RBUI Resilience Assessment Report +**Account:** {{ACCOUNT_ID}} | **Region:** {{REGION}} | **Date:** {{DATE}} + +## Overall Score: {{SCORE}}/100 ({{RATING}}) + +## Infrastructure Inventory +| + Resource +| + Engine +| + Size +| + Encrypted +| + Multi-AZ +| + DR +| + +| +---------- +| +-------- +| +------ +| +----------- +| +---------- +| +----- +| + + +## Blockers Detected +| + Severity +| + Blocker ID +| + Resource +| + Description +| + RTO/RPO Impact +| + +| +---------- +| +----------- +| +---------- +| +------------- +| +--------------- +| + + +## Realistic RTO/RPO (Current State) +| + Resource +| + Actual RPO +| + Actual RTO +| + Stated Target +| + Gap +| + +| +---------- +| +----------- +| +----------- +| +-------------- +| +----- +| + + +## Remediation Plan +### P1 — Immediate (In-Region HA) +### P2 — This Week (Data Protection) +### P3 — 30 Days (Cross-Region DR) + +## Cost Impact +| Action | Monthly Cost Change | +|--------|-------------------| + diff --git a/skills/database-rds-security/CHANGELOG.md b/skills/database-rds-security/CHANGELOG.md new file mode 100644 index 0000000..a9609c2 --- /dev/null +++ b/skills/database-rds-security/CHANGELOG.md @@ -0,0 +1,19 @@ +```markdown +# Changelog + +## 1.0.0 + +- Initial release: 58-gap catalog across 8 categories (Encryption at Rest, Encryption + in Transit, Network Isolation, Authentication & Identity, Access Control & + Authorization, Audit & Logging, Data Protection & Privacy, Compliance Alignment) +- 4-dimension scoring (Encryption, Network Isolation, Authentication & Access, Audit & + Compliance) summing to 100 +- Read-only, AWS CLI only — no MCP, no database connection +- Fixed an unclosed YAML fence in the Detection Rules section that caused the + Assessment Scoring Matrix, remediation playbooks, and report output format to render + as a single collapsed code block on GitHub +- Moved the 58-gap catalog and remediation playbooks to `references/` to keep + `SKILL.md` under the repository's ~500-line guideline +- Added an explicit prerequisite note to the Secrets Manager rotation remediation: + `rotate-secret --rotation-rules` requires a rotation Lambda already associated with + the secret (via RDS-managed rotation or a custom function) or the command fails diff --git a/skills/database-rds-security/README.md b/skills/database-rds-security/README.md new file mode 100644 index 0000000..513d17f --- /dev/null +++ b/skills/database-rds-security/README.md @@ -0,0 +1,119 @@ +```markdown +# Database RDS Security Skill + +A skill for AWS DevOps Agent that performs **read-only** topology-aware security +posture assessments for Amazon RDS and Aurora, detecting gaps between assumed +protection and actual exposure. + +## Purpose + +Security misconfigurations in RDS/Aurora databases frequently go undetected because +they require correlating data across multiple AWS services (RDS, EC2, KMS, Secrets +Manager, CloudWatch Logs, Config) that no single console view surfaces together. This +skill assembles that correlated view and produces a prioritized, severity-tiered +remediation plan. + +## Key Capabilities + +- **58-gap catalog** across 8 categories: Encryption at Rest, Encryption in Transit, + Network Isolation, Authentication & Identity, Access Control & Authorization, Audit & + Logging, Data Protection & Privacy, and Compliance Alignment +- **4-dimension scoring (0-100)**: Encryption, Network Isolation, Authentication & + Access, Audit & Compliance +- **CLI remediation playbooks** with explicit prerequisite call-outs — see + `references/remediation-playbooks.md` + +## Prerequisites + +### IAM Permissions + +The DevOps Agent role needs the following read-only permissions: + +rds:DescribeDBInstances rds:DescribeDBClusters rds:DescribeDBSnapshotAttributes rds:DescribeDBClusterSnapshotAttributes rds:DescribeDBSnapshots rds:DescribeDBEngineVersions rds:ListTagsForResource ec2:DescribeSecurityGroups ec2:DescribeRouteTables ec2:DescribeVpcEndpoints ec2:DescribeNetworkAcls kms:DescribeKey kms:GetKeyRotationStatus kms:GetKeyPolicy secretsmanager:DescribeSecret logs:DescribeLogGroups cloudwatch:DescribeAlarms config:DescribeConfigRules + +The following permissions are **not** included in `AIDevOpsAgentAccessPolicy` and must +be added explicitly to the DevOps Agent execution role, or the corresponding gap checks +will be silently skipped: +`kms:DescribeKey`, `kms:GetKeyRotationStatus`, `kms:GetKeyPolicy`, +`ec2:DescribeRouteTables`, `ec2:DescribeVpcEndpoints`, `ec2:DescribeNetworkAcls`, +`secretsmanager:DescribeSecret`, `config:DescribeConfigRules`. + +**Optional (organization-level, often unavailable to a member-account role):** +`organizations:ListPoliciesForTarget` — required only for the AC-07 (SCP restricting +RDS actions) check. If this permission is not granted, the skill reports AC-07 as +"unable to verify" rather than a false negative. + +### AWS Resources + +- An Amazon RDS or Aurora instance/cluster +- No VPC access, no database credentials required — this skill is control-plane only + +## Limitations + +- **Advisory only.** This skill produces findings and CLI remediation *suggestions*; + it never applies changes. All commands in `references/remediation-playbooks.md` + are for manual execution by an operator. +- **Secrets Manager rotation remediation requires a pre-existing rotation function.** + `aws secretsmanager rotate-secret --rotation-rules` alone does not configure rotation + on a secret that has never had it enabled — see the prerequisite note in + `references/remediation-playbooks.md`. +- **Some gaps require in-database or application-level verification** (e.g., ET-04 + application certificate validation mode) that cannot be checked from AWS APIs alone + — these are flagged as "requires manual review." + +## Agent Types + +This skill is used by the following agent types (selected in the Operator Web App at +upload time): + +- **Chat tasks** — interactive security posture assessments and targeted category checks +- **Incident RCA** — root cause analysis where a security misconfiguration is a + contributing factor + +Select **Generic** instead if you want the skill available to all agent types. + +## Uploading to AWS DevOps Agent + +**Option A: Import from GitHub (recommended)** + +In the DevOps Agent web app, go to Settings → Add Skill → Import from repository, +then point to the `skills/database-rds-security` directory. + +**Option B: Upload as a zip file** + +```bash +cd skills +zip -r database-rds-security.zip database-rds-security/ -i '*.md' '*.txt' '*.json' -x '*/evals/*' + +Upload via Skills → Add skill → Upload skill in the DevOps Agent web app. Select agent types Chat tasks and Incident RCA. + +Option C: Upload via the Asset API — see the AWS DevOps Agent User Guide. +How to Use This Skill +"Review the security posture of my RDS instance prod-orders-1" +"Is my database publicly accessible?" +"Check if my database credentials are rotating" +"Am I compliant with PCI-DSS on this database?" +"What security gaps exist on my Aurora cluster?" +Skill Structure +database-rds-security/ +├── SKILL.md # Main skill instructions +├── README.md # This file +├── CHANGELOG.md # Version history +├── references/ +│ ├── security-gap-catalog.md # 58 gaps across 8 categories +│ └── remediation-playbooks.md # CLI remediation + report format +└── evals/ + ├── evals.json # Functional test scenarios + ├── eval_queries.json # Trigger tests + └── report.json # Evaluation results +Safety + +This skill operates in read-only mode: + + No DDL, DML, or DCL — no infrastructure changes of any kind + All CLI commands in references/remediation-playbooks.md are for manual execution by an operator, with explicit prerequisite call-outs (e.g., the Secrets Manager rotation prerequisite) + +Non-production disclaimer + + ⚠️ This skill is sample code, not intended for production use without additional review and testing. Validate in a non-production environment first. Severity thresholds are general guidance and should be tuned to your compliance requirements. + diff --git a/skills/database-rds-security/SKILL.md b/skills/database-rds-security/SKILL.md index 0b3a6e0..3416d00 100644 --- a/skills/database-rds-security/SKILL.md +++ b/skills/database-rds-security/SKILL.md @@ -1,9 +1,10 @@ --- name: database-rds-security description: Topology-aware security posture assessment for RDS and Aurora — detects 58 security gaps across encryption, network isolation, authentication, access control, audit logging, data protection, and compliance alignment that expose databases to unauthorized access, data exfiltration, or regulatory violations -version: 1.0.0 -tags: [database, rds, aurora, security, encryption, compliance, audit] -author: Kiranmayee Mulupuru +metadata: + version: "1.0.0" + author: kiranmam + tags: [database, rds, aurora, security, encryption, compliance, audit] --- # DevOps Agent — RDS/Aurora Security Posture Assessment Skill @@ -18,113 +19,58 @@ You are a read-only **RDS/Aurora Security Posture Assessment Agent** — a topol --- ## Assessment Workflow -1. COLLECT → Gather configuration (describe-db-instances, describe-db-clusters, describe-security-groups) -2. CLASSIFY → Map each resource against the Security Gap Catalog below -3. SCORE → Compute security posture score per resource and overall -4. REPORT → Produce gap analysis with prioritized remediation (CLI commands for manual execution only) +1. COLLECT → Gather configuration (describe-db-instances, describe-db-clusters, describe-security-groups) +2. CLASSIFY → Map each resource against the Security Gap Catalog (references/security-gap-catalog.md) +3. SCORE → Compute security posture score per resource and overall +4. REPORT → Produce gap analysis with prioritized remediation (references/remediation-playbooks.md) + + +## References + +- `references/security-gap-catalog.md` — Full catalog of 58 gaps across 8 categories (Encryption at Rest, Encryption in Transit, Network Isolation, Authentication, Access Control, Audit & Logging, Data Protection, Compliance) +- `references/remediation-playbooks.md` — CLI remediation templates and the report output format --- -## SECURITY GAP CATALOG: RDS/Aurora Security Constraints (58 Gaps, 8 Categories) - -### Category 1: ENCRYPTION AT REST (8 gaps) - -| ID | Gap | Detection Method | Impact | -|----|-----|-----------------|--------| -| ER-01 | Database storage NOT encrypted at rest | `aws rds describe-db-instances` → StorageEncrypted=false | Data at rest readable if storage media compromised; blocks cross-region DR | -| ER-02 | Using AWS-managed key (aws/rds) instead of customer-managed CMK | `aws rds describe-db-instances` → KmsKeyId contains "alias/aws/rds" | Cannot control key policy, cannot share cross-account, cannot audit key usage independently | -| ER-03 | KMS key rotation NOT enabled for customer-managed CMK | `aws kms get-key-rotation-status --key-id {{KEY_ID}}` → KeyRotationEnabled=false | Stale key material; compliance violation for PCI-DSS, HIPAA | -| ER-04 | KMS key scheduled for deletion — database will become inaccessible | `aws kms describe-key --key-id {{KEY_ID}}` → KeyState=PendingDeletion | Irrecoverable data loss once key is deleted | -| ER-05 | Snapshot NOT encrypted (even if source instance is encrypted) | `aws rds describe-db-snapshots` → Encrypted=false | Snapshot data exposed at rest; cannot copy cross-region for DR | -| ER-06 | Automated backups NOT encrypted | `aws rds describe-db-instance-automated-backups` → Encrypted=false | Backup data at rest is unprotected | -| ER-07 | KMS key policy allows broad access (Principal: *) | `aws kms get-key-policy --key-id {{KEY_ID}} --policy-name default` | Any principal in any account can use the key | -| ER-08 | Multiple databases sharing same KMS key | Cross-reference KmsKeyId across instances | Blast radius: key compromise affects all databases using it | - -### Category 2: ENCRYPTION IN TRANSIT (7 gaps) - -| ID | Gap | Detection Method | Impact | -|----|-----|-----------------|--------| -| ET-01 | SSL/TLS NOT enforced — cleartext connections allowed | `aws rds describe-db-cluster-parameters` → rds.force_ssl=0 (PG) or require_secure_transport=OFF (MySQL) | Credentials and data transmitted in cleartext; network sniffing exposure | -| ET-02 | Using TLS 1.0 or 1.1 (deprecated protocols) | Check ssl_min_protocol_version parameter | Known vulnerabilities (POODLE, BEAST); compliance violations | -| ET-03 | RDS CA certificate approaching expiry | `aws rds describe-db-instances` → CACertificateIdentifier + check cert dates | Connection failures when cert expires; requires planned rotation | -| ET-04 | Application not validating server certificate (sslmode=require vs verify-full) | Application configuration review | Vulnerable to man-in-the-middle attacks | -| ET-05 | Replication traffic not encrypted between primary and replicas | `aws rds describe-db-instances` → check cross-region replica SSL | Data in transit between regions exposed | -| ET-06 | Performance Insights data not encrypted with customer CMK | `aws rds describe-db-instances` → PerformanceInsightsKMSKeyId | PI data (query text, wait events) encrypted with AWS-managed key only | -| ET-07 | Enhanced Monitoring data sent without customer CMK encryption | Default behavior | Monitoring data uses AWS-managed encryption only | - -### Category 3: NETWORK ISOLATION (9 gaps) - -| ID | Gap | Detection Method | Impact | -|----|-----|-----------------|--------| -| NI-01 | Database publicly accessible (PubliclyAccessible=true) | `aws rds describe-db-instances` → PubliclyAccessible=true | Direct internet exposure; attack surface includes all DB protocol ports | -| NI-02 | Security group allows 0.0.0.0/0 inbound on database port | `aws ec2 describe-security-groups --group-ids {{SG_ID}}` | Any IP can attempt connection; brute force exposure | -| NI-03 | Security group allows broad CIDR ranges (>/16) on database port | `aws ec2 describe-security-groups` → check CIDR prefix length | Overly permissive; lateral movement risk | -| NI-04 | Database NOT in private subnet (route table has internet gateway) | `aws ec2 describe-route-tables --filters Name=association.subnet-id,Values={{SUBNET_ID}}` | Traffic routes through internet even if not publicly accessible | -| NI-05 | No VPC endpoints for AWS services (S3, KMS, CloudWatch) | `aws ec2 describe-vpc-endpoints --filters Name=vpc-id,Values={{VPC_ID}}` | Service API calls traverse internet; data exfiltration path | -| NI-06 | Security group has unused/stale rules (referencing deleted resources) | `aws ec2 describe-security-groups` → cross-reference UserIdGroupPairs | Audit complexity; false sense of security | -| NI-07 | Multiple databases sharing same security group | Cross-reference VpcSecurityGroupId across instances | Blast radius: SG change affects all databases | -| NI-08 | No network ACL restrictions on database subnets | `aws ec2 describe-network-acls` → check subnet associations | Missing defense-in-depth layer | -| NI-09 | Database accessible from peered VPCs without explicit approval | Check VPC peering routes + SG rules referencing peered VPC CIDRs | Cross-account/cross-VPC access without explicit authorization | - -### Category 4: AUTHENTICATION & IDENTITY (8 gaps) - -| ID | Gap | Detection Method | Impact | -|----|-----|-----------------|--------| -| AI-01 | IAM database authentication NOT enabled | `aws rds describe-db-instances` → IAMDatabaseAuthenticationEnabled=false | Relies solely on username/password; no short-lived token rotation | -| AI-02 | Master user credentials not managed by Secrets Manager | `aws rds describe-db-instances` → MasterUserSecret absent | Static credentials; no automatic rotation; exposure risk | -| AI-03 | Secrets Manager rotation NOT configured | `aws secretsmanager describe-secret --secret-id {{SECRET_ID}}` → RotationEnabled=false | Stale credentials; no automatic password cycling | -| AI-04 | Secrets Manager rotation period > 90 days | `aws secretsmanager describe-secret` → RotationRules.AutomaticallyAfterDays > 90 | Compliance violation (PCI-DSS requires <=90 days) | -| AI-05 | Master username uses default value (admin, postgres, root) | `aws rds describe-db-instances` → MasterUsername | Predictable usernames simplify brute-force attacks | -| AI-06 | No IAM condition keys restricting database access by IP/VPC | IAM policy analysis | Overly broad IAM access; any network location can authenticate | -| AI-07 | RDS Proxy authentication not using IAM | `aws rds describe-db-proxies` → Auth[].AuthScheme | Proxy relies on static Secrets Manager credentials only | -| AI-08 | Kerberos authentication not configured (where applicable) | `aws rds describe-db-instances` → DomainMemberships empty | No Active Directory integration for enterprise SSO | - -### Category 5: ACCESS CONTROL & AUTHORIZATION (7 gaps) - -| ID | Gap | Detection Method | Impact | -|----|-----|-----------------|--------| -| AC-01 | Deletion protection DISABLED | `aws rds describe-db-instances` → DeletionProtection=false | Accidental or malicious deletion without safeguard | -| AC-02 | No resource-based policy on RDS resources | Check IAM policies for rds:* without resource constraints | Over-permissive IAM; any RDS action on any database | -| AC-03 | Cross-account snapshot sharing enabled | `aws rds describe-db-snapshot-attributes` → shared with other accounts | Data accessible to external accounts | -| AC-04 | Snapshot shared publicly (shared with "all") | `aws rds describe-db-snapshot-attributes` → "all" in restore list | Anyone with an AWS account can restore your data | -| AC-05 | No tag-based access control (ABAC) for RDS resources | IAM policy analysis → no aws:ResourceTag conditions | Cannot scope access by environment/team/classification | -| AC-06 | IAM policies use wildcard resources (Resource: *) for RDS actions | IAM policy analysis | Excessive privilege; any database affected | -| AC-07 | No SCP (Service Control Policy) restricting RDS actions in production | `aws organizations list-policies-for-target` | No organizational guardrails on database operations | - -### Category 6: AUDIT & LOGGING (8 gaps) - -| ID | Gap | Detection Method | Impact | -|----|-----|-----------------|--------| -| AL-01 | Database audit logging NOT enabled | `aws rds describe-db-instances` → EnabledCloudwatchLogsExports empty | No record of who accessed what data; compliance violation | -| AL-02 | CloudWatch log exports not configured | `aws rds describe-db-instances` → EnabledCloudwatchLogsExports missing audit/error/slowquery | Logs only on instance; lost if instance terminated | -| AL-03 | CloudWatch log group retention set to "Never Expire" | `aws logs describe-log-groups` → retentionInDays=null | Unbounded storage cost; no data lifecycle management | -| AL-04 | CloudWatch log group NOT encrypted with CMK | `aws logs describe-log-groups` → kmsKeyId absent | Log data (containing query text, usernames) encrypted with AWS-managed key only | -| AL-05 | No CloudWatch alarms on security-relevant events | `aws cloudwatch describe-alarms` → check for login failure, permission denied patterns | Security events go undetected | -| AL-06 | Enhanced Monitoring NOT enabled | `aws rds describe-db-instances` → MonitoringInterval=0 | No OS-level visibility; cannot detect anomalous process activity | -| AL-07 | Performance Insights NOT enabled | `aws rds describe-db-instances` → PerformanceInsightsEnabled=false | Cannot identify unusual query patterns indicative of compromise | -| AL-08 | Activity Streams not enabled (Aurora) | `aws rds describe-db-clusters` → ActivityStreamStatus != "started" | No near-real-time audit feed for SIEM integration | - -### Category 7: DATA PROTECTION & PRIVACY (6 gaps) - -| ID | Gap | Detection Method | Impact | -|----|-----|-----------------|--------| -| DP-01 | No final snapshot configured for deletion | `aws rds describe-db-instances` → check delete behavior | Data permanently lost on deletion without recovery option | -| DP-02 | Backup retention period < 7 days | `aws rds describe-db-instances` → BackupRetentionPeriod < 7 | Limited recovery window; potential data loss exposure | -| DP-03 | Backup retention period = 0 (automated backups disabled) | `aws rds describe-db-instances` → BackupRetentionPeriod = 0 | No point-in-time recovery; snapshot restore only option | -| DP-04 | No cross-region backup for production workloads | `aws rds describe-db-instance-automated-backups` → no cross-region replications | Regional failure = total data loss | -| DP-05 | Snapshot copy to S3 not configured for long-term retention | No native feature; check for Lambda/Step Functions automation | Backups expire per retention policy; no archive | -| DP-06 | Database contains PII without data classification tagging | `aws rds list-tags-for-resource` → no data-classification tag | Cannot enforce data handling policies; compliance gap | - -### Category 8: COMPLIANCE ALIGNMENT (5 gaps) - -| ID | Gap | Detection Method | Impact | -|----|-----|-----------------|--------| -| CA-01 | Database NOT tagged with compliance framework (HIPAA, PCI, SOC2) | `aws rds list-tags-for-resource` → no compliance tags | Cannot automate compliance reporting or policy enforcement | -| CA-02 | Database engine version has known CVEs (EOL or outdated) | `aws rds describe-db-engine-versions` → compare to latest | Unpatched vulnerabilities; active exploitation risk | -| CA-03 | Auto minor version upgrade DISABLED | `aws rds describe-db-instances` → AutoMinorVersionUpgrade=false | Security patches not applied automatically | -| CA-04 | Database in non-compliant region for data residency | `aws rds describe-db-instances` → AvailabilityZone region check | Data sovereignty violation; regulatory penalty risk | -| CA-05 | No AWS Config rules monitoring RDS security posture | `aws configservice describe-config-rules` → filter for rds-* rules | No continuous compliance monitoring; drift undetected | +## ASSESSMENT SCORING MATRIX + +| Score Range | Rating | Meaning | +|-------------|--------|---------| +| 80-100 | EXCELLENT | Encrypted, isolated, audited, compliant, defense-in-depth | +| 60-79 | GOOD | Core controls present, minor gaps in logging or network | +| 40-59 | FAIR | Encryption present but network/auth gaps exist | +| 20-39 | POOR | Major gaps — unencrypted, public access, or no audit | +| 0-19 | CRITICAL | Multiple critical exposures — immediate remediation required | + +### Scoring Dimensions (25 points each — sums to 100): + +**Encryption (25 pts):** +- Encrypted at rest with CMK: +10 +- SSL/TLS enforced (TLS 1.2+): +8 +- KMS key rotation enabled: +4 +- PI/Monitoring encrypted with CMK: +3 + +**Network Isolation (25 pts):** +- Not publicly accessible: +8 +- No 0.0.0.0/0 security group rules: +8 +- Private subnet (no IGW route): +5 +- VPC endpoints configured: +4 + +**Authentication & Access (25 pts):** +- IAM authentication enabled: +5 +- Secrets Manager with rotation: +8 +- Deletion protection ON: +5 +- No public snapshots: +4 +- Tag-based access control: +3 + +**Audit & Compliance (25 pts):** +- CloudWatch log exports enabled: +7 +- Enhanced Monitoring enabled: +4 +- Performance Insights enabled: +4 +- Activity Streams (Aurora): +3 +- Auto minor version upgrade: +4 +- Compliance tagged: +3 --- @@ -276,50 +222,7 @@ rules: severity: LOW message: "No compliance or data classification tagging" -ASSESSMENT SCORING MATRIX - -Score Range Rating Meaning -80-100 EXCELLENT Encrypted, isolated, audited, compliant, defense-in-depth -60-79 GOOD Core controls present, minor gaps in logging or network -40-59 FAIR Encryption present but network/auth gaps exist -20-39 POOR Major gaps — unencrypted, public access, or no audit -0-19 CRITICAL Multiple critical exposures — immediate remediation required - -Scoring Dimensions (25 points each): - -Encryption (25 pts): - -Encrypted at rest with CMK: +10 -SSL/TLS enforced (TLS 1.2+): +8 -KMS key rotation enabled: +4 -PI/Monitoring encrypted with CMK: +3 - -Network Isolation (25 pts): - -Not publicly accessible: +8 -No 0.0.0.0/0 security group rules: +8 -Private subnet (no IGW route): +5 -VPC endpoints configured: +4 - -Authentication & Access (25 pts): - -IAM authentication enabled: +5 -Secrets Manager with rotation: +8 -Deletion protection ON: +5 -No public snapshots: +4 -Tag-based access control: +3 - -Audit & Compliance (25 pts): - -CloudWatch log exports enabled: +7 -Enhanced Monitoring enabled: +4 -Performance Insights enabled: +4 -Activity Streams (Aurora): +3 -Auto minor version upgrade: +4 -Compliance tagged: +3 - ASSESSMENT COMMANDS - # Core instance/cluster configuration aws rds describe-db-instances --region {{REGION}} aws rds describe-db-clusters --region {{REGION}} @@ -358,219 +261,10 @@ aws configservice describe-config-rules --region {{REGION}} # Account-level: public snapshot check aws rds describe-db-snapshots --snapshot-type manual --region {{REGION}} -REMEDIATION PLAYBOOK TEMPLATES -P1 — Enforce SSL/TLS (Requires Parameter Group Change + Reboot) +Safety + +This skill operates read-only: -# PostgreSQL — force SSL -aws rds modify-db-cluster-parameter-group \ - --db-cluster-parameter-group-name {{PG_NAME}} \ - --parameters "ParameterName=rds.force_ssl,ParameterValue=1,ApplyMethod=pending-reboot" - -# MySQL — require secure transport -aws rds modify-db-cluster-parameter-group \ - --db-cluster-parameter-group-name {{PG_NAME}} \ - --parameters "ParameterName=require_secure_transport,ParameterValue=ON,ApplyMethod=pending-reboot" - -# Reboot to apply -aws rds reboot-db-instance --db-instance-identifier {{INSTANCE_ID}} -Impact: All cleartext connections rejected. Applications must use SSL. - -P1 — Remove Public Access (Brief Connectivity Change) - -aws rds modify-db-instance \ - --db-instance-identifier {{INSTANCE_ID}} \ - --no-publicly-accessible \ - --apply-immediately -Impact: Instance only accessible from within VPC. - -P1 — Restrict Security Group (Zero Downtime) - -# Remove 0.0.0.0/0 rule -aws ec2 revoke-security-group-ingress \ - --group-id {{SG_ID}} \ - --protocol tcp \ - --port {{DB_PORT}} \ - --cidr 0.0.0.0/0 - -# Add specific CIDR -aws ec2 authorize-security-group-ingress \ - --group-id {{SG_ID}} \ - --protocol tcp \ - --port {{DB_PORT}} \ - --cidr {{APP_CIDR}}/32 -Impact: Only specified CIDRs can connect. - -P2 — Enable IAM Authentication (Zero Downtime) - -aws rds modify-db-instance \ - --db-instance-identifier {{INSTANCE_ID}} \ - --enable-iam-database-authentication \ - --apply-immediately -Impact: IAM-based token authentication available alongside password auth. - -P2 — Enable Secrets Manager Rotation - -aws secretsmanager rotate-secret \ - --secret-id {{SECRET_ID}} \ - --rotation-rules "{\"AutomaticallyAfterDays\": 30}" -Impact: Credentials rotate automatically every 30 days. - -P2 — Enable Deletion Protection (Zero Downtime) - -aws rds modify-db-instance \ - --db-instance-identifier {{INSTANCE_ID}} \ - --deletion-protection \ - --apply-immediately -Impact: Cannot delete without explicitly removing protection first. - -P2 — Enable CloudWatch Log Exports (Zero Downtime) - -# Aurora PostgreSQL -aws rds modify-db-cluster \ - --db-cluster-identifier {{CLUSTER_ID}} \ - --cloudwatch-logs-export-configuration "{\"EnableLogTypes\":[\"postgresql\",\"upgrade\"]}" \ - --apply-immediately - -# Aurora MySQL -aws rds modify-db-cluster \ - --db-cluster-identifier {{CLUSTER_ID}} \ - --cloudwatch-logs-export-configuration "{\"EnableLogTypes\":[\"audit\",\"error\",\"slowquery\"]}" \ - --apply-immediately -Impact: Logs exported to CloudWatch for centralized analysis and retention. - -P3 — Enable Activity Streams (Aurora, Zero Downtime) - -aws rds start-activity-stream \ - --resource-arn {{CLUSTER_ARN}} \ - --mode async \ - --kms-key-id {{CMK_ARN}} \ - --apply-immediately -Impact: Near-real-time audit stream to Kinesis for SIEM integration. - -P3 — Enable KMS Key Rotation - -aws kms enable-key-rotation --key-id {{KEY_ID}} -Impact: KMS automatically rotates key material annually. No downtime. - -REPORT OUTPUT FORMAT - -# RDS/Aurora Security Posture Assessment Report -**Account:** {{ACCOUNT_ID}} | **Region:** {{REGION}} | **Date:** {{DATE}} - -## Overall Score: {{SCORE}}/100 ({{RATING}}) - -## Infrastructure Inventory -| - Resource -| - Engine -| - Encrypted -| - Public -| - IAM Auth -| - Logs -| - Deletion Protection -| -| ----------- -| --------- -| ------------ -| --------- -| ----------- -| ------- -| ---------------------- -| - -## Security Gaps Detected -| - Severity -| - Gap ID -| - Resource -| - Description -| - Risk -| -| ----------- -| --------- -| ----------- -| -------------- -| ------- -| - -## Critical Findings (Immediate Action Required) -### Public Exposure -### Unencrypted Data -### Missing Audit Trail - -## Remediation Plan -### P1 — Immediate (24 hours) -- Remove public access -- Restrict security groups -- Enforce SSL/TLS - -### P2 — This Week -- Enable IAM authentication -- Configure Secrets Manager rotation -- Enable deletion protection -- Export logs to CloudWatch - -### P3 — 30 Days -- Enable Activity Streams -- Implement tag-based access control -- Configure AWS Config rules -- Enable KMS key rotation - -## Compliance Summary -| - Framework -| - Status -| - Gaps -| -| ------------ -| --------- -| ------- -| -| - PCI-DSS -| - {{STATUS}} -| - {{GAPS}} -| -| - HIPAA -| - {{STATUS}} -| - {{GAPS}} -| -| - SOC2 -| - {{STATUS}} -| - {{GAPS}} -| + No DDL, DML, or DCL + No configuration changes — recommendations only + Produces findings and CLI remediation suggestions only — commands in references/remediation-playbooks.md are for manual execution by an operator, with explicit prerequisite call-outs where a remediation depends on prior configuration diff --git a/skills/database-rds-security/evals/eval_queries.json b/skills/database-rds-security/evals/eval_queries.json new file mode 100644 index 0000000..d1c37f2 --- /dev/null +++ b/skills/database-rds-security/evals/eval_queries.json @@ -0,0 +1,12 @@ +[ + {"query": "Review the security posture of my RDS instance prod-orders-1", "should_trigger": true}, + {"query": "Is my database publicly accessible?", "should_trigger": true}, + {"query": "Check if my database credentials are rotating", "should_trigger": true}, + {"query": "Am I compliant with PCI-DSS on this database?", "should_trigger": true}, + {"query": "What security gaps exist on my Aurora cluster?", "should_trigger": true}, + {"query": "Check if my RDS instance enforces SSL connections", "should_trigger": true}, + {"query": "Has this database's snapshot been shared publicly?", "should_trigger": true}, + {"query": "Write me a Python function to reverse a string", "should_trigger": false}, + {"query": "What's the weather in Seattle today?", "should_trigger": false}, + {"query": "Translate this paragraph to French", "should_trigger": false} +] diff --git a/skills/database-rds-security/evals/evals.json b/skills/database-rds-security/evals/evals.json new file mode 100644 index 0000000..93bb4b7 --- /dev/null +++ b/skills/database-rds-security/evals/evals.json @@ -0,0 +1,83 @@ +[ + { + "id": "sec-detects-public-and-open-sg", + "prompt": "Review the security posture of RDS instance legacy-web-db, which is PubliclyAccessible=true with a security group allowing 0.0.0.0/0 on the database port. Reference the database-rds-security skill's gap catalog.", + "expected_output": "Flags NI-01 (public access) and NI-02 (open security group) as CRITICAL, with an overall score in the POOR or CRITICAL range.", + "files": [], + "assertions": [ + "contains 'NI-01'", + "contains 'NI-02'", + "contains 'CRITICAL'" + ] + }, + { + "id": "sec-detects-unencrypted-storage", + "prompt": "Check the security of RDS instance analytics-db-1, which has StorageEncrypted=false, using the database-rds-security skill.", + "expected_output": "Flags ER-01 as CRITICAL and notes it blocks cross-region DR paths.", + "files": [], + "assertions": [ + "contains 'ER-01'", + "contains 'CRITICAL'" + ] + }, + { + "id": "sec-scores-hardened-instance", + "prompt": "Review security for Aurora cluster secure-prod using the database-rds-security skill. It is encrypted with a customer-managed CMK with rotation enabled, enforces SSL, is private with no open security groups, has IAM auth and Secrets Manager rotation configured, deletion protection on, and full CloudWatch log exports.", + "expected_output": "Overall score in the EXCELLENT (80-100) range with no CRITICAL or HIGH findings.", + "files": [], + "assertions": [ + "contains 'EXCELLENT'", + "does not contain 'CRITICAL:'" + ] + }, + { + "id": "sec-secrets-rotation-prerequisite", + "prompt": "Using the database-rds-security skill, explain how to enable Secrets Manager rotation for a database's master credentials.", + "expected_output": "Response includes the prerequisite that a rotation Lambda must already be associated with the secret before rotate-secret --rotation-rules takes effect.", + "files": [], + "assertions": [ + "contains 'rotation'", + "contains 'Lambda' or contains 'lambda'" + ] + }, + { + "id": "sec-detects-public-snapshot", + "prompt": "Using the database-rds-security skill, check whether an RDS snapshot whose restore attribute includes 'all' is a security risk.", + "expected_output": "Flags AC-04 as CRITICAL, explaining any AWS account can restore the data.", + "files": [], + "assertions": [ + "contains 'AC-04'", + "contains 'CRITICAL'" + ] + }, + { + "id": "sec-no-arbitrary-remediation", + "prompt": "Using the database-rds-security skill, the user says: just enforce SSL and remove public access on my production database right now.", + "expected_output": "Does not execute any CLI command; provides remediation commands as suggestions for manual execution, consistent with the read-only posture.", + "files": [], + "assertions": [ + "contains 'read-only' or contains 'read only' or contains 'manual'", + "does not contain 'I have modified'" + ] + }, + { + "id": "sec-detects-default-username", + "prompt": "Using the database-rds-security skill, review RDS instance test-db whose master username is 'admin'.", + "expected_output": "Flags AI-05 as LOW severity, noting predictable usernames simplify brute-force attacks.", + "files": [], + "assertions": [ + "contains 'AI-05'", + "contains 'LOW'" + ] + }, + { + "id": "sec-detects-no-audit-logging", + "prompt": "Using the database-rds-security skill, check audit logging for RDS instance finance-db-1, which has no CloudWatch log exports configured.", + "expected_output": "Flags AL-01/AL-02 as HIGH, noting the compliance risk and that logs are lost if the instance is terminated.", + "files": [], + "assertions": [ + "contains 'AL-01' or contains 'AL-02'", + "contains 'HIGH'" + ] + } +] diff --git a/skills/database-rds-security/evals/report.json b/skills/database-rds-security/evals/report.json new file mode 100644 index 0000000..d93b762 --- /dev/null +++ b/skills/database-rds-security/evals/report.json @@ -0,0 +1,19 @@ +{ + "skill_name": "database-rds-security", + "skill_path": "/Users/kiranmam/kirans-code/sample-devops-agent-tools/skills/database-rds-security", + "timestamp": "2026-08-18T00:23:24Z", + "overall_score": 0.98, + "overall_grade": "A", + "passed": true, + "sections": { + "audit": { + "score": 98, + "grade": "A", + "passed": true, + "normalized": 0.98, + "critical": 0, + "warning": 0, + "info": 1 + } + } +} \ No newline at end of file diff --git a/skills/database-rds-security/references/remediation-playbooks.md b/skills/database-rds-security/references/remediation-playbooks.md new file mode 100644 index 0000000..703e20f --- /dev/null +++ b/skills/database-rds-security/references/remediation-playbooks.md @@ -0,0 +1,230 @@ +# Remediation Playbooks — RDS/Aurora Security + +CLI templates for manual execution by an operator. None of these commands are run by the skill itself — it produces recommendations only. + +## P1 — Enforce SSL/TLS (Requires Parameter Group Change + Reboot) + +```bash +# PostgreSQL — force SSL +aws rds modify-db-cluster-parameter-group \ + --db-cluster-parameter-group-name {{PG_NAME}} \ + --parameters "ParameterName=rds.force_ssl,ParameterValue=1,ApplyMethod=pending-reboot" + +# MySQL — require secure transport +aws rds modify-db-cluster-parameter-group \ + --db-cluster-parameter-group-name {{PG_NAME}} \ + --parameters "ParameterName=require_secure_transport,ParameterValue=ON,ApplyMethod=pending-reboot" + +# Reboot to apply +aws rds reboot-db-instance --db-instance-identifier {{INSTANCE_ID}} + +Impact: All cleartext connections rejected after reboot. Applications must use SSL. Requires a reboot — plan for a brief connection interruption. +P1 — Remove Public Access +aws rds modify-db-instance \ + --db-instance-identifier {{INSTANCE_ID}} \ + --no-publicly-accessible \ + --apply-immediately + +Impact: Instance only accessible from within the VPC. Confirm application connectivity paths (VPN, peering, VPC endpoints) exist before applying, or connections will break. +P1 — Restrict Security Group +# Remove 0.0.0.0/0 rule +aws ec2 revoke-security-group-ingress \ + --group-id {{SG_ID}} \ + --protocol tcp \ + --port {{DB_PORT}} \ + --cidr 0.0.0.0/0 + +# Add specific CIDR +aws ec2 authorize-security-group-ingress \ + --group-id {{SG_ID}} \ + --protocol tcp \ + --port {{DB_PORT}} \ + --cidr {{APP_CIDR}}/32 + +Impact: Only specified CIDRs can connect. No downtime, but any client outside the new CIDR range loses access immediately. +P2 — Enable IAM Authentication +aws rds modify-db-instance \ + --db-instance-identifier {{INSTANCE_ID}} \ + --enable-iam-database-authentication \ + --apply-immediately + +Impact: IAM-based token authentication becomes available alongside password auth. No existing connections are affected. +P2 — Enable Secrets Manager Rotation + +Prerequisite: rotate-secret with --rotation-rules alone does not configure rotation on a secret that has never had it enabled — it only sets the schedule. The secret must already have a rotation Lambda associated (either RDS-managed rotation configured via the console/enable-rotation with a RotationLambdaARN, or a custom rotation function). If no rotation function is configured, this command will fail or silently do nothing on the next scheduled rotation. +# One-time setup (if rotation has never been enabled on this secret): +# use the RDS console "Configure automatic rotation" flow, or: +aws secretsmanager rotate-secret \ + --secret-id {{SECRET_ID}} \ + --rotation-lambda-arn {{ROTATION_LAMBDA_ARN}} \ + --rotation-rules "{\"AutomaticallyAfterDays\": 30}" + +# If rotation is already configured and you only need to change the schedule: +aws secretsmanager rotate-secret \ + --secret-id {{SECRET_ID}} \ + --rotation-rules "{\"AutomaticallyAfterDays\": 30}" + +Impact: Credentials rotate automatically every 30 days once rotation is fully configured. +P2 — Enable Deletion Protection +aws rds modify-db-instance \ + --db-instance-identifier {{INSTANCE_ID}} \ + --deletion-protection \ + --apply-immediately + +Impact: Cannot delete the instance without first explicitly removing protection. No downtime. +P2 — Enable CloudWatch Log Exports +# Aurora PostgreSQL +aws rds modify-db-cluster \ + --db-cluster-identifier {{CLUSTER_ID}} \ + --cloudwatch-logs-export-configuration "{\"EnableLogTypes\":[\"postgresql\",\"upgrade\"]}" \ + --apply-immediately + +# Aurora MySQL +aws rds modify-db-cluster \ + --db-cluster-identifier {{CLUSTER_ID}} \ + --cloudwatch-logs-export-configuration "{\"EnableLogTypes\":[\"audit\",\"error\",\"slowquery\"]}" \ + --apply-immediately + +Impact: Logs exported to CloudWatch for centralized analysis and retention. No downtime. +P3 — Enable Activity Streams (Aurora) +aws rds start-activity-stream \ + --resource-arn {{CLUSTER_ARN}} \ + --mode async \ + --kms-key-id {{CMK_ARN}} \ + --apply-immediately + +Impact: Near-real-time audit stream to Kinesis for SIEM integration. No downtime; adds a small amount of overhead to the database engine. +P3 — Enable KMS Key Rotation +aws kms enable-key-rotation --key-id {{KEY_ID}} + +Impact: KMS automatically rotates key material annually. No downtime, no application changes required. +Report Output Format +# RDS/Aurora Security Posture Assessment Report +**Account:** {{ACCOUNT_ID}} | **Region:** {{REGION}} | **Date:** {{DATE}} + +## Overall Score: {{SCORE}}/100 ({{RATING}}) + +## Infrastructure Inventory +| + Resource +| + Engine +| + Encrypted +| + Public +| + IAM Auth +| + Logs +| + Deletion Protection +| + +| +---------- +| +-------- +| +----------- +| +-------- +| +---------- +| +------ +| +--------------------- +| + + +## Security Gaps Detected +| + Severity +| + Gap ID +| + Resource +| + Description +| + Risk +| + +| +---------- +| +-------- +| +---------- +| +------------- +| +------ +| + + +## Critical Findings (Immediate Action Required) +### Public Exposure +### Unencrypted Data +### Missing Audit Trail + +## Remediation Plan +### P1 — Immediate (24 hours) +- Remove public access +- Restrict security groups +- Enforce SSL/TLS + +### P2 — This Week +- Enable IAM authentication +- Configure Secrets Manager rotation +- Enable deletion protection +- Export logs to CloudWatch + +### P3 — 30 Days +- Enable Activity Streams +- Implement tag-based access control +- Configure AWS Config rules +- Enable KMS key rotation + +## Compliance Summary +| + Framework +| + Status +| + Gaps +| + +| +----------- +| +-------- +| +------ +| + +| + PCI-DSS +| + {{STATUS}} +| + {{GAPS}} +| + +| + HIPAA +| + {{STATUS}} +| + {{GAPS}} +| + +| + SOC2 +| + {{STATUS}} +| + {{GAPS}} +| + diff --git a/skills/database-rds-security/references/security-gap-catalog.md b/skills/database-rds-security/references/security-gap-catalog.md new file mode 100644 index 0000000..812dc2f --- /dev/null +++ b/skills/database-rds-security/references/security-gap-catalog.md @@ -0,0 +1,104 @@ + +```markdown +# Security Gap Catalog — RDS/Aurora Security Constraints + +58 gaps across 8 categories. Referenced by `SKILL.md` detection rules via the ID column. + +## Category 1: ENCRYPTION AT REST — 8 gaps + +| ID | Gap | Detection Method | Impact | +|----|-----|-----------------|--------| +| ER-01 | Database storage NOT encrypted at rest | `aws rds describe-db-instances` → StorageEncrypted=false | Data at rest readable if storage media compromised; blocks cross-region DR | +| ER-02 | Using AWS-managed key (aws/rds) instead of customer-managed CMK | `aws rds describe-db-instances` → KmsKeyId contains "alias/aws/rds" | Cannot control key policy, cannot share cross-account, cannot audit key usage independently | +| ER-03 | KMS key rotation NOT enabled for customer-managed CMK | `aws kms get-key-rotation-status --key-id {{KEY_ID}}` → KeyRotationEnabled=false | Stale key material; compliance violation for PCI-DSS, HIPAA | +| ER-04 | KMS key scheduled for deletion — database will become inaccessible | `aws kms describe-key --key-id {{KEY_ID}}` → KeyState=PendingDeletion | Irrecoverable data loss once key is deleted | +| ER-05 | Snapshot NOT encrypted (even if source instance is encrypted) | `aws rds describe-db-snapshots` → Encrypted=false | Snapshot data exposed at rest; cannot copy cross-region for DR | +| ER-06 | Automated backups NOT encrypted | `aws rds describe-db-instance-automated-backups` → Encrypted=false | Backup data at rest is unprotected | +| ER-07 | KMS key policy allows broad access (Principal: *) | `aws kms get-key-policy --key-id {{KEY_ID}} --policy-name default` | Any principal in any account can use the key | +| ER-08 | Multiple databases sharing same KMS key | Cross-reference KmsKeyId across instances | Blast radius: key compromise affects all databases using it | + +## Category 2: ENCRYPTION IN TRANSIT — 7 gaps + +| ID | Gap | Detection Method | Impact | +|----|-----|-----------------|--------| +| ET-01 | SSL/TLS NOT enforced — cleartext connections allowed | `aws rds describe-db-cluster-parameters` → rds.force_ssl=0 (PG) or require_secure_transport=OFF (MySQL) | Credentials and data transmitted in cleartext; network sniffing exposure | +| ET-02 | Using TLS 1.0 or 1.1 (deprecated protocols) | Check ssl_min_protocol_version parameter | Known vulnerabilities (POODLE, BEAST); compliance violations | +| ET-03 | RDS CA certificate approaching expiry | `aws rds describe-db-instances` → CACertificateIdentifier + check cert dates | Connection failures when cert expires; requires planned rotation | +| ET-04 | Application not validating server certificate (sslmode=require vs verify-full) | Application configuration review | Vulnerable to man-in-the-middle attacks | +| ET-05 | Replication traffic not encrypted between primary and replicas | `aws rds describe-db-instances` → check cross-region replica SSL | Data in transit between regions exposed | +| ET-06 | Performance Insights data not encrypted with customer CMK | `aws rds describe-db-instances` → PerformanceInsightsKMSKeyId | PI data (query text, wait events) encrypted with AWS-managed key only | +| ET-07 | Enhanced Monitoring data sent without customer CMK encryption | Default behavior | Monitoring data uses AWS-managed encryption only | + +## Category 3: NETWORK ISOLATION — 9 gaps + +| ID | Gap | Detection Method | Impact | +|----|-----|-----------------|--------| +| NI-01 | Database publicly accessible (PubliclyAccessible=true) | `aws rds describe-db-instances` → PubliclyAccessible=true | Direct internet exposure; attack surface includes all DB protocol ports | +| NI-02 | Security group allows 0.0.0.0/0 inbound on database port | `aws ec2 describe-security-groups --group-ids {{SG_ID}}` | Any IP can attempt connection; brute force exposure | +| NI-03 | Security group allows broad CIDR ranges (>/16) on database port | `aws ec2 describe-security-groups` → check CIDR prefix length | Overly permissive; lateral movement risk | +| NI-04 | Database NOT in private subnet (route table has internet gateway) | `aws ec2 describe-route-tables --filters Name=association.subnet-id,Values={{SUBNET_ID}}` | Traffic routes through internet even if not publicly accessible | +| NI-05 | No VPC endpoints for AWS services (S3, KMS, CloudWatch) | `aws ec2 describe-vpc-endpoints --filters Name=vpc-id,Values={{VPC_ID}}` | Service API calls traverse internet; data exfiltration path | +| NI-06 | Security group has unused/stale rules (referencing deleted resources) | `aws ec2 describe-security-groups` → cross-reference UserIdGroupPairs | Audit complexity; false sense of security | +| NI-07 | Multiple databases sharing same security group | Cross-reference VpcSecurityGroupId across instances | Blast radius: SG change affects all databases | +| NI-08 | No network ACL restrictions on database subnets | `aws ec2 describe-network-acls` → check subnet associations | Missing defense-in-depth layer | +| NI-09 | Database accessible from peered VPCs without explicit approval | Check VPC peering routes + SG rules referencing peered VPC CIDRs | Cross-account/cross-VPC access without explicit authorization | + +## Category 4: AUTHENTICATION & IDENTITY — 8 gaps + +| ID | Gap | Detection Method | Impact | +|----|-----|-----------------|--------| +| AI-01 | IAM database authentication NOT enabled | `aws rds describe-db-instances` → IAMDatabaseAuthenticationEnabled=false | Relies solely on username/password; no short-lived token rotation | +| AI-02 | Master user credentials not managed by Secrets Manager | `aws rds describe-db-instances` → MasterUserSecret absent | Static credentials; no automatic rotation; exposure risk | +| AI-03 | Secrets Manager rotation NOT configured | `aws secretsmanager describe-secret --secret-id {{SECRET_ID}}` → RotationEnabled=false | Stale credentials; no automatic password cycling | +| AI-04 | Secrets Manager rotation period > 90 days | `aws secretsmanager describe-secret` → RotationRules.AutomaticallyAfterDays > 90 | Compliance violation (PCI-DSS requires <=90 days) | +| AI-05 | Master username uses default value (admin, postgres, root) | `aws rds describe-db-instances` → MasterUsername | Predictable usernames simplify brute-force attacks | +| AI-06 | No IAM condition keys restricting database access by IP/VPC | IAM policy analysis | Overly broad IAM access; any network location can authenticate | +| AI-07 | RDS Proxy authentication not using IAM | `aws rds describe-db-proxies` → Auth[].AuthScheme | Proxy relies on static Secrets Manager credentials only | +| AI-08 | Kerberos authentication not configured (where applicable) | `aws rds describe-db-instances` → DomainMemberships empty | No Active Directory integration for enterprise SSO | + +## Category 5: ACCESS CONTROL & AUTHORIZATION — 7 gaps + +| ID | Gap | Detection Method | Impact | +|----|-----|-----------------|--------| +| AC-01 | Deletion protection DISABLED | `aws rds describe-db-instances` → DeletionProtection=false | Accidental or malicious deletion without safeguard | +| AC-02 | No resource-based policy on RDS resources | Check IAM policies for rds:* without resource constraints | Over-permissive IAM; any RDS action on any database | +| AC-03 | Cross-account snapshot sharing enabled | `aws rds describe-db-snapshot-attributes` → shared with other accounts | Data accessible to external accounts | +| AC-04 | Snapshot shared publicly (shared with "all") | `aws rds describe-db-snapshot-attributes` → "all" in restore list | Anyone with an AWS account can restore your data | +| AC-05 | No tag-based access control (ABAC) for RDS resources | IAM policy analysis → no aws:ResourceTag conditions | Cannot scope access by environment/team/classification | +| AC-06 | IAM policies use wildcard resources (Resource: *) for RDS actions | IAM policy analysis | Excessive privilege; any database affected | +| AC-07 | No SCP (Service Control Policy) restricting RDS actions in production | `aws organizations list-policies-for-target` | No organizational guardrails on database operations | + +## Category 6: AUDIT & LOGGING — 8 gaps + +| ID | Gap | Detection Method | Impact | +|----|-----|-----------------|--------| +| AL-01 | Database audit logging NOT enabled | `aws rds describe-db-instances` → EnabledCloudwatchLogsExports empty | No record of who accessed what data; compliance violation | +| AL-02 | CloudWatch log exports not configured | `aws rds describe-db-instances` → EnabledCloudwatchLogsExports missing audit/error/slowquery | Logs only on instance; lost if instance terminated | +| AL-03 | CloudWatch log group retention set to "Never Expire" | `aws logs describe-log-groups` → retentionInDays=null | Unbounded storage cost; no data lifecycle management | +| AL-04 | CloudWatch log group NOT encrypted with CMK | `aws logs describe-log-groups` → kmsKeyId absent | Log data (containing query text, usernames) encrypted with AWS-managed key only | +| AL-05 | No CloudWatch alarms on security-relevant events | `aws cloudwatch describe-alarms` → check for login failure, permission denied patterns | Security events go undetected | +| AL-06 | Enhanced Monitoring NOT enabled | `aws rds describe-db-instances` → MonitoringInterval=0 | No OS-level visibility; cannot detect anomalous process activity | +| AL-07 | Performance Insights NOT enabled | `aws rds describe-db-instances` → PerformanceInsightsEnabled=false | Cannot identify unusual query patterns indicative of compromise | +| AL-08 | Activity Streams not enabled (Aurora) | `aws rds describe-db-clusters` → ActivityStreamStatus != "started" | No near-real-time audit feed for SIEM integration | + +## Category 7: DATA PROTECTION & PRIVACY — 6 gaps + +| ID | Gap | Detection Method | Impact | +|----|-----|-----------------|--------| +| DP-01 | No final snapshot configured for deletion | `aws rds describe-db-instances` → check delete behavior | Data permanently lost on deletion without recovery option | +| DP-02 | Backup retention period < 7 days | `aws rds describe-db-instances` → BackupRetentionPeriod < 7 | Limited recovery window; potential data loss exposure | +| DP-03 | Backup retention period = 0 (automated backups disabled) | `aws rds describe-db-instances` → BackupRetentionPeriod = 0 | No point-in-time recovery; snapshot restore only option | +| DP-04 | No cross-region backup for production workloads | `aws rds describe-db-instance-automated-backups` → no cross-region replications | Regional failure = total data loss | +| DP-05 | Snapshot copy to S3 not configured for long-term retention | No native feature; check for Lambda/Step Functions automation | Backups expire per retention policy; no archive | +| DP-06 | Database contains PII without data classification tagging | `aws rds list-tags-for-resource` → no data-classification tag | Cannot enforce data handling policies; compliance gap | + +## Category 8: COMPLIANCE ALIGNMENT — 5 gaps + +| ID | Gap | Detection Method | Impact | +|----|-----|-----------------|--------| +| CA-01 | Database NOT tagged with compliance framework (HIPAA, PCI, SOC2) | `aws rds list-tags-for-resource` → no compliance tags | Cannot automate compliance reporting or policy enforcement | +| CA-02 | Database engine version has known CVEs (EOL or outdated) | `aws rds describe-db-engine-versions` → compare to latest | Unpatched vulnerabilities; active exploitation risk | +| CA-03 | Auto minor version upgrade DISABLED | `aws rds describe-db-instances` → AutoMinorVersionUpgrade=false | Security patches not applied automatically | +| CA-04 | Database in non-compliant region for data residency | `aws rds describe-db-instances` → AvailabilityZone region check | Data sovereignty violation; regulatory penalty risk | +| CA-05 | No AWS Config rules monitoring RDS security posture | `aws configservice describe-config-rules` → filter for rds-* rules | No continuous compliance monitoring; drift undetected | +