Skip to content

CCM-14044 Setting prod defaults#439

Open
aidenvaines-cgi wants to merge 3 commits intomainfrom
CCM-14044_AddingAnomalyAlarm
Open

CCM-14044 Setting prod defaults#439
aidenvaines-cgi wants to merge 3 commits intomainfrom
CCM-14044_AddingAnomalyAlarm

Conversation

@aidenvaines-cgi
Copy link
Contributor

Description

Adding Anomaly alarms for Event Subscriptions
Bumping EventPub module to add Anomaly alarms for Event Publishing

Context

Type of changes

  • Refactoring (non-breaking change)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would change existing functionality)
  • Bug fix (non-breaking change which fixes an issue)

Checklist

  • I am familiar with the contributing guidelines
  • I have followed the code style of the project
  • I have added tests to cover my changes
  • I have updated the documentation accordingly
  • This PR is a result of pair or mob programming
  • If I have used the 'skip-trivy-package' label I have done so responsibly and in the knowledge that this is being fixed as part of a separate ticket/PR.

Sensitive Information Declaration

To ensure the utmost confidentiality and protect your and others privacy, we kindly ask you to NOT including PII (Personal Identifiable Information) / PID (Personal Identifiable Data) or any other sensitive data in this PR (Pull Request) and the codebase changes. We will remove any PR that do contain any sensitive information. We really appreciate your cooperation in this matter.

  • I confirm that neither PII/PID nor sensitive data are included in this PR and the codebase changes.

@aidenvaines-cgi aidenvaines-cgi requested a review from a team as a code owner March 3, 2026 16:21
variable "event_publishing_anomaly_band_width" {
type = number
description = "The width of the anomaly detection band. Higher values (e.g. 4-6) reduce sensitivity and noise, lower values (e.g. 2-3) increase sensitivity. Recommended: 2-4."
default = 5
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think 5 might be too high? From what I understand it's based on standard deviation. 5 standard deviations is like 99.9999% of all data


alarm_name = "${local.csi}-subscriber-anomaly"
alarm_description = "ANOMALY: Detects anomalous patterns in messages published to the SNS fanout topic"
comparison_operator = "LessThanLowerOrGreaterThanUpperThreshold"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

About LessThanLower - do we want to alert on drops too? I think there'll be periods when not in use?

return_data = true

metric {
metric_name = "NumberOfMessagesPublished"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want NumberOfNotificationsDelivered here instead?

metric_query {
id = "ad1"
expression = "ANOMALY_DETECTION_BAND(m1, ${var.event_anomaly_band_width})"
label = "NumberOfMessagesPublished (expected)"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NumberOfNotificationsDelivered?

Comment on lines +109 to +112
validation {
condition = var.event_anomaly_band_width >= 2 && var.event_anomaly_band_width <= 10
error_message = "Band width must be between 2 and 10"
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needed here? Is the anomaly threshold particularly important here? Again, I think 5 is already big

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants