Entropy Covariance and Mutual Information for System Governance
Abstract Conventional monitoring and alerting frameworks in distributed systems often rely on threshold-based metrics, which can produce excessive false positives and fail to capture complex dependencies between signals. This paper proposes an applied framework for using information-theoretic measures—specifically mutual information and the broader notion of entropy covariance—to detect anomalies and support governance in event-driven infrastructure. The approach shifts focus from isolated metrics to the relationships between uncertainties: when two signals that normally exhibit dependency diverge, the system can treat this as an indicator of instability. Leveraging existing mathematical foundations from information theory, this work evaluates the operational value of dependency-aware monitoring on real event streams (e.g., Kafka topics, MongoDB change data capture). Rolling measures of mutual information and covariance are integrated into monitoring pipelines, and their effectiveness is compared against conventional thresholds. ...