AWS Architecture BlogAWS 架构博客
Architecting AI-powered resilience framework on AWS在 AWS 上架构 AI 驱动的弹性框架
When your production system goes down, you often discover the hard way that your resilience testing missed critical dependencies. Building an AI-powered resilience framework on AWS helps you find those weaknesses before your customers do.当生产系统宕机时,你往往会痛苦地发现弹性测试遗漏了关键依赖。在 AWS 上构建 AI 驱动的弹性框架,能帮你赶在客户发现之前找出这些薄弱环节。
Your systems don’t fail because your infrastructure isn’t resilient. They fail because resilience is assumed, not proven. Every deployment introduces new dependencies, every configuration change creates untested paths, and every gap between design intent and runtime behavior is a risk waiting to surface. In a world where customers expect always-on availability, the cost of discovering these weaknesses in production isn’t just technical. It’s measured in revenue lost, trust eroded, and events that were entirely preventable.你的系统不会因为基础设施缺乏弹性而失败。它们失败是因为弹性只是假设,并未得到验证。每次部署都会引入新的依赖,每次配置变更都会产生未经测试的路径,每个设计意图与运行时行为之间的差距都是等待浮现的风险。在客户期望始终可用的世界里,在生产环境中发现这些弱点的成本不仅仅是技术性的,还体现在收入损失、信任侵蚀和那些完全可以预防的事故上。
In this post, you’ll learn how to architect and implement a five-layer AI-powered resilience framework that automatically discovers dependencies, generates targeted experiments, and integrates with your existing Continuous Integration/Continuous Deployment (CI/CD) pipelines. First, we’ll explore the key challenges in resilience testing. Then, we’ll walk through the five-layer architecture that solves these challenges. Finally, we’ll show you how to implement this, with phased rollout guidance for pilot, expansion, and organization-wide deployment.在本文中,你将了解如何架构并实现一个五层 AI 驱动的弹性框架,它能够自动发现依赖、生成有针对性的实验,并与你现有的持续集成/持续部署(CI/CD)流水线集成。首先,我们将探讨弹性测试面临的关键挑战。然后,我们介绍解决这些挑战的五层架构。最后,我们展示如何实施该框架,并提供试点、扩展和组织级部署的分阶段推出指南。
Traditional resilience testing can take weeks because the information needed is naturally spread across architecture diagrams, runbooks, code repositories, and team knowledge that evolves with every deployment. Designing meaningful chaos experiments on top of that requires specialized expertise that most teams don’t have on hand. Here’s how this AI-powered framework discovers your infrastructure dependencies in hours and generates targeted experiments without requiring specialized knowledge.传统的弹性测试可能需要数周时间,因为所需信息天然分散在架构图、运行手册、代码仓库以及随每次部署不断演变的团队知识中。在此基础上设计有意义的混沌实验,需要大多数团队并不具备的专门技能。下面介绍这个 AI 驱动的框架如何在不要求专业知识的情况下,在数小时内发现你的基础设施依赖并生成有针对性的实验。
Combining AWS Resilience Hub, AWS Fault Injection Service, Amazon Bedrock AgentCore, and AWS Systems Manager work together to discover infrastructure dependencies in hours, tailor experiments to your specific architecture, and identify weaknesses before they affect customers.AWS Resilience Hub、AWS Fault Injection Service、Amazon Bedrock AgentCore 与 AWS Systems Manager 协同工作,可在数小时内发现基础设施依赖,根据你的具体架构定制实验,并在问题影响客户之前识别弱点。
With the next generation of AWS Resilience Hub now providing native dependency discovery and generative AI-powered failure mode analysis, this framework extends those capabilities by automating experiment generation, embedding resilience testing into your CI/CD pipelines, and creating a continuous validation loop through custom AI agents hosted on Amazon Bedrock AgentCore.随着新一代 AWS Resilience Hub 提供原生依赖发现和生成式 AI 驱动的故障模式分析,该框架通过自动化实验生成、将弹性测试嵌入 CI/CD 流水线,并借助托管在 Amazon Bedrock AgentCore 上的自定义 AI 代理创建持续验证闭环,扩展了这些能力。
Key concepts关键概念
Before diving into the architecture, here are the key terms used throughout this post:在深入架构之前,先介绍本文使用的关键术语:
- Chaos engineering — The discipline of experimenting on a system to build confidence in its ability to withstand turbulent conditions in production.混沌工程(Chaos engineering)—— 对系统进行实验,以建立对其在生产环境中抵御动荡条件能力的信心的学科。
- Mean time to resolution (MTTR) — The average time to restore service after a failure.平均恢复时间(MTTR)—— 故障后恢复服务的平均时间。
- Recovery Time Objective (RTO) — The maximum acceptable downtime for your application.恢复时间目标(RTO)—— 应用程序可接受的最大停机时间。
- Recovery Point Objective (RPO) — The maximum acceptable data loss, measured in time.恢复点目标(RPO)—— 以时间衡量的最大可接受数据丢失量。
- Shift-left — Testing earlier in the development cycle to catch issues before they reach production.左移(Shift-left)—— 在开发周期早期进行测试,以便在问题进入生产环境之前发现它们。
- Circuit breaker — An automated mechanism that detects service failures and helps prevent cascading outages by temporarily blocking requests to failing services.熔断器(Circuit breaker)—— 一种自动机制,用于检测服务故障,并通过暂时阻止对故障服务的请求来帮助防止级联中断。
- Canary deployment — A technique where changes roll out to a small percentage of users first to validate functionality before full deployment.金丝雀部署(Canary deployment)—— 一种先向一小部分用户推出变更以验证功能,然后再全面部署的技术。
Who this is for目标读者
This post targets cloud architects, DevOps engineers, and Site Reliability Engineering (SRE) teams responsible for system reliability. You should understand AWS services, distributed systems architecture, and CI/CD pipelines. While chaos engineering experience helps, it’s not required. This framework reduces the expertise barrier that traditionally prevented adoption.本文面向负责系统可靠性的云架构师、DevOps 工程师和站点可靠性工程(SRE)团队。你需要了解 AWS 服务、分布式系统架构和 CI/CD 流水线。虽然有混沌工程经验会有所帮助,但并非必需。该框架降低了传统上阻碍采用的技能门槛。
Resilience testing challenges弹性测试的挑战
When infrastructure changes happen quickly, documentation tends to lag behind. A routing change to the payment service might not get reflected in the architecture diagrams, leaving single points of failure (like a single-Availability Zone authentication dependency) undocumented and untested. The payment service calls a legacy authentication API that only runs in one Availability Zone, a critical single point of failure. But documentation doesn’t reflect this because someone made a “quick fix” three weeks ago and forgot to update the diagrams.当基础设施变化很快时,文档往往滞后。对支付服务的路由更改可能不会反映在架构图中,导致单点故障(例如单可用区身份验证依赖)未被记录和测试。支付服务调用一个仅在单个可用区运行的旧版身份验证 API,这是一个关键单点故障。但文档没有反映这一点,因为三周前有人做了“快速修复”,却忘记更新图表。
Distributed systems contain hundreds of interconnected components. Tracking every dependency manually becomes impractical when you deploy changes continuously. Documentation created last month already misses dozens of new dependencies.分布式系统包含数百个相互关联的组件。当你持续部署变更时,手动跟踪每个依赖变得不切实际。上个月创建的文档已经遗漏了数十个新依赖。
Resilience testing can be challenging without dedicated specialists to design meaningful experiments. Resilience testing is most effective when it’s tailored to your actual architecture and runs continuously. Generic fault injection and one-off test runs can leave gaps, especially as your system changes over time.如果没有专职专家来设计有意义的实验,弹性测试可能具有挑战性。当针对你的实际架构定制并持续运行时,弹性测试才最为有效。通用的故障注入和一次性的测试运行可能会留下空白,尤其是当你的系统随时间变化时。
The expertise barrier stops many organizations entirely. Effective chaos engineering demands understanding distributed systems architecture, failure mode analysis, experiment scope management, and safe experiment design. Without this specialized knowledge, you either avoid resilience testing or run superficial tests that miss critical vulnerabilities.技能门槛使许多组织完全止步。有效的混沌工程需要理解分布式系统架构、故障模式分析、实验范围管理和安全实验设计。没有这些专业知识,你要么避免弹性测试,要么运行无法发现关键漏洞的表面测试。
According to the 2024 IBM Security Services Benchmark Report, organizations with mature response capabilities reduce their MTTR by approximately 50% and achieve cost savings of up to 58% per event when compared to organizations with less mature capabilities. Yet Gartner research shows that 68% of organizations cite increasing system complexity as their reason for adopting chaos engineering, while 50% admit they weren’t prepared when failures occurred.根据 2024 年 IBM Security Services Benchmark Report,与能力较不成熟的组织相比,拥有成熟响应能力的组织将 MTTR 降低了约 50%,每次事件的成本节省高达 58%。然而,Gartner 研究显示,68% 的组织将系统复杂性增加作为采用混沌工程的原因,而 50% 的组织承认他们在故障发生时并未做好准备。
This framework automates discovery. Automated discovery reduces infrastructure mapping from weeks to hours, typically completing initial assessment in 2–4 hours for single-account environments with thousands of resources. Subsequent runs process only the changes tracked by AWS Config, so your architecture map stays current without manual effort. Agents hosted on AgentCore Runtime analyze your AWS CloudFormation templates, code repositories, and runtime behavior to identify every connection, including hidden dependencies that manual audits miss. Experiment templates analyze your specific architecture and produce targeted tests that validate your actual failure modes, removing the need for specialized chaos engineering expertise. Continuous integration into your CI/CD pipelines catches regressions before they reach production, shifting resilience from a one-time project into an ongoing practice embedded in your development workflow.该框架自动化发现。自动发现将基础设施映射从数周缩短到数小时,对于拥有数千资源的单账户环境,通常在 2–4 小时内完成初始评估。后续运行只处理 AWS Config 跟踪的变更,因此你的架构图无需人工努力即可保持最新。托管在 AgentCore Runtime 上的代理会分析你的 AWS CloudFormation 模板、代码仓库和运行时行为,识别每一个连接,包括人工审计遗漏的隐藏依赖。实验模板分析你的具体架构并生成有针对性的测试,验证你的实际故障模式,从而无需专门的混沌工程专业知识。与 CI/CD 流水线的持续集成可在回归进入生产环境之前捕获它们,将弹性从一次性项目转变为嵌入开发工作流的持续实践。
Solution overview解决方案概览
The framework addresses these gaps by automatically discovering infrastructure dependencies and continuously validating resilience as the system changes. AI agents, hosted on AgentCore Runtime for secure, scalable execution, discover system dependencies automatically, analyze architectural patterns, and create targeted experiments based on your actual risk profiles. Testing scales across your application portfolio while lowering the expertise barrier.该框架通过自动发现基础设施依赖并在系统变化时持续验证弹性来解决这些差距。AI 代理托管在 AgentCore Runtime 上,以实现安全、可扩展的执行,自动发现系统依赖、分析架构模式,并根据你的实际风险状况创建有针对性的实验。测试可在整个应用程序组合中扩展,同时降低技能门槛。

Figure 1. Five-layer AI-powered resilience architecture图 1. 五层 AI 驱动的弹性架构
Five-layer architecture diagram with AWS Resilience Hub — labeled “Next-Gen” with native dependency discovery and generative AI failure mode analysis — as the central orchestration hub. Layers 1–4 (Discovery, Test Generation, Experimentation, Gap Analysis) sit across the top, each connecting down to the hub. Layer 5 (Continuous Validation) sits below with CI/CD, drift detection, and dashboards. Three dashed feedback loops overlay the diagram: Gap Analysis feeds “Architecture updates” back to Discovery, Continuous Validation sends “Experiment learnings” up to Test Generation, and SSM Docs feeds “Validated recovery procedures” into FIS Templates within the Test Generation layer.五层架构图,以 AWS Resilience Hub(标记为“新一代”,具备原生依赖发现和生成式 AI 故障模式分析)作为中央编排中心。第 1–4 层(发现、测试生成、实验、差距分析)位于顶部,各自向下连接到中心。第 5 层(持续验证)位于下方,包含 CI/CD、漂移检测和仪表板。图中覆盖三个虚线反馈循环:差距分析将“架构更新”反馈给发现层,持续验证将“实验经验”发送给测试生成层,SSM 文档将“经验证的恢复流程”输入测试生成层内的 FIS 模板。
Each layer builds on the previous one, creating a comprehensive validation strategy. This architecture aligns with the AWS Well-Architected Reliability Pillar, specifically the Test reliability best practice area. Discovery maps your infrastructure. Test generation creates relevant experiments from that map. Experimentation executes those tests safely. Gap analysis identifies what needs fixing. Continuous validation helps verify that improvements persist as your systems evolve.每一层都建立在前一层之上,形成全面的验证策略。该架构符合 AWS Well-Architected 可靠性支柱,尤其是测试可靠性最佳实践领域。发现层映射你的基础设施。测试生成层根据该映射创建相关实验。实验层安全地执行这些测试。差距分析层识别需要修复的问题。持续验证层帮助确认改进在你的系统演进过程中持续有效。
Now that you understand the overall strategy, let’s examine each layer in detail, starting with how the discovery layer automatically maps your infrastructure.既然你已经了解了整体策略,让我们详细审视每一层,首先了解发现层如何自动映射你的基础设施。
Discovery layer发现层
The discovery layer forms the foundation by automatically identifying infrastructure components and their dependencies. The next generation of AWS Resilience Hub provides native dependency discovery that identifies AWS services, internal endpoints, and third-party endpoints your applications rely on. A custom agent deployed on Amazon Bedrock AgentCore (with read permissions to AWS APIs) extends this native discovery with code-level analysis, scanning your code repositories for hard-coded dependencies, connection strings, timeout configurations, and retry logic that infrastructure-level discovery alone cannot detect. A custom agent deployed on Amazon Bedrock AgentCore (with read permissions to AWS APIs) handles infrastructure discovery. Amazon Bedrock AgentCore Runtime provides dedicated MicroVM session isolation, supports long-running discovery sessions up to eight hours, and handles scaling and security without requiring you to manage infrastructure. The runtime’s built-in observability (traces, logs, and metrics) integrates natively with your existing Amazon CloudWatch dashboards without additional instrumentation.发现层通过自动识别基础设施组件及其依赖来奠定基础。新一代 AWS Resilience Hub 提供原生依赖发现,可识别你的应用程序依赖的 AWS 服务、内部端点和第三方端点。部署在 Amazon Bedrock AgentCore 上的自定义代理(具有 AWS API 只读权限)通过代码级分析扩展这一原生发现,扫描你的代码仓库以查找硬编码依赖、连接字符串、超时配置和重试逻辑,这些仅靠基础设施级发现无法检测到。部署在 Amazon Bedrock AgentCore 上的自定义代理(具有 AWS API 只读权限)负责基础设施发现。Amazon Bedrock AgentCore Runtime 提供专用的 MicroVM 会话隔离,支持长达八小时的长时间发现会话,并处理扩展和安全,无需你管理基础设施。该运行时内置的可观测性(跟踪、日志和指标)无需额外检测即可原生集成到现有的 Amazon CloudWatch 仪表板中。
The AgentCore-hosted agent queries services including Amazon Elastic Compute Cloud (Amazon EC2), Amazon Relational Database Service (Amazon RDS), AWS Lambda, Amazon DynamoDB, and Amazon Simple Storage Service (Amazon S3) to build comprehensive inventory. It analyzes AWS CloudFormation templates and Terraform configurations to understand your intended architecture, and accesses code repositories to identify hard-coded dependencies, connection strings, and timeout configurations in your applications. AWS Config provides configuration data and tracks changes over time. This discovery completes in 2–4 hours for environments with thousands of resources.由 AgentCore 托管的代理会查询 Amazon Elastic Compute Cloud (Amazon EC2)、Amazon Relational Database Service (Amazon RDS)、AWS Lambda、Amazon DynamoDB 和 Amazon Simple Storage Service (Amazon S3) 等服务,以构建全面的清单。它分析 AWS CloudFormation 模板和 Terraform 配置以了解你的预期架构,并访问代码仓库以识别应用程序中的硬编码依赖、连接字符串和超时配置。AWS Config 提供配置数据并随时间跟踪变更。对于拥有数千资源的环境,此发现可在 2–4 小时内完成。

Figure 2. AI-powered discovery workflow图 2. AI 驱动的发现工作流
Workflow diagram organized in two swim lanes. Lane 1 (blue) shows AWS Resilience Hub native infrastructure-level discovery querying AWS service APIs (Amazon EC2, Amazon RDS, AWS Lambda, Amazon S3, Elastic Load Balancing, Amazon DynamoDB) for service topology, endpoints, and Multi-AZ configuration. Lane 2 (yellow) shows an agent on Amazon Bedrock AgentCore performing code-level discovery — scanning repositories (CodeCommit, GitHub, GitLab via IAM permissions) and analyzing CloudFormation/Terraform templates for connection strings, timeouts, and circuit breakers. Both lanes feed into a combined dependency map, which flows into the AWS Resilience Hub assessment engine. Outputs include single points of failure, dependency maps (infrastructure + code), baseline resilience scores, configuration drift, and resilience gaps. AWS Config monitors configurations in parallel. A dashed feedback arrow loops from the assessment engine back to the AgentCore agent labeled “Architecture updates & learnings.” Initial mapping completes in 2–4 hours for typical single-account environments; subsequent runs process only changes tracked by AWS Config.工作流图按两个泳道组织。泳道 1(蓝色)显示 AWS Resilience Hub 原生基础设施级发现,查询 AWS 服务 API(Amazon EC2、Amazon RDS、AWS Lambda、Amazon S3、Elastic Load Balancing、Amazon DynamoDB)以获取服务拓扑、端点和 Multi-AZ 配置。泳道 2(黄色)显示 Amazon Bedrock AgentCore 上的代理执行代码级发现——扫描仓库(通过 IAM 权限访问 CodeCommit、GitHub、GitLab)并分析 CloudFormation/Terraform 模板中的连接字符串、超时和熔断器。两个泳道汇入组合依赖图,然后流入 AWS Resilience Hub 评估引擎。输出包括单点故障、依赖图(基础设施+代码)、基线弹性评分、配置漂移和弹性差距。AWS Config 并行监控配置。一条虚线反馈箭头从评估引擎循环回 AgentCore 代理,标记为“架构更新与经验”。对于典型的单账户环境,初始映射在 2–4 小时内完成;后续运行仅处理 AWS Config 跟踪的变更。
Test generation layer测试生成层
While the next generation of Resilience Hub includes a generative AI-powered failure mode assessment that identifies potential weaknesses through static analysis, this test generation layer converts those recommendations into executable AWS Fault Injection Service experiment templates, complete with safety guardrails, progressive scope expansion, and business impact scoring tailored to your specific architecture.虽然新一代 Resilience Hub 包含由生成式 AI 驱动的故障模式评估,可通过静态分析识别潜在弱点,但该测试生成层将这些建议转化为可执行的 AWS Fault Injection Service 实验模板,并带有安全护栏、渐进式范围扩展以及针对你的特定架构量身定制的业务影响评分。
Building on the discovered infrastructure, the test generation layer creates targeted chaos experiments for your specific architecture. The agent hosted on AgentCore Runtime uses Amazon Bedrock foundation models to analyze your infrastructure context, combined with the RTO, RPO, and availability targets you define in AWS Resilience Hub, to identify single points of failure and produce hypothesis-driven test scenarios aligned with your business requirements. Each hypothesis is scored by potential business impact, prioritizing experiments for customer-facing systems and components where architectural patterns indicate high-availability intent. Each experiment includes business impact scoring based on your application tier definitions in AWS Resilience Hub, architectural patterns (such as internet-facing load balancers and Amazon API Gateway endpoints), dependency analysis, and AWS resource tags. This makes sure experiments prioritize your customer-facing systems and highest-impact components. Using the code repository analysis from the discovery layer, the system detects when your applications use Amazon RDS Multi-AZ but lack proper connection retry handling, and designs database failover tests that validate your actual recovery mechanisms rather than generic network disruption tests.在已发现的基础设施之上,测试生成层针对你的特定架构创建有目标的混沌实验。托管在 AgentCore Runtime 上的代理使用 Amazon Bedrock 基础模型分析你的基础设施上下文,并结合你在 AWS Resilience Hub 中定义的 RTO、RPO 和可用性目标,识别单点故障并生成与业务需求一致的假设驱动测试场景。每个假设都根据潜在业务影响进行评分,优先考虑面向客户的系统和架构模式表明高可用性意图的组件。每个实验都包含基于你在 AWS Resilience Hub 中的应用程序层级定义、架构模式(如面向 Internet 的负载均衡器和 Amazon API Gateway 端点)、依赖分析和 AWS 资源标签的业务影响评分。这确保实验优先考虑面向客户的系统和影响最大的组件。系统利用发现层的代码仓库分析,检测到你的应用程序使用 Amazon RDS Multi-AZ 但缺少正确的连接重试处理时,会设计数据库故障转移测试,以验证你的实际恢复机制,而不是通用的网络中断测试。
For your production environments, implement a manual approval workflow where your infrastructure teams review experiment templates before execution. AWS Step Functions orchestrates approval gates. Step Functions is a workflow management service that coordinates multiple AWS services into serverless workflows.对于生产环境,请实施手动审批工作流,让基础设施团队在执行前审查实验模板。AWS Step Functions 负责编排审批门。Step Functions 是一项工作流管理服务,可将多个 AWS 服务协调到无服务器工作流中。
Experimentation layer实验层
After creating targeted experiments, the experimentation layer runs chaos tests with multi-layered safety guardrails on your infrastructure. AWS Fault Injection Service executes chaos tests with built-in safety mechanisms. Experiments start with minimal scope (affecting only 1% of your resources) and expand progressively based on your risk tolerance and validation results (for example, 1% → 5% → 10% → 25%). This follows progressive deployment strategies recommended in the AWS Well-Architected Reliability Pillar, similar to canary deployments where changes roll out incrementally to limit blast radius. Amazon CloudWatch alarms serve as stop conditions that halt experiments before they violate your Service Level Agreements (SLAs), which are contracts defining expected uptime and performance. Set alarm thresholds well below your SLA limits. If your SLA allows 1% error rate, configure stop conditions to trigger at 0.1%.创建有目标的实验后,实验层在你的基础设施上运行带有多层安全护栏的混沌测试。AWS Fault Injection Service 使用内置安全机制执行混沌测试。实验从最小范围开始(仅影响 1% 的资源),并根据你的风险承受能力和验证结果逐步扩展(例如 1% → 5% → 10% → 25%)。这遵循 AWS Well-Architected 可靠性支柱中推荐的渐进式部署策略,类似于金丝雀部署,增量推出变更以限制爆炸半径。Amazon CloudWatch 警报作为停止条件,在实验违反服务水平协议(SLA,即定义预期正常运行时间和性能的合同)之前中止实验。将警报阈值设置为远低于 SLA 限制。如果你的 SLA 允许 1% 的错误率,请将停止条件配置为在 0.1% 时触发。
Gap analysis layer差距分析层
After your experiments complete, the gap analysis layer processes results to identify weaknesses and prioritize remediation. AWS Resilience Hub correlates experiment outcomes with your resilience policies, categorizing gaps across architectural, operational, data protection, and testing dimensions. Each gap receives a priority score based on severity (how badly this violates your resilience policy), likelihood (how often this failure mode occurs), and business impact (the cost if this failure occurs in your environment).实验完成后,差距分析层处理结果以识别弱点并确定修复优先级。AWS Resilience Hub 将实验结果与你的弹性政策相关联,在架构、运营、数据保护和测试维度对差距进行分类。每个差距都会根据严重性(对弹性政策的违反程度)、可能性(该故障模式发生的频率)和业务影响(该故障在你的环境中发生时的成本)获得优先级评分。
Continuous validation layer持续验证层
The continuous validation layer integrates resilience testing into your development workflow. The right approach depends on your deployment velocity and testing goals.持续验证层将弹性测试集成到你的开发工作流中。正确的方法取决于你的部署速度和测试目标。
For most teams, a lightweight policy-as-code check (using tools like Open Policy Agent to validate Infrastructure as Code and Dockerfiles) runs in seconds and fits naturally in your CI/CD pipeline for every commit. This catches basic configuration issues, like missing health checks or single-AZ deployments, before code reaches staging.对于大多数团队,轻量级的策略即代码检查(使用 Open Policy Agent 等工具验证基础设施即代码和 Dockerfile)可在几秒钟内运行,并自然适用于每次提交的 CI/CD 流水线。它能在代码进入预发布环境之前捕获基本配置问题,例如缺少健康检查或单可用区部署。
Full resilience assessments are better suited as a pre-production gate, triggered on significant architectural changes rather than every commit. For routine deployments, lightweight resilience regression tests (validating a focused set of critical failure scenarios like database failover, Availability Zone loss, and circuit breaker activation) run automatically to catch unintended resilience degradation from code or configuration changes. This two-tiered approach gives you comprehensive safety validation for major changes and continuous regression coverage for everyday deployments, without slowing your pipeline. Your new code and infrastructure changes trigger automated resilience assessments that identify potential weaknesses during development rather than after deployment, embedding this shift-left strategy directly into your CI/CD workflow.全面的弹性评估更适合作为生产前门禁,在重大架构变更时触发,而不是每次提交都触发。对于常规部署,轻量级弹性回归测试(验证一组重点关键故障场景,如数据库故障转移、可用区丢失和熔断器激活)会自动运行,以捕获代码或配置变更导致的意外弹性退化。这种两层方法为重大变更提供全面的安全验证,为日常部署提供持续回归覆盖,同时不会拖慢流水线。你的新代码和基础设施变更会触发自动弹性评估,在开发期间而非部署后识别潜在弱点,从而将这种左移策略直接嵌入 CI/CD 工作流。
The policy-as-code check adds seconds to each pipeline run. Full resilience assessments add approximately 2–3 minutes per experiment.策略即代码检查为每次流水线运行增加几秒钟。全面弹性评估每个实验增加约 2–3 分钟。
AWS Config drift detection identifies manual changes that bypass your deployment pipelines, helping keep your architecture aligned with tested configurations.AWS Config 漂移检测可识别绕过部署流水线的手动更改,帮助你的架构与经过测试的配置保持一致。

Figure 3. Resilience testing in the CI/CD pipeline图 3. CI/CD 流水线中的弹性测试
Flowchart showing a CI/CD pipeline with two-tiered resilience testing. The top row shows the standard pipeline flow: Developer Commits Code → Build & Unit Tests → Deploy to Test Environment → Resilience Regression Tests (green hexagon gate, runs 3–5 critical scenarios in ~2–3 minutes on every deployment) → Integration Tests → Deploy to Staging. From staging, if an architectural change is detected, the flow drops to a second row: Full Resilience Assessment (orange hexagon gate, runs 15–20 experiments over ~15–45 minutes using AWS Resilience Hub, FIS, and Bedrock) → Manual Approval Gate → Deploy to Production → Continuous Monitoring. If no architectural change occurred, staging skips directly to the approval gate. Both resilience gates have failure paths (red) that block deployment and loop back to the developer. A legend and a side panel summarize the two tiers.流程图显示带两层弹性测试的 CI/CD 流水线。顶行显示标准流水线流程:开发人员提交代码 → 构建与单元测试 → 部署到测试环境 → 弹性回归测试(绿色六边形门,每次部署运行 3–5 个关键场景,约 2–3 分钟)→ 集成测试 → 部署到预发布环境。从预发布环境开始,如果检测到架构变更,流程进入第二行:全面弹性评估(橙色六边形门,使用 AWS Resilience Hub、FIS 和 Bedrock 运行 15–20 个实验,约 15–45 分钟)→ 手动审批门 → 部署到生产环境 → 持续监控。如果未发生架构变更,预发布环境直接跳到审批门。两个弹性门都有失败路径(红色),会阻止部署并循环回开发人员。图例和侧面板总结了两层。
Continuous improvement through feedback通过反馈实现持续改进
Experiment results feed back into the discovery and test generation layers, creating a continuous improvement cycle. When experiments reveal undocumented dependencies, the discovery layer updates your architecture map. When remediation actions successfully resolve failure patterns, Systems Manager automation documents capture these procedures for future use. The Bedrock agent analyzes experiment outcomes to refine hypothesis generation, deprioritizing consistently passing scenarios and focusing on emerging risk areas as your architecture evolves.实验结果反馈到发现层和测试生成层,形成持续改进循环。当实验揭示未记录的依赖时,发现层会更新你的架构图。当修复措施成功解决故障模式时,Systems Manager 自动化文档会捕获这些流程以备将来使用。Bedrock 代理分析实验结果以优化假设生成,降低持续通过场景的优先级,并随着架构演进聚焦于新兴风险领域。
Key benefits主要优势
Now that you’ve seen how each layer works together, let’s examine the concrete benefits this architecture brings to your organization.既然你已经了解了各层如何协同工作,下面让我们看看该架构为你的组织带来的具体好处。
Faster infrastructure discovery: Manual infrastructure discovery requires significant effort across distributed teams, including cataloging resources, tracing dependencies, and validating configurations. Automated discovery reduces this from weeks to hours by programmatically querying cloud service APIs, analyzing infrastructure-as-code templates, and mapping dependencies. After implementation, the framework scales across your application portfolio without proportionally increasing staffing.更快的基础设施发现:手动基础设施发现需要在分布式团队中付出大量努力,包括编目资源、跟踪依赖和验证配置。自动发现通过以编程方式查询云服务 API、分析基础设施即代码模板和映射依赖,将这一过程从数周缩短到数小时。实施后,该框架可在整个应用程序组合中扩展,而无需按比例增加人员。
Removes expertise barrier: Without chaos engineering specialists, you can implement resilience testing using automated scenarios. Start with the AWS Fault Injection Service Scenarios Library for common failure patterns, then expand with scenarios specific to your architecture and customize based on your application’s specific failure modes.消除技能门槛:即使没有混沌工程专家,你也可以使用自动化场景实施弹性测试。从 AWS Fault Injection Service Scenarios Library 中的常见故障模式入手,然后扩展针对你的架构的场景,并根据应用程序的具体故障模式进行定制。
Proactive risk identification: Automated discovery reveals critical single points of failure that manual audits consistently miss, including hard-coded endpoints, missing circuit breakers, and absent health checks. The system identifies vulnerabilities across your infrastructure and prioritizes them by business impact, so your teams can focus remediation on the highest-risk items first.主动风险识别:自动发现揭示人工审计经常遗漏的关键单点故障,包括硬编码端点、缺失的熔断器和缺失的健康检查。系统识别整个基础设施中的漏洞,并按业务影响排列优先级,以便你的团队优先修复最高风险项。
Faster recovery through automated remediation: Automated remediation reduces your mean time to resolution by removing manual intervention for common failure patterns in your environments. AWS Systems Manager automation documents codify recovery procedures discovered during chaos experiments. When Amazon CloudWatch alarms detect failure patterns in your systems, AWS Systems Manager automatically executes remediation actions, handling issues faster than manual response.通过自动修复实现更快恢复:自动修复消除了环境中常见故障模式的人工干预,从而缩短平均恢复时间。AWS Systems Manager 自动化文档将混沌实验期间发现的恢复流程固化。当 Amazon CloudWatch 警报检测到系统中的故障模式时,AWS Systems Manager 会自动执行修复操作,比人工响应更快地处理问题。
Continuous resilience validation: Integrating resilience assessments into your CI/CD pipelines catches regressions before production deployment, maintaining resilience as your system evolves rather than treating it as a one-time validation.持续弹性验证:将弹性评估集成到 CI/CD 流水线中,可在生产部署前捕获回归,随着系统演进保持弹性,而不是将其视为一次性验证。
Framework components框架组件
AWS Resilience Hub serves as the central orchestration layer, defining your resilience policies, running assessments, and tracking improvements. Define RTO and RPO targets for each application tier based on your business impact analysis.AWS Resilience Hub 作为中央编排层,定义弹性策略、运行评估并跟踪改进。根据业务影响分析,为每个应用程序层级定义 RTO 和 RPO 目标。
Amazon Bedrock delivers the AI capabilities that power discovery and test creation. AgentCore Runtime provides the managed hosting layer, handling session isolation, scaling, identity management, and observability, so your agent runs securely in production without infrastructure overhead.Amazon Bedrock 提供支持发现和测试创建的 AI 能力。AgentCore Runtime 提供托管托管层,处理会话隔离、扩展、身份管理和可观测性,使你的代理无需基础设施开销即可在生产中安全运行。
Deploy a custom agent on Amazon Bedrock AgentCore, a framework-agnostic managed runtime that supports agents built with Strands, LangChain, or custom Python. The agent uses Amazon Bedrock to analyze your infrastructure context against architectural patterns, AWS documentation, and best practices. AgentCore Runtime’s built-in tool gateway provides controlled, secure access to your AWS APIs during discovery.在 Amazon Bedrock AgentCore 上部署自定义代理,这是一个与框架无关的托管运行时,支持使用 Strands、LangChain 或自定义 Python 构建的代理。该代理使用 Amazon Bedrock 分析你的基础设施上下文,对照架构模式、AWS 文档和最佳实践。AgentCore Runtime 的内置工具网关在发现期间提供对 AWS API 的受控、安全访问。
AWS Fault Injection Service runs controlled chaos experiments with built-in safety mechanisms on your infrastructure. Pre-built actions cover common failure scenarios: terminating Amazon EC2 instances, injecting network latency, throttling API calls, failing over Amazon RDS databases, and disrupting Availability Zone connectivity in your environments.AWS Fault Injection Service 在你的基础设施上运行具有内置安全机制的受控混沌实验。预构建操作涵盖常见故障场景:终止 Amazon EC2 实例、注入网络延迟、限制 API 调用、故障转移 Amazon RDS 数据库以及中断环境中的可用区连接。
AWS Systems Manager extends your resilience framework beyond the default AWS Fault Injection Service actions. You can create custom automation documents that codify recovery procedures and transform manual runbooks into automated self-healing responses. When you build custom actions, you take on responsibility for proper rollback procedures and service state restoration. Design these with the same rigor you’d apply to your production runbooks.AWS Systems Manager 将你的弹性框架扩展到默认 AWS Fault Injection Service 操作之外。你可以创建自定义自动化文档,将恢复流程固化,并把手动运行手册转变为自动化自愈响应。在构建自定义操作时,你需要负责正确的回滚流程和服务状态恢复。请像对待生产运行手册一样严谨地设计这些操作。
AWS Config continuously monitors your resource configurations and tracks changes. AWS Config rules validate that your resources comply with resilience policies. For example, they verify your Amazon RDS instances use Multi-AZ deployment and confirm your Auto Scaling groups span multiple Availability Zones.AWS Config 持续监控你的资源配置并跟踪变更。AWS Config 规则验证你的资源是否符合弹性策略。例如,它们验证你的 Amazon RDS 实例使用 Multi-AZ 部署,并确认你的 Auto Scaling 组跨多个可用区。
Prerequisites前提条件
To implement this framework, you’ll need:要实现此框架,你需要:
- AWS account with administrative access.具有管理员访问权限的 AWS 账户。
- AWS Identity and Access Management (IAM) permissions for: AWS Resilience Hub, AWS Fault Injection Service, Amazon Bedrock AgentCore, AWS Systems Manager, and AWS Config. For each service, follow the principle of least privilege. Refer to the respective service documentation for minimum required permissions.以下服务的 AWS Identity and Access Management (IAM) 权限:AWS Resilience Hub、AWS Fault Injection Service、Amazon Bedrock AgentCore、AWS Systems Manager 和 AWS Config。对于每个服务,请遵循最低权限原则。有关最低所需权限,请参阅相应服务文档。
- The Amazon Bedrock AgentCore Starter Toolkitcreates broad dev/test permissions by default. Scope these down to least-privilege before production deployment.Amazon Bedrock AgentCore Starter Toolkit 默认会创建宽泛的开发/测试权限。在生产部署之前,请将这些权限缩小到最低权限。
- AWS Command Line Interface (AWS CLI) installed and configured. Basic understanding of AWS CloudFormation or Terraform. Non-critical application available for testing.已安装并配置 AWS Command Line Interface (AWS CLI)。基本了解 AWS CloudFormation 或 Terraform。有可用于测试的非关键应用程序。
- Estimated time: 4–6 hours for pilot implementation (with a team of 2–3 engineers who have working knowledge of your AWS environment).预计时间:试点实施需要 4–6 小时(由 2–3 名熟悉 AWS 环境的工程师组成团队)。
- Cost awareness: This implementation creates billable AWS resources including AWS Resilience Hub, AWS Fault Injection Service, Amazon Bedrock AgentCore, AWS Systems Manager, AWS Config, and Amazon CloudWatch. Follow the cleanup procedures after testing to avoid ongoing charges.成本提醒:此实施会创建可计费的 AWS 资源,包括 AWS Resilience Hub、AWS Fault Injection Service、Amazon Bedrock AgentCore、AWS Systems Manager、AWS Config 和 Amazon CloudWatch。测试后请遵循清理流程,避免持续产生费用。
Getting started入门
A phased rollout builds confidence before expanding scope if you’re new to chaos engineering.如果你是混沌工程新手,分阶段推出可在扩大范围之前建立信心。
Pilot phase (1–2 weeks, 2–3 engineers)试点阶段(1–2 周,2–3 名工程师)
- Select a non-critical application with well-understood architecture from your portfolio.从你的应用组合中选择一个架构清晰、非关键的应用程序。
- Enable AWS Config across the regions where your application runs.在应用程序运行的所有区域启用 AWS Config。
- Package your discovery agent code using Strands, LangChain, or custom Python.使用 Strands、LangChain 或自定义 Python 打包你的发现代理代码。
- Deploy the packaged agent on Amazon Bedrock AgentCore using the Amazon Bedrock AgentCore Starter Toolkit. AgentCore Runtime handles the compute, session management, and security so you can focus on the agent logic and discovery scope. The runtime maintains stateful working context (including tool state and memory) across the multi-step infrastructure discovery workflow.使用 Amazon Bedrock AgentCore Starter Toolkit 在 Amazon Bedrock AgentCore 上部署打包好的代理。AgentCore Runtime 负责计算、会话管理和安全,以便你可以专注于代理逻辑和发现范围。该运行时在多步骤基础设施发现工作流中维护有状态的工作上下文(包括工具状态和内存)。
- Run a baseline resilience assessment in AWS Resilience Hub to identify initial architectural gaps. Resilience Hub evaluates your architecture against Well-Architected best practices and establishes your starting resilience posture. The Bedrock agent you deploy in the next step builds on this baseline, discovering undocumented dependencies and generating targeted experiments that go beyond standard recommendations.在 AWS Resilience Hub 中运行基线弹性评估,以识别初始架构差距。Resilience Hub 根据 Well-Architected 最佳实践评估你的架构,并确定你的初始弹性态势。你在下一步部署的 Bedrock 代理以此基线为基础,发现未记录的依赖并生成超越标准建议的有针对性的实验。
Verify: Check the AWS Resilience Hub console for a completed assessment report showing baseline resilience scores and identified gaps. You should see a resilience score for each disruption type (AZ, Region, Application) and a list of recommended actions.验证:在 AWS Resilience Hub 控制台中查看已完成的评估报告,报告应显示基线弹性评分和已识别的差距。你应该看到每种故障类型(可用区、区域、应用程序)的弹性评分以及建议操作列表。
Expansion phase (4–6 weeks, cross-functional team)扩展阶段(4–6 周,跨职能团队)
- Expand to 3–5 applications across different tiers after validating safety with your pilot.在通过试点验证安全性后,扩展到不同层级的 3–5 个应用程序。
- Configure automated test creation to develop experiments specific to each of your application’s architectures.配置自动化测试创建,为每个应用程序的架构开发特定实验。
- Run controlled chaos experiments starting with 1% scope during your low-traffic periods.在低流量时段运行从 1% 范围开始的受控混沌实验。
Verify: Review the AWS Fault Injection Service console for experiment status “Completed” and check Amazon CloudWatch metrics to confirm the 1% scope was applied without triggering stop conditions.验证:在 AWS Fault Injection Service 控制台中查看实验状态是否为“已完成”,并检查 Amazon CloudWatch 指标,确认 1% 范围已应用且未触发停止条件。
- Analyze results to identify common patterns across your applications.分析结果,识别你的应用程序中的常见模式。
Enterprise scale (8–12 weeks, dedicated resilience team)企业级扩展(8–12 周,专职弹性团队)
- Expand resilience assessments in your CI/CD pipelines to comprehensive, multi-account validation.将 CI/CD 流水线中的弹性评估扩展为全面的多账户验证。
- Configure centralized reporting across organizational units. Set up cross-account experiment coordination.配置跨组织单元的集中式报告。设置跨账户实验协调。
- Distribute shared experiment templates across organizational units using AWS Organizations.使用 AWS Organizations 在组织单元之间分发共享实验模板。
- Deploy distributed worker pools for parallel testing across your multiple applications.部署分布式工作线程池,以便在多个应用程序之间进行并行测试。
- Implement executive dashboards using Amazon QuickSight to track resilience trends across your portfolio.使用 Amazon QuickSight 实施管理层仪表板,跟踪整个应用组合的弹性趋势。
When to move to enterprise patterns何时迁移到企业级模式
Consider adopting the enterprise deployment patterns in the next section when you meet any of these criteria:当你满足以下任一条件时,请考虑采用下一节中的企业级部署模式:
- You manage dozens of applications across multiple AWS accounts.你在多个 AWS 账户中管理数十个应用程序。
- Multiple business units require differentiated resilience policies based on varying risk tolerances, compliance requirements, or customer SLAs.多个业务部门根据不同的风险承受能力、合规要求或客户 SLA 需要差异化的弹性策略。
- Compliance requirements demand centralized audit trails across accounts.合规要求需要跨账户的集中审计跟踪。
- Your testing cadence exceeds what a single-account setup can handle.你的测试节奏超出了单账户设置所能处理的范围。
Design considerations for enterprise deployment企业级部署的设计考虑
When you’re ready to scale beyond initial pilots, consider these enterprise deployment patterns that handle the unique challenges of managing resilience testing across large organizations.当你准备扩展到初始试点之外时,请考虑这些企业级部署模式,它们可应对在大型组织中管理弹性测试的独特挑战。

Figure 4. Enterprise scalability patterns图 4. 企业级可扩展性模式
Diagram showing multi-account structure with native AWS Resilience Hub and AWS Organizations integration. A central management account connects to Production and Non-Production organizational units with distributed application accounts. The framework extends this native integration with cross-account FIS coordination (coordinated experiments across OUs) and shared FIS experiment templates distributed via Organizations. Priority-based scheduling shows Tier 1 Mission-Critical (weekly assessments, 100+ applications), Tier 2 Business-Critical (monthly assessments, 500+ applications), and Tier 3 Non-Critical (quarterly assessments, 1000+ applications). Distributed worker pools operate across multiple AWS regions (us-east-1, us-west-2, eu-west-1, ap-southeast-1) with centralized monitoring via Amazon EventBridge, Amazon QuickSight dashboards, Amazon CloudWatch logs, and Amazon SNS notifications.图显示多账户结构,具有原生 AWS Resilience Hub 和 AWS Organizations 集成。中央管理账户连接到包含分布式应用程序账户的生产和非生产组织单元。该框架通过跨账户 FIS 协调(跨 OU 的协调实验)和通过 Organizations 分发的共享 FIS 实验模板扩展了这一原生集成。基于优先级的调度显示:第 1 层任务关键型(每周评估,100+ 个应用程序)、第 2 层业务关键型(每月评估,500+ 个应用程序)和第 3 层非关键型(每季度评估,1000+ 个应用程序)。分布式工作线程池在多个 AWS 区域(us-east-1、us-west-2、eu-west-1、ap-southeast-1)运行,并通过 Amazon EventBridge、Amazon QuickSight 仪表板、Amazon CloudWatch 日志和 Amazon SNS 通知进行集中监控。
Multi-account architecture多账户架构
The next generation of AWS Resilience Hub supports modular resilience policies that you can assign at the system, user journey, or service level. Choose this multi-account strategy if you manage more than 100 applications across different business units. When you have thousands of applications, assessing your workloads simultaneously from a single account becomes impractical. Implement a hub-and-spoke model (a centralized architecture pattern where a central “hub” account manages shared services while “spoke” accounts contain individual workloads) with centralized resilience testing infrastructure and distributed application ownership. Deploy AWS Resilience Hub and AWS Fault Injection Service in your central management account. Your production accounts contain application workloads and local AWS Config recorders. The next generation of AWS Resilience Hub natively integrates with AWS Organizations, enabling central teams to define resilience policies and monitor posture across all accounts and regions from a single dashboard. This framework extends native multi-account visibility with cross-account experiment coordination and shared experiment templates across organizational units. For guidance on structuring your multi-account environment, see Best practices for a multi-account environment and the Organizing Your AWS Environment Using Multiple Accounts whitepaper.新一代 AWS Resilience Hub 支持模块化弹性策略,你可以在系统、用户旅程或服务级别进行分配。如果你在不同业务部门管理超过 100 个应用程序,请选择此多账户策略。当你有数千个应用程序时,从单个账户同时评估工作负载变得不切实际。实施中心辐射模型(一种集中式架构模式,中央“中心”账户管理共享服务,而“辐射”账户包含各个工作负载),并采用集中式弹性测试基础设施和分布式应用程序所有权。在中央管理账户中部署 AWS Resilience Hub 和 AWS Fault Injection Service。你的生产账户包含应用程序工作负载和本地 AWS Config 记录器。新一代 AWS Resilience Hub 原生集成 AWS Organizations,使中央团队能够在单个仪表板中定义弹性策略并监控所有账户和区域的态势。该框架通过跨账户实验协调和跨组织单元共享实验模板,扩展了原生多账户可见性。有关构建多账户环境的指导,请参阅 Best practices for a multi-account environment 和 Organizing Your AWS Environment Using Multiple Accounts 白皮书。
Tiered resilience policies分层弹性策略
Not every workload justifies the same resilience investment. Implement tiered resilience based on your business impact analysis. For example, mission-critical applications might target stricter recovery objectives (such as RTO < 15 minutes, RPO < 5 minutes, 99.99% availability) with comprehensive quarterly chaos experiments, while business-critical applications might set moderate targets (such as RTO < 1 hour, RPO < 15 minutes, 99.9% availability) with monthly validation, and non-critical applications might accept longer recovery windows with quarterly assessments. This tiered strategy optimizes costs by focusing resilience investments on your highest-impact workloads.并非每个工作负载都需要相同的弹性投入。根据业务影响分析实施分层弹性。例如,任务关键型应用程序可能设定更严格的恢复目标(如 RTO < 15 分钟、RPO < 5 分钟、99.99% 可用性),并进行全面的季度混沌实验;业务关键型应用程序可能设定中等目标(如 RTO < 1 小时、RPO < 15 分钟、99.9% 可用性),并进行月度验证;而非关键应用程序可能接受较长的恢复窗口,并进行季度评估。这种分层策略将弹性投资集中在影响最大的工作负载上,从而优化成本。
Security and compliance安全与合规
Under the AWS Shared Responsibility Model, AWS is responsible for security of the cloud (infrastructure), while you are responsible for security in the cloud (your configurations, data, and access management). The controls described below are your responsibility to configure and maintain. Encrypt your data at rest using AWS Key Management Service (AWS KMS) with customer-managed keys. Implement separate encryption keys per environment to limit the scope of unauthorized access in your infrastructure. Enforce Transport Layer Security 1.3 (TLS 1.3), a cryptographic protocol that secures data transmission, for data in transit. AWS CloudTrail delivers complete audit trails of your resilience operations. Automated compliance monitoring through AWS Security Hub (which continuously evaluates resources against standards including CIS AWS Foundations Benchmark, PCI DSS, and AWS Foundational Security Best Practices), combined with AWS Config conformance packs, streamlines evidence collection for Service Organization Control 2 Type II (SOC 2 Type II), International Organization for Standardization 27001 (ISO 27001), Payment Card Industry Data Security Standard (PCI DSS), and Digital Operational Resilience Act (DORA), an EU regulation requiring financial institutions to test operational resilience.根据 AWS 共享责任模型,AWS 负责云本身的安全(基础设施),而你负责云中的安全(你的配置、数据和访问管理)。以下描述的控件由你负责配置和维护。使用 AWS Key Management Service (AWS KMS) 和客户托管密钥加密静态数据。为每个环境实施单独的加密密钥,以限制基础设施中未授权访问的范围。对传输中的数据强制执行传输层安全性 1.3 (TLS 1.3)(一种保护数据传输的加密协议)。AWS CloudTrail 提供你的弹性操作的完整审计跟踪。通过 AWS Security Hub(持续根据 CIS AWS Foundations Benchmark、PCI DSS 和 AWS Foundational Security Best Practices 等标准评估资源)进行自动化合规监控,结合 AWS Config 一致性包,可简化 Service Organization Control 2 Type II (SOC 2 Type II)、International Organization for Standardization 27001 (ISO 27001)、Payment Card Industry Data Security Standard (PCI DSS) 和 Digital Operational Resilience Act (DORA)(一项要求金融机构测试运营弹性的欧盟法规)的证据收集。
AI agent security: The Amazon Bedrock AgentCore-hosted agent operates with scoped IAM roles following least-privilege principles. AgentCore Runtime’s MicroVM session isolation makes sure that each discovery session runs in a dedicated, ephemeral environment with no cross-session data leakage. The agent’s infrastructure access is read-only during discovery and cannot modify resources. Amazon Bedrock interactions occur within your AWS account boundary, and no customer data is used for training purposes. For additional guardrails, you can configure Amazon Bedrock Guardrails to filter agent outputs and enforce responsible AI policies.AI 代理安全:由 Amazon Bedrock AgentCore 托管的代理使用遵循最低权限原则的受限 IAM 角色运行。AgentCore Runtime 的 MicroVM 会话隔离确保每个发现会话在专用、临时环境中运行,不会发生跨会话数据泄漏。代理在发现期间对基础设施的访问是只读的,不能修改资源。Amazon Bedrock 交互发生在你的 AWS 账户边界内,并且不会使用客户数据进行训练。如需额外护栏,你可以配置 Amazon Bedrock Guardrails 来过滤代理输出并强制执行负责任 AI 策略。
Note: This framework supports your compliance efforts but does not guarantee compliance with any regulatory framework. Compliance is a shared responsibility. Consult your legal and compliance teams and qualified auditors to validate that your implementation meets your specific regulatory obligations.注意:此框架支持你的合规工作,但不保证符合任何监管框架。合规是共同责任。请咨询你的法律和合规团队以及合格的审计人员,以验证你的实现是否符合你的具体监管义务。
Addressing common concerns解决常见问题
Progressive scope expansion and automated stop conditions help you verify experiments reveal weaknesses without causing outages in your environments. Starting with 1% of your resources limits potential impact to statistically insignificant traffic. Organizations have validated this strategy using progressive scope expansion and automated stop conditions.渐进式范围扩展和自动化停止条件帮助您验证实验是否揭示弱点,而不会在您的环境中造成中断。从您资源的1%开始,可将潜在影响限制在统计上不显著的流量水平。组织已验证这一策略,即使用渐进式范围扩展和自动化停止条件。
Automated scenarios remove the expertise barrier. The AI-powered analysis examines your specific architecture to develop targeted experiments rather than demanding you design tests manually. AWS CloudTrail provides comprehensive audit trails of your chaos experiments, which can help support due diligence documentation for resilience testing. This evidence can contribute to your compliance documentation for SOC 2, ISO 27001, and other frameworks relevant to your organization. For financial services, the framework supports DORA scenario testing requirements.自动化场景消除了专业知识的障碍。AI驱动的分析会检查您的特定架构,以制定有针对性的实验,而不是要求您手动设计测试。AWS CloudTrail为您的混沌实验提供全面的审计跟踪,这有助于支持韧性测试的尽职调查文档。这些证据可纳入您的合规文档中,适用于SOC 2、ISO 27001以及与您组织相关的其他框架。对于金融服务,该框架支持DORA场景测试要求。
Clean up清理
To avoid ongoing charges, delete the resources you created during implementation:为避免持续产生费用,请删除您在实施期间创建的资源:
- Delete your AWS Fault Injection Service experiment templates. Warning: This permanently removes experiment history and results. Consider exporting experiment data before deletion if you need to retain this information for compliance or analysis purposes.删除您的AWS Fault Injection Service实验模板。警告:这将永久删除实验历史和结果。如果您需要保留这些信息以用于合规或分析目的,请考虑在删除前导出实验数据。
- Remove your Amazon Bedrock AgentCore agent deployment, runtime endpoints, and associated configurations.移除您的Amazon Bedrock AgentCore代理部署、运行时端点及相关配置。
- Delete your AWS Systems Manager automation documents. Warning: This removes your automation runbooks permanently. Back up any custom runbooks you may want to reuse in future implementations.删除您的AWS Systems Manager自动化文档。警告:这将永久删除您的自动化运行手册。请备份您可能希望在将来实施中复用的任何自定义运行手册。
- Remove your Amazon CloudWatch alarms created for stop conditions.移除您为停止条件创建的Amazon CloudWatch警报。
- Delete any AWS Step Functions state machines created for approval workflows.删除为审批工作流创建的任何AWS Step Functions状态机。
- Remove any Open Policy Agent configurations deployed for IaC validation.移除为IaC验证部署的任何Open Policy Agent配置。
- Delete Amazon QuickSight dashboards created for resilience tracking (if applicable). Warning: This removes resilience trend data and operational insights. Export dashboard data or save analysis snapshots before deletion.删除为韧性跟踪创建的Amazon QuickSight仪表板(如适用)。警告:这将删除韧性趋势数据和运营洞察。请在删除前导出仪表板数据或保存分析快照。
- Remove Amazon EventBridge rules and Amazon SNS topics created for notifications (if applicable).移除为通知创建的Amazon EventBridge规则和Amazon SNS主题(如适用)。
Your AWS Resilience Hub and AWS Config continue incurring minimal costs. Consider retaining them for ongoing resilience validation.您的AWS Resilience Hub和AWS Config会继续产生少量费用。请考虑保留它们以进行持续的韧性验证。
Conclusion结论
In this post, I showed you how to build a five-layer AI-powered resilience framework that automatically discovers dependencies, generates targeted experiments, and integrates with your CI/CD pipelines. Building on the next generation of AWS Resilience Hub’s native dependency discovery and generative AI-powered failure mode analysis, this framework adds automated experiment generation through Amazon Bedrock AgentCore, controlled execution via AWS Fault Injection Service, and continuous CI/CD validation through AWS Systems Manager, creating an end-to-end resilience pipeline that goes from discovery to prevention.在本文中,我展示了如何构建一个五层AI驱动的韧性框架,该框架可自动发现依赖关系、生成有针对性的实验,并与您的CI/CD管道集成。该框架基于下一代AWS Resilience Hub的原生依赖发现和生成式AI驱动的故障模式分析,通过Amazon Bedrock AgentCore增加了自动化实验生成,通过AWS Fault Injection Service实现受控执行,并通过AWS Systems Manager进行持续CI/CD验证,从而打造一个从发现到预防的端到端韧性管道。
The next frontier is shifting even earlier, scanning your Infrastructure as Code and application code for resilience anti-patterns before a single resource is deployed. When your CI/CD pipeline can flag a missing circuit breaker or a single-AZ dependency at the pull request stage, prevention becomes truly proactive.下一个前沿领域将更进一步提前,在部署任何资源之前,扫描您的基础设施即代码和应用程序代码中的韧性反模式。当您的CI/CD管道能够在拉取请求阶段标记缺失的断路器或单可用区依赖时,预防就真正变得主动了。
The progressive strategy (starting with your single application pilot, expanding to multiple applications, then scaling organization-wide) builds confidence while demonstrating value at each phase. Organizations often realize positive return on investment through prevented events and reduced MTTR.渐进式策略(从单个应用程序试点开始,扩展到多个应用程序,然后在组织范围内推广)在展示每个阶段价值的同时建立信心。组织往往通过预防事件和缩短MTTR来实现正向投资回报。
The framework makes resilience testing accessible to you, removing the expertise barrier that traditionally prevented adoption. Start with the pilot phase outlined earlier and expand to your mission-critical systems as confidence builds.该框架使韧性测试对您而言更加便捷,消除了传统上阻碍采用的专家知识壁垒。从前面概述的试点阶段开始,随着信心的建立,扩展到您关键任务系统。
Ready to get started? Pick a non-critical application from your portfolio, deploy the discovery agent, and run your first assessment this week. Then share what you found. We’d love to hear about the hidden dependencies your team uncovered.准备好开始了吗?从您的应用组合中挑选一个非关键应用程序,部署发现代理,并在本周运行您的首次评估。然后分享您的发现。我们很乐意了解您的团队发现的隐藏依赖关系。
Have you implemented chaos engineering in your organization? What challenges did you face? Share your experience in the comments below.您是否在组织中实施了混沌工程?您遇到了哪些挑战?欢迎在下方评论中分享您的经验。
Next steps后续步骤
For the latest Resilience Hub capabilities including native dependency discovery and generative AI-powered failure mode analysis, see Introducing the next generation of AWS Resilience Hub and the next generation documentation.要了解最新的Resilience Hub功能,包括原生依赖发现和生成式AI驱动的故障模式分析,请参阅《Introducing the next generation of AWS Resilience Hub》以及下一代文档。
Start with the resources most relevant to where you are in your resilience journey:从与您韧性之旅所处阶段最相关的资源开始:
If you’re just getting started:如果您刚刚开始:
- For more information about resilience policies and assessment capabilities, see AWS Resilience Hub documentation.有关韧性策略和评估功能的更多信息,请参阅AWS Resilience Hub文档。
- For hands-on experience with chaos engineering, see AWS Fault Injection Service Workshop.如需获得混沌工程的实践经验,请参阅AWS Fault Injection Service Workshop。
If you’re ready to build:如果您准备构建:
- For information about creating fault injection experiments using natural language through Amazon Bedrock, see Chaos engineering made clear: Generate AWS FIS experiments using natural language through Amazon Bedrock.有关通过Amazon Bedrock使用自然语言创建故障注入实验的信息,请参阅《Chaos engineering made clear: Generate AWS FIS experiments using natural language through Amazon Bedrock》。
- For information about assessing application resilience with AWS Resilience Hub and AWS CodePipeline, see Continually assessing application resilience with AWS Resilience Hub and AWS CodePipeline.有关使用AWS Resilience Hub和AWS CodePipeline评估应用程序韧性的信息,请参阅《Continually assessing application resilience with AWS Resilience Hub and AWS CodePipeline》。
- For additional fault injection experiment templates, see AWS Fault Injection Service Template Library.有关其他故障注入实验模板,请参阅AWS Fault Injection Service Template Library。
If you’re scaling to production: 6. For information about hosting production AI agents at scale, see Amazon Bedrock AgentCore 7. For sample agent code and deployment templates, see the Amazon Bedrock AgentCore Starter Toolkit.如果您要扩展到生产环境:6. 有关大规模托管生产AI代理的信息,请参阅Amazon Bedrock AgentCore。7. 有关示例代理代码和部署模板,请参阅Amazon Bedrock AgentCore Starter Toolkit。
Systems will face failures. Discover and fix weaknesses before your customers experience them by shifting your resilience testing from reactive response to proactive prevention.系统会面临故障。通过将韧性测试从被动响应转变为主动预防,在客户遇到问题之前发现并修复弱点。
If you have questions or need guidance implementing the framework, contact AWS or reach out to your AWS Solutions Architect.如果您有任何问题或在实施该框架时需要指导,请联系AWS或联系您的AWS解决方案架构师。