Building Resilient AI-powered Software, And Planning For When Things Go Wrong
AIThis post was created with the assistance of artificial intelligence (AI).

TL;DR

Before you orderOffer from Amazon

Get privacy and security gear delivered free with Prime

  • Fast, free delivery on millions of items
  • Prime Video, Amazon Music and more included
  • Member-only deals all year
Start your free Prime trial Free trial for eligible customers · Cancel anytime
As an affiliate, we earn on qualifying purchases.

The Pragmatic Engineer published excerpts from Sam Newman’s 2026 book, Building Resilient Distributed Systems, covering disaster recovery and the reliability challenges of AI-powered software. The material outlines recovery targets and infrastructure patterns, while warning that model changes, resource shortages and vendor constraints complicate resilience planning.

The Pragmatic Engineer has published excerpts from Sam Newman’s new book, Building Resilient Distributed Systems, explaining how engineering teams can plan for major outages and reliability risks in AI-powered software. The excerpts set out disaster recovery targets and approaches, and discuss how changing models and constrained computing resources affect system resilience.

Newman’s book took three years to complete, one year longer than he initially planned, according to The Pragmatic Engineer. The newsletter said he reworked sections to account for the relevance of AI to resilient system design. It published material from two chapters, including a chapter on disaster recovery, with Newman’s permission.

The excerpt describes recovery point objectives (RPO) and recovery time objectives (RTO) as measurable requirements for disaster recovery. RPO sets the maximum acceptable period of potential data loss, based on how recently data was backed up or synchronized. RTO sets how long a system can remain unavailable while services are restored, computing resources are made available and traffic is redirected.

The report also outlines a range of recovery patterns, from backup and restore to pilot light, warm failover and multisite active-active deployments. The newsletter’s summary says the book addresses infrastructure availability amid GPU and CPU shortages, as well as resilience strategies for AI software, including observability, multiple models or vendors, transparent costs and regular vendor reviews.

At a glance
reportWhen: Published last week, according to The P…
The developmentThe Pragmatic Engineer published excerpts from Sam Newman’s new book on disaster recovery and resilience in distributed and AI-powered systems.

Recovery Targets Shape Outage Plans

Disaster recovery choices affect both service downtime and potential data loss. Setting explicit RTO and RPO targets gives teams a basis for choosing a recovery pattern and weighing its cost and operational complexity, Newman writes in the excerpt.

AI services add dependencies that can change over time. The newsletter’s account highlights model-provider resilience and model changes as issues for system designers. It also points to limited GPU and CPU availability, which can constrain attempts to restore workloads quickly. These factors make it necessary to plan around the capacity and services that vendors can actually provide.

The report says the book recommends approaches such as using multiple models and vendors and monitoring systems closely. Those are presented as ways to manage uncertainty; the excerpt does not provide performance comparisons or evidence that any one approach guarantees availability.

Amazon

disaster recovery backup software

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

From Local Failures to Site Outages

The excerpt places disaster recovery alongside earlier techniques for dealing with failures in individual service instances. It focuses on more extensive events, including fires, floods, power problems and failures in underlying cloud infrastructure that can disable large parts of a system at once.

Newman uses “site” as a general term for the location of computing infrastructure, whether that means a public-cloud region, a data center or another deployment location. The newsletter says the book considers both public- and private-cloud settings and flags cross-cloud-vendor issues for a later chapter.

The published excerpts are a selection from a book with 16 chapters, according to The Pragmatic Engineer. The newsletter identifies O’Reilly Media as the publisher and says the excerpts are used with permission.

““The recovery point objective (RPO) defines the maximum amount of time that can have passed since the data was last backed up or synchronized.””

— Sam Newman, in an excerpt from Building Resilient Distributed Systems

Amazon

AI system resilience tools

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

Costs and Capacity Remain System-Specific

The published material does not set universal RPO or RTO targets. Appropriate limits depend on each system’s needs, and the excerpt says reducing tolerated downtime or data loss raises cost and complexity.

The newsletter raises GPU and CPU shortages and vendor constraints as planning concerns, but the supplied report does not quantify the shortages, identify specific affected providers or describe particular outages. It also does not give comparative results for the recovery patterns or AI resilience tactics discussed.

It remains unclear from the excerpts how teams should choose among those options for a particular workload. The report presents concepts and approaches, not a single prescribed design or a claim that multi-vendor strategies eliminate service interruptions.

Amazon

multi-cloud failover solution

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

Further Guidance Sits in the Book

The newsletter presents the excerpts as a portion of Newman’s book, which contains 16 chapters. Its summary points readers to additional coverage of disaster recovery patterns, infrastructure availability and AI-powered software resilience across the selected chapters.

The report does not announce a release schedule for further excerpts or any upcoming industry milestone. For engineering teams applying the material, the immediate planning step described in the excerpt is to define acceptable data loss and downtime, then use those targets to evaluate recovery options and their operational costs.

Amazon

system monitoring and observability tools

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

Key Questions

What is the news development?

The Pragmatic Engineer published excerpts from Sam Newman’s Building Resilient Distributed Systems, covering disaster recovery and resilience concerns for AI-powered software.

What do RPO and RTO measure?

RPO sets the maximum period of potential data loss, based on the last backup or synchronization. RTO sets the acceptable time for restoring service after an incident.

Which disaster recovery approaches does the report mention?

The newsletter lists backup and restore, pilot light failover, warm failover and multisite active-active as patterns covered in the book.

The report’s summary points to model-provider resilience, models changing over time, observability, limited GPU and CPU capacity, and strategies such as using multiple models or vendors. It does not claim these measures guarantee reliability.

What remains unknown from the published excerpts?

The excerpts do not prescribe universal recovery targets, quantify infrastructure shortages or compare the effectiveness and costs of the listed approaches across specific systems.

Source: rss

COLUMBUS DAY / I

Columbus Day / Indigenous Peoples' Day Picks

As an affiliate, we earn on qualifying purchases.

You May Also Like

What Evidence Preservation Means After a Breach

Learn how to properly preserve digital evidence after a breach. Practical steps, recent advances, and legal tips to protect your organization.

What Containment Means During a Security Incident

Learn how containment limits damage during security breaches. Discover strategies, recent trends, and practical tips to respond effectively.

Parenting Signal Monitor: Albert Einstein’s Advice To His Son Is Applicable Wisdom For Parents Today Raising Resil

Einstein’s advice to his son is now seen as valuable wisdom for modern parents raising resilient children, according to recent discussions.

How to Communicate During a Security Event

Learn what to say, when to update people, and how to help them act during a security event without guessing or adding confusion.