Extracted Topic

Identifying, prioritizing, and engineering trade-offs among software architecture characteristics (the "-ilities") - chosen over any single characteristic (e.g. scalability alone) because the post's core argument is that the skill is ranking and trading off characteristics against each other, not mastering any one in isolation.

The Vital 20%

• The 12 characteristics grouped into families that are easy to confuse: availability/reliability/fault tolerance/recoverability (resilience family) vs. scalability/elasticity/performance (load family) vs. deployability/testability/agility/learnability (velocity family) vs. security (cross-cutting).
• The explicit-prioritization habit: writing an ADR that ranks 3-4 characteristics per system before building, instead of implicitly assuming "we need all of them."
• Converting adjectives to numbers via SLOs (MTBF/MTTR for availability, RTO/RPO for recoverability, p99 latency for performance) so trade-offs are testable, not vibes.
• The three core mechanisms that implement most of these characteristics in practice: circuit breakers (fault tolerance), liveness/readiness probes (recoverability/availability), and deploy pipelines with feature flags/canary/rollback (deployability).
• Recognizing the four recurring trade-off pairs: security↔performance, scalability↔testability, agility↔reliability, elasticity↔capacity-planning (the "elasticity is not capacity planning" trap).
• Verification over declaration: chaos engineering, load testing, and security scanning as the only way to confirm a claimed characteristic actually holds.
• Revisiting priorities as systems mature (launch-stage learnability/agility vs. scale-stage availability/performance).

20-Hour Plan

Session 1 - Explain what architectural characteristics are and why they're invisible until violated

• Core concept(s): Structural vs. operational concerns; ISO/IEC 25010 quality model; why functional tests don't catch "-ility" failures.
• Resource(s): Fundamentals of Software Architecture (Richards & Ford, O'Reilly) - Ch. 4 "Architecture Characteristics Defined"; ISO/IEC 25010:2011 overview page (iso.org or a summary from the ISO 25000 series portal).
• Hands-on task: Pick a real system you work on (or a known open-source project) and list which of the 12 characteristics it currently has NO explicit target for.
• 15-min review:
  • Can you state the difference between a functional requirement and an architectural characteristic in one sentence?
  • Name two characteristics from ISO/IEC 25010 not covered as headline items in this post.
  • What made your chosen system's gaps invisible until now?

Session 2 - Distinguish the resilience-family characteristics from each other precisely

• Core concept(s): Availability (MTBF/MTTR) vs. reliability (correctness) vs. fault tolerance (anticipated failure containment) vs. recoverability (RTO/RPO).
• Resource(s): Site Reliability Engineering (Beyer et al., Google, free at sre.google/sre-book) - Ch. 4 "Service Level Objectives" and Ch. 22 "Addressing Cascading Failures."
• Hands-on task: Write four one-sentence definitions in your own words, then classify 3 past incidents from your team's history (or hypothetical ones) into which of the four each one violated.
• 15-min review:
  • Give an example of a system that is available but not reliable.
  • What's the difference between RTO and RPO in a concrete scenario (e.g., database corruption)?
  • Why can two systems share an availability number but have opposite failure profiles?

Session 3 - Implement a circuit breaker and understand fault containment

• Core concept(s): Circuit breaker state machine (CLOSED/OPEN/HALF_OPEN); why retries alone can cascade failures.
• Resource(s): Martin Fowler's "CircuitBreaker" article (martinfowler.com/bliki/CircuitBreaker.html); resilience4j documentation (resilience4j.readme.io) for a production-grade reference implementation.
• Hands-on task: Reproduce the TypeScript circuit breaker from the source post, then intentionally break it (e.g., inject a bug where HALF_OPEN never trips back to OPEN) and observe the failure mode.
• 15-min review:
  • Draw the three states and transitions from memory.
  • What happens to downstream load if you remove the OPEN state entirely?
  • What's the purpose of resetTimeoutMs?

Session 4 - Implement liveness/readiness probes and understand graceful degradation

• Core concept(s): Liveness vs. readiness; degraded vs. down dependency states; caching health checks to avoid probe storms.
• Resource(s): Kubernetes docs, "Configure Liveness, Readiness, and Startup Probes" (kubernetes.io); the Python ReadinessProbe example in the source post.
• Hands-on task: Extend the post's Python readiness probe to add a third dependency and simulate one going DOWN mid-run; verify is_ready() correctly flips.
• 15-min review:
  • Why does a DEGRADED dependency still allow traffic but a DOWN one doesn't?
  • What production incident would a missing readiness probe (only a liveness probe) cause during a rolling deploy?
  • Why cache health check results, and what's the risk of caching too long?

Session 5 - Understand the load family: scalability, elasticity, performance

• Core concept(s): Vertical vs. horizontal scaling; elasticity as "scale down too"; percentile latency (p50/p95/p99) vs. averages.
• Resource(s): AWS Well-Architected Framework - "Performance Efficiency Pillar" (aws.amazon.com/architecture/well-architected); SRE book Ch. 6 "Monitoring Distributed Systems" (for percentile latency reasoning).
• Hands-on task: Take a latency dataset (real or synthetic, e.g., generate 1000 random samples with a long tail in Python/numpy) and compute p50/p95/p99 vs. mean; plot the gap.
• 15-min review:
  • Why does p99 matter more than average for user-perceived slowness?
  • Give one concrete reason horizontal scaling is preferred over vertical in cloud-native systems.
  • What's the practical difference between "scalable" and "elastic"?

Session 6 - Understand deployability mechanics and how they enable reliability, not oppose it

• Core concept(s): Feature flags decoupling deploy from release; blue-green/canary deployments; automated rollback triggers.
• Resource(s): Continuous Delivery (Humble & Farley, Addison-Wesley) - Ch. 1-2; DORA "State of DevOps Reports" (dora.dev) for the deployment-frequency/reliability correlation data.
• Hands-on task: Design (on paper or as a config sketch) a canary rollout for a hypothetical service with an automated rollback trigger tied to a specific error-rate threshold.
• 15-min review:
  • Why does DORA research treat deployment frequency as a performance indicator, not just a speed metric?
  • What's the difference between "deploy" and "release" when using feature flags?
  • What metric would trigger your automated rollback, and why that one?

Session 7 - Understand security as cross-cutting and its collision with performance

• Core concept(s): CIA triad; defense in depth; where to place expensive checks (boundaries vs. everywhere).
• Resource(s): NIST SP 800-53 overview (csrc.nist.gov) - focus on the control families relevant to defense in depth, not the full 500-page document; OWASP "Defense in Depth" cheat sheet-style overview.
• Hands-on task: Take one API endpoint you know well and map where you'd place authN, authZ, input validation, and audit logging - mark which are cheap (per-request) vs. expensive (boundary-only).
• 15-min review:
  • Name the three components of the CIA triad and one architectural control for each.
  • Why shouldn't full audit logging run on every internal call uniformly?
  • What's one measurable performance cost of adding TLS/mTLS between internal services?

Session 8 - Diagnose the four core trade-off pairs and the elasticity/capacity-planning trap

• Core concept(s): Security↔performance, scalability↔testability, agility↔reliability, elasticity≠capacity planning.
• Resource(s): The source blog post's "Trade-offs and Common Pitfalls" section (primary source, most direct treatment); Pact Foundation docs (docs.pact.io) for contract testing as the scalability↔testability mitigation.
• Hands-on task: For a system you know, pick two of the four trade-off pairs and write down (a) which side you're currently favoring, (b) evidence that's a conscious choice or an accident.
• 15-min review:
  • Why does horizontal scale make deterministic testing harder, and what pattern mitigates it?
  • Describe the "availability failure wearing an elasticity costume" scenario in your own words.
  • What's one reliability task teams commonly defer, and why is that a false economy?

Session 9 - Convert priorities into SLOs and write an ADR

• Core concept(s): SLOs/error budgets; ranking 3-4 characteristics explicitly; ADR format for recording the ranking.
• Resource(s): SRE book Ch. 4 "Service Level Objectives" (sre.google/sre-book); Michael Nygard's original ADR format post ("Documenting Architecture Decisions," cognitect.com/blog).
• Hands-on task: Write one real ADR for a system you own: rank your top 3-4 characteristics, state explicit SLOs for each, and name which characteristics you're consciously deprioritizing.
• 15-min review:
  • What makes "99.95% availability measured monthly" different from "highly available" as an engineering statement?
  • What happens organizationally when an error budget is exhausted?
  • Which characteristic did you deprioritize in your ADR, and why is that defensible?

Session 10 - Verify claimed characteristics under real failure and load, and plan for revisiting priorities

• Core concept(s): Chaos engineering; load testing; security scanning; revisiting the priority list as the system matures.
• Resource(s): Netflix Technology Blog, "The Netflix Simian Army" (netflixtechblog.com); Principles of Chaos Engineering (principlesofchaos.org, maintained by the original Netflix/Gremlin authors).
• Hands-on task: Design one chaos experiment for your system (e.g., "kill one instance behind the load balancer during peak synthetic load") including the hypothesis, blast radius limit, and success criteria.
• 15-min review:
  • What's the difference between "claiming" fault tolerance and "verifying" it?
  • Why should launch-stage priorities (learnability, agility) differ from scale-stage priorities (availability, performance)?
  • Name the one experiment you designed and what specific claim it tests.

What This Plan Deliberately Skips

Detailed treatment of learnability and testability as standalone disciplines (e.g., specific test-pyramid strategies, onboarding-doc design) and deep dives into any single cloud provider's implementation specifics (AWS Auto Scaling internals, Kubernetes HPA tuning parameters) - these are useful but secondary to the core skill of ranking and trading off characteristics deliberately.