WhoBuilt

A Conversation

Nov 11, 2018· 56 min· 30K views·Watch on YouTube

Speaker

Amazon CTO Werner Vogels joins Y Combinator to discuss Amazon’s engineering culture, the creation of AWS, product development, and lessons for startups.

From academia to Amazon

Before Amazon, Vogels spent 10 years as a research scientist at Cornell building distributed systems and had experience with both a successful startup and a failed one. He joined Amazon in 2004 after seeing that its technology challenges were far larger than he expected. He says the company had to develop its own practices for operating at scale and moving quickly.

Building reliable systems and independent teams

Vogels describes bringing more rigor to performance measurement, including tracking 99th and 99.9th percentile latency and connecting improvements to business outcomes. Amazon tested its resilience with surprise data center failures, then repeated the exercises until failover became automated. He also credits small teams, strong ownership, and room to accept technical debt when teams plan to pay it down.

How Amazon’s architecture led to AWS

Amazon moved from a brittle monolith to independent services with their own data stores, then split oversized services into smaller components with distinct scaling and reliability needs. As teams spent too much time managing infrastructure, Amazon built shared services and virtualized resources for its own engineers. S3 launched in spring 2006, followed by EC2 that fall, making storage and compute available through programmable services.

Customer-led product development

Vogels says about 95% of AWS features and services respond to direct customer requests. Teams launch with a small but reliable feature set, observe how customers use it, and adjust the roadmap; DynamoDB customers, for example, prioritized item-level access management over secondary indexes. Amazon’s working-backwards process uses a press release, FAQ, user experience document, and user manual outline to clarify the customer problem before engineering begins.

Security and startup choices

Vogels argues that protecting customer data must be a core responsibility from the start, supported by encryption and automated checks in continuous integration and deployment. He warns that treating AWS like a traditional data center misses the productivity benefits of higher-level services. Startups should also decide whether they prioritize rapid growth or sustainable operations, since the choice affects architecture, cost control, and how they use cloud services.

Takeaways

  • Measure tail latency, not just median performance, and connect improvements to customer and business outcomes.
  • Practice failure scenarios so recovery procedures work under real conditions and can be automated.
  • Give small teams ownership of services and the freedom to move quickly while tracking technical debt.
  • Learn from customer use before expanding a product’s feature set, while keeping the initial service dependable.
  • Build security into development from the beginning, and align cloud architecture with the company’s growth and cost goals.

Mentioned in this talk

Search inside this talk

What was claimed 19

Show all 19 facts

Related talks