Things That Don't Scale, The Software Edition

Seibel and Caldwell frame the episode around Paul Buchheit’s question: how can you get 90% of the benefit with 10% of the work? Buchheit began Gmail by reading his own email through an existing Google Groups interface, then added features as he needed them. When a coworker wanted to use it with his own email, Buchheit built that next step.

Caldwell recalls seeing Facebook’s early servers in a Santa Clara data center, with separate servers and databases for each university. The setup relied on copied code and school-specific instances, which avoided scaling one database across every school at once. Seibel describes a similar shortcut at Justin.tv: making popular pages static during traffic spikes so the video could keep playing.

Caldwell contrasts Friendster’s expensive calculation of users’ extended networks with Myspace’s simpler labels for friends and extended connections. Seibel describes Imeem using video files and a Flash video player to play audio, and Twitch using a brief delay to distribute celebrity streams across enough video servers. These workarounds delivered useful features before the team built more complete systems.

Twitch reduced video delivery costs through free peering with internet providers, and blocked access for viewers whose provider would not peer, asking them to contact the ISP. The team also built a community translation site where volunteers translated product strings. Seibel says multiple translations could be compared, and the approach eventually translated the product without paying a translation company.

Seibel shares a secondhand account that Google’s web indexing batch process failed as the web grew, leaving search results stale for months. He says this pressure led to MapReduce, which broke crawling and indexing into parallel tasks. His closing advice is to launch, observe what breaks, and fix those problems; teams earn the opportunity to build scalable systems by first making something people want.