Lee traces four stages of working with AI, from copying code out of ChatGPT to agents that run in the background. These agents can use their own cloud computers, keep memory, respond to events such as Slack messages, and decide when an update is worth sending.
Stanford CS146S: How always-on agents work
#Agent Architecture#Workflow Automation#Multi-Agent Systems#Context Engineering#AI Cost Optimization#Agent Memory#Computer-Use Agents#Agent Security#Evals
Speaker
Lee, a founder or engineer at SpaceX by context, explains how to build always-on agents during Stanford CS146S; the transcript does not give his full name or exact role.
The agent and its virtual computer can sleep between tasks, with conversation history, memories, and routines saved for later. Lee describes routing incoming events, prioritizing direct user messages, and using durable workflows such as Temporal so tasks can resume after failures.
The client and server communicate through a single “send to user” tool, while heavier work runs on the server. Dynamic tool discovery saves context space, and subagents handle expensive or lengthy work without filling the main agent’s conversation. For browser tasks, agents can inspect HTML or accessibility data before resorting to screenshots and clicks.
Lee recommends reviewing commands, treating emails and webhooks as untrusted input, and requiring approval for irreversible actions. Prompt caching, file-based notes, and periodic summaries help control context and cost over long sessions. He also describes using evaluations to fix product issues and train models, while pointing to open challenges in long-term memory, security, and when an agent should speak up.
Takeaways
- Keep the agent loop on a server so its computer can be restarted independently and its state can persist.
- Use durable workflows for long tasks so failures do not force the agent to repeat completed steps.
- Keep the main conversation lean by discovering tools when needed and delegating bulky work to helpers.
- Treat incoming content as untrusted, review commands, and require approval before irreversible actions.
- Turn agent failures into evaluations, then use the results to improve the harness and future models.
Mentioned in this talk
Search inside this talk
What was claimed 12
- feature
SpaceX: Some of the discussed always-on agent work is in progress there“some of the things we'll talk about are in progress at SpaceX”
YouTube Unverified1 source· Talk
- feature
Grokbot: Always-on agents can run in the background with their own computer and memory“always-on agents. They can run ambiently in the background, with their own computer and memory”
YouTube Unverified1 source· Talk
- integrates with
“we have this view in both Slack and soon Microsoft Teams”
5:03inGrok Bot for AdminsYouTube Verified5 sources· Talk
- featureGrokbot: Supports plugins that use skills or servers
“In Grokbot, there's a whole system of plugins”
YouTube Unverified1 source· Talk
- featureGrokbot: The Linux computer can be snapshotted, shut down, and restarted
“makes it easy to snapshot the machine's state, shut it down, and start it back up”
YouTube Unverified1 source· Talk
- usesGrokbot uses Firecracker
“It runs a Firecracker virtual machine”
YouTube Unverified1 source· Talk
- feature
Temporal: Durable workflows resume after a failure without replaying completed steps“After a failure, it picks right up at step three”
YouTube Unverified1 source· Talk
- uses
“we use an open-source tool called Temporal”
YouTube Unverified1 source· Talk
- featureGrokbot: Uses a thin-client, thick-server architecture with one client-server communication tool, send to user
“there's only one tool that communicates between the client and the server: send to user”
YouTube Unverified1 source· Talk
- feature
OpenClaw: Early always-on agent usage had many prompt-cache misses“There were a lot of cache misses”
YouTube Unverified1 source· Talk
- role at
“One principle we had at Cursor, and now at SpaceX”
YouTube Unverified1 source· Talk
- previously at
“One principle we had at Cursor, and now at SpaceX”
YouTube Unverified1 source· Talk
