1. The New Paradigm of AI Agent Management [0]

thumbnail The rise of AI agents means that individuals must become skilled managers of these agents to succeed, requiring new technical expertise and efficient decision-making.

1.1. The Role of an Agent Manager [0]

  1. The ability to run teams of AI agents will allow individuals to outperform others in the current AI era [0].
  2. By the end of this discussion, listeners will understand cloud agents, how to set up automations, and how to create a software factory that produces software 24/7 [2].
  3. Becoming an "agent pro" involves mastering three key areas: running Cloud Agents, implementing automations, and shipping faster [7].
  4. The new professional role is that of an agent manager, and individuals need to excel at it, regardless of their background (e.g., VC, founder, IC, parent, or college student) [13].

1.2. Background and Current Operations [16]

  1. Ryan Carson, a founder and CEO for 25 years, is currently working on his fourth company, Untangle, an AI divorce agent for family law firms [19].
    1. His previous company, Treehouse, had 110 employees and taught a million people to code, providing extensive experience in managing organizations [21].
    2. Untangle currently operates with one employee (Ryan) but is scaling rapidly, with revenue projected to quadruple this month [23].
  2. Ryan has learned to scale himself and will explain how to manage many agents effectively, including checking their work, automating tasks, and working from a phone [29].
    1. He performs almost 50% of his work, or even more, from his phone [30].
  3. The methods and tools for managing AI agents are constantly evolving, changing every 3 to 6 months, so hands-on experience is crucial for learning [33].

1.3. Desk Setup for Agent Management [41]

  1. Ryan's home office setup in Connecticut includes a large 52-inch Dell monitor, replacing multiple smaller screens, to facilitate multitasking with many agents [43].
    1. He organizes his monitor into eight screens to manage multiple agents simultaneously [49].
    2. He uses a paper-based UG Monk system for to-dos and a vertical Razer mouse to prevent wrist pain [52].
    3. A button connected to Whisper Flow is used for most of his tasks [57].
  2. His screen layout includes:
    1. Slack in the top left for agent notifications and updates [59].
    2. 1Password always open to securely manage production write keys, which are never directly given to agents [63].
      1. Agents must request keys for production writing, and Ryan manually copies and pastes them into the session to ensure clarity and security [68].
    3. Devon, a software factory by Cognition, is used for its cloud agents and is considered one of the best in the industry, despite its cost [72].
    4. Two screens are dedicated to testing his application [78].
    5. X (formerly Twitter) is open in the top left and top right [80].
    6. CodeX is used for most local desktop work, typically in the bottom right [81].
  3. A significant portion of Ryan's work, over 50%, is done on his iPhone, allowing him to manage tasks like checking or landing pull requests from anywhere [85].

1.4. The Importance of Technical Skills in the Agent Age [91]

  1. Devon is considered a powerful coding agent because its cloud-based harness works effectively with improved models, unlike two and a half years ago when models were insufficient [91].
  2. Users have choices for agent harnesses, including models from OpenAI, Cloud Code from Anthropic, and independent agent harnesses like AMP, Cursor (now owned by Elon), Cognition (maker of Devon), and Factory [96].
  3. Ryan chose Devon to scale himself horizontally, running many agents without worrying about code collisions or technical overhead like Git worktrees, which hinder shipping [101].
  4. The belief that engineering and technical skills are becoming obsolete is a misconception; instead, becoming a better agent manager requires increased technical proficiency [111].
    1. This role is analogous to an engineering manager, requiring an understanding of concepts like Postgres, production versus development environments, and migrations [112].
    2. These technical skills are learned through the process of using agents, making individuals more technical than ever before [114].

2. Cloud-Based Development and Agent Management Strategies [117]

thumbnail Cloud-based development environments, using virtual machines, enable parallel agent operations, reduce mental overhead, and significantly boost output compared to traditional local setups.

2.1. Cloud Development vs. Local Development [117]

  1. Working in the cloud is essential for running more agents and increasing productivity, rather than relying on a local machine [119].
  2. Local development involves editing and hosting code on a personal computer, requiring the setup of an entire local environment, including databases and authentication [123].
    1. Performing multiple tasks simultaneously locally is difficult, requiring either technical solutions like Git worktrees or creating multiple copies of code, which becomes unmanageable for more than a few tasks [126].
  3. Devon and other independent agent harnesses (e.g., Cursor, CodeEx) create Virtual Machines (VMs) in the cloud, which are small servers that act as development environments [131].
    1. These VMs can be spun up infinitely, allowing Ryan to run 5 to 10 cloud agents simultaneously without the risk of code collisions [134].
    2. Cloud development eliminates the mental burden of coordinating code, synchronizing updates, and managing different work trees [137].
    3. Working locally is considered outdated and inefficient, as it significantly limits output compared to cloud-based methods [140].
  4. While some heavy front-end work or new UI features might initially require a local agent for wireframing, the goal is to quickly move this work to the cloud for a cloud agent to take over [149].
    1. Much of the traditional knowledge about how to work is now considered outdated [154].

2.2. Managing High-Stakes Decisions and Output [157]

  1. The nature of work has shifted to making many high-stakes decisions throughout the day, unlike the past where only a few such decisions were made daily [158].
    1. Individuals must mentally pace themselves, recognizing that they might make 10 to 20 high-stakes decisions by lunchtime [162].
  2. To manage this increased decision-making, Ryan organizes his agent threads by pinning the most important tasks for the day, separating them from smaller bug fixes [164].
  3. He sets a cadence for checking on agents and making high-stakes decisions, typically every 25 minutes, to allow for mental rest [168].
    1. This approach aligns with the phrase "slow is smooth, and smooth is fast," emphasizing deliberate action [167].
  4. The new work environment is demanding, requiring individuals to work more, not less, to survive [171].
    1. Ryan's daily output often ranges from 22 to 25 pull requests (PRs), sometimes reaching 40, with most being merged [173].
    2. Even during a hiking trip where he couldn't access his phone, he managed to ship eight PRs in the morning [176].
  5. Working from a phone for half the time is crucial for competing in this new world, despite being tiring [186].
    1. This involves exercising a "new muscle" for making high-stakes decisions constantly, including on a phone [188].
    2. Ryan uses an analog, written to-do list daily to prioritize the most important tasks and avoid getting distracted by numerous fire-fighting PRs [191].
  6. Being available to provide real-time feedback and make decisions is critical for achieving product-market fit and rapid revenue growth [197].
    1. Agents are infinitely scalable and work quickly, but they are often blocked by human decisions, which cannot yet be delegated [201].
    2. While advanced models like Fable can manage child sessions (e.g., spinning up five child Devon sessions), a human manager must still pay attention [203].
    3. The analogy used is that of a manager of a big team: if you're always out to lunch, you need to get into the office, and now "the office is your phone" [209].

2.3. Essential Skills for the Agent Age [215]

  1. The system for managing tasks in the agent age is similar to that of a highly organized person before AI, emphasizing basic principles like pinning important items [219].
  2. Being an excellent engineering manager is one of the most valuable skills to develop [223].
  3. Key behaviors for success in this new world include:
    1. Managing as many agents as possible [226].
    2. Working in the cloud [228].
    3. Prioritizing tasks effectively [228].
    4. Learning to make high-stakes decisions quickly [229].
    5. Becoming more technical [229].
  4. The idea that engineering is disappearing or becoming less technical is false; those who claim this are not actively engaged in the work [230].
    1. An analogy is provided: a carpenter would be a better manager of robot carpenters because they understand the craft, such as where to cut or nail [233].
  5. The main takeaways are to embrace cloud agents, practice using them, and become comfortable working from a phone interface [237].

3. Automations and Software Factory Development [242]

thumbnail

Automations are crucial for continuous improvement, bug fixing, and monitoring production, while building a software factory involves agents writing, reviewing, and shipping all code.

3.1. Implementing Automations [242]

  1. The second key step after working in the cloud is to implement automations for tasks that would typically involve team meetings or daily monitoring [247].
  2. An example of an automation is setting up an agent to automate the entire user signup and onboarding process for Untangle in a browser [249].
    1. This is basic user testing that good agents can perform [252].
    2. The goal is to enable agents to improve themselves, rather than requiring manual system building [254].
  3. In Devon, this is called a playbook, which is a list of tasks and how to perform them correctly, distinct from a skill [255].
    1. Ryan has an "End-to-End signup test" playbook that runs three times a week [258].
    2. This automation is expensive, costing about $60 in tokens per run due to extensive browser testing [259].
  4. It is vital for agents to perform browser testing effectively, ideally out of the box [261].
    1. Devon excels at this, having used cloud agents for browser testing for two years [263].
    2. It records and annotates videos of its testing, then uses these videos to identify and fix bugs, creating an agentic loop [264].
  5. When an automation fails (e.g., a test fails), it should trigger a triage session where another agent is spun up to fix the issue [266].
    1. The manager needs to understand how to monitor these failures and fixes, and how to be notified (e.g., via Slack) [270].
  6. This specific automation primarily focuses on finding bugs introduced into production that were missed by other automated test suites, ensuring critical functions like signups don't fail unnoticed [276].
    1. It is not primarily for UX testing; for UX, there is currently no substitute for a human using the app themselves [281].
    2. Ryan personally goes through the entire app periodically to find bugs and quickly spins up sessions to fix them [284].

3.2. Other Automation Examples [289]

  1. Beyond QA and bug fixing, Ryan implements other automations:
    1. Production Watchdog: This automation runs daily at 9:00 a.m., reviewing all database events for paid customers of Untangle [291].
      1. It summarizes the activity in a JSON file, which is accessible in the admin panel [295].
      2. This provides a daily summary of customer actions, similar to a chief of staff reporting important events, successes, and bugs [297].
      3. The report includes links to view specific customer actions in the production instance, allowing Ryan to investigate anything that seems "off" [306].
      4. This has helped identify user experience issues that agents built but were not quite right [310].

3.3. Self-Improvement Loops [315]

  1. Untangle uses an agent named Grace, who acts as a paralegal and chats with attorneys, paralegals, and clients going through divorce [316].
  2. A daily automation grades these chats based on a predefined rubric [319].
    1. The agent is instructed on how to judge whether a conversation is good or bad [320].
    2. If a chat falls below a certain score, the automation kicks off a child session to fix the issue [322].
  3. This self-improvement loop has been remarkably effective at identifying and fixing small UX details, often "paper cuts" that a human might overlook or deem too minor to address [323].
    1. Ryan estimates that about three such fixes are shipped daily due to this automation [328].

3.4. Managing Token Costs and Model Routing [330]

  1. Self-improvement loops can be expensive; Ryan previously spent $20,000 in tokens in one month, which is not sustainable [330].
    1. The viable cost for real engineering work per employee is estimated to be around $5,000 per month [334].
    2. To manage costs, model routing is essential, which involves using the right model for the right task [339].
  2. Independent agent labs (e.g., Cursor, Cognition, Factory, AMP) understand the need for cost-effective solutions and are working to optimize token budgets [335].
    1. In contrast, heavily funded companies like CloudCode and CodeX may subsidize tokens, making them seem cheap, but this is not sustainable long-term [343].
  3. For reinforcement loops, Ryan uses SUI 1.7, a fine-tuned model by Cognition specifically for coding, which is much cheaper than models like GPT-4 or GPT-5-6 [347].
    1. These sessions typically cost around $5 each [351].
    2. Investing $15 a day to improve core features is considered a worthwhile expense for a real company [353].
  4. Building engineering operations on a Frontier Lab stack (e.g., Cloud Code, CodeX) is not recommended because these labs are not incentivized to provide long-term affordability [358].
    1. They tend to lock users into their models and processes [360].
    2. Independent agent labs, however, optimize for affordable engineering long-term by routing tasks to the best and most cost-effective models (e.g., GPT-4 for complex tasks, SWE 1.7 for loops, Fable for checks) [361].

3.5. Building a Software Factory [368]

  1. Individuals should avoid building their own software factory unless they reach a significant scale, like Ramp, which developed its custom in-house agent, Inspect [368].
    1. For a one-person operation, using a Frontier Lab for about $200 a month might suffice [371].
    2. However, once a product achieves market fit and begins hiring, transitioning to an independent agent lab is crucial [373].
    3. Otherwise, one would either have to build a software factory from scratch (which is inefficient) or be locked into a "walled garden" [376].
  2. Working with an independent lab is analogous to using a mortgage broker or travel agent; they make calls to secure the best prices and solutions [380].
  3. While Ryan uses Cloud Code and Cursor, he is interested in Devon [382].
    1. He also uses the CodeX Mac app because it is visually appealing and offers heavily subsidized tokens, making it almost free for personal information work like opening browser tabs or checking email [388].
    2. However, he advises against building a company's software factory on CodeX due to the long-term cost implications [391].

4. Key Takeaways and Reputation Building [394]

thumbnail

The main takeaways are to embrace cloud agents and automations, recognize that agents will handle all code, and build a strong reputation by sharing knowledge on X.

4.1. Core Principles for Agent Management [394]

  1. The primary takeaways are to utilize cloud agents and automations [399].
  2. The future involves a software factory where agents will write, review, and ship 100% of the code, and the goal is to align with this direction as much as possible [402].

4.2. Building a Credible Reputation on X [405]

  1. Building a credible reputation by sharing knowledge publicly, especially on X (Twitter), yields significant dividends [405].
    1. Writing helpful articles on X is particularly effective, as the algorithm rewards such content [407].
    2. Being active on X, regardless of personal feelings about Elon, is important for sharing knowledge [409].
  2. Sharing knowledge fosters a network and relationships, which can open doors [411].
    1. The host's relationship with Ryan, leading to his appearance on the show, was built through interactions on X [412].
  3. Sharing doesn't require polished content; simply being useful to people and sharing what you're learning is sufficient [414].
    1. The rewards may not be immediate but will materialize over time [417].
    2. It's acceptable to admit "I don't know" or to ask an agent for help [419].
  4. An example is Sahil Bloom, who grew his audience to over a million Twitter followers by writing about topics of interest during COVID, leading to a bestselling book, companies, and a fund [423].
    1. His strategy was to share what he was learning, not to claim omniscience [427].
  5. The advice is to "just get out there and do, learn," echoing Yoda's quote: "There is no try. There is only do" [432].