In the rapidly evolving landscape of artificial intelligence (AI), the conversation surrounding deployment strategies has shifted dramatically. While rising compute costs have long been cited as a barrier to AI adoption, leading companies are now prioritizing speed and flexibility over cost considerations. This shift reflects a broader industry trend where the challenges of latency, capacity, and operational agility take precedence over financial constraints.
Two notable examples of this trend can be seen in the operations of Wonder, a food delivery service, and Recursion, a biotech firm. Both companies illustrate how AI leaders are navigating the complexities of scaling their operations while maintaining a focus on rapid deployment and experimentation.
Wonder, which leverages AI for various functions including logistics and customer recommendations, has found that the incremental cost of AI per order is relatively low—currently estimated at just 2 to 3 cents, with projections indicating a rise to 5 to 8 cents. Despite these manageable costs, the company’s primary concern lies not in the expense of AI itself but in the capacity of its cloud infrastructure to meet surging demand. Initially built on the assumption of “unlimited” cloud resources, Wonder faced unexpected challenges as it grew. Six months ago, the company received warnings from its cloud providers about potential capacity issues, prompting a strategic pivot to a multi-region infrastructure sooner than anticipated.
James Chen, the Chief Technology Officer (CTO) of Wonder, expressed surprise at the need to adapt their infrastructure plans earlier than expected. The company had initially envisioned a timeline that allowed for further growth before addressing capacity concerns. However, as demand surged, it became clear that a more robust infrastructure was necessary to sustain operations. This experience underscores the importance of flexibility in cloud strategy, particularly for companies experiencing rapid growth.
Looking ahead, Wonder aims to develop hyper-personalized micro-models tailored to individual users, utilizing AI agents or concierges based on purchase history and clickstream data. However, Chen noted that the current costs associated with creating such customized models render them economically unfeasible at this time. The challenge lies in balancing the desire for personalization with the realities of operational costs.
Budgeting for AI initiatives presents another layer of complexity for Wonder. The company strives to provide its developers and data scientists with the freedom to experiment while also implementing internal reviews to monitor usage and prevent excessive compute costs. Chen described budgeting in this context as an art rather than a science, highlighting the unpredictable nature of token-based systems. A significant portion of AI costs—estimated between 50% to 80%—arises from resending the same contextual information with each request. This inefficiency complicates budgeting efforts, as the company seeks to leverage new models without incurring prohibitive expenses.
On the other side of the spectrum, Recursion has adopted a hybrid approach to its AI infrastructure, combining on-premises clusters with cloud-based solutions. This strategy allows the company to address a wide range of compute needs effectively. When Recursion first sought to build its AI capabilities, it encountered limitations in the offerings available from cloud providers. As CTO Ben Mabey recounted, the company’s initial foray into AI infrastructure involved setting up its own clusters, which included Nvidia gaming GPUs launched in 2016. Remarkably, some of these early GPUs remain in use today, challenging the common perception that a GPU’s lifespan is limited to three years.
Recursion’s infrastructure strategy is informed by the specific demands of its operations. For large training jobs that require extensive data processing, the company relies on its on-premises clusters, which offer a fully connected network and access to high-performance parallel file systems. Conversely, shorter inference tasks are executed in the cloud, allowing for a flexible allocation of resources based on workload requirements.
One of the key advantages of Recursion’s approach is its ability to optimize costs through a practice known as “pre-emption.” This involves interrupting ongoing GPU tasks to prioritize higher-urgency workloads, particularly when dealing with biological data uploads, such as images or DNA sequencing data. Mabey explained that the company can afford to wait for non-critical tasks, allowing them to manage compute resources efficiently without sacrificing the quality of their experiments.
From a financial perspective, moving large workloads to on-premises infrastructure is significantly more cost-effective for Recursion. Mabey estimated that on-prem compute is conservatively ten times cheaper for substantial workloads, with a five-year total cost of ownership (TCO) being half that of cloud solutions. However, he acknowledged that for smaller storage needs, cloud options can still be competitive.
Mabey emphasized the importance of commitment when it comes to investing in AI infrastructure. He urged tech leaders to evaluate their willingness to make long-term investments in compute resources, as cost-effective solutions often require multi-year buy-ins. He observed that some peers in the industry hesitate to invest in compute capabilities due to concerns about escalating cloud bills, resulting in teams that underutilize available resources. This reluctance can stifle innovation, as teams may avoid pursuing ambitious projects out of fear of incurring high costs.
The experiences of both Wonder and Recursion highlight a critical turning point in the AI landscape. As organizations increasingly recognize that the economics of AI are not the primary constraint, the focus has shifted toward how quickly and flexibly AI solutions can be deployed and scaled. This evolution necessitates a reevaluation of traditional budgeting practices and resource allocation strategies.
For companies operating at scale, the ability to deploy AI rapidly and adapt to changing demands is paramount. The lessons learned from Wonder and Recursion serve as valuable insights for other organizations navigating similar challenges. By prioritizing flexibility and capacity over immediate cost concerns, companies can position themselves for success in an increasingly competitive environment.
As the AI industry continues to mature, the emphasis on deployment speed and operational agility will likely shape the future of technology development. Companies that embrace this shift and invest in scalable infrastructure will be better equipped to harness the full potential of AI, driving innovation and delivering value to their customers.
In conclusion, the landscape of AI deployment is undergoing a significant transformation. The focus has shifted from cost as a primary concern to the imperative of rapid and flexible deployment. Organizations like Wonder and Recursion exemplify this trend, demonstrating that the ability to adapt to changing demands and leverage advanced technologies is crucial for success in the AI space. As the industry evolves, companies must continue to prioritize agility and scalability, ensuring they remain at the forefront of innovation in an ever-changing technological landscape.