Elon Musk Advances in AI to Compete with Mark Zuckerberg

• Elon Musk has activated a significant number of Nvidia chips.
• On Monday, he revealed that xAI launched a new training cluster named Colossus.
• This cluster, utilizing Nvidia’s H100 GPUs, aims to help Musk compete with Meta.

Despite distractions from external matters, including the Brazilian Supreme Court’s recent decision to ban X, Elon Musk remains focused on his ambitious AI goals. On Monday, he announced that xAI, the venture he launched in 2023, has successfully activated a substantial new cluster of chips over the weekend. He referred to this system as “the most powerful AI training system in the world.”

Known as Colossus, this advanced system was established in Memphis, Tennessee, and is composed of an impressive 100,000 Nvidia H100 GPUs. Musk noted that the cluster was constructed in a rapid 122 days and anticipates that it will “double in size” within months as additional GPUs are incorporated.

This milestone is significant for Musk, particularly as it allows him to close the gap with his rival, Mark Zuckerberg. Musk had previously confirmed the intended size of the cluster back in July, but bringing it online is a critical step forward in his aspirations for xAI. The plan involves leveraging this technology to enhance “our collective understanding of the universe” through innovations like the Grok chatbot.

The ability to harness high-performance GPUs is crucial for both Musk and Zuckerberg to drive their AI initiatives forward. These powerful chips deliver the computational capabilities necessary for training advanced AI models. However, acquiring these GPUs has been a challenging endeavor, especially since the launch of ChatGPT in late 2022, which sparked a surge in demand. Companies have faced shortages due to the frantic rush for these resources, with some chips reportedly fetching prices exceeding $40,000.

Despite these hurdles, technology companies are relentlessly vying to secure GPU supplies to maintain a competitive edge. Nathan Benaich, founder and general partner of Air Street Capital, has been tracking the acquisition of H100 GPUs among leading tech firms. He estimates that Meta has amassed around 350,000 H100 GPUs, whereas xAI holds 100,000. In comparison, Tesla, another Musk enterprise, has acquired approximately 35,000 GPUs.

Zuckerberg previously asserted that Meta aims to stockpile 600,000 GPUs by the end of the year, with commitments for 350,000 of those being Nvidia’s H100s. Other major players in the AI field, such as Microsoft, OpenAI, and Amazon, have not publicly disclosed the number of H100 GPUs in their possession.

Though Meta has not specified the exact count of GPUs secured for its year-end goal, a research paper released in July indicated that the largest iteration of its Llama 3 AI model was trained using 16,000 H100 GPUs. Earlier this year, Meta announced a strategic investment to bolster its AI efforts, including the establishment of two clusters equipped with 24,000 GPUs each to aid in Llama 3’s progression.

This information implies that xAI’s new training cluster, which boasts 100,000 H100 GPUs, surpasses the capabilities of the cluster used for training Meta’s most extensive AI model. The technological strides made by xAI have been recognized widely across the industry.

In a post on X, Nvidia’s data-center account expressed excitement about Colossus, dubbing it the world’s largest GPU supercomputer brought online in record time. Similarly, Greg Yang, a co-founder of xAI, offered a creative response to this news, referencing popular culture.

Investors are closely monitoring xAI’s developments, with Shaun Maguire from Sequoia noting that the team now possesses the world’s most powerful training cluster for advancing its Grok chatbot. He highlighted that in recent weeks, Grok-2 has notably caught up to leading models in terms of capability.

However, like many other AI companies, xAI faces significant uncertainties regarding the commercial viability of its technology. Benaich remarked on the impressive achievements of xAI under Musk’s leadership, but he pointed out that the product strategy remains somewhat ambiguous. Musk has mentioned that the forthcoming version of Grok, trained on the 100,000 H100s, is expected to be particularly special.

As developments unfold, it remains to be seen how this massive investment in AI will shape Musk’s competitive landscape with Zuckerberg in the tech industry.

Source: Business Insider