UC Davis Alumnus Shubhabrata Sengupta Helps Build Meta’s Newest and Largest AI Supercomputer

UC Davis’ engineering alumnus Shubhabrata Sengupta (class of 2010) helped design the internal architecture for Meta’s new supercomputer, known as the AI Research SuperCluster (RSC). 

The RSC will help Meta’s AI researchers build new and better AI models that can integrate information from trillions of examples at once, learn hundreds of different languages, seamlessly analyze text, images, and video, develop new augmented reality tools and much more. Meta’s researchers will train the largest models needed to develop advanced AI models for applications, including computer vision, NLP, and speech recognition. 

RSC currently has 6,080 GPUs, putting it on par with the National Energy Research Scientific Computing Center’s Perlmutter supercomputer, which is No. 5 on the November 2021 TOP 500 list. The TOP 500 lists the 500 fastest computer systems in use today. When fully built out, the cluster will almost triple in size, which Meta believes will make it the fastest AI supercomputer in the world. This computer will speed up AI research. 

Building on His College Experience

Sengupta reflects on how his time at UC Davis prepared him for his computing work at Meta. 

“One key skill I brought was being able to take a small idea to something as big as RSC over five years. When I started as a grad student at UC Davis, graphical processing units (GPUs) were barely functional as a computing platform. In [ECE professor] John Owens’ group, we were able to build some of the early software infrastructures that expanded the scope of GPUs beyond computer games.”

He outlines how working with John Owens on GPUs serendipitously gave him experience with the tools he would need for artificial intelligence. 

“John Owens’ research group at UC Davis was one of the pioneers in developing the algorithms that expanded the scope of GPUs from graphics to general-purpose computation. As part of that group, I was fortunate to be part of that vision, which laid much of the foundation for GPU computing on which AI now stands. I did not know where GPUs would find widespread use outside of graphics when I graduated, but I was lucky to continue working on them at various jobs. 

When AI started getting traction in 2012, GPUs became an excellent fit for training neural networks. I transitioned to AI in 2014, primarily on my experience of working on GPUs for close to a decade. This decade-long experience also helped me bring ideas from High-Performance Computing (HPC) to AI training, especially to scale-out training to thousands of GPUs.”

While at UC Davis, Sengupta learned to persevere through the early days of new technology. He says he has brought design principles and grit, both of which he learned while in college, into his work with artificial intelligence and the RSC. 

“The RSC’s evolution can be traced back to 2017, to a cluster that was significantly smaller in scale but which allowed us to iterate on many new ideas, much like GPUs in 2004. There were many times when I felt that it was miraculous when it worked at all, and there were many low points when things didn’t work. But the conviction that it was eventually going to work kept me going,” says Sengupta. 

The Power and Promise of AI

The AI supercomputer processes information at high speed because it uses a different method for calculating numbers with a decimal point that lessens the number of digits that have to be carried. This method reduces the amount of resources needed for each calculation without compromising the accuracy of the results. 

In the future, Sengupta hopes that this AI supercomputer will have the power for tasks that remain elusive today. 

“We hope the RSC will help us build entirely new AI systems that can, for example, power real-time voice translations to large groups of people, each speaking a different language, so they could seamlessly collaborate on a research project or play an AR game together.”

He advises engineering students who want to work with supercomputers to pursue AI since supercomputers are increasingly used to train AI models.