Latte3D: NVIDIA's Revolutionary AI for Instant Text-to-3D Model Creation

On March 27, 2024, Xpiens learned that NVIDIA announced the launch of an AI model named Latte3D, capable of converting brief text descriptions into precise three-dimensional models of objects and animals.

This breakthrough is akin to virtual 3D printing, where the generated 3D shapes are crafted in popular standard rendering application formats, facilitating their easy application in virtual environments.

The technology is poised for widespread use across video game development, advertising design, engineering projects, and robotic virtual training scenarios, signaling a future where reliable text-to-3D printing tools could emerge.

Sanja Fidler, NVIDIA's Vice President of Artificial Intelligence Research, stated, "A year ago, AI models took an hour to generate 3D visuals of this quality, but with Latte3D, it takes only 10 to 12 seconds. We can now produce results much faster, enabling creators across various industries to achieve near-real-time text-to-3D generation."

NVIDIA researchers demonstrated that using a single RTX A6000 GPU, 3D shapes could be generated almost instantaneously.

Creators can now instantly generate detailed objects with LATTE3D, avoiding the need to start from scratch or sift through 3D asset libraries. The model offers several different three-dimensional shape options for each text prompt, with selected objects being optimized for higher quality within minutes.

Users can then export these shapes into graphic software applications or platforms like NVIDIA Omniverse, facilitating 3D workflows and applications based on Universal Scene Description (OpenUSD).

Although LATTE3D was trained on datasets of animals and everyday objects, developers can train other types of data using the same model architecture. For instance, a version of LATTE3D trained on a 3D plant dataset could assist landscape designers in quickly populating garden visuals with trees, flower beds, and succulents during brainstorming sessions with clients.

Similarly, if trained on household items, the model could generate items to fill 3D simulated homes, aiding developers in training personal assistant robots before testing and deploying them in the real world.

LATTE3D was trained using NVIDIA A100 Tensor Core GPUs and was also trained on various text prompts generated by ChatGPT to enhance the model's ability to handle various phrases users might use to describe specific 3D objects.

For example, prompts related to canine animals should generate dog-like shapes.

Subscribe to Xpiens to learn more about the latest news in the 3D printing industry.