Point-E

Recent breakthroughs in text-to-image synthesis have been driven by diffusion models trained on billions of image-text pairs. Adapting this approach to 3D synthesis would require large-scale datasets of labeled 3D assets and efficient architectures for denoising 3D data, neither of which currently exist. In this work, we circumvent these limitations by using a pre-trained 2D text-to-image diffusion model to perform text-to-3D synthesis. We introduce a loss based on probability density distillation that enables the use of a 2D diffusion model as a prior for optimization of a parametric image generator. Using this loss in a DeepDream-like procedure, we optimize a randomly-initialized 3D model (a Neural Radiance Field, or NeRF) via gradient descent such that its 2D renderings from random angles achieve a low loss. The resulting 3D model of the given text can be viewed from any angle, relit by arbitrary illumination, or composited into any 3D environment.

This 3D avatar diffusion model is an AI system that automatically produces highly detailed 3D digital avatars. The generated avatars can be freely viewed in 360 degrees with unprecedented quality. The model significantly accelerates traditionally sophisticated 3D modeling process and opens new opportunities for 3D artists. This 3D avatar diffusion model is trained to generate 3D digital avatars represented as neural radiance fields. We build on the state-of-the-art generative technique (diffusion models) for 3D modeling. We use tri-plane representation to factorize the neural radiance field of avatars, which can be explicitly modeled by diffusion models and rendered to images via volumetric rendering. The proposed 3D-aware convolution brings the much-needed computational efficiency while preserving the integrity of diffusion modeling in 3D. The whole generation is a hierarchical process with cascaded diffusion models for multi-scale modeling.