Farm3D: Learning Articulated 3D Animals by Distilling 2D Diffusionhttps://farm3d.github.io/

Farm3D: Learning Articulated 3D Animals by Distilling 2D Diffusion

🐾🔮 Exciting new tool alert! Farm3D revolutionizes 3D animal reconstruction & synthesis from 2D diffusion. Quickly create controllable 3D assets with lighting tweaks & texture swaps from a single image! 🦁🦓 #AI #3DReconstruction #Farm3D

  • Farm3D is a framework for learning articulated 3D animals from 2D diffusion.
  • It uses Stable Diffusion to create training data for reconstructing 3D shapes from images.
  • The diffusion model is also used as a scoring mechanism to enhance learning.
  • This method enables generating controllable 3D assets quickly from a single input image.
  • The approach involves training a model to factorize objects into shape, appearance, viewpoint, and light direction.
  • Farm3D can reconstruct detailed shapes like legs and ears without real image training.
  • It allows for adjusting lighting, swapping textures, and animating shapes of 3D assets.
  • The input images can be either real or synthesized using Stable Diffusion.
  • Synthetic views critiqued by Stable Diffusion guide the learning process effectively.