How do we generate 3D models out of images?

Automating the generation of 3D models based on just a few images is a  difficult problem to solve. Even more difficult if you want to perpare 3D models at a same quality level as a human creator would. This difficulty is rooted in the fact that the inputs do not visualize the entire object, so any approach for solving this 3D generation problem must be capable of dealing with missing information.

At 3DFY.ai, we tackle this challenge by separating the AI pipeline into object categories. We use category-specific datasets to train our AI based 3D generation pipeline for each class of objects. Using this methodology, our Deep Learning Models are capable of understanding category specific priors which enables them to complete the missing information in a plausible manner.

3DFY in three simple steps:

1

Pre-processing

The input images are searched for recognized objects. The objects are extracted from the images and the background is removed.

2

Analysis

The processed images are further analyzed by a Deep Neural Network to produce its object code which is  a compact, proprietary representation of the imaged object.

3

Synthesis

A high-quality 3D model is generated according to the object code and predefined control parameters - user gets control over traits such as level of detail and texture resolution

Example 3D models created by 3dfy.ai pipeline

Below you can inspect 4 example models created by 3dfy.ai pipeline
for four different categories: Lamps, Sofas, Dressers, Ottomans.

The comparison above presents a real image of ottoman collected from vendor website (HOMCOM Storage Ottoman with Removable Lid - blue version) compared to render of our 3d model reconstruction with mesh wireframe overlaid on the model. The camera position, lens properties and 3d model location was manually adjusted to allow for a better comparison.

Slide the middle bar to compare.

Small data

Create a 3D model using as few as 1 image or text prompt

Scalable

The 3D model creation is highly automated

Effortless

Use the customer’s existing images. No need to scan or take additional photos of the product/object

Fast

3DFY 2D images or text prompt into a 3D model in seconds