We present a neural network structure, ControlNet, to control pretrained large diffusion models to support additional input conditions. The ControlNet learns task-specific conditions in an end-to-end way, and the learning is robust even when the training dataset is small (< 50k).

2024年7月7日 · ControlNet’s depth map has a higher resolution than depth-to-image’s. How does ControlNet work? This tutorial won’t be complete without explaining how ControlNet works under the hood.

These are controlnet weights trained on stabilityai/stable-diffusion-xl-base-1.0 with depth conditioning. You can find some example images in the following. prompt: spiderman lecture, …

2023年2月11日 · In this mode, the ControlNet encoder will try best to recognize the content of the input control map, like depth map, edge map, scribbles, etc, even if you remove all prompts. Let's have fun with some very challenging experimental settings!

We report that large diffusion models like Stable Diffusion can be augmented with ControlNets to enable conditional inputs like edge maps, depthmentation maps, keypoints, etc. This may enrich the methods to control large diffusion models and further facilitate related applications.

2023年12月21日 · A depth map is a 2D grayscale representation of a 3D scene where each of the pixel’s values corresponds to the distance or depth of objects in the scene from the observer’s viewpoint. With a depth map, you can recreate compositions, poses, and reimagine constant shapes into new ideas.

2023年2月26日 · It is used with "depth" models. (e.g. control_depth-fp16) In a depth map (which is the actual name of the kind of detectmap image this preprocessor creates), lighter areas are "closer" and darker areas are "further away"

2023年11月15日 · A depth map is a 2D grayscale representation of a 3D scene where each of the pixel’s values corresponds to the distance or depth of objects in the scene from the observer’s viewpoint. There are four Depth preprocessors available for you to …

2023年8月14日 · The ControlNet Depth model is designed to use depth information to control the image outputs of diffusion models. Often used alongside Stable Diffusion, ControlNet Depth is a text-to-image generation model that focuses on depth information and estimation.

2023年5月15日 · This package contains 900 images of hands for the use of depth maps, Depth Library and ControlNet. Usage: Place the files in folder \extensions\sd-...