library_name: diffusers
license: openrail++
datasets:
  - ShinoharaHare/Danbooru-2024-Filtered-1M
language:
  - en
base_model:
  - ShinoharaHare/Waifu-Inpaint-XL
pipeline_tag: image-to-image
tags:
  - anime
  - art
  - inpaint
  - inpainting
  - stable-diffusion
  - stable-diffusion-xl

Waifu-Colorize-XL

Waifu-Colorize-XL is an SDXL-based model specifically designed for colorizing anime-themed images.

Task	Model
Generation	WAI-NSFW-illustrious-SDXL-V14.0-V-Prediction
Inpainting	Waifu-Inpaint-XL
Colorizing	Waifu-Colorize-XL

Overview

Waifu-Colorize-XL is further trained from Waifu-Inpaint-XL, enabling it to colorize grayscale or binarized images.

Model Details

Developed by: ShinoharaHare
Model type: Diffusion based image-to-image generative model for colorizing
Language(s) (NLP): English
License: CreativeML Open RAIL++-M
Finetuned from: Waifu-Inpaint-XL

🧨 Diffusers

Grayscale Colorized

import torch
from diffusers import StableDiffusionXLInpaintPipeline
from diffusers.utils import load_image

pipeline = StableDiffusionXLInpaintPipeline.from_pretrained(
  'ShinoharaHare/Waifu-Colorize-XL',
  torch_dtype=torch.half
)
pipeline.to('cuda')

grayscale_image = load_image('https://cdn-uploads.huggingface.co/production/uploads/630ed69a31970d1cd4fd575d/ZusF3zcH-auhPgSxY0LKs.png')
prompt = '' # Write your own prompt if you want.

masked_image_latents = pipeline.vae.encode(
  pipeline.image_processor.preprocess(grayscale_image).to(pipeline.device, pipeline.dtype)
)[0].sample() * pipeline.vae.config.scaling_factor

colorized_image = pipeline(
  prompt=prompt,
  image=grayscale_image,
  mask_image=torch.ones((1, grayscale_image.height, grayscale_image.width)),
  masked_image_latents=masked_image_latents,
  num_inference_steps=28,
  strength=1.0,
  guidance_scale=5.0 if prompt else 0.0,
  height=grayscale_image.height,
  width=grayscale_image.width
).images[0]

colorized_image.show()

ComfyUI

This model requires a custom node to function correctly.

Please install InpaintModelConditioningAdvanced first.