je-suis-tm/vanessa_kirby_lora_flux_nf4

Vanessa Kirby Lora Flux NF4

Prompt

Training With QLoRA: The image portrays a scene of refined elegance, likely from a historical drama. Vanessa Kirby in the foreground commands attention, dressed in a stunning, light blue or grey strapless gown. The dress features a flattering sweetheart neckline and fitted bodice, complemented by long sleeves concealed beneath grey gloves. A silver necklace and earrings enhance her regal appearance, which is further emphasized by a crown or tiara atop her intricately styled updo. Behind her, a second woman is partially visible, also in formal attire, though less elaborate than the foreground figure's gown. She stands before a wall featuring a large mirror or painting, contributing to the overall sense of grandeur and luxury. The lighting and formal attire create an atmosphere of opulence and drama, reinforcing the image's sophisticated and refined aesthetic.

Prompt

Training Without QLoRA: The image portrays a scene of refined elegance, likely from a historical drama. Vanessa Kirby in the foreground commands attention, dressed in a stunning, light blue or grey strapless gown. The dress features a flattering sweetheart neckline and fitted bodice, complemented by long sleeves concealed beneath grey gloves. A silver necklace and earrings enhance her regal appearance, which is further emphasized by a crown or tiara atop her intricately styled updo. Behind her, a second woman is partially visible, also in formal attire, though less elaborate than the foreground figure's gown. She stands before a wall featuring a large mirror or painting, contributing to the overall sense of grandeur and luxury. The lighting and formal attire create an atmosphere of opulence and drama, reinforcing the image's sophisticated and refined aesthetic.

Prompt

Testing With QLoRA: Vanessa Kirby in a white kimono of cherry blossom patterns

Prompt

Testing Without QLoRA: Vanessa Kirby in a white kimono of cherry blossom patterns

The QLoRA fine-tuning process of vanessa_kirby_lora_flux_nf4 takes inspiration from this post (https://huggingface.co/blog/diffusers-quantization). The training was executed on a local computer with 1000 timesteps and the same parameters as the link mentioned above, which took around 6 hours on 8GB VRAM 4060. The peak VRAM usage was around 7.7GB. To avoid running low on VRAM, both transformers and text_encoder were quantized. All the images generated here are using the below parameters

• Height: 512
• Width: 512
• Guidance scale: 5
• Num inference steps: 20
• Max sequence length: 512
• Seed: 0

Usage

import torch
from diffusers import FluxPipeline, FluxTransformer2DModel
from transformers import T5EncoderModel

text_encoder_4bit = T5EncoderModel.from_pretrained(
    "hf-internal-testing/flux.1-dev-nf4-pkg", subfolder="text_encoder_2",torch_dtype=torch.float16,)

transformer_4bit = FluxTransformer2DModel.from_pretrained(
        "hf-internal-testing/flux.1-dev-nf4-pkg", subfolder="transformer",torch_dtype=torch.float16,)

pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", torch_dtype=torch.float16,
                                    transformer=transformer_4bit,text_encoder_2=text_encoder_4bit)

pipe.load_lora_weights("je-suis-tm/vanessa_kirby_lora_flux_nf4",
                       weight_name='pytorch_lora_weights.safetensors')

prompt="Vanessa Kirby in a white kimono of cherry blossom patterns"

image = pipe(
            prompt,
            height=512,
            width=512,
            guidance_scale=5,
            num_inference_steps=20,
            max_sequence_length=512,
            generator=torch.Generator("cpu").manual_seed(0),            
        ).images[0]

image.save("vanessa_kirby_lora_flux_nf4.png")

Trigger words

You should use Vanessa Kirby to trigger the image generation.

Download model

Download them in the Files & versions tab.