---
tags:
- text-to-image
- lora
- qlora
- flux
- nf4
- diffusers
- template:diffusion-lora
widget:
- output:
    url: images/2025-08-25_02-09-53.png
  text: >-
    Training With QLoRA: The image portrays a scene of refined elegance, likely from a historical drama. Vanessa Kirby in the foreground commands attention, dressed in a stunning, light blue or grey strapless gown. The dress features a flattering sweetheart neckline and fitted bodice, complemented by long sleeves concealed beneath grey gloves. A silver necklace and earrings enhance her regal appearance, which is further emphasized by a crown or tiara atop her intricately styled updo. Behind her, a second woman is partially visible, also in formal attire, though less elaborate than the foreground figure's gown. She stands before a wall featuring a large mirror or painting, contributing to the overall sense of grandeur and luxury. The lighting and formal attire create an atmosphere of opulence and drama, reinforcing the image's sophisticated and refined aesthetic.
- output:
    url: images/2025-08-25_02-44-57.png
  text: >-
    Training Without QLoRA: The image portrays a scene of refined elegance, likely from a historical drama. Vanessa Kirby in the foreground commands attention, dressed in a stunning, light blue or grey strapless gown. The dress features a flattering sweetheart neckline and fitted bodice, complemented by long sleeves concealed beneath grey gloves. A silver necklace and earrings enhance her regal appearance, which is further emphasized by a crown or tiara atop her intricately styled updo. Behind her, a second woman is partially visible, also in formal attire, though less elaborate than the foreground figure's gown. She stands before a wall featuring a large mirror or painting, contributing to the overall sense of grandeur and luxury. The lighting and formal attire create an atmosphere of opulence and drama, reinforcing the image's sophisticated and refined aesthetic.
- output:
    url: images/2025-08-25_01-50-35.png
  text: >-
    Testing With QLoRA: Vanessa Kirby in a white kimono of cherry blossom patterns
- output:
    url: images/2025-08-25_02-27-11.png
  text: >-
    Testing Without QLoRA: Vanessa Kirby in a white kimono of cherry blossom patterns
base_model: black-forest-labs/FLUX.1-dev
instance_prompt: Vanessa Kirby, lora, qlora, flux, nf4
license: mit
datasets:
- je-suis-tm/vanessa_kirby_lora_flux_nf4
---

# Vanessa Kirby Lora Flux NF4

<Gallery />

The QLoRA fine-tuning process of `vanessa_kirby_lora_flux_nf4` takes inspiration from [this post (https://huggingface.co/blog/diffusers-quantization)](https://huggingface.co/blog/diffusers-quantization). The training was executed on a local computer with 1000 timesteps and the same parameters as the link mentioned above, which took around 6 hours on 8GB VRAM 4060. The peak VRAM usage was around 7.7GB. To avoid running low on VRAM, **both transformers and text_encoder were quantized.** All the images generated here are using the below parameters

* Height: 512
* Width: 512
* Guidance scale: 5
* Num inference steps: 20
* Max sequence length: 512
* Seed: 0
  
## Usage

```python
import torch
from diffusers import FluxPipeline, FluxTransformer2DModel
from transformers import T5EncoderModel

text_encoder_4bit = T5EncoderModel.from_pretrained(
    "hf-internal-testing/flux.1-dev-nf4-pkg", subfolder="text_encoder_2",torch_dtype=torch.float16,)

transformer_4bit = FluxTransformer2DModel.from_pretrained(
        "hf-internal-testing/flux.1-dev-nf4-pkg", subfolder="transformer",torch_dtype=torch.float16,)

pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", torch_dtype=torch.float16,
                                    transformer=transformer_4bit,text_encoder_2=text_encoder_4bit)

pipe.load_lora_weights("je-suis-tm/vanessa_kirby_lora_flux_nf4",
                       weight_name='pytorch_lora_weights.safetensors')

prompt="Vanessa Kirby in a white kimono of cherry blossom patterns"

image = pipe(
            prompt,
            height=512,
            width=512,
            guidance_scale=5,
            num_inference_steps=20,
            max_sequence_length=512,
            generator=torch.Generator("cpu").manual_seed(0),            
        ).images[0]

image.save("vanessa_kirby_lora_flux_nf4.png")
```

## Trigger words

You should use `Vanessa Kirby` to trigger the image generation.


## Download model


[Download](/je-suis-tm/vanessa_kirby_lora_flux_nf4/tree/main) them in the Files & versions tab.