ByteDance
/

FaceCLIP

@@ -1,5 +1,5 @@
 ---
-license: mit
 language:
 - en
 library_name: face-clip
@@ -21,7 +21,7 @@ base_model:
 This repository provides the official models for the following paper:
-[**Learning Joint ID-Textual Representation for ID-Preserving Image Synthesis**](https://arxiv.org/abs/2504.14202)<br />
 [**Zichuan Liu**](https://scholar.google.com/citations?user=-H18WY8AAAAJ)&nbsp;&nbsp;&nbsp;&nbsp;
 [**Liming Jiang**](https://liming-jiang.com/)&nbsp;&nbsp;&nbsp;&nbsp;
 [**Qing Yan**](https://scholar.google.com/citations?user=0TIYjPAAAAAJ)&nbsp;&nbsp;&nbsp;&nbsp;
@@ -32,9 +32,9 @@ This repository provides the official models for the following paper:
 > **Abstract:** *Recent progress in text-to-image (T2I) diffusion models has greatly improved image quality and flexibility. However, a major challenge in personalized generation remains: preserving the subject’s identity (ID) while allowing diverse visual changes. We address this with a new framework for ID-preserving image generation. Instead of relying on adapter modules to inject identity features into pre-trained models, we propose a unified multi-modal encoding strategy that jointly captures identity and text information. Our method, called FaceCLIP, learns a shared embedding space for facial identity and textual semantics. Given a reference face image and a text prompt, FaceCLIP produces a joint representation that guides the generative model to synthesize images consistent with both the subject’s identity and the prompt. To train FaceCLIP, we introduce a multi-modal alignment loss that aligns features across face, text, and image domains. We then integrate FaceCLIP with existing UNet and Diffusion Transformer (DiT) architectures, forming a complete synthesis pipeline FaceCLIP-x. Compared to existing ID-preserving approaches, our method produces more photorealistic portraits with better identity retention and text alignment. Extensive experiments demonstrate that FaceCLIP-x outperforms prior methods in both qualitative and quantitative evaluations.*
-![teaser](asset/readme/teaser.jpeg)
-![teaser](asset/readme/demo.jpeg)
-![teaser](asset/readme/arch.jpeg)
 ## 🔧 Installation and Usage
@@ -53,7 +53,7 @@ Please clone our [GitHub code repository](https://github.com/bytedance/FaceCLIP)
 The images used in this repository and related demos are sourced from consented subjects or generated by the models.
 These pictures are intended solely to showcase the capabilities of our research. If you have any concerns, please feel free to contact us, and we will promptly remove any inappropriate content.
-Our model is released under the [Creative Commons Attribution-NonCommercial 4.0 International Public License](./LICENSE) for academic research purposes only. Any manual or automatic downloading of the face models from [Arc2Face](https://github.com/foivospar/Arc2Face), the [OpenAI-CLIP-L-14](https://huggingface.co/openai/clip-vit-large-patch14), [OpenCLIP-bigG-14](https://github.com/mlfoundations/open_clip), the [SDXL-base-1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0) base model, and the [FLUX.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev) base model, *etc.*, must follow their original licenses and be used only for academic research purposes.
 This research aims to positively impact the field of Generative AI. Any usage of this method must be responsible and comply with local laws. The developers do not assume any responsibility for any potential misuse.

 ---
+license: cc-by-nc-4.0
 language:
 - en
 library_name: face-clip
 This repository provides the official models for the following paper:
+[**Learning Joint ID-Textual Representation for ID-Preserving Image Synthesis**](https://arxiv.org/abs/2503.16418)<br />
 [**Zichuan Liu**](https://scholar.google.com/citations?user=-H18WY8AAAAJ)&nbsp;&nbsp;&nbsp;&nbsp;
 [**Liming Jiang**](https://liming-jiang.com/)&nbsp;&nbsp;&nbsp;&nbsp;
 [**Qing Yan**](https://scholar.google.com/citations?user=0TIYjPAAAAAJ)&nbsp;&nbsp;&nbsp;&nbsp;
 > **Abstract:** *Recent progress in text-to-image (T2I) diffusion models has greatly improved image quality and flexibility. However, a major challenge in personalized generation remains: preserving the subject’s identity (ID) while allowing diverse visual changes. We address this with a new framework for ID-preserving image generation. Instead of relying on adapter modules to inject identity features into pre-trained models, we propose a unified multi-modal encoding strategy that jointly captures identity and text information. Our method, called FaceCLIP, learns a shared embedding space for facial identity and textual semantics. Given a reference face image and a text prompt, FaceCLIP produces a joint representation that guides the generative model to synthesize images consistent with both the subject’s identity and the prompt. To train FaceCLIP, we introduce a multi-modal alignment loss that aligns features across face, text, and image domains. We then integrate FaceCLIP with existing UNet and Diffusion Transformer (DiT) architectures, forming a complete synthesis pipeline FaceCLIP-x. Compared to existing ID-preserving approaches, our method produces more photorealistic portraits with better identity retention and text alignment. Extensive experiments demonstrate that FaceCLIP-x outperforms prior methods in both qualitative and quantitative evaluations.*
+![teaser](../asset/readme/teaser.jpeg)
+![teaser](../asset/readme/demo.jpeg)
+![teaser](../asset/readme/arch.jpeg)
 ## 🔧 Installation and Usage
 The images used in this repository and related demos are sourced from consented subjects or generated by the models.
 These pictures are intended solely to showcase the capabilities of our research. If you have any concerns, please feel free to contact us, and we will promptly remove any inappropriate content.
+Our model is released under the [Creative Commons Attribution-NonCommercial 4.0 International Public License](./LICENSE) for academic research purposes only. Any manual or automatic downloading of the face models from the [OpenAI-CLIP-L-14](https://huggingface.co/openai/clip-vit-large-patch14), [OpenCLIP-bigG-14](https://github.com/mlfoundations/open_clip), the [SDXL-base-1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0) base model, and the [FLUX.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev) base model, *etc.*, must follow their original licenses and be used only for academic research purposes.
 This research aims to positively impact the field of Generative AI. Any usage of this method must be responsible and comply with local laws. The developers do not assume any responsibility for any potential misuse.