Files changed (1) hide show
  1. README.md +3 -46
README.md CHANGED
@@ -32,6 +32,9 @@ library_name: PaddlePaddle
32
 
33
  # ERNIE-4.5-0.3B
34
 
 
 
 
35
  ## ERNIE 4.5 Highlights
36
 
37
  The advanced capabilities of the ERNIE 4.5 models, particularly the MoE-based A47B and A3B series, are underpinned by several key technical innovations:
@@ -88,52 +91,6 @@ python -m fastdeploy.entrypoints.openai.api_server \
88
  --max-num-seqs 32
89
  ```
90
 
91
- ### Using `transformers` library
92
-
93
- The following contains a code snippet illustrating how to use the model generate content based on given inputs.
94
-
95
- ```python
96
- from transformers import AutoModelForCausalLM, AutoTokenizer
97
-
98
- model_name = "baidu/ERNIE-4.5-0.3B-PT"
99
- model_name = "baidu/ERNIE-4.5-0.3B-PT"
100
-
101
- # load the tokenizer and the model
102
- tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
103
- model = AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code=True)
104
-
105
- # prepare the model input
106
- prompt = "Give me a short introduction to large language model."
107
- messages = [
108
- {"role": "user", "content": prompt}
109
- ]
110
- text = tokenizer.apply_chat_template(
111
- messages,
112
- tokenize=False,
113
- add_generation_prompt=True
114
- )
115
- model_inputs = tokenizer([text], add_special_tokens=False, return_tensors="pt").to(model.device)
116
-
117
- # conduct text completion
118
- generated_ids = model.generate(
119
- model_inputs.input_ids,
120
- max_new_tokens=1024
121
- )
122
- output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist()
123
-
124
- # decode the generated ids
125
- generate_text = tokenizer.decode(output_ids, skip_special_tokens=True).strip("\n")
126
- print("generate_text:", generate_text)
127
- ```
128
-
129
- ### vLLM inference
130
-
131
- vLLM is currently being adapted, priority can be given to using our forked repository [vllm](https://github.com/CSWYF3634076/vllm/tree/ernie). We are working with the community to fully support ERNIE4.5 models, stay tuned.
132
-
133
- ```bash
134
- vllm serve baidu/ERNIE-4.5-0.3B-PT --trust-remote-code
135
- ```
136
-
137
  ## License
138
 
139
  The ERNIE 4.5 models are provided under the Apache License 2.0. This license permits commercial use, subject to its terms and conditions. Copyright (c) 2025 Baidu, Inc. All Rights Reserved.
 
32
 
33
  # ERNIE-4.5-0.3B
34
 
35
+ > [!NOTE]
36
+ > Note: "**-Paddle**" models use [PaddlePaddle](https://github.com/PaddlePaddle/Paddle) weights, while "**-PT**" models use Transformer-style PyTorch weights.
37
+
38
  ## ERNIE 4.5 Highlights
39
 
40
  The advanced capabilities of the ERNIE 4.5 models, particularly the MoE-based A47B and A3B series, are underpinned by several key technical innovations:
 
91
  --max-num-seqs 32
92
  ```
93
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
94
  ## License
95
 
96
  The ERNIE 4.5 models are provided under the Apache License 2.0. This license permits commercial use, subject to its terms and conditions. Copyright (c) 2025 Baidu, Inc. All Rights Reserved.