Add example of chat templates for function calling (#3)
Browse files- Add example of chat templates for function calling (975d07b5c9e889b8f6d9c3c8d84dc245d3a87022)
Co-authored-by: Matthew Carrigan <Rocketknight1@users.noreply.huggingface.co>
README.md
CHANGED
|
@@ -131,6 +131,51 @@ The stock fundamentals data for Tesla (TSLA) are as follows:
|
|
| 131 |
This information provides a snapshot of Tesla's financial position and performance based on the fundamental data obtained from the yfinance API. It shows that Tesla has a substantial market capitalization and a relatively high P/E and P/B ratio compared to other stocks in its industry. The company does not pay a dividend at the moment, which is reflected by a 'Dividend Yield' of 'None'. The Beta value indicates that Tesla's stock has a moderate level of volatility relative to the market. The 52-week high and low prices give an idea of the stock's range over the past year. This data can be useful when assessing investment opportunities and making investment decisions.<|im_end|>
|
| 132 |
```
|
| 133 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 134 |
## Prompt Format for JSON Mode / Structured Outputs
|
| 135 |
|
| 136 |
Our model was also trained on a specific system prompt for Structured Outputs, which should respond with **only** a json object response, in a specific json schema.
|
|
|
|
| 131 |
This information provides a snapshot of Tesla's financial position and performance based on the fundamental data obtained from the yfinance API. It shows that Tesla has a substantial market capitalization and a relatively high P/E and P/B ratio compared to other stocks in its industry. The company does not pay a dividend at the moment, which is reflected by a 'Dividend Yield' of 'None'. The Beta value indicates that Tesla's stock has a moderate level of volatility relative to the market. The 52-week high and low prices give an idea of the stock's range over the past year. This data can be useful when assessing investment opportunities and making investment decisions.<|im_end|>
|
| 132 |
```
|
| 133 |
|
| 134 |
+
## Chat Templates for function calling
|
| 135 |
+
|
| 136 |
+
You can also use chat templates for function calling. For more information, please see the relevant section of the [chat template documentation](https://huggingface.co/docs/transformers/en/chat_templating#advanced-tool-use--function-calling).
|
| 137 |
+
|
| 138 |
+
Here is a brief example of this approach:
|
| 139 |
+
|
| 140 |
+
```python
|
| 141 |
+
def multiply(a: int, b: int):
|
| 142 |
+
"""
|
| 143 |
+
A function that multiplies two numbers
|
| 144 |
+
|
| 145 |
+
Args:
|
| 146 |
+
a: The first number to multiply
|
| 147 |
+
b: The second number to multiply
|
| 148 |
+
"""
|
| 149 |
+
return int(a) * int(b)
|
| 150 |
+
|
| 151 |
+
tools = [multiply] # Only one tool in this example, but you probably want multiple!
|
| 152 |
+
|
| 153 |
+
model_input = tokenizer.apply_chat_template(
|
| 154 |
+
messages,
|
| 155 |
+
tools=tools
|
| 156 |
+
)
|
| 157 |
+
```
|
| 158 |
+
|
| 159 |
+
The docstrings and type hints of the functions will be used to generate a function schema that will be read by the chat template and passed to the model.
|
| 160 |
+
Please make sure you include a docstring in the same format as this example!
|
| 161 |
+
|
| 162 |
+
If the model makes a tool call, you can append the tool call to the conversation like so:
|
| 163 |
+
|
| 164 |
+
```python
|
| 165 |
+
tool_call_id = "vAHdf3" # Random ID, should be unique for each tool call
|
| 166 |
+
tool_call = {"name": "multiply", "arguments": {"a": "6", "b": "7"}}
|
| 167 |
+
messages.append({"role": "assistant", "tool_calls": [{"id": tool_call_id, "type": "function", "function": tool_call}]})
|
| 168 |
+
```
|
| 169 |
+
|
| 170 |
+
Next, call the tool function and append the tool result:
|
| 171 |
+
|
| 172 |
+
```python
|
| 173 |
+
messages.append({"role": "tool", "tool_call_id": tool_call_id, "name": "multiply", "content": "42"})
|
| 174 |
+
```
|
| 175 |
+
|
| 176 |
+
And finally apply the chat template to the updated `messages` list and `generate()` text once again to continue the conversation.
|
| 177 |
+
|
| 178 |
+
|
| 179 |
## Prompt Format for JSON Mode / Structured Outputs
|
| 180 |
|
| 181 |
Our model was also trained on a specific system prompt for Structured Outputs, which should respond with **only** a json object response, in a specific json schema.
|