Spaces:
Running
on
Zero
Running
on
Zero
adding companion blog post link
Browse files
app.py
CHANGED
@@ -44,7 +44,7 @@ class Config:
|
|
44 |
f"1. **Byte Latent Transformer (BLT):** Entropy-based patching plot and patched text. Spaces are replaced by '_' for viz purposes. Using `blt_main_entropy_100m_512w`.\n"
|
45 |
f"2. **Tiktoken (GPT-4):** Text segmented by `{TIKTOKEN_ENCODING_NAME}` tokens.\n"
|
46 |
f"3. **Llama 3:** Text segmented by the `{LLAMA3_MODEL_NAME}` tokenizer.\n\n\n"
|
47 |
-
"
|
48 |
)
|
49 |
|
50 |
# --- Bytelatent Processor ---
|
|
|
44 |
f"1. **Byte Latent Transformer (BLT):** Entropy-based patching plot and patched text. Spaces are replaced by '_' for viz purposes. Using `blt_main_entropy_100m_512w`.\n"
|
45 |
f"2. **Tiktoken (GPT-4):** Text segmented by `{TIKTOKEN_ENCODING_NAME}` tokens.\n"
|
46 |
f"3. **Llama 3:** Text segmented by the `{LLAMA3_MODEL_NAME}` tokenizer.\n\n\n"
|
47 |
+
"Companion blog post [can be found here](https://lucalp.dev/bitter-lesson-tokenization-and-blt)."
|
48 |
)
|
49 |
|
50 |
# --- Bytelatent Processor ---
|