Hugging Face's logo
AbstractPhila's picture
🤝 Open to Collab

AbstractPhila PRO

AbstractPhil
AbstractPowered
19 6 30
P(doom) Rejects the premise
xziayro's profile picture Ujjwal-Tyagi's profile picture tegridydev's profile picture
·
https://civitai.com/user/AbstractPhila

AI & ML interests

datasets, research papers, experimentation, vision, classification, text encoders, tokenization, llms, diffusion, distillation, and more.

Recent Activity

posted an update 3 minutes ago
Mini-Beatrix-V2-control-softmax is complete. The model is substantially more intelligent than the counterparts. https://huggingface.co/AbstractPhil/alephllm-mini-beatrix-training/tree/main/mini-beatrix-2s-control-attn-fp16-r10k --- With that information I have built the tests required to create a smaller and substantially more powerful variant. Nano-Beatrix-V4 will have hundreds of trains as intelligent as the V3 version within a few days. I will be slowly pushing up the capacity as each will train a hundred thousand steps within a couple hours on a colab card. Roughly 400k byte/s or so, these shallow models have nearly perfect recon at 4096 context length. HOWEVER, a full matching corpus train that fed Beatrix V3 60 billion bytes - is only about 2 days or less on a colab card vs the near 20 for the v3. RECON won't guarantee intelligence, however these tiny models here will train so quickly that I can directly test which is eating what space and why, then increase that with minimal cost through direct minimal expansion and then genetic interpolation, until the model hits around 200k token/s or so. Essentially these lightning quick little models will be capable of attaching to other models, training miniature modular routing systems with the quiet loss structure attached to other models. They won't control a big model, but that's what the mini version is for.
updated a model 44 minutes ago
AbstractPhil/alephllm-mini-beatrix-training
updated a bucket about 2 hours ago
AbstractPhil/alephllm-chat-storage
View all activity

Organizations

DeepGHS's profile picture Blog-explorers's profile picture BangumiBase's profile picture Abstract Powered Research's profile picture