@sometimesanotion Maybe this is useful to you! :)
python generate-rag-qav4.py \
--input-dir ./rag-input/ \
--output-dir ./rag-output/ \
--output-filename finetuning_qa_dataset \
--gen-model google/gemma-3-4b \
--gen-api-base http://127.0.0.1:1234/v1 \
--judge-model google/gemma-3-4b \
--judge-api-base http://127.0.0.1:1234/v1 \
--min-chunk-len 200 \
--question-chars 20 \
--answer-chars 5 \
--lang en
Oh also definitely look into this! Don't know how I forgot to mention it in my first post it's SUPER useful for RAG:
I know it's something very different from what you described, but have you read about AnythingLLM and their browser extension? I have been using it a lot and it works very well.
I also have been looking into MCP a lot lately (it seems to be very promising and imo is the next big thing happening right now) which could be used for this.
Finally, just because I found it super useful (although a bit unrelated), this python script that can turn pretty much any text data into a LLM-dataset is something I wanted to share with you as well (even though technically not RAG-related. It's been a while since we talked haha): https://www.reddit.com/r/LocalLLaMA/comments/1ai2gby/comment/korunem/?share_id=DFUUUr1ZD2ZCKFGXwccvF
username/repo ID.
OK nevermind I clicked that blog link and this is hella damn interesting how come I never heard of this haha. It states some really promising things right there... :o
the model that calls itself "Qwenconceited-14B-v13-DeepSuffering". <-- That cracked me up, lol!
And yeah very interesting but I'm going to have to read that again at another moment to fully understand all it is saying haha. Sounds like interesting stuff though!
Oh yeah for sure I'll hit you up sometime! Just to be clear I wasn't asking you to upload all your personal tweaks that you've spent probably weeks on to improve them haha. I was just curious about some of the things you said. For example, when you said " Extract a small LoRA from this" I was a little bit confused actually haha. As in: I have no idea how to do that, let alone apply it to smoothen out other models in the merge.
I know about adapter models and that you can create those with LoRA fine-tuning which you can either load on top during inference or you can merge with the base model, but extracting a LoRA from an existing model is kinda confusing me haha (sorry!). It sounds interesting though! Do I understand correctly that this would enable you to kind of "operate" on the model more precisely and with a lot less compute required (aka: more merges you can make and test in a given time window)?
Would you mind doing a writeup about your customized mergekit workflow, or do you prefer to keep some of the secret sauce to yourself? ;)
Or I guess the ReadMe as nobody can read that lol: https://huggingface.co/spaces/CultriX/MultiAgent-CodeTask/blob/main/README.md