Model Card for org-chatbot-id/gemma3-sealion-27b-multiturn-mentalhealth-id

This model is finetuned and aligned (Supervised Finetuning and DPO-based Preference Tuning) on a new constructed mental health dataset in Indonesian language. The finetuned LLM is then used as a backbone model for a prototype of friendly and emphatic Chatbot application based on Low Intensity Psychological Interventions intended for mental health problems. The current target users for this prototype are high school students (adolescents) in Garut, West Java, Indonesia.

Model Details

Base model before finetuning: aisingapore/Gemma-SEA-LION-v4-27B-IT

Model Description

• Developed by: BRIN Indonesia
• Funded by [optional]: BRIN Indonesia and a collaboration with netmind.ai for inference resources (GPUs for inference)
• Shared by [optional]: BRIN Indonesia and a collaboration with netmind.ai for inference resources (GPUs for inference)
• Model type: Chatbot for Multi-turn conversational tasks
• Language(s) (NLP): Indonesian
• License: cc-by-nc-4.0
• Finetuned from model [optional]: aisingapore/Gemma-SEA-LION-v4-27B-IT

Model Sources [optional]

• Repository: Github-Frontend
• Paper [optional]: Rini Wijayanti, Iftitahu Ni’mah, Ekasari Nugraheni, and Ana Heriyana. “Explaining Mental Disorder Classification in Dialogues: Turn-Level Analysis and Label Dynamics”. In 2025 International Conference on Computer, Control, Informatics and its Applications (IC3INA), Jakarta, Indonesia, October 2025. IEEE
• Demo [optional]: temancerita.online

Installation (Merging)

Reassemble the file: cat gemma3-sealion-27b-part-* > gemma3-sealion-27b-dpo-BF16.gguf

Verify integrity (if you have checksums): md5sum gemma3-sealion-27b-dpo-BF16.gguf

Upload as ollama model (in the same directory of GGUF model): ollama create gemma3-sealion-27b-multiturn-mentalhealth-id-dpo -f ./Modelfile

Check whether model has been uploaded under ollama service: ollama list

Out-of-Scope Use

This model is still under a constant development.

Bias, Risks, and Limitations

USE WITH PRECAUTIONS: This model has not been aligned with nuances, cultures, social norm, religion values that can be occured in a conversation between high school students in Indonesian and AI Chatbot.

Training Details

Training Data

1. Supervised Finetuning Data

A. Synthetic Augmented Multi-Turn Conversation Dataset.

Our augmentation steps for preparing the finetuning dataset are as follows:

• We translated the single-turn dialogue from English to Indonesian language.
• We classify samples that have relevant topics within the categories for adolescent minor mental health (manuscript is currently under reviewed in IC3INA2025 conference):
  (i) Personality Disorders (ii) Anxiety Disorders (iii) Depressive Disorders (iv) Impulse-Control Disorders (v) Substance-Related and Addictive Disorders (vi) Trauma- and Stressor-Related Disorders (vii) Neurodevelopmental Disorders (viii) Feeding and Eating Disorders
• From the above filtered samples, we augment the single-turn dialogue samples to multi-turn dialogue samples based on our constructed prompt for OpenAI gpt-4.1.
• The prompt was carefully designed with the guidance of public health researchers (BRIN Indonesia), doctors (BRIN Indonesia), and psychologists (Universitas Indonesia). The prompt contains rules from the expertise to converse with the patients diagnosed with minor mental health problems, particularly for high school students.
• We iteratively created n-version of prompts or LLM instructions for data augmentation and confered with the experts before deciding the final system prompt for generating a multi-turn dialogue sample.
• Finally, we split the finetuning data into: training (2484 samples), validation (712), and test (890) subset.

B. Augmented Interview Dataset from two high schools in Garut, West Java, Indonesia (18 samples).

C. Augmented conversation from ESCONV dataset, translated in Indonesian language and sampled based on relevant topics for high school students (150 samples).

2. Preference Tuning Data

We instructed the experts to converse with the chatbot version 1.0, and correspondingly provide the human preferred chatbot responses (Chosen versus Rejected response pairs).

Training Procedure

Training Hyperparameters

• Training regime: [More Information Needed]

• • LORA parameters: r = 16; target_modules = ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj",]; lora_alpha = 16; lora_dropout = 0;

• Epoch: 1; per_device_train_batch_size = 2; gradient_accumulation_steps = 4; max_steps=-1; learning_rate = 2e-5; warmup_steps = 5; logging_steps = 1; eval_strategy="steps"; eval_steps=100; max_seq_length = 2048; lr_scheduler_type = "linear".

• Precision: torch.bfloat16, load_in_4bit=True.

• GPU: 1 node A100 40GB.

Evaluation

The evaluation is an ongoing work.

Testing Data, Factors & Metrics

Testing Data

[More Information Needed]

Factors

[More Information Needed]

Metrics

[More Information Needed]

Results

[More Information Needed]

Summary

Model Examination [optional]

[More Information Needed]

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: [More Information Needed]
• Hours used: [More Information Needed]
• Cloud Provider: [More Information Needed]
• Compute Region: [More Information Needed]
• Carbon Emitted: [More Information Needed]

Technical Specifications [optional]

Model Architecture and Objective

[More Information Needed]

Compute Infrastructure

[More Information Needed]

Hardware

[More Information Needed]

Software

[More Information Needed]

Citation [optional]

BibTeX:

[More Information Needed]

APA:

[More Information Needed]

Glossary [optional]

[More Information Needed]

More Information [optional]

[More Information Needed]

Model Card Authors [optional]

[More Information Needed]

Model Card Contact

[More Information Needed]