Foundational model architecture for Mixture of Experts by aquif. State-of-the-art performance and economical training.