Xperience albert.

albert. is loading: reading the architecture and building the 3D parts. This takes a few seconds.

albert. lädt: Die Architektur wird gelesen und die 3D-Teile werden gebaut. Das dauert einige Sekunden.

albert. is a ternary dual-stream mixture-of-experts language model. Every weight is −1, 0 or +1. It is a research model by RFI-IRFOS in Graz. This page shows its structure, the path the data takes through it, and its weights.

46 layers per stream, 2 streams
12 shared experts per layer; for each word a router picks 3 of them
7 exchange gates, where the streams swap information
331,140,622 parameters

Source code and model card · Model on Hugging Face