view post Post 465 no ai slop posted here today i just feel like posting what i did for todaywrote a little framework for turning multiple dense models (llama based) into Sparse MoEs.. i found it fun, spent the whole day and a half on it.code @ https://gist.github.com/cappuch/6a454ec8d2d349a27f9fd84f6ac90554 See translation
MoE Experiments (proper sparse MoEs) aldigobbler/smollmv2-360Mx4E-MoE-v0.1-unaligned-gates Updated May 27, 2025 • 1 aldigobbler/smollmv2-360Mx4E-MoE-v0.1 Updated May 28, 2025 • 3
MoE Experiments (proper sparse MoEs) aldigobbler/smollmv2-360Mx4E-MoE-v0.1-unaligned-gates Updated May 27, 2025 • 1 aldigobbler/smollmv2-360Mx4E-MoE-v0.1 Updated May 28, 2025 • 3
aldigobbler/canyon-racer-lfm2-230m-v1.5-gguf Reinforcement Learning • 0.2B • Updated about 21 hours ago • 84