He/Him. I like LLM's and VLM's. I work with my other friends to make stuff. We are in year 7 and we are enthusiastic about AI. We are based in Australia š¦šŗ
G1-MINI has now seen around 8B tokens during its current run, and pretraining is still going strong.
Our E1 (Efficiency-1) prototype has also reached 15B pretraining tokens. E1 has 1B total parameters while activating under 100M parameters per token. It features adaptive activation, meaning easier tokens can use less compute while harder tokens receive more.
We plan to open-source E1 ASAP! š
Weāre also excited to announce Project Prism, which will provide limited access to our upcoming Orion Flagship model, powered by our T2 architecture.
Note: T2 here refers to the architecture, not our T2 (Thinker-2) model.
Applications for Project Prism are available through the org page, with more details coming soon!
Finally, welcome @soyL061215, who joined the Hugging Face org today! š
Hello Everyone! I am happy to announce a few things. 1. G1-MINI G1-MINI is now in pretraining and is training at a steady pace. Our current ETAs state completion and launch in about 15-20 days, somewhere near the end of September. 2. G1-NANO G1-NANO is also being pretrained as we speak at a pace of over 400K tokens per second processing more than 10B tokens in 12 hours. This allows us to train extremely fast, and we will launch it somewhere around 15th September. 3. We have begun work on FrameShot, a dual image and video generation model at around 4B dense parameters. This is expected to launch around late December with no promised date. - Bc-AI