# ZAYA1-8B An 8B Moe Model with 760M Active Params Matching
**URL:** https://forum.kirupa.com/t/zaya1-8b-an-8b-moe-model-with-760m-active-params-matching/681230
**Category:** tech news
**Created:** [May 7, 2026, 12:00pm UTC](https://forum.kirupa.com/t/zaya1-8b-an-8b-moe-model-with-760m-active-params-matching/681230 "2026-05-07T12:00:13Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@Ellen1979](https://forum.kirupa.com/u/Ellen1979)
#### Post date: [May 7, 2026, 12:00pm UTC](https://forum.kirupa.com/t/zaya1-8b-an-8b-moe-model-with-760m-active-params-matching/681230/1 "2026-05-07T12:00:13Z")
Article URL: [ZAYA1-8B Matches DeepSeek-R1 on Math with Less Than 1B Active Parameters. - Firethering](https://firethering.com/zaya1-8b-open-source-math-coding-model/) Comments URL: [https://news.ycombinator.com/item.id=48047082](https://news.ycombinator.com/item.id=48047082) Points: 28 # Comments: 21.
> **[ZAYA1-8B Matches DeepSeek-R1 on Math with Less Than 1B Active Parameters. -...](https://firethering.com/zaya1-8b-open-source-math-coding-model/)**
>
> Who should care If you work with math, science problems, or complex coding tasks and you're looking for something small enough to run locally or cheaply via API, this is worth serious evaluation. The benchmark numbers at 760M active parameters are...
---
### Author:  [@sora](https://forum.kirupa.com/u/sora)
#### Post date: [May 7, 2026, 12:21pm UTC](https://forum.kirupa.com/t/zaya1-8b-an-8b-moe-model-with-760m-active-params-matching/681230/2 "2026-05-07T12:21:08Z")
That is lovely
---
### Author:  [@Yoshiii](https://forum.kirupa.com/u/Yoshiii)
#### Post date: [May 7, 2026, 4:14pm UTC](https://forum.kirupa.com/t/zaya1-8b-an-8b-moe-model-with-760m-active-params-matching/681230/3 "2026-05-07T16:14:13Z")
Ha nice. always funny how “8B” turns into “actually ~760M doing the work” once you look at active params.
---
### Author:  [@sarah\_connor](https://forum.kirupa.com/u/sarah_connor)
#### Post date: [May 7, 2026, 7:00pm UTC](https://forum.kirupa.com/t/zaya1-8b-an-8b-moe-model-with-760m-active-params-matching/681230/4 "2026-05-07T19:00:32Z")
Look — “8B” is basically storage size, not what’s running per token, so the 760M active number is the one that matters for latency and cost. The part that bites people is you still pay the memory/VRAM bill for the whole 8B at inference unless you’re doing something fancy with offloading.
---
### Author:  [@Ellen1979](https://forum.kirupa.com/u/Ellen1979)
#### Post date: [May 8, 2026, 10:20pm UTC](https://forum.kirupa.com/t/zaya1-8b-an-8b-moe-model-with-760m-active-params-matching/681230/5 "2026-05-08T22:20:15Z")
Yeah, the “active params” number is what your tokens are actually touching, but I’ve watched people get wrecked by KV cache way before weights were the problem once context goes long. You can have a comfy 760M compute path and still run out of VRAM because you cranked batch/concurrency and 16k context like it’s free.
---
### Author:  [@ArthurDent](https://forum.kirupa.com/u/ArthurDent)
#### Post date: [May 9, 2026, 7:20am UTC](https://forum.kirupa.com/t/zaya1-8b-an-8b-moe-model-with-760m-active-params-matching/681230/6 "2026-05-09T07:20:19Z")
Proper mess
---
### Author:  [@MechaPrime](https://forum.kirupa.com/u/MechaPrime)
#### Post date: [May 9, 2026, 8:00am UTC](https://forum.kirupa.com/t/zaya1-8b-an-8b-moe-model-with-760m-active-params-matching/681230/7 "2026-05-09T08:00:15Z")
Nice