Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
tardellirsΒ 
posted an update 3 days ago
Post
7176
I built Model Pulse: daily download history for every model on the Hub πŸ“ˆ

A model page tells you its downloads for the last 30 days, but not how it got there. Model Pulse rebuilds the day-by-day history for 1.6M models, back to July 2024, from the daily snapshots in @cfahlgren1 's hub-stats dataset.

A few things the data shows:
- The Hub now serves ~100M model downloads a day, up from ~62M a year ago
- Vision-language models grew about 7x in that year, to 10.7M downloads a day
- 37% of Qwen3-8B's monthly downloads go to its 6,283 quantizations, fine-tunes and merges, not to the original

What you can do with it:
- Open any model, e.g. tardellirs/model-pulse
- Compare up to five models on one chart
- Browse weekly rankings: fastest growing, new this month, biggest families, top organizations
- Add a live badge to your model card (monthly downloads, sparkline and weekly trend)

The data is open too: modelpulse/model-pulse-data

I'd love feedback, especially from model authors: what would you want to see about your own models?

hub_series.parquet is the one file where "totals are unaffected" doesn't hold.

After a gap it books the per-day average on the first day back, and the missing days get no row. Checked against your own series/2026-08.parquet (sum of dl_all diffs per day):

  • normal days: hub / series = 1.000
  • 08-11, after a 3-day gap: 0.334
  • 08-27, after a 2-day gap: 0.500

So August sums 228M short. Across the whole file, 17 gaps hide 92 days. If the other 15 behave like these two, summing rows misses about 6.5B downloads, ~15% of the total.

Daily means are fine, so the 100M vs 62M headline holds. A monthly or yearly total built from this file doesn't. series/ and author_series/ carry cumulative dl_all, so they're unaffected.

Two easy fixes: write a row for each missing day at the average, or add a days column so sums can weight by it. Which fits the Space's charts better?

Β·

Thanks a lot, you're right, and your numbers check out exactly: 17 gaps, 92 missing days, 6.48B downloads (14.6%) missing from row sums. I went with your first option. hub_series.parquet now has one row per calendar day, with each gap's per-day average written to every day it covers, so sums are exact and the charts get a continuous series. The file in modelpulse/model-pulse-data is already updated (583 consecutive days, total 44.4B), and the daily job writes gaps the same way from now on. The report's headline numbers came from month-end cumulative totals, so they're unchanged. Really appreciate you checking it against series/, that's exactly why the data is open.

Checked 2f11da1d against the old file and series/2026-08: 583 days, step 1 everywhere, 44.376B. Every pre-existing row is untouched, the 92 new rows carry the average, and August now matches series/ window by window.

One leftover the calendar rule can't see: days that exist but are empty.

  • 2025-03-04, 2025-08-16, 2025-08-17, 2025-10-15: all 54 tags at exactly 0
  • 2025-08-19, 2026-07-20, 2026-08-28: 0.03 to 0.06M, against 47M to 189M on the days either side

Looks like two snapshots landing close together. Sums stay exact, since those downloads land on a neighbouring day, but the daily chart gets a hole where nothing happened.

Same averaging would cover it if the job could tell a short interval from a quiet day. Does each snapshot keep its fetch time, so a day could be weighted by hours elapsed?

Β·

Thanks again, this was a good catch. I checked fetch times first: each day is the last hub-stats commit of that UTC day, so I recovered the timestamps from the commit history. Around these days the snapshots are still about 24h apart (about 13:10 UTC), only 2025-03-04 comes 10h after the previous one, and the files differ every day (new models keep landing).
So it's not two snapshots close together: the Hub's download counters stood still that day and caught up over the next day or two, sometimes on the day before. Weighting by elapsed hours wouldn't fix it.

What's in now: a day below 30% of its 15-day median is a stall. It's grouped with the catch-up days around it (above 1.3Γ— the median), and the window's total is spread evenly across its days, tag by tag. That gives 10 windows, your 7 days plus partial ones on 2025-05-03/04, 2025-09-10, 2026-04-01, 2026-06-12/13 and 2026-09-09.
Sums are unchanged (44.376B, rounding aside), and no day is left under 30% of its median.
Per-model series skip the snapshots inside each window (listed as skip_days in meta.json), so model, family and author charts spread those days the same way. New stalls get settled automatically once their catch-up arrives.

Dataset commit: https://huggingface.co/datasets/modelpulse/model-pulse-data/commit/aaf4ef4c417ccad18d998e496772fba205337049

Checked aaf4ef4c against 2f11da1d. 30 days changed, in exactly your 10 windows. Every window's total is unchanged, 44.376B overall, all 56 tags flat inside each window, and no day is under 0.305 of its 15-day median. skip_days is each window minus its last day, which is the right snapshot to keep for the per-model series. The 10-04 daily job left all of it intact.

Thanks for pulling the fetch times. That kills my two-snapshot story.

What the 30% rule leaves behind is the half stalls. 34 days sit between 0.3 and 0.7 of their median, and 12 of them have a 1.3x to 3.3x day right next to them:

  • 2025-10-08 at 0.50, 10-09 at 1.86
  • 2025-11-26 at 0.46, 11-27 at 1.95
  • 2026-07-29 at 0.44, 07-30 at 1.78
  • 2026-08-12 at 0.56, 08-13 at 3.28

Same shape as the full stalls, half the depth. A pair rule would fold them in: a day under 0.7 next to a day over 1.3, where the two sum to about twice the median. It would leave 2025-05-21 at 7.45x alone, since that one has no low neighbour.

Is 08-13 a spike you can name, or a counter catching up?

Β·

Thanks, this was worth digging into, and it changed the data.

The half stalls are real, and they're the same events as the dataset stalls. On those days the dataset total goes to zero, with every dataset frozen. On the model side only part of the counters freeze, and it splits by repo age: about 98% of models created before 2022 keep counting (BERT, CLIP, all-MiniLM-L6-v2), while almost no repo from mid-2025 on does. The old giants are what hold the total at half. 20 of the 21 such days are Wednesdays, and snapshot spacing is 24.0 h on all of them, so it's not timing.

08-13 is a counter catching up. On 08-12, 80% of models with 2k+ daily downloads were frozen at exactly zero (bge-small-en-v1.5, bge-m3, chronos-2, MiniMax-H3), while all-MiniLM-L6-v2 kept counting at twice its usual rate. On 08-13, 94% of them were above 1.8x their own median, and the top 20 explain only 30% of the excess. Frozen models do 2–4x the next day, so these downloads are delayed, not lost.

Your pair rule catches most of them but not all. The share of frozen counters also catches stalls that barely dip the total (2025-12-17, 2026-05-12, 2026-09-02), and misses ones where counters crawled instead of stopping. 06-25 and 08-01 have nothing frozen and normal datasets, but every model sits at about 0.5x and then 1.6–1.9x, so they read as a shifted count too. A day is now a stall if any of these holds: total under 0.3x of the median, at least 25% of models with 60K+ monthly downloads frozen, or your pair test (under 0.7x, made up by a neighbour to about 2x). Datasets use a 90% frozen bar, since they freeze all at once.

And 2025-05-21 wasn't a spike. On 05-19 and 05-20, dl_all went down for 16–19% of models and came back days later. We clipped the drop but counted the rebound: 440M phantom downloads. The same happened from 2026-06-13, another 537M. Snapshots during a rollback are now set aside, and downloads are measured across them.

It's live. The model side has 27 windows and skip_days went from 20 to 59 (datasets 57 to 65). The Hub total dropped 0.99B to 43.48B, all of it from the two rollbacks, and days outside 0.7–1.3x of the median went from 70 to 40. 08-01/02 is the one pair that still slips through, right at the edge of the sum band.

Checked d48742f7 against 9bbf6b62. 43.476B, down 0.986B, and all of it sits in the two rollback windows: 05-19 to 05-23 gives back 440.2M, 06-12 to 06-19 gives back 536.9M. The per-model series agree: dl_all went down for 16.0% of models on 05-19, 19.0% on 05-20, and 39.3% on 06-13. Days outside 0.7 to 1.3x went 70 to 40. skip_days is 59, every stall window minus its last snapshot, with 04-01 running straight into the gap behind it. The Wednesday pattern holds at the window level too: 18 of 26 stall windows start on one.

What the three rules leave is 13 runs under 0.7x, 20 days, 769M below the median, 1.8% of the total. The next week brings back 486M of it. A normal week brings back about 13M on its own, so call it 300M made up and 450M that never reappears.

The clearest case is 01-28 to 02-01: five days, 137M under the median, 28M back the week after. 01-28 was a freeze, 48% of the 60K+ models flat, so your rule fires and pairs it with 01-29. But 01-29 was itself a low day, and 01-30 lower: 1% frozen on both, the median 60K+ model at 0.56x and then 0.37x of its own usual day. A crawl, with nothing on either side making it up.

A uniform 0.37x across 2,158 models is hard to read as demand. But spreading only moves downloads around, it cannot bring back the missing ones. Would you mark these days in meta.json, a low_days list beside skip_days, so a chart can grey them out instead of smoothing over them?

Β·

Thanks for checking it against both commits. Your numbers match mine: 43.476B, 13 runs under 0.7x, 769M below the median, 486M back the week after.

Before adding low_days I went through the rest of the series the same way, and it changed a few things:

  • 2025-11-24 was a partial snapshot, 289k of 1.13M models. It's now left out and listed under a new partial key; the total moves to 43.494B.
  • A stall window could stop at the edge of days without a snapshot, leaving a hole followed by a flat high stretch. Windows now take the whole stretch, up to the snapshot that closes it. That's what 2026-04-01 was: it ran into 15 days without a snapshot, and is now one window at 81M/day.
  • 13 of the 24 days under 0.7x were made up within three days (08-01 and 08-02 sum to 2.403x, just over the pair cap), and a few were ordinary Sundays and Mondays, which run about 0.87x.

So low_days is now: under 0.7 of a weekday-adjusted 15-day median, in runs the three days on each side don't make up by at least half. That leaves 8 days for models: 2025-03-04/05, 2025-11-08/09 and your 2026-01-28 to 01-31 (repos_meta.json has 12 for datasets). They keep their measured values, the Hub chart greys them out, and the dataset card describes the rule.

Your 01-28 case holds: a crawl that nothing on either side made up.

Checked 69090f3b. low_days is the 8 you list, partial is 2025-11-24, and the card states the rule.

The rule also passes the case I would have flagged next. 2025-12-04 and 12-05 sit at 0.40 and 0.42 of the 15-day median, then four days at 1.19x bring it back. Left out, correctly.

One thing the two lists say together. 5 of the 8 model low days are dataset low days too: 2025-03-04/05 and 2026-01-28 to 01-30. Two kinds of repo, short on the same days, never made up. That reads like requests the Hub never counted, not a counter running late. 2025-11-08/09 is the odd one: models only, on a weekend.

Did anything else you track dip on 01-30, Spaces or the paper pages?

Β·

Neither dipped on 01-30, and the likes in the same snapshots say more than Spaces or papers do.

Model likes gained per day, from the same hub-stats snapshots as the downloads, were 1.19, 1.14 and 1.05 of their 15-day median on 01-28 to 01-30, and the number of models gaining a like was normal (1.05, 1.10, 0.95), while downloads ran at 0.47, 0.43 and 0.31. Dataset likes: 0.91, 1.04, 0.91, with downloads at 0, 1.34 and 0.56. The snapshots were on time too (13:30 to 13:39 UTC, about 24h apart). So the snapshots were fresh and people were on the site; only the download counters fell behind, and since dl_all never caught up, those requests were never counted. That fits your reading.

Spaces likes: 1.00, 0.99, 1.03. Daily Papers upvotes (from hysts-bot-data/daily-papers-stats, which I now follow in https://huggingface.co/spaces/tardellirs/paper-pulse): 0.71, 1.32, 1.27 after a weekday adjustment, though at ~200 upvotes a day that's a weak signal.

One correction to the overlap: 2025-03-04/05 is a different thing. hub-stats moved its collection from about 14:20 to 00:35 UTC that day, so the 03-04 snapshot came 10.3 hours after the one before. Likes ran at 0.30 as well and no download counter moved at all: a short day from the schedule change, not uncounted requests. That leaves 01-28 to 01-30 as the shared one.

2025-11-08/09: snapshots on time, model likes slightly low (0.93, 0.86), paper upvotes too (0.65, 0.68), downloads lower (0.68, 0.60). Mostly a quiet weekend, though I can't rule out a few uncounted downloads. Spaces can't tell: hub-stats has no Space snapshots from 11-04 to 11-23.

Checked a91d3cc3. short_days is in, 03-04 is off low_days, and the card says why.

I take 03-04/05 back from the overlap. Your series agrees with you: the two days carry 35.8M each, identical on all 54 pipeline tags, and together they make 71.7M against a 15-day median of 66 to 69M. One ordinary day on two labels. Nothing missing.

That leaves 01-28 to 01-30, and your likes numbers made me cut it one more way. By pipeline tag, from hub_series, those three days against each tag's own 15-day median:

  • All 18 tags above 0.5M downloads a day are under 0.7. Range 0.39 to 0.67, median 0.44.
  • Two weeks earlier (01-14 to 01-16) the same 18 tags run 1.04 to 1.31.

So it was not one family or one client going quiet. sentence-similarity (0.47), text-generation (0.54) and image-classification (0.41) lost about the same share on the same days, while likes ran above median. A change in usage does not halve every task at once. A counter can.

One thing I can't settle from the totals. 01-31 is on the list and 02-01 is not, yet both read 0.69 of the plain median in my cut. Is 02-01 the tail of the same event, or an ordinary Sunday once the weekday adjustment is in?

Β·

Neither, quite: there was no snapshot on 01-31. The 02-01 snapshot came 47.6 hours after 01-30's, so 01-31 and 02-01 are one measurement spread over two days, 42.3M each. The weekday adjustment then split it: Saturday's factor is 1.00 and Sunday's 0.88, so the same number read 0.69 on 01-31 and 0.78 on 02-01. That split was the bug. As one measurement the weekend comes out at 0.735, so neither day is low. So the event is the three weekdays; whether part of the weekend's shortfall belongs to it, a two-day average can't tell.

The same mistake was in two more places, so low_days and the stall rules now treat a value spread over days without a snapshot as one measurement throughout. The pair rule now catches datasets 2026-03-17/18 (0.51 over two days, then 2.21 on 03-19): one window now. The weekday pattern is estimated from days with their own snapshot, which moves three borderline dataset days across 0.7. Models: low_days is 2025-11-08/09 and 2026-01-28 to 01-30 (dce0806f), and the card says why.

Your tag cut checks out on my side: tags above 0.5M a day sit at 0.39 to 0.67 (median 0.44) on 01-28 to 01-30, and 1.04 to 1.31 two weeks earlier. I count 16 tags rather than 18 at that threshold, probably a different window for the cutoff.