No auth required. Trending data is public — send the standard required URL parameters (
client_id, app-name, app-version) and a User-Agent header, but no user Authorization token.Which IDs can I send/expect? All accepted input identifiers and the keys you’ll see echoed back in responses are listed at Standard media objects → Supported ID keys. Send every ID you have on writes — Simkl picks the first that resolves and ignores the rest. Reminder:
slug is response-only (never send it on a request).Attribution required. When you display trending data in your app or website, the section title must include Simkl alongside Trending (or an equivalent like Most Watched / Popular). Any sensible combination works — feel free to invent your own wording, as long as both ideas appear together. A few examples to get you started:
For commercial use without attribution, contact us — we’re happy to discuss licensing.
Simkl Trending Movies | Trending Movies on Simkl | What's Trending on Simkl |
Simkl Trending TV Shows | Trending TV on Simkl | Now Trending — Simkl |
Simkl Trending Anime | Trending Anime on Simkl | Trending Now · Powered by Simkl |
Simkl Trending Today | Trending Today on Simkl | Today on Simkl |
Simkl Trending This Week | Trending This Week on Simkl | This Week's Top Picks — Simkl |
Simkl Most Watched | Most Watched on Simkl | Popular on Simkl |
Simkl Top 100 Movies | Top 100 on Simkl | Hot Right Now — Simkl |
Simkl Top Charts | Top Charts on Simkl | Charting on Simkl |
Linking back (websites only)
If your client can render hyperlinks — websites, browser extensions, web apps — link the title to the matching Simkl Most Watched page. TV apps, consoles, CLIs, and other contexts that can’t open external URLs are exempt; the title alone is enough.Most Watched Movies
simkl.com/movies/best-movies/most-watched
Most Watched TV
simkl.com/tv/best-shows/most-watched
Most Watched Anime
simkl.com/anime/best-anime/most-watched
_100) or top 500 (_500) items. Titles with the most watchers are returned first.
At a glance
Sizes
Top 100 (
_100) and Top 500 (_500) per file.Last-Modified
Each file’s
Last-Modified response header tells you exactly when it was generated.Update frequency
| Data | Description | Update frequency |
|---|---|---|
| Today | Most watched titles over the last 24 hours | Every hour |
| Week | Most watched titles over the last 7 days | Once a day |
| Month | Most watched titles over the last 30 days | Once a day |
| DVD releases | Latest popular DVD releases (Movies only) | Once a day |
Regeneration is best-effort, not strictly precise. The cadences above are targets — the actual job can take a few extra minutes (and occasionally longer) to finish. If your scheduled refresh hits a
304 Not Modified right at the expected time, the new file just hasn’t been generated yet; back off and retry in a few minutes rather than waiting another full cycle. See SimklTrendingClient — drop-in SDK for the recommended pattern.The file URLs ignore all query strings. Don’t add
?random=... or ?nocache=... — the CDN treats every variant as the same resource. Simkl regenerates the files on the schedule above and automatically clears them from the Cloudflare cache, so you’ll always get the latest version on your next request — client-side cache-busting won’t deliver newer data.All trending data files
Append?client_id=YOUR_CLIENT_ID&app-name=my-app-name&app-version=1.0 to every URL below — the SimklTrendingClient does this for you from CONFIG; if you’re rolling your own, see Without the SDK.
Combined
Movies + TV + anime in a single response. Top-level keys:movies, tv, anime. Best when you want all categories at once and don’t want three round-trips.
| Timeframe | Top 100 | Top 500 |
|---|---|---|
| Today | today_100.json | today_500.json |
| This week | week_100.json | week_500.json |
| This month | month_100.json | month_500.json |
By type — Movies, TV, Anime
One category per file. Top-level response is the array directly (no wrapper object). Best when you only render one category, or want to fetch them independently.| Category | Timeframe | Top 100 | Top 500 |
|---|---|---|---|
| Movies | Today | movies/today_100.json | movies/today_500.json |
| Movies | This week | movies/week_100.json | movies/week_500.json |
| Movies | This month | movies/month_100.json | movies/month_500.json |
| TV | Today | tv/today_100.json | tv/today_500.json |
| TV | This week | tv/week_100.json | tv/week_500.json |
| TV | This month | tv/month_100.json | tv/month_500.json |
| Anime | Today | anime/today_100.json | anime/today_500.json |
| Anime | This week | anime/week_100.json | anime/week_500.json |
| Anime | This month | anime/month_100.json | anime/month_500.json |
DVD releases (movies only)
Latest popular DVD releases — mirrors Simkl DVD Releases. No timeframe split; refreshes once a day.| Top 100 | Top 500 |
|---|---|
| dvd/releases_100.json | dvd/releases_500.json |
Copy-paste JSON
Drop this object into your app as a constant. ReplaceYOUR_CLIENT_ID with your registered app’s client_id and my-app-name / 1.0 with your app’s name and version — both the URL params (required_query) AND the matching User-Agent header (user_agent_header) are required on every request, free or paid plan.
Each entry has a title (attribution-compliant display label), a url (bare path — append ? + required_query when you fetch), and a refresh_seconds (how often the file is regenerated server-side — set your refresh timer to this value). The structure is category → timeframe → size so you can filter or iterate by any axis. DVD has no timeframe.
{
"required_query": "client_id=YOUR_CLIENT_ID&app-name=my-app-name&app-version=1.0",
"user_agent_header": "my-app-name/1.0",
"urls": {
"combined": {
"today": {
"top_100": { "title": "Trending on Simkl — Today (Top 100)", "url": "https://data.simkl.in/discover/trending/today_100.json", "refresh_seconds": 3600 },
"top_500": { "title": "Trending on Simkl — Today (Top 500)", "url": "https://data.simkl.in/discover/trending/today_500.json", "refresh_seconds": 3600 }
},
"week": {
"top_100": { "title": "Trending on Simkl — This Week (Top 100)", "url": "https://data.simkl.in/discover/trending/week_100.json", "refresh_seconds": 86400 },
"top_500": { "title": "Trending on Simkl — This Week (Top 500)", "url": "https://data.simkl.in/discover/trending/week_500.json", "refresh_seconds": 86400 }
},
"month": {
"top_100": { "title": "Trending on Simkl — This Month (Top 100)", "url": "https://data.simkl.in/discover/trending/month_100.json", "refresh_seconds": 86400 },
"top_500": { "title": "Trending on Simkl — This Month (Top 500)", "url": "https://data.simkl.in/discover/trending/month_500.json", "refresh_seconds": 86400 }
}
},
"movies": {
"today": {
"top_100": { "title": "Trending Movies on Simkl — Today (Top 100)", "url": "https://data.simkl.in/discover/trending/movies/today_100.json", "refresh_seconds": 3600 },
"top_500": { "title": "Trending Movies on Simkl — Today (Top 500)", "url": "https://data.simkl.in/discover/trending/movies/today_500.json", "refresh_seconds": 3600 }
},
"week": {
"top_100": { "title": "Trending Movies on Simkl — This Week (Top 100)", "url": "https://data.simkl.in/discover/trending/movies/week_100.json", "refresh_seconds": 86400 },
"top_500": { "title": "Trending Movies on Simkl — This Week (Top 500)", "url": "https://data.simkl.in/discover/trending/movies/week_500.json", "refresh_seconds": 86400 }
},
"month": {
"top_100": { "title": "Trending Movies on Simkl — This Month (Top 100)", "url": "https://data.simkl.in/discover/trending/movies/month_100.json", "refresh_seconds": 86400 },
"top_500": { "title": "Trending Movies on Simkl — This Month (Top 500)", "url": "https://data.simkl.in/discover/trending/movies/month_500.json", "refresh_seconds": 86400 }
}
},
"tv": {
"today": {
"top_100": { "title": "Trending TV Shows on Simkl — Today (Top 100)", "url": "https://data.simkl.in/discover/trending/tv/today_100.json", "refresh_seconds": 3600 },
"top_500": { "title": "Trending TV Shows on Simkl — Today (Top 500)", "url": "https://data.simkl.in/discover/trending/tv/today_500.json", "refresh_seconds": 3600 }
},
"week": {
"top_100": { "title": "Trending TV Shows on Simkl — This Week (Top 100)", "url": "https://data.simkl.in/discover/trending/tv/week_100.json", "refresh_seconds": 86400 },
"top_500": { "title": "Trending TV Shows on Simkl — This Week (Top 500)", "url": "https://data.simkl.in/discover/trending/tv/week_500.json", "refresh_seconds": 86400 }
},
"month": {
"top_100": { "title": "Trending TV Shows on Simkl — This Month (Top 100)", "url": "https://data.simkl.in/discover/trending/tv/month_100.json", "refresh_seconds": 86400 },
"top_500": { "title": "Trending TV Shows on Simkl — This Month (Top 500)", "url": "https://data.simkl.in/discover/trending/tv/month_500.json", "refresh_seconds": 86400 }
}
},
"anime": {
"today": {
"top_100": { "title": "Trending Anime on Simkl — Today (Top 100)", "url": "https://data.simkl.in/discover/trending/anime/today_100.json", "refresh_seconds": 3600 },
"top_500": { "title": "Trending Anime on Simkl — Today (Top 500)", "url": "https://data.simkl.in/discover/trending/anime/today_500.json", "refresh_seconds": 3600 }
},
"week": {
"top_100": { "title": "Trending Anime on Simkl — This Week (Top 100)", "url": "https://data.simkl.in/discover/trending/anime/week_100.json", "refresh_seconds": 86400 },
"top_500": { "title": "Trending Anime on Simkl — This Week (Top 500)", "url": "https://data.simkl.in/discover/trending/anime/week_500.json", "refresh_seconds": 86400 }
},
"month": {
"top_100": { "title": "Trending Anime on Simkl — This Month (Top 100)", "url": "https://data.simkl.in/discover/trending/anime/month_100.json", "refresh_seconds": 86400 },
"top_500": { "title": "Trending Anime on Simkl — This Month (Top 500)", "url": "https://data.simkl.in/discover/trending/anime/month_500.json", "refresh_seconds": 86400 }
}
},
"dvd": {
"top_100": { "title": "Latest DVD Releases on Simkl (Top 100)", "url": "https://data.simkl.in/discover/dvd/releases_100.json", "refresh_seconds": 86400 },
"top_500": { "title": "Latest DVD Releases on Simkl (Top 500)", "url": "https://data.simkl.in/discover/dvd/releases_500.json", "refresh_seconds": 86400 }
}
},
"catalogs": [
{ "id": "movies_today", "source": "movies", "timeframe": "today" },
{ "id": "tv_today", "source": "tv", "timeframe": "today" },
{ "id": "anime_today", "source": "anime", "timeframe": "today" },
{ "id": "dvd", "source": "dvd" },
{ "id": "tv_premieres", "source": "tv", "title": "Trending TV Premieres on Simkl (Last 30 Days)", "recipe": "justReleased", "recipe_opts": { "withinDays": 30 } },
{ "id": "movies_premieres", "source": "movies", "title": "Trending Movie Premieres on Simkl (Last 30 Days)", "recipe": "justReleased", "recipe_opts": { "withinDays": 30 } },
{ "id": "tv_ongoing", "source": "tv", "title": "Currently Airing TV on Simkl", "recipe": "ongoing" },
{ "id": "movies_this_year", "source": "movies", "title": "Movies This Year on Simkl", "recipe": "currentYear" },
{ "id": "tv_this_year", "source": "tv", "title": "TV This Year on Simkl", "recipe": "currentYear" },
{ "id": "anime_this_year", "source": "anime", "title": "Anime This Year on Simkl", "recipe": "currentYear" },
{ "id": "movies_last_year", "source": "movies", "title": "Movies of Last Year on Simkl", "recipe": "lastYear" },
{ "id": "tv_last_year", "source": "tv", "title": "TV of Last Year on Simkl", "recipe": "lastYear" },
{ "id": "movies_anticipated", "source": "movies", "title": "Most Watchlisted Movies on Simkl", "recipe": "mostWatchlisted" },
{ "id": "tv_anticipated", "source": "tv", "title": "Most Watchlisted TV on Simkl", "recipe": "mostWatchlisted" },
{ "id": "anime_anticipated", "source": "anime", "title": "Most Watchlisted Anime on Simkl", "recipe": "mostWatchlisted" },
{ "id": "movies_top_rated", "source": "movies", "title": "Top Rated Movies on Simkl", "recipe": "highestRated" },
{ "id": "tv_top_rated", "source": "tv", "title": "Top Rated TV on Simkl", "recipe": "highestRated" },
{ "id": "anime_top_rated", "source": "anime", "title": "Top Rated Anime on Simkl", "recipe": "highestRated", "recipe_opts": { "source": "mal" } },
{ "id": "movies_by_rank", "source": "movies", "title": "All-Time Best Movies on Simkl", "recipe": "byRank" },
{ "id": "tv_by_rank", "source": "tv", "title": "All-Time Best TV on Simkl", "recipe": "byRank" },
{ "id": "anime_by_rank", "source": "anime", "title": "All-Time Best Anime on Simkl", "recipe": "byRank" },
{ "id": "movies_box_office", "source": "movies", "title": "Top Box Office Movies on Simkl", "recipe": "bestBoxOffice" },
{ "id": "movies_hidden_gems", "source": "movies", "title": "Hidden Gem Movies on Simkl", "recipe": "hiddenGems" },
{ "id": "tv_hidden_gems", "source": "tv", "title": "Hidden Gem TV on Simkl", "recipe": "hiddenGems" },
{ "id": "anime_hidden_gems", "source": "anime", "title": "Hidden Gem Anime on Simkl", "recipe": "hiddenGems", "recipe_opts": { "source": "mal" } },
{ "id": "tv_marathon", "source": "tv", "title": "Marathon-Worthy TV on Simkl", "recipe": "marathonWorthy" },
{ "id": "movies_quick", "source": "movies", "title": "Quick Watches on Simkl (≤ 90 min)", "recipe": "quickWatches" },
{ "id": "tv_best_netflix", "source": "tv", "title": "Best of Netflix on Simkl", "recipe": "bestOfNetwork", "recipe_opts": { "network": "Netflix" } },
{ "id": "tv_best_hbo", "source": "tv", "title": "Best of HBO on Simkl", "recipe": "bestOfNetwork", "recipe_opts": { "network": "HBO" } },
{ "id": "tv_best_disney", "source": "tv", "title": "Best of Disney+ on Simkl", "recipe": "bestOfNetwork", "recipe_opts": { "network": "Disney+" } },
{ "id": "tv_best_prime", "source": "tv", "title": "Best of Prime Video on Simkl", "recipe": "bestOfNetwork", "recipe_opts": { "network": "Prime Video" } },
{ "id": "tv_best_apple", "source": "tv", "title": "Best of Apple TV on Simkl", "recipe": "bestOfNetwork", "recipe_opts": { "network": "Apple TV" } },
{ "id": "movies_best_action", "source": "movies", "title": "Best Action Movies on Simkl", "recipe": "bestOfGenre", "recipe_opts": { "genre": "Action" } },
{ "id": "movies_best_drama", "source": "movies", "title": "Best Drama Movies on Simkl", "recipe": "bestOfGenre", "recipe_opts": { "genre": "Drama" } },
{ "id": "movies_best_comedy", "source": "movies", "title": "Best Comedy Movies on Simkl", "recipe": "bestOfGenre", "recipe_opts": { "genre": "Comedy" } },
{ "id": "movies_best_scifi", "source": "movies", "title": "Best Sci-Fi Movies on Simkl", "recipe": "bestOfGenre", "recipe_opts": { "genre": "Science Fiction" } },
{ "id": "movies_best_thriller", "source": "movies", "title": "Best Thriller Movies on Simkl", "recipe": "bestOfGenre", "recipe_opts": { "genre": "Thriller" } },
{ "id": "movies_best_horror", "source": "movies", "title": "Best Horror Movies on Simkl", "recipe": "bestOfGenre", "recipe_opts": { "genre": "Horror" } },
{ "id": "tv_best_drama", "source": "tv", "title": "Best Drama TV on Simkl", "recipe": "bestOfGenre", "recipe_opts": { "genre": "Drama" } },
{ "id": "tv_best_comedy", "source": "tv", "title": "Best Comedy TV on Simkl", "recipe": "bestOfGenre", "recipe_opts": { "genre": "Comedy" } },
{ "id": "tv_best_crime", "source": "tv", "title": "Best Crime TV on Simkl", "recipe": "bestOfGenre", "recipe_opts": { "genre": "Crime" } },
{ "id": "tv_best_scifi", "source": "tv", "title": "Best Sci-Fi TV on Simkl", "recipe": "bestOfGenre", "recipe_opts": { "genre": "Science-Fiction" } },
{ "id": "anime_best_shounen", "source": "anime", "title": "Best Shounen Anime on Simkl", "recipe": "bestOfGenre", "recipe_opts": { "genre": "Shounen", "source": "mal" } },
{ "id": "anime_best_isekai", "source": "anime", "title": "Best Isekai Anime on Simkl", "recipe": "bestOfGenre", "recipe_opts": { "genre": "Isekai", "source": "mal" } },
{ "id": "anime_best_romance", "source": "anime", "title": "Best Romance Anime on Simkl", "recipe": "bestOfGenre", "recipe_opts": { "genre": "Romance", "source": "mal" } },
{ "id": "movies_best_2020s", "source": "movies", "title": "Best Movies of the 2020s on Simkl", "recipe": "bestOfDecade", "recipe_opts": { "decade": 2020 } },
{ "id": "movies_best_2010s", "source": "movies", "title": "Best Movies of the 2010s on Simkl", "recipe": "bestOfDecade", "recipe_opts": { "decade": 2010 } },
{ "id": "movies_best_2000s", "source": "movies", "title": "Best Movies of the 2000s on Simkl", "recipe": "bestOfDecade", "recipe_opts": { "decade": 2000 } },
{ "id": "movies_best_90s", "source": "movies", "title": "Best Movies of the 90s on Simkl", "recipe": "bestOfDecade", "recipe_opts": { "decade": 1990 } },
{ "id": "tv_best_2020s", "source": "tv", "title": "Best TV of the 2020s on Simkl", "recipe": "bestOfDecade", "recipe_opts": { "decade": 2020 } },
{ "id": "tv_best_2010s", "source": "tv", "title": "Best TV of the 2010s on Simkl", "recipe": "bestOfDecade", "recipe_opts": { "decade": 2010 } },
{ "id": "anime_best_2020s", "source": "anime", "title": "Best Anime of the 2020s on Simkl", "recipe": "bestOfDecade", "recipe_opts": { "decade": 2020, "source": "mal" } },
{ "id": "anime_best_2010s", "source": "anime", "title": "Best Anime of the 2010s on Simkl", "recipe": "bestOfDecade", "recipe_opts": { "decade": 2010, "source": "mal" } },
{ "id": "anime_movies", "source": "anime", "title": "Trending Anime Movies on Simkl", "recipe": "byAnimeType", "recipe_opts": { "animeType": "movie" } },
{ "id": "anime_ovas", "source": "anime", "title": "Trending Anime OVAs on Simkl", "recipe": "byAnimeType", "recipe_opts": { "animeType": "ova" } }
]
}
- Titles are pre-composed to satisfy the attribution requirement (every label includes both “Trending” / “Most Watched” and “Simkl”). Use them as-is or adapt the wording — just keep both halves.
catalogsis the default row list the SDK renders on a home screen — 44 rows out of the box, all titled and ready to display. Mix of: trending (movies/tv/anime today + DVD), sort variants (byRank,highestRated,mostAnticipated× 3 types), composed catalogs (hiddenGems× 3,criticallyAcclaimed× 2,marathonWorthy,bestBoxOffice), Best of (Netflix / HBO / Disney+ / Prime Video / Apple TV), Best of for movies (Action / Sci-Fi / Comedy / Horror / Thriller / Drama), TV (Drama / Comedy / Crime / Sci-Fi), anime (Shounen / Isekai / Romance), and time-based filters (Quick Watches, In Theaters Now, Just Released on DVD, Currently Airing TV, Anime Movies, Anime OVAs). Edit the JSON to remove rows you don’t want, change titles, or swap recipe opts. Per-app override still works viacfg.catalogsin the SDK config. Currently the recipes ship in the Node + Browser tabs below; Python / Swift / Kotlin / Dart will render the derived rows as raw items until those tabs gain recipe ports (or you supply atransformclosure).refresh_secondsis3600(1 hour) fortodayfiles and86400(1 day) for everything else, matching the Update frequency table. This is the nominal regeneration cadence, not a guarantee — server regeneration can take a few minutes longer. For bandwidth-optimal refresh, pair this with the response’sLast-Modifiedheader and sendIf-Modified-Sinceon subsequent requests — the server returns304 Not Modifiedwhen nothing has changed. If a refresh scheduled forrefresh_secondsreturns 304, back off and retry rather than assuming the cache is final (see SimklTrendingClient — drop-in SDK).
simkl-trending-urls.json in your project — the SDK class loads it back into a SIMKL_TRENDING_URLS symbol. Each language tab’s top comment shows the runtime-appropriate one-liner.
SimklTrendingClient — drop-in SDK
Two-step setup before pasting the SDK class below. First, save the Copy-paste JSON block above as
simkl-trending-urls.json in your project (or as an asset your runtime can load). Second, load it back into a SIMKL_TRENDING_URLS symbol — each tab’s top comment shows the runtime-appropriate one-liner (import / require / fetch for JS, open() for Python, Bundle.main for Swift, context.assets for Android, rootBundle for Flutter). The constructor throws if neither this symbol nor an explicit urls argument is in scope.CONFIG block at the top of each tab — everything else is plumbing you should not need to edit. What’s built in:
- Auto-picks the Combined endpoint when you ask for 2+ of movies/tv/anime — one HTTP call per timeframe instead of two or three.
- Persistent cache keyed on the URL — relaunch can paint the UI instantly from the last persisted snapshot before any network call fires. JS auto-detects the host (browser →
localStorage[storageKey], Node → file atpersistPath); other languages use platform-native paths. Cache entries older thanpersistMaxAgeSecondsare dropped on restore.client.clearCache()wipes both in-memory and persistent state (and notifies subscribers so the UI repaints empty). For server-side runtimes with many processes, swap the_persist/_restorePersisted/clearCachetrio for a Redis / KV adapter. - Conditional revalidation with
If-Modified-Sinceso refreshes hit the CDN with a cheap 304 when nothing has changed. - Regen-lag backoff — when the server hasn’t regenerated by the time
refresh_secondselapses, the client backs off via a[5min, 15min, 30min]ladder instead of hammering on the nominal cadence. - Jitter — ±5% random jitter on every refresh + ±10% on retry delays, so all clients don’t hit the CDN at exactly
:00and don’t retry-storm in lockstep. - Retry with exponential backoff + per-request timeout +
Retry-Afterhonored for network blips and429s. - Permanent-error classification —
404/400/401/403are not retried (they’d waste the backoff schedule on a request that will never succeed); the entry’s loop halts and you see the error in the log. - Process-wide
minIntervalMsfloor — N entries refreshing at the same tick are serialized through a 250 ms (configurable) gate so you never burst N parallel requests at the CDN. - Concurrent fetch dedup — two callers asking for the same URL fold into one in-flight request.
- Cancellation —
client.stop()aborts every in-flight request and clears every timer. - Subscribe / get — your UI subscribes to a type and receives the current cached items immediately, plus a callback on every successful refresh.
get(type)is synchronous — paginate the returned array locally for UI pages instead of refetching. client.tick()for background schedulers — call from aBGTaskScheduler/WorkManager/workmanager/ cron periodic task to perform a single revalidate-and-persist pass.- Namespace-safe — every public symbol is prefixed
Simkl…(SimklTrendingClient,SimklTrendingConfig,SimklTrendingError,SIMKL_TRENDING_URLS, etc.) so dropping the class into an existing app doesn’t collide with your ownConfig/Client/Entrytypes.
Try it live
A self-contained vanilla-JS demo runs in CodePen below — same SDK, all 55 default catalog rows, two live fetches againstdata.simkl.in: trending/today_500.json (combined movies/TV/anime in one round-trip) and dvd/releases_500.json. Click Result to see the rendered home screen; JS / HTML / CSS tabs show the wiring.
Open & fork in CodePen
Edit the catalog config, try other recipes, swap the test
client_id for your own.<!--
Save this whole file as `index.html` next to two siblings:
simkl-trending-urls.json ← the "Copy-paste JSON" block above
simkl-trending-client.js ← the "JavaScript (save as simkl-trending-client.js)" tab →
Serve over http:// (ES modules don't load over file://). For local dev: `python -m http.server`.
No build step, no framework — what you see is what you get.
-->
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Simkl Trending</title>
<style>
:root { color-scheme: dark; }
body { margin: 0; background: #0b0d12; color: #e6e8ee; font-family: system-ui, -apple-system, Segoe UI, Roboto, Helvetica, Arial, sans-serif; }
header { padding: 22px 26px 14px; border-bottom: 1px solid #1d2330; }
header h1 { margin: 0 0 4px; font-size: 18px; letter-spacing: .02em; }
header .sub { color: #8a93a6; font-size: 13px; }
main { padding: 18px 0 60px; }
.row { padding: 18px 0 6px; }
.row h2 { margin: 0 0 10px; padding: 0 26px; font-size: 15px; font-weight: 600; color: #d6dbe8; }
.row h2 .count { color: #6b7280; font-weight: 400; font-size: 12px; margin-left: 8px; }
.scroller { display: flex; gap: 12px; overflow-x: auto; padding: 4px 26px 14px; scrollbar-width: thin; }
.scroller::-webkit-scrollbar { height: 8px; }
.scroller::-webkit-scrollbar-thumb { background: #2a3142; border-radius: 4px; }
.card { flex: 0 0 auto; width: 132px; cursor: pointer; }
.card .poster { width: 132px; height: 198px; background: #1d2330 center/cover no-repeat; border-radius: 6px; overflow: hidden; position: relative; }
.card .poster .rank { position: absolute; top: 6px; left: 6px; background: rgba(0,0,0,.7); color: #fff; font-size: 11px; padding: 2px 6px; border-radius: 3px; }
.card .poster .rating { position: absolute; bottom: 6px; right: 6px; background: rgba(0,0,0,.7); color: #f7c948; font-size: 11px; padding: 2px 6px; border-radius: 3px; }
.card .title { margin-top: 6px; font-size: 12px; line-height: 1.3; color: #d6dbe8; max-height: 32px; overflow: hidden; }
.card .meta { font-size: 11px; color: #8a93a6; margin-top: 2px; }
.empty { padding: 4px 26px 14px; color: #6b7280; font-size: 12px; font-style: italic; }
.toolbar { padding: 14px 26px; color: #8a93a6; font-size: 12px; border-top: 1px solid #1d2330; background: #0e1117; position: sticky; bottom: 0; }
.toolbar button { background: #1d2330; color: #d6dbe8; border: 1px solid #2a3142; padding: 6px 10px; border-radius: 4px; cursor: pointer; font: inherit; }
.toolbar button:hover { background: #232a3a; }
a, a:visited { color: #6ea8fe; }
</style>
</head>
<body>
<header>
<h1>Trending on Simkl</h1>
<div class="sub">Loads <code>simkl-trending-urls.json</code>, wires up <code>SimklTrendingClient</code>, subscribes to all configured catalogs, renders one carousel per row.</div>
</header>
<main id="rows"></main>
<footer class="toolbar">
<span id="status">booting…</span>
·
<button id="refresh">Refresh now</button>
·
<button id="clearcache">Clear cache</button>
·
Cache backend: <span id="cacheBackend"></span>
</footer>
<script type="module">
import { SimklTrendingClient } from './simkl-trending-client.js';
// ─── Render config — tweak here, not down inside render() ──────────────────
const MAX_ITEMS_PER_ROW = 50; // posters per carousel row (each row can hold hundreds; this just caps DOM cost)
const $rows = document.getElementById('rows');
const $status = document.getElementById('status');
document.getElementById('cacheBackend').textContent = (typeof localStorage !== 'undefined') ? 'localStorage' : '(none)';
// 1. Load the URL catalog (JSON catalog + 44 default rows).
const data = await (await fetch('./simkl-trending-urls.json')).json();
// 2. Pull CONFIG out of the JSON so one file drives both URLs and client_id / app-name / app-version.
const cfg = Object.fromEntries(new URLSearchParams(data.required_query));
// 3. Instantiate. Defaults pull supportedTypes from SDK CONFIG, catalogs from JSON's catalogs[] array.
const client = new SimklTrendingClient({
clientId: cfg.client_id,
appName: cfg['app-name'],
appVersion: cfg['app-version'],
userAgent: data.user_agent_header, // browsers ignore this anyway — set for non-browser runtimes
size: 500,
}, data);
// 4. Render — diff-by-id so unchanged rows aren't rebuilt on every refresh.
function render(rows) {
for (const row of rows) {
let el = document.getElementById(`row-${row.id}`);
if (!el) {
el = document.createElement('section');
el.id = `row-${row.id}`;
el.className = 'row';
el.innerHTML = `<h2>${escapeHtml(row.title)} <span class="count"></span></h2><div class="scroller"></div>`;
$rows.appendChild(el);
}
el.querySelector('.count').textContent = `(${row.items.length})`;
const $sc = el.querySelector('.scroller');
if (row.items.length === 0) {
$sc.innerHTML = '';
if (!el.querySelector('.empty')) {
const m = document.createElement('div'); m.className = 'empty';
m.textContent = 'no items yet — waiting for first fetch';
el.appendChild(m);
}
} else {
el.querySelector('.empty')?.remove();
$sc.innerHTML = row.items.slice(0, MAX_ITEMS_PER_ROW).map(itemHtml).join('');
}
}
$status.textContent = `${rows.length} rows · ${rows.reduce((s, r) => s + r.items.length, 0)} items total · last update ${new Date().toLocaleTimeString()}`;
}
function itemHtml(item) {
const posterUrl = item.poster
? `https://wsrv.nl/?url=https://simkl.in/posters/${item.poster}_m.webp&q=90`
: 'https://simkl.in/poster_no_pic.png';
const ratingTxt = item.ratings?.simkl?.rating
? `★ ${item.ratings.simkl.rating.toFixed(1)}`
: '';
return `
<a class="card" href="https://simkl.com${item.url ?? ''}" target="_blank" rel="noopener">
<div class="poster" style="background-image:url('${posterUrl}')">
${item.rank ? `<span class="rank">#${item.rank}</span>` : ''}
${ratingTxt ? `<span class="rating">${ratingTxt}</span>` : ''}
</div>
<div class="title">${escapeHtml(item.title || '')}</div>
<div class="meta">${escapeHtml(formatMeta(item))}</div>
</a>
`;
}
function formatMeta(item) {
const year = (item.release_date || '').match(/(\d{4})/)?.[1];
const bits = [];
if (year) bits.push(year);
if (item.runtime) bits.push(item.runtime);
if (item.network) bits.push(item.network);
if (item.country && !item.network) bits.push(item.country.toUpperCase());
return bits.join(' · ');
}
function escapeHtml(s) {
return String(s).replace(/[&<>"']/g, c => ({ '&':'&','<':'<','>':'>','"':'"',"'":''' }[c]));
}
// 5. Subscribe — fires immediately from cache (if any), then on every refresh.
client.subscribeCatalogs(render);
// 6. Start. Boots fetches in the background.
$status.textContent = 'starting…';
await client.start();
$status.textContent = 'running — refreshing on cadence';
// 7. Wire toolbar.
document.getElementById('refresh').addEventListener('click', () => {
$status.textContent = 'refreshing…';
client.tick()
.then(() => { $status.textContent = 'refreshed via tick()'; })
.catch(e => { $status.textContent = 'refresh failed: ' + e.message; });
});
document.getElementById('clearcache').addEventListener('click', async () => {
$status.textContent = 'clearing cache…';
await client.clearCache();
$status.textContent = 'cache cleared — re-fetching…';
client.tick()
.then(() => { $status.textContent = 'cache cleared + re-fetched via tick()'; })
.catch(e => { $status.textContent = 're-fetch failed: ' + e.message; });
});
window.simklTrending = client; // exposed for console poking
</script>
</body>
</html>
// === Universal ES module — works in any modern browser AND Node 22+ ===
// This is the single SDK file. Save the entire block below as `simkl-trending-client.js`.
// The "Browser (vanilla JS)" tab above imports from this exact file.
// For Node: also add `"type": "module"` to your package.json (or save as `.mjs`).
//
// Then pass it the URL catalog object — three ways to load that:
// Browser / Deno / CF Workers: const data = await (await fetch('./simkl-trending-urls.json')).json();
// Node 22+ (ESM): import data from './simkl-trending-urls.json' with { type: 'json' };
// Node (CommonJS .cjs) const data = require('./simkl-trending-urls.json');
//
// Then: `new SimklTrendingClient(config, data)` or set `globalThis.SIMKL_TRENDING_URLS = data` first.
// === SIMKL_TRENDING_DEFAULTS — tune to match your app ===
export const SIMKL_TRENDING_DEFAULTS = {
// What to load
supportedTypes: ['movies', 'tv', 'anime', 'dvd'], // movies / tv / anime / dvd — drop ones your app doesn't surface
timeframes: ['today'], // today / week / month (ignored for dvd)
size: 100, // 100 or 500 — use 500 if you render derived catalogs (see "Build your own catalogs" below)
// HTTP — replace with YOUR values
clientId: 'YOUR_CLIENT_ID',
appName: 'my-app-name',
appVersion: '1.0',
userAgent: 'my-app-name/1.0',
requestTimeoutMs: 8_000,
minIntervalMs: 250, // process-wide floor between consecutive requests
fetchRetry: {
maxAttempts: 3,
baseBackoffMs: 1_000,
maxBackoffMs: 16_000,
retryStatuses: [408, 425, 429, 500, 502, 503, 504], // never retry other 4xx — they're permanent
},
// Refresh
regenRetrySeconds: [300, 900, 1_800], // 5 / 15 / 30 min after a 304 (server not regenerated yet)
jitterPercent: 0.05, // ±5% so all clients don't hit the CDN at exactly :00
// Persistent cache. Browser: localStorage[storageKey]. Node: file at persistPath.
// Set BOTH to null to disable. For server-side runtimes with many processes, swap
// this for a Redis / KV adapter (see _persist / _restorePersisted).
storageKey: 'simkl-trending-cache',
persistPath: './.simkl-trending-cache.json',
persistMaxAgeSeconds: 7 * 86_400, // discard cache older than 7d
};
export class SimklTrendingError extends Error {
constructor(message, { status, retryAfterMs, permanent } = {}) {
super(message);
this.name = 'SimklTrendingError';
this.status = status ?? null;
this.retryAfterMs = retryAfterMs ?? null;
this.permanent = Boolean(permanent);
}
}
export class SimklTrendingClient {
constructor(config = {}, data = globalThis.SIMKL_TRENDING_URLS) {
if (!data) throw new SimklTrendingError('Pass the URL catalog (SIMKL_TRENDING_URLS) as the second arg or set it globally.');
this.cfg = { ...SIMKL_TRENDING_DEFAULTS, ...config };
this.urls = data.urls ?? data; // unwrap if { urls: {...}, catalogs: [...] } shape
this.defaultCatalogs = data.catalogs ?? null; // pre-defined rows from the JSON
this._cache = new Map();
this._inflight = new Map();
this._subs = new Map();
this._timers = new Set();
this._abort = new AbortController();
this._stopped = false;
this._lastRequestAt = 0;
}
// === Public API ===
async start() {
await this._restorePersisted();
for (const t of new Set(this._subs.keys())) this._notify(t); // paint immediately from restored cache
for (const entry of this._plan()) this._loop(entry);
}
stop() {
this._stopped = true;
this._abort.abort();
for (const t of this._timers) clearTimeout(t);
this._timers.clear();
}
get(type) {
for (const v of this._cache.values()) if (v.itemsByType[type]) return v.itemsByType[type];
return [];
}
subscribe(type, cb) {
if (!this._subs.has(type)) this._subs.set(type, new Set());
this._subs.get(type).add(cb);
const items = this.get(type);
if (items.length) cb(items); // immediate paint from cache
return () => this._subs.get(type)?.delete(cb);
}
// === Catalogs — rows on the home screen ===
catalogs() { return this._catalogConfigs().map(c => this._buildCatalog(c)); }
subscribeCatalogs(cb) {
const sources = new Set(this._catalogConfigs().map(c => c.source));
const unsubs = [];
let primed = false;
// Coalesce: a combined fetch notifies 3 types (movies/tv/anime) in one tick;
// cache-restore fires per subscribed source. queueMicrotask flushes once per tick.
let queued = false;
const fire = () => {
if (queued) return;
queued = true;
queueMicrotask(() => { queued = false; cb(this.catalogs()); });
};
for (const s of sources) unsubs.push(this.subscribe(s, () => { primed = true; fire(); }));
if (!primed) fire(); // empty cache — skeleton paint
return () => unsubs.forEach(u => u());
}
_catalogConfigs() {
if (Array.isArray(this.cfg.catalogs)) return this.cfg.catalogs; // explicit override wins
if (Array.isArray(this.defaultCatalogs)) return this.defaultCatalogs; // pulled from JSON's catalogs[] (recommended path)
const tf0 = this.cfg.timeframes?.[0] ?? 'today'; // last resort: auto-build
return this.cfg.supportedTypes.map(t => t === 'dvd'
? { id: 'dvd', source: 'dvd' }
: { id: `${t}_${tf0}`, source: t, timeframe: tf0 });
}
_fileTitleFor(source, timeframe) {
const sizeKey = `top_${this.cfg.size}`;
const file = source === 'dvd' ? this.urls.dvd?.[sizeKey] : this.urls?.[source]?.[timeframe]?.[sizeKey];
return file?.title ?? `Trending ${source} on Simkl`;
}
_buildCatalog(c) {
if (!c.id || !c.source) throw new SimklTrendingError('catalogs[] entries need `id` and `source`');
const items = this.get(c.source);
let out = items;
if (typeof c.transform === 'function') { // inline closure (per-app overrides)
out = c.transform(items);
} else if (typeof c.recipe === 'string') { // JSON catalogs reference by name
const fn = SimklTrendingRecipes[c.recipe];
if (typeof fn === 'function') out = fn(items, c.recipe_opts || {});
else console.warn(`SimklTrendingClient: unknown recipe "${c.recipe}" for catalog "${c.id}" — rendering raw items.`);
}
return { id: c.id, title: c.title ?? this._fileTitleFor(c.source, c.timeframe ?? this.cfg.timeframes?.[0] ?? 'today'), items: out };
}
// Call from a background scheduler (node-cron, Cloudflare Cron, AWS EventBridge) for a single pass.
async tick() {
await this._restorePersisted();
for (const entry of this._plan()) {
const cached = this._cache.get(entry.url);
const r = await this._fetch(entry, cached?.lastModified);
if (!r.notModified) this._storeAndNotify(entry, r);
}
await this._persist();
}
// === Internals ===
_requiredQuery() {
const { clientId, appName, appVersion } = this.cfg;
return `client_id=${clientId}&app-name=${encodeURIComponent(appName)}&app-version=${encodeURIComponent(appVersion)}`;
}
_plan() {
const { supportedTypes, timeframes, size } = this.cfg;
if (size !== 100 && size !== 500) throw new SimklTrendingError('size must be 100 or 500');
const key = `top_${size}`;
const tvma = supportedTypes.filter(t => ['movies','tv','anime'].includes(t));
const useCombined = tvma.length >= 2; // 1 round-trip < 2-3 per timeframe
const plan = [];
if (tvma.length) {
for (const tf of timeframes) {
const file = useCombined ? this.urls.combined[tf][key]
: this.urls[tvma[0]][tf][key];
plan.push({ ...file, types: tvma });
}
}
if (supportedTypes.includes('dvd')) plan.push({ ...this.urls.dvd[key], types: ['dvd'] });
return plan;
}
// Process-wide rate-limit floor (so N entries refreshing at once don't fire N simultaneous requests).
async _throttle() {
const gap = Date.now() - this._lastRequestAt;
if (gap < this.cfg.minIntervalMs) await this._sleep((this.cfg.minIntervalMs - gap) / 1_000);
this._lastRequestAt = Date.now();
}
static _parseRetryAfter(value) {
if (!value) return null;
const secs = Number(value);
if (Number.isFinite(secs)) return Math.max(0, secs * 1_000);
const dateMs = Date.parse(value);
return Number.isFinite(dateMs) ? Math.max(0, dateMs - Date.now()) : null;
}
// _fetch — dedup + retry + timeout + conditional revalidation + Retry-After. Normalizes array vs object response into itemsByType.
async _fetch(entry, lastModified) {
const sep = entry.url.includes('?') ? '&' : '?';
const url = entry.url + sep + this._requiredQuery();
if (this._inflight.has(url)) return this._inflight.get(url);
// In Node/Deno/Bun/CF Workers, set If-Modified-Since manually for conditional revalidation.
// In BROWSERS, the manual header would trigger a CORS preflight (If-Modified-Since isn't
// CORS-safelisted), and most CDNs respond to OPTIONS with 405 → preflight fails → fetch
// fails. Instead we use `cache: 'no-cache'` so the browser handles revalidation via its
// own HTTP cache (browser-added If-None-Match / If-Modified-Since don't count as author
// headers and skip preflight). User-Agent is dropped in browsers too — it's a forbidden
// header per Fetch spec and silently ignored.
const isBrowser = typeof window !== 'undefined';
const headers = { 'Accept': 'application/json' };
if (!isBrowser) {
headers['User-Agent'] = this.cfg.userAgent;
if (lastModified) headers['If-Modified-Since'] = lastModified;
}
const { fetchRetry, requestTimeoutMs } = this.cfg;
const run = async () => {
let lastErr;
for (let attempt = 0; attempt < fetchRetry.maxAttempts; attempt++) {
await this._throttle();
const ctl = new AbortController();
const time = setTimeout(() => ctl.abort(), requestTimeoutMs);
const off = () => ctl.abort();
this._abort.signal.addEventListener('abort', off);
try {
const fetchOpts = { headers, signal: ctl.signal };
if (isBrowser) fetchOpts.cache = 'no-cache';
const res = await fetch(url, fetchOpts);
if (res.status === 304) return { itemsByType: null, lastModified, notModified: true };
if (!res.ok) {
const retryAfterMs = SimklTrendingClient._parseRetryAfter(res.headers.get('Retry-After'));
const permanent = !fetchRetry.retryStatuses.includes(res.status);
throw new SimklTrendingError(`${res.status} ${res.statusText} — ${entry.title}`,
{ status: res.status, retryAfterMs, permanent });
}
// In browsers with cache:'no-cache', fetch resolves to 200 + cached body when the
// server returned 304 to the browser's conditional request. Detect that via
// Last-Modified equality so we skip a no-op re-render.
const newLastModified = res.headers.get('Last-Modified');
if (isBrowser && lastModified && newLastModified === lastModified) {
return { itemsByType: null, lastModified, notModified: true };
}
const raw = await res.json();
// Combined endpoint returns {movies, tv, anime}. Keep ALL keys (not just wanted types) so
// a dev who later adds 'anime' to supportedTypes can serve it from the same cached response.
const itemsByType = Array.isArray(raw)
? { [entry.types[0]]: raw } // per-type / dvd → array
: raw; // combined → store full object
return { itemsByType, lastModified: newLastModified, notModified: false };
} catch (e) {
if (this._stopped) throw e;
if (e instanceof SimklTrendingError && e.permanent) throw e; // don't waste backoff on 404/400/etc
lastErr = e;
const honored = (e instanceof SimklTrendingError && e.retryAfterMs != null) ? e.retryAfterMs : null;
const backoff = honored ?? Math.min(fetchRetry.baseBackoffMs * (2 ** attempt), fetchRetry.maxBackoffMs);
const jitter = backoff * (Math.random() * 0.2 - 0.1); // ±10% jitter on retry delay
await this._sleep(Math.max(0, backoff + jitter) / 1_000);
} finally {
clearTimeout(time);
this._abort.signal.removeEventListener('abort', off);
}
}
throw lastErr;
};
const p = run().finally(() => this._inflight.delete(url));
this._inflight.set(url, p);
return p;
}
async _loop(entry) {
let lastModified = this._cache.get(entry.url)?.lastModified ?? null;
let retry = 0;
while (!this._stopped) {
let r;
try { r = await this._fetch(entry, lastModified); }
catch (e) {
if (this._stopped) return;
if (e instanceof SimklTrendingError && e.permanent) { // 404 / 401 / etc — halt this entry's loop
console.error(`${entry.title}: ${e.message} (permanent, halting loop)`);
return;
}
console.error(`${entry.title}:`, e.message);
await this._sleep(30); continue;
}
let wait;
if (r.notModified) {
wait = this.cfg.regenRetrySeconds[Math.min(retry++, this.cfg.regenRetrySeconds.length - 1)];
} else {
this._storeAndNotify(entry, r);
lastModified = r.lastModified;
retry = 0;
await this._persist();
const j = this.cfg.jitterPercent;
wait = Math.round(entry.refresh_seconds * (1 + (Math.random() - 0.5) * 2 * j));
}
await this._sleep(wait);
}
}
_storeAndNotify(entry, r) {
this._cache.set(entry.url, { ...r, types: entry.types, fetchedAt: Date.now() });
for (const t of entry.types) this._notify(t);
}
_notify(type) { const items = this.get(type); for (const cb of (this._subs.get(type) ?? [])) cb(items); }
_sleep(seconds) {
return new Promise(resolve => {
const t = setTimeout(() => { this._timers.delete(t); resolve(); }, seconds * 1_000);
this._timers.add(t);
});
}
// Wipe in-memory + persistent cache and notify subscribers (so UI repaints empty).
// Call client.tick() after this if you want to immediately re-fetch.
async clearCache() {
this._cache.clear();
const ls = this._localStorage();
if (ls && this.cfg.storageKey) { try { ls.removeItem(this.cfg.storageKey); } catch { /* ignore */ } }
if (this.cfg.persistPath && !ls) {
try { const fs = await import('node:fs/promises'); await fs.unlink(this.cfg.persistPath); }
catch { /* no file — fine */ }
}
for (const t of this._subs.keys()) this._notify(t);
}
// Persistence — auto-detects environment. Browser → localStorage. Node → file.
// Swap these two methods (plus clearCache) for a Redis / KV adapter in server-side runtimes.
_localStorage() { try { return globalThis.localStorage ?? null; } catch { return null; } }
async _persist() {
const data = JSON.stringify(Object.fromEntries(this._cache));
const ls = this._localStorage();
if (ls && this.cfg.storageKey) { try { ls.setItem(this.cfg.storageKey, data); } catch { /* quota etc */ } return; }
if (!this.cfg.persistPath) return;
try {
const fs = await import('node:fs/promises');
await fs.writeFile(this.cfg.persistPath, data);
} catch { /* best-effort */ }
}
async _restorePersisted() {
const cutoff = Date.now() - this.cfg.persistMaxAgeSeconds * 1_000;
let raw = null;
const ls = this._localStorage();
if (ls && this.cfg.storageKey) {
try { const s = ls.getItem(this.cfg.storageKey); if (s) raw = JSON.parse(s); } catch { /* corrupt — ignore */ }
}
if (!raw && this.cfg.persistPath && !ls) {
try {
const fs = await import('node:fs/promises');
raw = JSON.parse(await fs.readFile(this.cfg.persistPath, 'utf8'));
} catch { /* no cache yet — fine */ }
}
if (!raw) return;
for (const [url, val] of Object.entries(raw)) {
if (val.fetchedAt && val.fetchedAt >= cutoff) this._cache.set(url, val);
}
}
}
// ─────────────────────────────────────────────────────────────────────────────
// SimklTrendingRecipes — the catalog transforms referenced by name from the
// JSON's `catalogs[]` array. Pure functions: `(items, opts?) => items`.
// ─────────────────────────────────────────────────────────────────────────────
export const SimklTrendingRecipes = {
// Sorts
byRank(items) { return [...items].sort((a, b) => (a.rank ?? Infinity) - (b.rank ?? Infinity)); },
// Sorts by plan_to_watch desc — i.e. how many users have added the item to their watchlist.
// Despite the historical "mostAnticipated" name, this fires for already-released titles too
// (Game of Thrones, Avatar, One Piece — all rank high because they have huge backlog appeal).
// `mostWatchlisted` is the accurate name; `mostAnticipated` is kept as an alias for back-compat.
mostWatchlisted(items) { return [...items].sort((a, b) => (b.plan_to_watch ?? 0) - (a.plan_to_watch ?? 0)); },
mostAnticipated(items) { return [...items].sort((a, b) => (b.plan_to_watch ?? 0) - (a.plan_to_watch ?? 0)); }, // alias of mostWatchlisted
newest(items) { return [...items].filter(i => i.release_date).sort((a, b) => +new Date(b.release_date) - +new Date(a.release_date)); },
stickiest(items) { return [...items].sort((a, b) => parseFloat(a.drop_rate ?? 'Infinity') - parseFloat(b.drop_rate ?? 'Infinity')); },
highestRated(items, { source = 'simkl', minVotes = 500 } = {}) {
return items.filter(i => (i.ratings?.[source]?.votes ?? 0) >= minVotes)
.sort((a, b) => (b.ratings?.[source]?.rating ?? 0) - (a.ratings?.[source]?.rating ?? 0));
},
bestBoxOffice(items) { // movies only
const bo = i => { const m = (i.metadata || '').match(/Box office \$([\d.]+)([MBK])/);
return m ? parseFloat(m[1]) * { K: 1e3, M: 1e6, B: 1e9 }[m[2]] : null; };
return items.map(i => ({ i, v: bo(i) })).filter(x => x.v != null).sort((a, b) => b.v - a.v).map(x => x.i);
},
// Composed (multiple signals)
// Hidden gems = well-rated titles OUTSIDE Simkl's top N by all-time popularity. The
// `watched` field on trending feeds is last-24h viewers (not lifetime), so filtering on
// it surfaces mainstream titles (The Office, Interstellar) instead of actual gems.
// `rank` is the right gate: mainstream shows rank low (~50-500), obscure-but-loved
// ones rank in the thousands. Sort by rating so the best gems surface first.
hiddenGems(items, { minRating = 7.5, minVotes = 500, minRank = 2000, source = 'simkl' } = {}) {
return items
.filter(i => (i.rank ?? 0) > minRank &&
(i.ratings?.[source]?.rating ?? 0) >= minRating &&
(i.ratings?.[source]?.votes ?? 0) >= minVotes)
.sort((a, b) => (b.ratings?.[source]?.rating ?? 0) -
(a.ratings?.[source]?.rating ?? 0));
},
criticallyAcclaimed(items, { minRating = 8.5, minVotes = 1000, source = 'simkl' } = {}) {
return items.filter(i => (i.ratings?.[source]?.rating ?? 0) >= minRating &&
(i.ratings?.[source]?.votes ?? 0) >= minVotes)
.sort((a, b) => (b.ratings?.[source]?.rating ?? 0) - (a.ratings?.[source]?.rating ?? 0));
},
marathonWorthy(items, { minEpisodes = 20, maxEpisodes = 100, minRating = 8, source = 'simkl' } = {}) {
return items.filter(i => i.status === 'ended' &&
(i.total_episodes ?? 0) >= minEpisodes && (i.total_episodes ?? 0) <= maxEpisodes &&
(i.ratings?.[source]?.rating ?? 0) >= minRating);
},
crossPlatformConsensus(items, { simklMin = 8, otherMin = 8 } = {}) {
return items.filter(i => (i.ratings?.simkl?.rating ?? 0) >= simklMin &&
((i.ratings?.imdb?.rating ?? i.ratings?.mal?.rating) ?? 0) >= otherMin);
},
massiveAnticipation(items, { minRatio = 300 } = {}) {
return items.filter(i => (i.plan_to_watch ?? 0) / ((i.watched ?? 0) + 1) >= minRatio)
.sort((a, b) => (b.plan_to_watch / (b.watched + 1)) - (a.plan_to_watch / (a.watched + 1)));
},
bestOfNetwork(items, { network, minRating = 7.5, requireEnded = false, source = 'simkl' } = {}) {
return items.filter(i => i.network === network &&
(!requireEnded || i.status === 'ended') &&
(i.ratings?.[source]?.rating ?? 0) >= minRating)
.sort((a, b) => (b.ratings?.[source]?.rating ?? 0) - (a.ratings?.[source]?.rating ?? 0));
},
bestOfGenre(items, { genre, minRating = 7.5, minVotes = 300, source = 'simkl' } = {}) {
return items.filter(i => i.genres?.includes(genre) &&
(i.ratings?.[source]?.rating ?? 0) >= minRating &&
(i.ratings?.[source]?.votes ?? 0) >= minVotes)
.sort((a, b) => (b.ratings?.[source]?.rating ?? 0) - (a.ratings?.[source]?.rating ?? 0));
},
// Filters (all use { ...opts } so they're JSON-describable)
byGenre (items, { genre } = {}) { return items.filter(i => i.genres?.includes(genre)); },
byCountry (items, { country } = {}) { const c = String(country || '').toLowerCase(); return items.filter(i => i.country === c); },
byNetwork (items, { network } = {}) { return items.filter(i => i.network === network); },
byStatus (items, { status } = {}) { return items.filter(i => i.status === status); },
byAnimeType(items, { animeType } = {}) { return items.filter(i => i.anime_type === animeType); },
withTrailer(items) { return items.filter(i => i.trailer); },
ongoing(items) { return items.filter(i => i.status === 'ongoing'); },
recentlyEnded(items) { return items.filter(i => i.status === 'ended'); },
// `status === 'premiere'` flags items whose first episode JUST aired (typically within ~30 days).
// It is NOT an "upcoming premieres" filter — those items have status 'ongoing' or no status set.
justPremiered(items) { return items.filter(i => i.status === 'premiere'); },
premiering(items) { return items.filter(i => i.status === 'premiere'); }, // alias of justPremiered (kept for back-compat)
miniSeries(items, { maxEpisodes = 6 } = {}) { return items.filter(i => (i.total_episodes ?? Infinity) <= maxEpisodes); },
justReleased(items, { withinDays = 30 } = {}) {
const cutoff = Date.now() - withinDays * 86_400_000;
return items.filter(i => i.release_date && +new Date(i.release_date) >= cutoff);
},
inTheatersNow(items, { withinDays = 30 } = {}) { // movies only
const cutoff = Date.now() - withinDays * 86_400_000;
return items.filter(i => i.theater && +new Date(i.theater) >= cutoff);
},
recentlyOnDvd(items, { withinDays = 30 } = {}) { // movies only
const cutoff = Date.now() - withinDays * 86_400_000;
return items.filter(i => i.dvd_date && +new Date(i.dvd_date) >= cutoff);
},
// Era filters — release_date in trending feeds is "MM/DD/YYYY"; the regex grabs the 4-digit year either way.
byYear (items, { year } = {}) { return items.filter(i => +String(i.release_date||'').match(/\d{4}/)?.[0] === year); },
byDecade (items, { decade } = {}) { return items.filter(i => { const y = +String(i.release_date||'').match(/\d{4}/)?.[0]; return y >= decade && y < decade + 10; }); },
currentYear(items) { const y = new Date().getFullYear(); return items.filter(i => +String(i.release_date||'').match(/\d{4}/)?.[0] === y); },
lastYear (items) { const y = new Date().getFullYear()-1; return items.filter(i => +String(i.release_date||'').match(/\d{4}/)?.[0] === y); },
bestOfDecade(items, { decade, source = 'simkl', minVotes = 100 } = {}) { // decade + rating sort, mirrors bestOfNetwork/bestOfGenre
return items.filter(i => { const y = +String(i.release_date||'').match(/\d{4}/)?.[0];
return y >= decade && y < decade + 10 && (i.ratings?.[source]?.votes ?? 0) >= minVotes; })
.sort((a, b) => (b.ratings?.[source]?.rating ?? 0) - (a.ratings?.[source]?.rating ?? 0));
},
quickWatches(items, { maxMinutes = 90 } = {}) { // movies only
const m = s => { if (!s) return Infinity;
const x = s.match(/^(?:(\d+)h)?\s*(?:(\d+)m)?$/);
return x ? ((parseInt(x[1]||'0')*60) + parseInt(x[2]||'0')) || Infinity : Infinity; };
return items.filter(i => m(i.runtime) <= maxMinutes);
},
};
/* === Sample usage (put this in a SEPARATE file — app.js / main.mjs / etc.) ===
import data from './simkl-trending-urls.json' with { type: 'json' };
import { SimklTrendingClient } from './simkl-trending-client.js';
const client = new SimklTrendingClient({ clientId: 'YOUR_CLIENT_ID', appName: 'my-app', appVersion: '1.0' }, data);
client.subscribeCatalogs(rows => {
for (const row of rows) console.log(`${row.title}: ${row.items.length} items`);
});
await client.start();
// process.on('SIGINT', () => { client.stop(); process.exit(0); });
=== Background scheduling (server-side runtimes) ===
For long-running servers, the in-process loop is enough. For serverless / cron-triggered
runtimes (Cloudflare Workers, AWS Lambda, GitHub Actions), wire `await client.tick()` to
your scheduler instead of calling `start()`. For browser usage, see the Browser tab above. */
import email.utils, json, os, random, threading, time, urllib.parse
import requests
# === Load the URL catalog ===
# 1. Save the "Copy-paste JSON" block above as `simkl-trending-urls.json` in your project.
# 2. Load it into SIMKL_TRENDING_URLS:
# with open('simkl-trending-urls.json') as f: SIMKL_TRENDING_URLS = json.load(f)
# Or for a packaged module:
# import importlib.resources, json
# SIMKL_TRENDING_URLS = json.loads(importlib.resources.files('your_pkg').joinpath('simkl-trending-urls.json').read_text())
# You can also pass it explicitly: SimklTrendingClient(urls=urls_dict)
# SimklTrendingClient raises SimklTrendingError if it can't find the catalog in either place.
# === SIMKL_TRENDING_DEFAULTS — tune to match your app ===
SIMKL_TRENDING_DEFAULTS = {
# What to load
'supported_types': ['movies', 'tv', 'anime', 'dvd'], # movies / tv / anime / dvd — drop ones your app doesn't surface
'timeframes': ['today'], # today / week / month (ignored for dvd)
'size': 100, # 100 or 500 — use 500 if you render derived catalogs
# HTTP — replace with YOUR values
'client_id': 'YOUR_CLIENT_ID',
'app_name': 'my-app-name',
'app_version': '1.0',
'user_agent': 'my-app-name/1.0',
'request_timeout_s': 8,
'min_interval_s': 0.25, # process-wide floor between consecutive requests
'fetch_retry': {
'max_attempts': 3,
'base_backoff_s': 1,
'max_backoff_s': 16,
'retry_statuses': [408, 425, 429, 500, 502, 503, 504], # never retry other 4xx — they're permanent
},
# Refresh
'regen_retry_s': [300, 900, 1800], # 5 / 15 / 30 min after a 304 (server not regenerated yet)
'jitter_percent': 0.05, # ±5% so all clients don't hit the CDN at exactly :00
# Persistent cache — set persist_path to None to disable. For server-side runtimes,
# swap _persist / _restore_persisted for a Redis / KV adapter.
'persist_path': './.simkl-trending-cache.json',
'persist_max_age_s': 7 * 86400, # discard cache older than 7d
# Catalogs — the rows on your home screen. Each becomes a {'id', 'title', 'items'} dict
# returned by client.catalogs() / pushed to client.subscribe_catalogs(cb). If None,
# one row per supported_type with the JSON's pre-composed title.
# { 'id', 'source', 'timeframe'?, 'title'?, 'transform'? }
'catalogs': None,
}
class SimklTrendingError(Exception):
def __init__(self, message, status=None, retry_after_s=None, permanent=False):
super().__init__(message)
self.status = status
self.retry_after_s = retry_after_s
self.permanent = bool(permanent)
class SimklTrendingClient:
def __init__(self, config=None, data=None):
data = data or globals().get('SIMKL_TRENDING_URLS')
if not data:
raise SimklTrendingError('Pass the URL catalog (SIMKL_TRENDING_URLS) as the second arg or define it globally.')
self.cfg = {**SIMKL_TRENDING_DEFAULTS, **(config or {})}
self.urls = data.get('urls', data) # unwrap {urls: {...}, catalogs: [...]} shape
self.default_catalogs = data.get('catalogs') # pre-defined rows from JSON
self._cache = {}
self._inflight = {}
self._subs = {}
self._lock = threading.RLock()
self._stopped = threading.Event()
self._threads = []
self._last_request_at = 0.0
# === Public API ===
def start(self):
self._restore_persisted()
with self._lock:
for t in list(self._subs):
self._notify(t) # paint immediately from restored cache
for entry in self._plan():
th = threading.Thread(target=self._loop, args=(entry,), daemon=True)
th.start()
self._threads.append(th)
def stop(self):
self._stopped.set()
# Wipe in-memory + persistent cache and notify subscribers (so UI repaints empty).
# Call client.tick() after this to immediately re-fetch.
def clear_cache(self):
with self._lock:
self._cache.clear()
subs_snapshot = {t: list(cbs) for t, cbs in self._subs.items()}
if self.cfg.get('persist_path'):
try: os.remove(self.cfg['persist_path'])
except OSError: pass # no file — fine
for t, cbs in subs_snapshot.items():
for cb in cbs: cb([])
def get(self, type_):
with self._lock:
for v in self._cache.values():
if type_ in v['items_by_type']:
return v['items_by_type'][type_]
return []
def subscribe(self, type_, cb):
with self._lock:
self._subs.setdefault(type_, []).append(cb)
items = self.get(type_)
if items:
cb(items) # immediate paint from cache
def unsub():
with self._lock:
if cb in self._subs.get(type_, []): self._subs[type_].remove(cb)
return unsub
# Resolve all configured catalogs to [{'id','title','items'}, ...] — TV-home-screen rows.
def catalogs(self):
return [self._build_catalog(c) for c in self._catalog_configs()]
# Subscribe to ALL catalog rows at once. cb([{...}, ...]) fires on every refresh.
def subscribe_catalogs(self, cb):
sources = {c['source'] for c in self._catalog_configs()}
unsubs = []
def fire(_): cb(self.catalogs())
for s in sources:
unsubs.append(self.subscribe(s, fire))
if not any(self.get(s) for s in sources):
cb(self.catalogs()) # empty cache — fire once for skeleton paint
def unsub():
for u in unsubs: u()
return unsub
# Call from a background scheduler (cron, Celery beat, APScheduler) for a single pass.
def tick(self):
self._restore_persisted()
for entry in self._plan():
with self._lock: cached = self._cache.get(entry['url'])
r = self._fetch(entry, cached and cached['last_modified'])
if not r['not_modified']: self._store_and_notify(entry, r)
self._persist()
# === Internals ===
def _required_query(self):
c = self.cfg
return (f"client_id={c['client_id']}"
f"&app-name={urllib.parse.quote(c['app_name'])}"
f"&app-version={urllib.parse.quote(c['app_version'])}")
def _plan(self):
c = self.cfg
if c['size'] not in (100, 500): raise SimklTrendingError('size must be 100 or 500')
key = f"top_{c['size']}"
tvma = [t for t in c['supported_types'] if t in ('movies', 'tv', 'anime')]
use_combined = len(tvma) >= 2 # 1 round-trip < 2-3 per timeframe
plan = []
if tvma:
for tf in c['timeframes']:
f = (self.urls['combined'][tf][key] if use_combined
else self.urls[tvma[0]][tf][key])
plan.append({**f, 'types': tvma})
if 'dvd' in c['supported_types']:
plan.append({**self.urls['dvd'][key], 'types': ['dvd']})
return plan
# Process-wide rate-limit floor (so N entries refreshing at once don't fire N simultaneous requests).
def _throttle(self):
with self._lock:
gap = time.monotonic() - self._last_request_at
if gap < self.cfg['min_interval_s']:
if self._stopped.wait(self.cfg['min_interval_s'] - gap): return False
self._last_request_at = time.monotonic()
return True
@staticmethod
def _parse_retry_after(value):
if not value: return None
try: return max(0.0, float(value))
except (TypeError, ValueError): pass
try:
dt = email.utils.parsedate_to_datetime(value)
return max(0.0, dt.timestamp() - time.time())
except (TypeError, ValueError): return None
# _fetch — dedup + retry + timeout + If-Modified-Since + Retry-After. Normalizes list vs dict into items_by_type.
def _fetch(self, entry, last_modified):
sep = '&' if '?' in entry['url'] else '?'
url = entry['url'] + sep + self._required_query()
with self._lock:
inflight = self._inflight.get(url)
if inflight:
inflight['done'].wait()
if inflight.get('error'): raise inflight['error']
return inflight['result']
done = threading.Event()
slot = {'done': done, 'result': None, 'error': None}
with self._lock: self._inflight[url] = slot
try:
headers = {'User-Agent': self.cfg['user_agent'], 'Accept': 'application/json'}
if last_modified: headers['If-Modified-Since'] = last_modified
retry_cfg = self.cfg['fetch_retry']
last_err = None
for attempt in range(retry_cfg['max_attempts']):
self._throttle()
try:
res = requests.get(url, headers=headers, timeout=self.cfg['request_timeout_s'])
if res.status_code == 304:
slot['result'] = {'items_by_type': None, 'last_modified': last_modified, 'not_modified': True}
return slot['result']
if not res.ok:
permanent = res.status_code not in retry_cfg['retry_statuses']
retry_after_s = self._parse_retry_after(res.headers.get('Retry-After'))
raise SimklTrendingError(f"{res.status_code} {res.reason} — {entry['title']}",
status=res.status_code, retry_after_s=retry_after_s, permanent=permanent)
raw = res.json()
items_by_type = ({entry['types'][0]: raw} if isinstance(raw, list) # per-type / dvd → list
else {t: raw.get(t, []) for t in entry['types']}) # combined → dict, filter to wanted types
slot['result'] = {'items_by_type': items_by_type,
'last_modified': res.headers.get('Last-Modified'),
'not_modified': False}
return slot['result']
except SimklTrendingError as e:
if e.permanent: slot['error'] = e; raise # don't waste backoff on permanent errors
last_err = e
honored = e.retry_after_s if e.retry_after_s is not None else None
backoff = honored if honored is not None else min(retry_cfg['base_backoff_s'] * (2 ** attempt), retry_cfg['max_backoff_s'])
backoff += backoff * (random.random() * 0.2 - 0.1) # ±10% jitter
if self._stopped.wait(max(0, backoff)): raise RuntimeError('stopped')
except Exception as e:
if self._stopped.is_set(): raise
last_err = e
backoff = min(retry_cfg['base_backoff_s'] * (2 ** attempt), retry_cfg['max_backoff_s'])
if self._stopped.wait(backoff): raise RuntimeError('stopped')
slot['error'] = last_err
raise last_err
finally:
done.set()
with self._lock: self._inflight.pop(url, None)
def _loop(self, entry):
with self._lock: c = self._cache.get(entry['url'])
last_modified = c['last_modified'] if c else None
retry = 0
while not self._stopped.is_set():
try:
r = self._fetch(entry, last_modified)
except SimklTrendingError as e:
if self._stopped.is_set(): return
if e.permanent: # 404 / 401 / etc — halt this entry's loop
print(f"{entry['title']}: {e} (permanent, halting loop)")
return
print(f"{entry['title']}: {e}")
if self._stopped.wait(30): return
continue
except Exception as e:
if self._stopped.is_set(): return
print(f"{entry['title']}: {e}")
if self._stopped.wait(30): return
continue
if r['not_modified']:
wait = self.cfg['regen_retry_s'][min(retry, len(self.cfg['regen_retry_s']) - 1)]
retry += 1
else:
self._store_and_notify(entry, r)
last_modified = r['last_modified']
retry = 0
self._persist()
j = self.cfg['jitter_percent']
wait = round(entry['refresh_seconds'] * (1 + (random.random() - 0.5) * 2 * j))
if self._stopped.wait(wait): return
def _store_and_notify(self, entry, r):
with self._lock:
self._cache[entry['url']] = {**r, 'types': entry['types'], 'fetched_at': time.time()}
for t in entry['types']: self._notify(t)
def _notify(self, type_):
with self._lock:
cbs = list(self._subs.get(type_, []))
items = self.get(type_)
for cb in cbs: cb(items)
# === Catalog helpers ===
def _catalog_configs(self):
if self.cfg.get('catalogs'): return self.cfg['catalogs'] # explicit override wins
if self.default_catalogs: return self.default_catalogs # JSON's catalogs (recommended path)
tf0 = (self.cfg.get('timeframes') or ['today'])[0] # last resort: auto-build
out = []
for t in self.cfg['supported_types']:
if t == 'dvd': out.append({'id': 'dvd', 'source': 'dvd'})
else: out.append({'id': f'{t}_{tf0}', 'source': t, 'timeframe': tf0})
return out
def _file_title_for(self, source, timeframe=None):
size_key = f"top_{self.cfg['size']}"
if source == 'dvd':
file = (self.urls.get('dvd') or {}).get(size_key)
else:
file = ((self.urls.get(source) or {}).get(timeframe) or {}).get(size_key)
return (file or {}).get('title', f'Trending {source} on Simkl')
def _build_catalog(self, c):
if not c.get('id') or not c.get('source'):
raise SimklTrendingError("catalogs[] entries need 'id' and 'source'")
items = self.get(c['source'])
transform = c.get('transform')
out = transform(items) if callable(transform) else items
title = c.get('title') or self._file_title_for(c['source'], c.get('timeframe') or (self.cfg.get('timeframes') or ['today'])[0])
return {'id': c['id'], 'title': title, 'items': out}
# Persistence — swap these two methods for a Redis / KV adapter in server-side runtimes.
def _persist(self):
if not self.cfg['persist_path']: return
try:
with self._lock: dump = json.dumps(self._cache)
tmp = self.cfg['persist_path'] + '.tmp'
with open(tmp, 'w') as f: f.write(dump)
os.replace(tmp, self.cfg['persist_path'])
except Exception: pass # best-effort
def _restore_persisted(self):
if not self.cfg['persist_path'] or not os.path.exists(self.cfg['persist_path']): return
try:
with open(self.cfg['persist_path']) as f: raw = json.load(f)
cutoff = time.time() - self.cfg['persist_max_age_s']
with self._lock:
for url, val in raw.items():
if val.get('fetched_at', 0) >= cutoff: self._cache[url] = val
except Exception: pass # no cache yet — fine
# === Run ===
client = SimklTrendingClient()
client.subscribe_catalogs(lambda rows: [print(f"{r['title']}: {len(r['items'])} items") for r in rows])
client.start()
# import signal; signal.signal(signal.SIGINT, lambda *_: (client.stop(), exit(0)))
# === Background scheduling (server-side) ===
# For long-running processes, the in-process threads are enough. For cron-style runtimes
# (Celery beat, APScheduler, GitHub Actions), wire `client.tick()` to your scheduler instead of start().
import Foundation
// === Load the URL catalog ===
// 1. Save the "Copy-paste JSON" block above as `simkl-trending-urls.json` in your app bundle (e.g.
// drag it into Xcode and tick "Copy items if needed" + your app target's "Copy Bundle Resources").
// 2. Load it into a global `SIMKL_TRENDING_URLS: [String: Any]` (and pass it to the constructor, or
// let the actor pick it up from the global on first use):
// let url = Bundle.main.url(forResource: "simkl-trending-urls", withExtension: "json")!
// let data = try Data(contentsOf: url)
// let SIMKL_TRENDING_URLS = try JSONSerialization.jsonObject(with: data) as! [String: Any]
// SimklTrendingClient(config:, trending: SIMKL_TRENDING_URLS) — fatalErrors if neither global nor arg is set.
// === CONFIG — tune to match your app ===
struct SimklTrendingConfig {
// What to load
var supportedTypes: [String] = ["movies", "tv", "anime", "dvd"] // movies / tv / anime / dvd — drop ones your app doesn't surface
var timeframes: [String] = ["today"] // today / week / month (ignored for dvd)
var size: Int = 100 // 100 or 500 — use 500 if you render derived catalogs (see "Build your own catalogs" below)
// HTTP — replace with YOUR values
var clientId: String = "YOUR_CLIENT_ID"
var appName: String = "my-app-name"
var appVersion: String = "1.0"
var userAgent: String = "my-app-name/1.0"
var requestTimeout: TimeInterval = 8
var minIntervalMs: Int = 250 // process-wide floor between consecutive requests
var fetchMaxAttempts: Int = 3
var fetchBaseBackoff: TimeInterval = 1
var fetchMaxBackoff: TimeInterval = 16
var retryStatuses: Set<Int> = [408, 425, 429, 500, 502, 503, 504] // never retry other 4xx — they're permanent
// Refresh
var regenRetrySeconds: [Int] = [300, 900, 1800] // 5 / 15 / 30 min after a 304
var jitterPercent: Double = 0.05 // ±5%
// Persistent cache — set persistFile to nil to disable
var persistFile: URL? = FileManager.default
.urls(for: .cachesDirectory, in: .userDomainMask).first?.appendingPathComponent("simkl-trending-cache.json")
var persistMaxAgeSeconds: TimeInterval = 7 * 86_400
// Catalogs — the rows on your home screen. Nil = auto-build one per supportedType.
var catalogs: [SimklTrendingCatalogConfig]? = nil
}
struct SimklTrendingCatalogConfig {
let id: String // unique row key
let source: String // 'movies' | 'tv' | 'anime' | 'dvd'
var timeframe: String? = nil // defaults to cfg.timeframes[0]
var title: String? = nil // defaults to JSON's pre-composed title
var transform: (([[String: Any]]) -> [[String: Any]])? = nil
}
struct SimklTrendingCatalog {
let id: String
let title: String
let items: [[String: Any]]
}
struct SimklTrendingError: Error {
let message: String
let status: Int?
let retryAfter: TimeInterval?
let permanent: Bool
var localizedDescription: String { message }
}
struct SimklTrendingEntry: Codable {
let title: String, url: String
let refreshSeconds: Int
let types: [String]
}
actor SimklTrendingClient {
private var cfg: SimklTrendingConfig
private let trending: [String: Any]
private var cache: [String: _SimklCached] = [:]
private var inflight: [String: Task<SimklTrendingFetchResult, Error>] = [:]
private var subs: [String: [UUID: ([[String: Any]]) -> Void]] = [:]
private var loops: [Task<Void, Never>] = []
private var stopped: Bool = false
private var lastRequestAt: Date? = nil // for minIntervalMs floor
struct _SimklCached { var itemsByType: [String: [[String: Any]]]; var lastModified: String?; var types: [String]; var fetchedAt: Date }
struct SimklTrendingFetchResult { let itemsByType: [String: [[String: Any]]]?; let lastModified: String?; let notModified: Bool }
private var defaultCatalogs: [[String: Any]]? = nil
init(config: SimklTrendingConfig = SimklTrendingConfig(), trending: [String: Any]? = nil) {
self.cfg = config
let data = trending ?? (SIMKL_TRENDING_URLS as [String: Any])
// Accept both shapes: { urls: {...}, catalogs: [...] } from JSON OR the legacy unwrapped { combined, movies, ... }.
self.trending = (data["urls"] != nil) ? data : ["urls": data]
self.defaultCatalogs = data["catalogs"] as? [[String: Any]]
}
// === Public API ===
func start() async {
restorePersisted()
for t in subs.keys { notify(type: t) } // paint immediately from restored cache
for entry in plan() {
let task = Task { await loop(entry) }
loops.append(task)
}
}
func stop() {
stopped = true
for task in loops { task.cancel() }
for (_, t) in inflight { t.cancel() }
loops.removeAll(); inflight.removeAll()
}
// Wipe in-memory + persistent cache and notify subscribers (so UI repaints empty).
// Call client.tick() after this to immediately re-fetch.
func clearCache() {
cache.removeAll()
if let url = cfg.persistFile { try? FileManager.default.removeItem(at: url) }
for (t, _) in subs { notify(type: t) }
}
func get(_ type: String) -> [[String: Any]] {
for v in cache.values { if let arr = v.itemsByType[type] { return arr } }
return []
}
func subscribe(_ type: String, _ cb: @escaping ([[String: Any]]) -> Void) -> UUID {
let id = UUID()
subs[type, default: [:]][id] = cb
let items = get(type); if !items.isEmpty { cb(items) } // immediate paint from cache
return id
}
func unsubscribe(_ type: String, _ id: UUID) { subs[type]?.removeValue(forKey: id) }
// === Catalogs — rows on the home screen ===
func catalogs() -> [SimklTrendingCatalog] {
return catalogConfigs().map { buildCatalog($0) }
}
func subscribeCatalogs(_ cb: @escaping ([SimklTrendingCatalog]) -> Void) -> [UUID] {
let sources = Set(catalogConfigs().map { $0.source })
var ids: [UUID] = []
for source in sources {
ids.append(subscribe(source) { _ in cb(self.catalogs()) })
}
if sources.allSatisfy({ get($0).isEmpty }) { cb(catalogs()) } // empty cache — skeleton paint
return ids
}
private func catalogConfigs() -> [SimklTrendingCatalogConfig] {
if let cs = cfg.catalogs { return cs } // explicit override wins
if let raw = defaultCatalogs { // JSON's catalogs[] (recommended)
return raw.compactMap { d in
guard let id = d["id"] as? String, let src = d["source"] as? String else { return nil }
return SimklTrendingCatalogConfig(id: id, source: src,
timeframe: d["timeframe"] as? String, title: d["title"] as? String, transform: nil)
}
}
let tf0 = cfg.timeframes.first ?? "today" // last resort: auto-build
return cfg.supportedTypes.map { t in
t == "dvd"
? SimklTrendingCatalogConfig(id: "dvd", source: "dvd")
: SimklTrendingCatalogConfig(id: "\(t)_\(tf0)", source: t, timeframe: tf0)
}
}
private func fileTitleFor(_ source: String, _ timeframe: String?) -> String {
let sizeKey = "top_\(cfg.size)"
let urls = trending["urls"] as? [String: Any] ?? [:]
let file: [String: Any]?
if source == "dvd" {
file = (urls["dvd"] as? [String: Any])?[sizeKey] as? [String: Any]
} else {
let bucket = (urls[source] as? [String: Any])?[timeframe ?? "today"] as? [String: Any]
file = bucket?[sizeKey] as? [String: Any]
}
return (file?["title"] as? String) ?? "Trending \(source) on Simkl"
}
private func buildCatalog(_ c: SimklTrendingCatalogConfig) -> SimklTrendingCatalog {
let items = get(c.source)
let out = c.transform?(items) ?? items
let title = c.title ?? fileTitleFor(c.source, c.timeframe ?? cfg.timeframes.first)
return SimklTrendingCatalog(id: c.id, title: title, items: out)
}
// Call from BGTaskScheduler for a single revalidate-and-persist pass.
func tick() async {
restorePersisted()
for entry in plan() {
let lm = cache[entry.url]?.lastModified
if let r = try? await fetchDedupd(entry, lastModified: lm), !r.notModified, r.itemsByType != nil {
storeAndNotify(entry: entry, result: r)
}
}
persist()
}
// === Internals ===
private func requiredQuery() -> String {
let an = cfg.appName.addingPercentEncoding(withAllowedCharacters: .urlQueryAllowed) ?? cfg.appName
let av = cfg.appVersion.addingPercentEncoding(withAllowedCharacters: .urlQueryAllowed) ?? cfg.appVersion
return "client_id=\(cfg.clientId)&app-name=\(an)&app-version=\(av)"
}
private func plan() -> [SimklTrendingEntry] {
precondition(cfg.size == 100 || cfg.size == 500, "size must be 100 or 500")
let key = "top_\(cfg.size)"
let urls = trending["urls"] as! [String: Any]
let tvma = cfg.supportedTypes.filter { ["movies","tv","anime"].contains($0) }
let useCombined = tvma.count >= 2 // 1 round-trip < 2-3 per timeframe
func make(_ raw: Any, types: [String]) -> SimklTrendingEntry {
let d = raw as! [String: Any]
return SimklTrendingEntry(title: d["title"] as! String, url: d["url"] as! String,
refreshSeconds: d["refresh_seconds"] as! Int, types: types)
}
var out: [SimklTrendingEntry] = []
if !tvma.isEmpty {
for tf in cfg.timeframes {
let bucket = (useCombined ? urls["combined"] : urls[tvma[0]]) as! [String: Any]
out.append(make((bucket[tf] as! [String: Any])[key]!, types: tvma))
}
}
if cfg.supportedTypes.contains("dvd") {
out.append(make((urls["dvd"] as! [String: Any])[key]!, types: ["dvd"]))
}
return out
}
// In-flight dedup: actor-serialized lookup of inflight tasks.
private func fetchDedupd(_ entry: SimklTrendingEntry, lastModified: String?) async throws -> SimklTrendingFetchResult {
let sep = entry.url.contains("?") ? "&" : "?"
let url = entry.url + sep + requiredQuery()
if let t = inflight[url] { return try await t.value }
let task = Task<SimklTrendingFetchResult, Error> { try await self.runFetch(url: url, entry: entry, lastModified: lastModified) }
inflight[url] = task
defer { inflight[url] = nil }
return try await task.value
}
// Process-wide rate-limit floor (so N entries refreshing at once don't fire N simultaneous requests).
private func throttle() async {
if let last = lastRequestAt {
let gapMs = Date().timeIntervalSince(last) * 1_000
let floor = Double(cfg.minIntervalMs)
if gapMs < floor { try? await Task.sleep(nanoseconds: UInt64((floor - gapMs) * 1_000_000)) }
}
lastRequestAt = Date()
}
private static func parseRetryAfter(_ value: String?) -> TimeInterval? {
guard let v = value else { return nil }
if let s = TimeInterval(v) { return max(0, s) }
let f = DateFormatter(); f.dateFormat = "EEE, dd MMM yyyy HH:mm:ss zzz"; f.locale = Locale(identifier: "en_US_POSIX")
if let d = f.date(from: v) { return max(0, d.timeIntervalSinceNow) }
return nil
}
// runFetch — throttle + retry with exponential backoff + Retry-After + permanent-error classify. Normalizes both response shapes.
private func runFetch(url: String, entry: SimklTrendingEntry, lastModified: String?) async throws -> SimklTrendingFetchResult {
var lastErr: Error?
for attempt in 0..<cfg.fetchMaxAttempts {
await throttle()
var req = URLRequest(url: URL(string: url)!, timeoutInterval: cfg.requestTimeout)
req.setValue(cfg.userAgent, forHTTPHeaderField: "User-Agent")
req.setValue("application/json", forHTTPHeaderField: "Accept")
if let lm = lastModified { req.setValue(lm, forHTTPHeaderField: "If-Modified-Since") }
do {
let (data, response) = try await URLSession.shared.data(for: req)
let http = response as! HTTPURLResponse
if http.statusCode == 304 { return SimklTrendingFetchResult(itemsByType: nil, lastModified: lastModified, notModified: true) }
guard 200..<300 ~= http.statusCode else {
let retryAfter = Self.parseRetryAfter(http.value(forHTTPHeaderField: "Retry-After"))
let permanent = !cfg.retryStatuses.contains(http.statusCode)
throw SimklTrendingError(message: "\(http.statusCode) — \(entry.title)",
status: http.statusCode, retryAfter: retryAfter, permanent: permanent)
}
let raw = try JSONSerialization.jsonObject(with: data)
var ibt: [String: [[String: Any]]] = [:]
if let arr = raw as? [[String: Any]] { ibt[entry.types[0]] = arr } // per-type / dvd → array
else if let obj = raw as? [String: Any] { // combined → object, filter to wanted types
for t in entry.types { ibt[t] = (obj[t] as? [[String: Any]]) ?? [] }
}
return SimklTrendingFetchResult(itemsByType: ibt, lastModified: http.value(forHTTPHeaderField: "Last-Modified"), notModified: false)
} catch let e as SimklTrendingError where e.permanent {
throw e // don't waste backoff on permanent errors
} catch {
if Task.isCancelled || stopped { throw error }
lastErr = error
let honored = (error as? SimklTrendingError)?.retryAfter
let backoff = honored ?? min(cfg.fetchBaseBackoff * pow(2, Double(attempt)), cfg.fetchMaxBackoff)
let jitter = backoff * (Double.random(in: 0..<1) * 0.2 - 0.1) // ±10% jitter
try? await Task.sleep(nanoseconds: UInt64(max(0, backoff + jitter) * 1_000_000_000))
}
}
throw lastErr ?? SimklTrendingError(message: "fetch failed", status: nil, retryAfter: nil, permanent: false)
}
private func loop(_ entry: SimklTrendingEntry) async {
var lastModified = cache[entry.url]?.lastModified
var retry = 0
while !stopped && !Task.isCancelled {
let r: SimklTrendingFetchResult
do { r = try await fetchDedupd(entry, lastModified: lastModified) }
catch let e as SimklTrendingError where e.permanent { // 404 / 401 / etc — halt this entry's loop
print("\(entry.title): \(e.message) (permanent, halting loop)"); return
} catch {
if stopped { return }
try? await Task.sleep(nanoseconds: 30 * 1_000_000_000); continue
}
let wait: Int
if r.notModified {
wait = cfg.regenRetrySeconds[min(retry, cfg.regenRetrySeconds.count - 1)]; retry += 1
} else if r.itemsByType != nil {
storeAndNotify(entry: entry, result: r)
lastModified = r.lastModified; retry = 0
persist()
wait = Int(Double(entry.refreshSeconds) * (1 + (Double.random(in: 0..<1) - 0.5) * 2 * cfg.jitterPercent))
} else { wait = 30 }
try? await Task.sleep(nanoseconds: UInt64(wait) * 1_000_000_000)
}
}
private func storeAndNotify(entry: SimklTrendingEntry, result r: SimklTrendingFetchResult) {
cache[entry.url] = _SimklCached(itemsByType: r.itemsByType ?? [:], lastModified: r.lastModified, types: entry.types, fetchedAt: Date())
for t in entry.types { notify(type: t) }
}
private func notify(type: String) {
let items = get(type)
for cb in (subs[type] ?? [:]).values { cb(items) }
}
private func persist() {
guard let url = cfg.persistFile else { return }
var dump: [String: [String: Any]] = [:]
for (k, v) in cache {
dump[k] = ["itemsByType": v.itemsByType, "lastModified": v.lastModified as Any,
"types": v.types, "fetchedAt": v.fetchedAt.timeIntervalSince1970]
}
if let data = try? JSONSerialization.data(withJSONObject: dump) { try? data.write(to: url, options: .atomic) }
}
private func restorePersisted() {
guard let url = cfg.persistFile,
let data = try? Data(contentsOf: url),
let raw = (try? JSONSerialization.jsonObject(with: data)) as? [String: [String: Any]] else { return }
let cutoff = Date().addingTimeInterval(-cfg.persistMaxAgeSeconds)
for (k, v) in raw {
guard let fa = v["fetchedAt"] as? TimeInterval, Date(timeIntervalSince1970: fa) >= cutoff,
let ibt = v["itemsByType"] as? [String: [[String: Any]]],
let types = v["types"] as? [String] else { continue }
cache[k] = _SimklCached(itemsByType: ibt, lastModified: v["lastModified"] as? String,
types: types, fetchedAt: Date(timeIntervalSince1970: fa))
}
}
}
// === Run ===
Task {
let client = SimklTrendingClient()
_ = await client.subscribeCatalogs { rows in
for row in rows { print("\(row.title): \(row.items.count) items") }
}
await client.start()
}
// === Background scheduling (iOS) ===
// Register a BGAppRefreshTask in your AppDelegate. In the handler call `await client.tick()`
// to perform a single revalidate-and-persist pass — Task.sleep timers do not fire while the
// app is suspended, so BGTaskScheduler is what keeps the cache warm in the background.
import kotlinx.coroutines.*
import kotlinx.coroutines.sync.Mutex
import kotlinx.coroutines.sync.withLock
import okhttp3.OkHttpClient
import okhttp3.Request
import org.json.JSONArray
import org.json.JSONObject
import java.io.File
import java.net.URLEncoder
import java.util.concurrent.TimeUnit
import kotlin.random.Random
import kotlin.math.pow
// === Load the URL catalog ===
// 1. Save the "Copy-paste JSON" block above as `simkl-trending-urls.json` in `app/src/main/assets/`.
// 2. Load it into a global `SIMKL_TRENDING_URLS: JSONObject` once at app startup
// (e.g. in your Application.onCreate or DI module):
// val SIMKL_TRENDING_URLS = JSONObject(
// context.assets.open("simkl-trending-urls.json").bufferedReader().use { it.readText() }
// )
// You can also skip the global and pass it directly: `SimklTrendingClient(cfg, trending = urls)`.
// The constructor throws if neither the global nor the `trending` arg is provided.
// === CONFIG — tune to match your app ===
data class SimklTrendingConfig(
// What to load
val supportedTypes: List<String> = listOf("movies", "tv", "anime", "dvd"), // movies / tv / anime / dvd — drop ones your app doesn't surface
val timeframes: List<String> = listOf("today"), // today / week / month (ignored for dvd)
val size: Int = 100, // 100 or 500 — use 500 if you render derived catalogs (see "Build your own catalogs" below)
// HTTP — replace with YOUR values
val clientId: String = "YOUR_CLIENT_ID",
val appName: String = "my-app-name",
val appVersion: String = "1.0",
val userAgent: String = "my-app-name/1.0",
val requestTimeoutMs: Long = 8_000,
val minIntervalMs: Long = 250, // process-wide floor between consecutive requests
val fetchMaxAttempts: Int = 3,
val fetchBaseBackoffMs: Long = 1_000,
val fetchMaxBackoffMs: Long = 16_000,
val retryStatuses: Set<Int> = setOf(408, 425, 429, 500, 502, 503, 504), // never retry other 4xx — they're permanent
// Refresh
val regenRetrySeconds: List<Int> = listOf(300, 900, 1_800), // 5 / 15 / 30 min after a 304
val jitterPercent: Double = 0.05, // ±5%
// Persistent cache — set persistFile to null to disable. In Android pass context.cacheDir.
val persistFile: File? = null, // e.g. File(context.cacheDir, "simkl-trending-cache.json")
val persistMaxAgeMs: Long = 7L * 86_400 * 1_000,
// Catalogs — the rows on your home screen. Null = auto-build one per supportedType.
val catalogs: List<SimklTrendingCatalogConfig>? = null,
)
data class SimklTrendingCatalogConfig(
val id: String,
val source: String, // "movies" | "tv" | "anime" | "dvd"
val timeframe: String? = null, // defaults to cfg.timeframes[0]
val title: String? = null, // defaults to JSON's pre-composed title
val transform: ((List<JSONArray>) -> List<JSONArray>)? = null,
)
data class SimklTrendingCatalog(val id: String, val title: String, val items: List<JSONArray>)
data class SimklTrendingEntry(val title: String, val url: String, val refreshSeconds: Int, val types: List<String>)
class SimklTrendingError(message: String, val status: Int? = null, val retryAfterMs: Long? = null, val permanent: Boolean = false) : RuntimeException(message)
class SimklTrendingClient(
private val cfg: SimklTrendingConfig = SimklTrendingConfig(),
data: JSONObject = SIMKL_TRENDING_URLS,
) {
// Accept both JSON shapes: { urls: {...}, catalogs: [...] } OR the legacy unwrapped { combined, movies, ... }.
private val trending: JSONObject = if (data.has("urls")) data else JSONObject().put("urls", data)
private val defaultCatalogs: JSONArray? = data.optJSONArray("catalogs")
private data class _SimklCached(val itemsByType: Map<String, JSONArray>, val lastModified: String?, val types: List<String>, val fetchedAt: Long)
data class SimklTrendingFetchResult(val itemsByType: Map<String, JSONArray>?, val lastModified: String?, val notModified: Boolean)
private val http = OkHttpClient.Builder()
.callTimeout(cfg.requestTimeoutMs, TimeUnit.MILLISECONDS).build()
private val cache = mutableMapOf<String, _SimklCached>()
private val inflight = mutableMapOf<String, Deferred<SimklTrendingFetchResult>>()
private val subs = mutableMapOf<String, MutableMap<Int, (List<JSONArray>) -> Unit>>()
private var subIdSeq = 0
private val mu = Mutex()
private val scope = CoroutineScope(SupervisorJob() + Dispatchers.IO)
private var lastRequestAt = 0L // for minIntervalMs floor
// === Public API ===
suspend fun start() {
restorePersisted()
mu.withLock { subs.keys.toList() }.forEach { notify(it) } // paint immediately from restored cache
plan().forEach { entry -> scope.launch { loop(entry) } }
}
fun stop() { scope.cancel() }
/** Wipe in-memory + persistent cache and notify subscribers (UI repaints empty). Call client.tick() after to re-fetch. */
suspend fun clearCache() {
val snapshot = mu.withLock {
cache.clear()
subs.mapValues { it.value.values.toList() }
}
cfg.persistFile?.takeIf { it.exists() }?.delete()
snapshot.forEach { (_, cbs) -> cbs.forEach { it(emptyList()) } }
}
suspend fun get(type: String): List<JSONArray> = mu.withLock {
for (v in cache.values) v.itemsByType[type]?.let { return@withLock listOf(it) }
emptyList()
}
suspend fun subscribe(type: String, cb: (List<JSONArray>) -> Unit): Int {
val id = mu.withLock {
val i = ++subIdSeq
subs.getOrPut(type) { mutableMapOf() }[i] = cb
i
}
val items = get(type); if (items.isNotEmpty()) cb(items) // immediate paint from cache
return id
}
suspend fun unsubscribe(type: String, id: Int) { mu.withLock { subs[type]?.remove(id) } }
// === Catalogs — rows on the home screen ===
suspend fun catalogs(): List<SimklTrendingCatalog> = catalogConfigs().map { buildCatalog(it) }
suspend fun subscribeCatalogs(cb: (List<SimklTrendingCatalog>) -> Unit): List<Int> {
val sources = catalogConfigs().map { it.source }.toSet()
val ids = mutableListOf<Int>()
for (s in sources) {
ids.add(subscribe(s) { _ -> scope.launch { cb(catalogs()) } })
}
if (sources.all { get(it).isEmpty() }) cb(catalogs()) // empty cache — skeleton paint
return ids
}
private fun catalogConfigs(): List<SimklTrendingCatalogConfig> {
cfg.catalogs?.let { return it } // explicit override wins
defaultCatalogs?.let { arr -> // JSON's catalogs[]
return (0 until arr.length()).mapNotNull { i ->
val o = arr.optJSONObject(i) ?: return@mapNotNull null
val id = o.optString("id"); val src = o.optString("source")
if (id.isEmpty() || src.isEmpty()) null
else SimklTrendingCatalogConfig(id = id, source = src,
timeframe = if (o.isNull("timeframe")) null else o.optString("timeframe"),
title = if (o.isNull("title")) null else o.optString("title"),
transform = null)
}
}
val tf0 = cfg.timeframes.firstOrNull() ?: "today" // last resort: auto-build
return cfg.supportedTypes.map { t ->
if (t == "dvd") SimklTrendingCatalogConfig(id = "dvd", source = "dvd")
else SimklTrendingCatalogConfig(id = "${t}_${tf0}", source = t, timeframe = tf0)
}
}
private fun fileTitleFor(source: String, timeframe: String?): String {
val sizeKey = "top_${cfg.size}"
val urls = trending.optJSONObject("urls") ?: return "Trending $source on Simkl"
val file = if (source == "dvd") urls.optJSONObject("dvd")?.optJSONObject(sizeKey)
else urls.optJSONObject(source)?.optJSONObject(timeframe ?: "today")?.optJSONObject(sizeKey)
return file?.optString("title") ?: "Trending $source on Simkl"
}
private suspend fun buildCatalog(c: SimklTrendingCatalogConfig): SimklTrendingCatalog {
val items = get(c.source)
val out = c.transform?.invoke(items) ?: items
val title = c.title ?: fileTitleFor(c.source, c.timeframe ?: cfg.timeframes.firstOrNull())
return SimklTrendingCatalog(c.id, title, out)
}
// Call from a WorkManager PeriodicWorker for a single revalidate-and-persist pass.
suspend fun tick() {
restorePersisted()
plan().forEach { entry ->
val lm = mu.withLock { cache[entry.url]?.lastModified }
runCatching { fetchDedupd(entry, lm) }.getOrNull()?.let { r ->
if (!r.notModified && r.itemsByType != null) storeAndNotify(entry, r)
}
}
persist()
}
// === Internals ===
private fun requiredQuery(): String =
"client_id=${cfg.clientId}" +
"&app-name=${URLEncoder.encode(cfg.appName, "UTF-8")}" +
"&app-version=${URLEncoder.encode(cfg.appVersion, "UTF-8")}"
private fun plan(): List<SimklTrendingEntry> {
require(cfg.size == 100 || cfg.size == 500) { "size must be 100 or 500" }
val key = "top_${cfg.size}"
val urls = trending.getJSONObject("urls")
val tvma = cfg.supportedTypes.filter { it in listOf("movies", "tv", "anime") }
val useCombined = tvma.size >= 2 // 1 round-trip < 2-3 per timeframe
fun make(o: JSONObject, types: List<String>) =
SimklTrendingEntry(o.getString("title"), o.getString("url"), o.getInt("refresh_seconds"), types)
val out = mutableListOf<SimklTrendingEntry>()
if (tvma.isNotEmpty()) cfg.timeframes.forEach { tf ->
val bucket = if (useCombined) urls.getJSONObject("combined") else urls.getJSONObject(tvma[0])
out.add(make(bucket.getJSONObject(tf).getJSONObject(key), tvma))
}
if ("dvd" in cfg.supportedTypes)
out.add(make(urls.getJSONObject("dvd").getJSONObject(key), listOf("dvd")))
return out
}
// In-flight dedup: only one outstanding network call per URL.
private suspend fun fetchDedupd(entry: SimklTrendingEntry, lastModified: String?): SimklTrendingFetchResult {
val sep = if (entry.url.contains('?')) "&" else "?"
val url = entry.url + sep + requiredQuery()
val existing = mu.withLock { inflight[url] }
if (existing != null) return existing.await()
val task = scope.async { runFetch(url, entry, lastModified) }
mu.withLock { inflight[url] = task }
try { return task.await() } finally { mu.withLock { inflight.remove(url) } }
}
// Process-wide rate-limit floor (so N entries refreshing at once don't fire N simultaneous requests).
private suspend fun throttle() {
val gap = System.currentTimeMillis() - lastRequestAt
if (gap < cfg.minIntervalMs) delay(cfg.minIntervalMs - gap)
lastRequestAt = System.currentTimeMillis()
}
private fun parseRetryAfter(value: String?): Long? {
if (value == null) return null
value.toLongOrNull()?.let { return maxOf(0L, it * 1000L) }
return runCatching {
val fmt = java.time.format.DateTimeFormatter.RFC_1123_DATE_TIME
val dt = java.time.ZonedDateTime.parse(value, fmt)
maxOf(0L, dt.toInstant().toEpochMilli() - System.currentTimeMillis())
}.getOrNull()
}
// runFetch — throttle + retry with exponential backoff + Retry-After + permanent-error classify. Normalizes both response shapes.
private suspend fun runFetch(url: String, entry: SimklTrendingEntry, lastModified: String?): SimklTrendingFetchResult {
var lastErr: Exception? = null
for (attempt in 0 until cfg.fetchMaxAttempts) {
throttle()
try {
val req = Request.Builder().url(url)
.header("User-Agent", cfg.userAgent)
.header("Accept", "application/json")
.apply { lastModified?.let { header("If-Modified-Since", it) } }.build()
http.newCall(req).execute().use { res ->
if (res.code == 304) return SimklTrendingFetchResult(null, lastModified, notModified = true)
if (!res.isSuccessful) {
val retryAfter = parseRetryAfter(res.header("Retry-After"))
val permanent = res.code !in cfg.retryStatuses
throw SimklTrendingError("${res.code} ${res.message} — ${entry.title}",
status = res.code, retryAfterMs = retryAfter, permanent = permanent)
}
val body = res.body!!.string().trim()
val ibt = mutableMapOf<String, JSONArray>()
if (body.startsWith("[")) ibt[entry.types[0]] = JSONArray(body) // per-type / dvd → array
else { // combined → object, filter to wanted types
val obj = JSONObject(body)
entry.types.forEach { ibt[it] = obj.optJSONArray(it) ?: JSONArray() }
}
return SimklTrendingFetchResult(ibt, res.header("Last-Modified"), notModified = false)
}
} catch (e: CancellationException) { throw e }
catch (e: SimklTrendingError) {
if (e.permanent) throw e // don't waste backoff on permanent errors
lastErr = e
val honored = e.retryAfterMs
val backoff = honored ?: (cfg.fetchBaseBackoffMs * 2.0.pow(attempt.toDouble())).toLong().coerceAtMost(cfg.fetchMaxBackoffMs)
val jitter = (backoff * (Random.nextDouble() * 0.2 - 0.1)).toLong() // ±10% jitter
delay(maxOf(0L, backoff + jitter))
}
catch (e: Exception) {
lastErr = e
val backoff = (cfg.fetchBaseBackoffMs * 2.0.pow(attempt.toDouble())).toLong().coerceAtMost(cfg.fetchMaxBackoffMs)
delay(backoff)
}
}
throw lastErr ?: SimklTrendingError("fetch failed")
}
private suspend fun loop(entry: SimklTrendingEntry) {
var lastModified = mu.withLock { cache[entry.url]?.lastModified }
var retry = 0
while (scope.isActive) {
val r = try { fetchDedupd(entry, lastModified) }
catch (e: CancellationException) { return }
catch (e: SimklTrendingError) {
if (e.permanent) { println("${entry.title}: ${e.message} (permanent, halting loop)"); return }
delay(30_000L); continue
}
catch (e: Exception) { delay(30_000L); continue }
val wait = if (r.notModified) {
cfg.regenRetrySeconds[minOf(retry, cfg.regenRetrySeconds.lastIndex)].also { retry++ }
} else {
storeAndNotify(entry, r)
lastModified = r.lastModified; retry = 0
persist()
(entry.refreshSeconds * (1.0 + (Random.nextDouble() - 0.5) * 2 * cfg.jitterPercent)).toInt()
}
delay(wait * 1_000L)
}
}
private suspend fun storeAndNotify(entry: SimklTrendingEntry, r: SimklTrendingFetchResult) {
mu.withLock {
cache[entry.url] = _SimklCached(r.itemsByType ?: emptyMap(), r.lastModified, entry.types, System.currentTimeMillis())
}
entry.types.forEach { notify(it) }
}
private suspend fun notify(type: String) {
val items = get(type)
val cbs = mu.withLock { subs[type]?.values?.toList().orEmpty() }
cbs.forEach { it(items) }
}
private suspend fun persist() {
val f = cfg.persistFile ?: return
val dump = JSONObject()
mu.withLock {
for ((url, v) in cache) {
val o = JSONObject()
val ibt = JSONObject(); v.itemsByType.forEach { (k, arr) -> ibt.put(k, arr) }
o.put("itemsByType", ibt)
o.put("lastModified", v.lastModified ?: JSONObject.NULL)
o.put("types", JSONArray(v.types)); o.put("fetchedAt", v.fetchedAt)
dump.put(url, o)
}
}
runCatching {
val tmp = File(f.parentFile, f.name + ".tmp")
tmp.writeText(dump.toString()); tmp.renameTo(f)
}
}
private suspend fun restorePersisted() {
val f = cfg.persistFile ?: return
if (!f.exists()) return
runCatching {
val raw = JSONObject(f.readText())
val cutoff = System.currentTimeMillis() - cfg.persistMaxAgeMs
mu.withLock {
raw.keys().forEach { url ->
val o = raw.getJSONObject(url)
val fa = o.optLong("fetchedAt", 0L); if (fa < cutoff) return@forEach
val ibtO = o.getJSONObject("itemsByType")
val ibt = mutableMapOf<String, JSONArray>()
ibtO.keys().forEach { t -> ibt[t] = ibtO.getJSONArray(t) }
val types = (0 until o.getJSONArray("types").length()).map { o.getJSONArray("types").getString(it) }
cache[url] = _SimklCached(ibt, if (o.isNull("lastModified")) null else o.getString("lastModified"), types, fa)
}
}
}
}
}
// === Run ===
runBlocking {
val client = SimklTrendingClient()
client.subscribeCatalogs { rows ->
for (row in rows) println("${row.title}: ${row.items.size} items")
}
client.start()
}
// === Background scheduling (Android) ===
// In-process coroutines stop when the app process is killed. Schedule a
// WorkManager.PeriodicWorkRequest whose Worker calls `client.tick()` to keep the cache warm
// in the background. Use cfg.persistFile = File(context.cacheDir, "simkl-trending-cache.json")
// so the next foreground launch can paint the UI from disk instantly.
import 'dart:async';
import 'dart:convert';
import 'dart:io';
import 'dart:math';
import 'package:http/http.dart' as http;
// import 'package:path_provider/path_provider.dart'; // for persistFile in Flutter — see Run block
// === Load the URL catalog ===
// 1. Save the "Copy-paste JSON" block above as `assets/simkl-trending-urls.json`, then declare
// it under `flutter: assets:` in pubspec.yaml.
// 2. Load it into a global `SIMKL_TRENDING_URLS` once at app startup (after WidgetsFlutterBinding.ensureInitialized):
// final SIMKL_TRENDING_URLS = jsonDecode(
// await rootBundle.loadString('assets/simkl-trending-urls.json')
// ) as Map<String, dynamic>;
// For pure-Dart (server / CLI / tests), use:
// final SIMKL_TRENDING_URLS = jsonDecode(await File('simkl-trending-urls.json').readAsString()) as Map<String, dynamic>;
// You can also skip the global and pass it explicitly: `SimklTrendingClient(trending: urlsMap)`.
// === CONFIG — tune to match your app ===
class SimklTrendingConfig {
// What to load
final List<String> supportedTypes; // movies / tv / anime / dvd
final List<String> timeframes; // today / week / month (ignored for dvd)
final int size; // 100 or 500 — use 500 if you render derived catalogs (see "Build your own catalogs" below)
// HTTP — replace with YOUR values
final String clientId, appName, appVersion, userAgent;
final Duration requestTimeout;
final Duration minInterval; // process-wide floor between consecutive requests
final int fetchMaxAttempts;
final Duration fetchBaseBackoff, fetchMaxBackoff;
final Set<int> retryStatuses; // never retry other 4xx — they're permanent
// Refresh
final List<int> regenRetrySeconds; // 5 / 15 / 30 min after a 304
final double jitterPercent; // ±5%
// Persistent cache — set persistFile to null to disable.
final File? persistFile;
final Duration persistMaxAge;
// Catalogs — the rows on your home screen. Null = auto-build one per supportedType.
final List<SimklTrendingCatalogConfig>? catalogs;
const SimklTrendingConfig({
this.supportedTypes = const ['movies', 'tv', 'anime', 'dvd'],
this.timeframes = const ['today'],
this.size = 100,
this.clientId = 'YOUR_CLIENT_ID',
this.appName = 'my-app-name',
this.appVersion = '1.0',
this.userAgent = 'my-app-name/1.0',
this.requestTimeout = const Duration(seconds: 8),
this.minInterval = const Duration(milliseconds: 250),
this.fetchMaxAttempts = 3,
this.fetchBaseBackoff = const Duration(seconds: 1),
this.fetchMaxBackoff = const Duration(seconds: 16),
this.retryStatuses = const {408, 425, 429, 500, 502, 503, 504},
this.regenRetrySeconds = const [300, 900, 1800],
this.jitterPercent = 0.05,
this.persistFile,
this.persistMaxAge = const Duration(days: 7),
this.catalogs,
});
}
class SimklTrendingCatalogConfig {
final String id;
final String source; // 'movies' | 'tv' | 'anime' | 'dvd'
final String? timeframe; // defaults to cfg.timeframes.first
final String? title; // defaults to JSON's pre-composed title
final List<dynamic> Function(List<dynamic>)? transform;
SimklTrendingCatalogConfig({required this.id, required this.source, this.timeframe, this.title, this.transform});
}
class SimklTrendingCatalog {
final String id, title;
final List<dynamic> items;
SimklTrendingCatalog(this.id, this.title, this.items);
}
class SimklTrendingError implements Exception {
final String message;
final int? status;
final Duration? retryAfter;
final bool permanent;
SimklTrendingError(this.message, {this.status, this.retryAfter, this.permanent = false});
@override String toString() => 'SimklTrendingError($status): $message';
}
class SimklTrendingEntry {
final String title, url;
final int refreshSeconds;
final List<String> types;
SimklTrendingEntry(this.title, this.url, this.refreshSeconds, this.types);
}
class _SimklCached {
Map<String, List<dynamic>> itemsByType; String? lastModified; List<String> types; int fetchedAt;
_SimklCached(this.itemsByType, this.lastModified, this.types, this.fetchedAt);
}
class SimklTrendingFetchResult {
final Map<String, List<dynamic>>? itemsByType;
final String? lastModified;
final bool notModified;
SimklTrendingFetchResult({this.itemsByType, this.lastModified, this.notModified = false});
}
class SimklTrendingClient {
final SimklTrendingConfig cfg;
final Map<String, dynamic> trending;
final _cache = <String, _SimklCached>{};
final _inflight = <String, Future<SimklTrendingFetchResult>>{};
final _subs = <String, Map<int, void Function(List<dynamic>)>>{};
final _client = http.Client();
int _subIdSeq = 0;
bool _stopped = false;
int _lastRequestAt = 0; // for minInterval floor
final _loops = <Future<void>>[];
final List<dynamic>? defaultCatalogs;
// Accept both JSON shapes: { urls: {...}, catalogs: [...] } OR the legacy unwrapped { combined, movies, ... }.
factory SimklTrendingClient({SimklTrendingConfig? config, Map<String, dynamic>? trending}) {
final raw = trending ?? (SIMKL_TRENDING_URLS as Map<String, dynamic>);
final urlsMap = raw['urls'] != null ? raw : { 'urls': raw };
return SimklTrendingClient._internal(config ?? const SimklTrendingConfig(),
urlsMap as Map<String, dynamic>,
raw['catalogs'] as List<dynamic>?);
}
SimklTrendingClient._internal(this.cfg, this.trending, this.defaultCatalogs);
// === Public API ===
Future<void> start() async {
await _restorePersisted();
for (final t in _subs.keys.toList()) _notify(t); // paint immediately from restored cache
for (final entry in _plan()) _loops.add(_loop(entry));
}
void stop() { _stopped = true; _client.close(); }
/// Wipe in-memory + persistent cache and notify subscribers (UI repaints empty). Call client.tick() after to re-fetch.
Future<void> clearCache() async {
_cache.clear();
final f = cfg.persistFile;
if (f != null && await f.exists()) { try { await f.delete(); } catch (_) {/* ignore */} }
for (final t in _subs.keys.toList()) _notify(t);
}
List<dynamic> get(String type) {
for (final v in _cache.values) if (v.itemsByType.containsKey(type)) return v.itemsByType[type]!;
return const [];
}
int subscribe(String type, void Function(List<dynamic>) cb) {
final id = ++_subIdSeq;
(_subs[type] ??= {})[id] = cb;
final items = get(type); if (items.isNotEmpty) cb(items); // immediate paint from cache
return id;
}
void unsubscribe(String type, int id) { _subs[type]?.remove(id); }
// === Catalogs — rows on the home screen ===
List<SimklTrendingCatalog> catalogs() => _catalogConfigs().map(_buildCatalog).toList();
List<int> subscribeCatalogs(void Function(List<SimklTrendingCatalog>) cb) {
final sources = _catalogConfigs().map((c) => c.source).toSet();
final ids = <int>[];
for (final s in sources) {
ids.add(subscribe(s, (_) => cb(catalogs())));
}
if (sources.every((s) => get(s).isEmpty)) cb(catalogs()); // empty cache — skeleton paint
return ids;
}
List<SimklTrendingCatalogConfig> _catalogConfigs() {
if (cfg.catalogs != null) return cfg.catalogs!; // explicit override wins
if (defaultCatalogs != null) { // JSON's catalogs[]
return defaultCatalogs!.whereType<Map>().map((d) => SimklTrendingCatalogConfig(
id: d['id'] as String, source: d['source'] as String,
timeframe: d['timeframe'] as String?, title: d['title'] as String?,
)).toList();
}
final tf0 = cfg.timeframes.isNotEmpty ? cfg.timeframes.first : 'today'; // last resort: auto-build
return cfg.supportedTypes.map((t) {
if (t == 'dvd') return SimklTrendingCatalogConfig(id: 'dvd', source: 'dvd');
return SimklTrendingCatalogConfig(id: '${t}_$tf0', source: t, timeframe: tf0);
}).toList();
}
String _fileTitleFor(String source, String? timeframe) {
final sizeKey = 'top_${cfg.size}';
final urls = trending['urls'] as Map?;
if (urls == null) return 'Trending $source on Simkl';
final file = source == 'dvd'
? (urls['dvd'] as Map?)?[sizeKey]
: ((urls[source] as Map?)?[timeframe ?? 'today'] as Map?)?[sizeKey];
return (file as Map?)?['title'] as String? ?? 'Trending $source on Simkl';
}
SimklTrendingCatalog _buildCatalog(SimklTrendingCatalogConfig c) {
final items = get(c.source);
final out = c.transform != null ? c.transform!(items) : items;
final title = c.title ?? _fileTitleFor(c.source, c.timeframe ?? (cfg.timeframes.isNotEmpty ? cfg.timeframes.first : 'today'));
return SimklTrendingCatalog(c.id, title, out);
}
// Call from a `workmanager` periodic task for a single revalidate-and-persist pass.
Future<void> tick() async {
await _restorePersisted();
for (final entry in _plan()) {
final lm = _cache[entry.url]?.lastModified;
try {
final r = await _fetchDedupd(entry, lm);
if (!r.notModified && r.itemsByType != null) _storeAndNotify(entry, r);
} catch (_) { /* swallow — best-effort */ }
}
await _persist();
}
// === Internals ===
String _requiredQuery() =>
'client_id=${cfg.clientId}'
'&app-name=${Uri.encodeQueryComponent(cfg.appName)}'
'&app-version=${Uri.encodeQueryComponent(cfg.appVersion)}';
List<SimklTrendingEntry> _plan() {
if (cfg.size != 100 && cfg.size != 500) throw ArgumentError('size must be 100 or 500');
final key = 'top_${cfg.size}';
final urls = trending['urls'] as Map;
final tvma = cfg.supportedTypes.where((t) => ['movies','tv','anime'].contains(t)).toList();
final useCombined = tvma.length >= 2; // 1 round-trip < 2-3 per timeframe
SimklTrendingEntry make(Map e, List<String> types) =>
SimklTrendingEntry(e['title'], e['url'], e['refresh_seconds'], types);
final out = <SimklTrendingEntry>[];
if (tvma.isNotEmpty) for (final tf in cfg.timeframes) {
final bucket = (useCombined ? urls['combined'] : urls[tvma[0]]) as Map;
out.add(make((bucket[tf] as Map)[key], tvma));
}
if (cfg.supportedTypes.contains('dvd')) out.add(make((urls['dvd'] as Map)[key], ['dvd']));
return out;
}
// In-flight dedup: only one outstanding network call per URL.
Future<SimklTrendingFetchResult> _fetchDedupd(SimklTrendingEntry entry, String? lastModified) {
final sep = entry.url.contains('?') ? '&' : '?';
final url = entry.url + sep + _requiredQuery();
final existing = _inflight[url]; if (existing != null) return existing;
final task = _runFetch(url, entry, lastModified).whenComplete(() { _inflight.remove(url); });
_inflight[url] = task;
return task;
}
// Process-wide rate-limit floor (so N entries refreshing at once don't fire N simultaneous requests).
Future<void> _throttle() async {
final gap = DateTime.now().millisecondsSinceEpoch - _lastRequestAt;
if (gap < cfg.minInterval.inMilliseconds) {
await Future.delayed(Duration(milliseconds: cfg.minInterval.inMilliseconds - gap));
}
_lastRequestAt = DateTime.now().millisecondsSinceEpoch;
}
static Duration? _parseRetryAfter(String? value) {
if (value == null) return null;
final secs = num.tryParse(value);
if (secs != null) return Duration(milliseconds: (secs.toDouble() * 1000).round().clamp(0, 1 << 31));
try {
final dt = HttpDate.parse(value);
return Duration(milliseconds: (dt.millisecondsSinceEpoch - DateTime.now().millisecondsSinceEpoch).clamp(0, 1 << 31));
} catch (_) { return null; }
}
// _runFetch — throttle + retry with exponential backoff + Retry-After + permanent-error classify. Normalizes both response shapes.
Future<SimklTrendingFetchResult> _runFetch(String url, SimklTrendingEntry entry, String? lastModified) async {
Object? lastErr;
for (var attempt = 0; attempt < cfg.fetchMaxAttempts; attempt++) {
await _throttle();
try {
final headers = {'User-Agent': cfg.userAgent, 'Accept': 'application/json'};
if (lastModified != null) headers['If-Modified-Since'] = lastModified;
final res = await _client.get(Uri.parse(url), headers: headers).timeout(cfg.requestTimeout);
if (res.statusCode == 304) return SimklTrendingFetchResult(itemsByType: null, lastModified: lastModified, notModified: true);
if (res.statusCode != 200) {
final retryAfter = _parseRetryAfter(res.headers['retry-after']);
final permanent = !cfg.retryStatuses.contains(res.statusCode);
throw SimklTrendingError('${res.statusCode} ${res.reasonPhrase} — ${entry.title}',
status: res.statusCode, retryAfter: retryAfter, permanent: permanent);
}
final raw = jsonDecode(res.body);
final ibt = <String, List<dynamic>>{};
if (raw is List) ibt[entry.types[0]] = raw; // per-type / dvd → array
else if (raw is Map) for (final t in entry.types) ibt[t] = (raw[t] as List?) ?? []; // combined → object, filter to wanted types
return SimklTrendingFetchResult(itemsByType: ibt, lastModified: res.headers['last-modified']);
} on SimklTrendingError catch (e) {
if (e.permanent) rethrow; // don't waste backoff on permanent errors
lastErr = e;
final honoredMs = e.retryAfter?.inMilliseconds;
final backoffMs = honoredMs ?? min(cfg.fetchBaseBackoff.inMilliseconds * pow(2, attempt).toInt(),
cfg.fetchMaxBackoff.inMilliseconds);
final jitterMs = (backoffMs * (Random().nextDouble() * 0.2 - 0.1)).round(); // ±10% jitter
await Future.delayed(Duration(milliseconds: max(0, backoffMs + jitterMs)));
} catch (e) {
if (_stopped) rethrow;
lastErr = e;
final backoffMs = min(cfg.fetchBaseBackoff.inMilliseconds * pow(2, attempt).toInt(),
cfg.fetchMaxBackoff.inMilliseconds);
await Future.delayed(Duration(milliseconds: backoffMs));
}
}
throw lastErr ?? SimklTrendingError('fetch failed');
}
Future<void> _loop(SimklTrendingEntry entry) async {
var lastModified = _cache[entry.url]?.lastModified;
var retry = 0;
while (!_stopped) {
late SimklTrendingFetchResult r;
try { r = await _fetchDedupd(entry, lastModified); }
on SimklTrendingError catch (e) {
if (_stopped) return;
if (e.permanent) { print('${entry.title}: ${e.message} (permanent, halting loop)'); return; }
await Future.delayed(const Duration(seconds: 30)); continue;
}
catch (e) {
if (_stopped) return;
await Future.delayed(const Duration(seconds: 30)); continue;
}
int wait;
if (r.notModified) {
wait = cfg.regenRetrySeconds[min(retry, cfg.regenRetrySeconds.length - 1)]; retry++;
} else {
_storeAndNotify(entry, r);
lastModified = r.lastModified; retry = 0;
await _persist();
wait = (entry.refreshSeconds * (1 + (Random().nextDouble() - 0.5) * 2 * cfg.jitterPercent)).round();
}
await Future.delayed(Duration(seconds: wait));
}
}
void _storeAndNotify(SimklTrendingEntry entry, SimklTrendingFetchResult r) {
_cache[entry.url] = _SimklCached(r.itemsByType ?? {}, r.lastModified, entry.types, DateTime.now().millisecondsSinceEpoch);
for (final t in entry.types) _notify(t);
}
void _notify(String type) {
final items = get(type);
for (final cb in (_subs[type]?.values ?? const Iterable.empty())) cb(items);
}
Future<void> _persist() async {
final f = cfg.persistFile; if (f == null) return;
try {
final dump = {
for (final e in _cache.entries) e.key: {
'itemsByType': e.value.itemsByType, 'lastModified': e.value.lastModified,
'types': e.value.types, 'fetchedAt': e.value.fetchedAt,
},
};
final tmp = File('${f.path}.tmp');
await tmp.writeAsString(jsonEncode(dump));
await tmp.rename(f.path);
} catch (_) { /* best-effort */ }
}
Future<void> _restorePersisted() async {
final f = cfg.persistFile; if (f == null || !await f.exists()) return;
try {
final raw = jsonDecode(await f.readAsString()) as Map<String, dynamic>;
final cutoff = DateTime.now().millisecondsSinceEpoch - cfg.persistMaxAge.inMilliseconds;
for (final entry in raw.entries) {
final v = entry.value as Map;
final fa = (v['fetchedAt'] as num?)?.toInt() ?? 0;
if (fa < cutoff) continue;
final ibt = (v['itemsByType'] as Map).map((k, val) => MapEntry(k as String, List<dynamic>.from(val as List)));
_cache[entry.key] = _SimklCached(ibt, v['lastModified'] as String?, List<String>.from(v['types'] as List), fa);
}
} catch (_) { /* no cache yet — fine */ }
}
}
// === Run ===
Future<void> main() async {
// For Flutter, set persistFile via path_provider:
// final dir = await getApplicationCacheDirectory();
// final cfg = SimklTrendingConfig(persistFile: File('${dir.path}/simkl-trending-cache.json'));
final client = SimklTrendingClient();
client.subscribeCatalogs((rows) {
for (final row in rows) print('${row.title}: ${row.items.length} items');
});
await client.start();
}
// === Background scheduling (Flutter) ===
// Future.delayed stops when the app is killed. Schedule a `workmanager` package periodic task
// whose callback calls `await client.tick()` to keep the cache warm in the background. Use the
// path_provider snippet above for persistFile so the next foreground launch paints from disk.
Node 22+, Deno, Bun, React Native, Cloudflare Workers, and other fetch-based JS runtimes — the JavaScript tab above (the SDK file) is universal: save it verbatim as Bundle
simkl-trending-client.js, then wire it up from your app entry point in ~6 lines. The Browser tab right above it is the HTML+DOM wrapper you’d skip in a headless runtime; here’s the non-browser equivalent:app.mjs (or index.js App.tsx)
// Pick the loader your runtime prefers:
import data from './simkl-trending-urls.json' with { type: 'json' }; // Node 22+ ESM
// const data = require('./simkl-trending-urls.json'); // Node CommonJS
// const data = await (await fetch('/simkl-trending-urls.json')).json(); // Deno / Bun / RN / CF Workers
import { SimklTrendingClient } from './simkl-trending-client.js';
const client = new SimklTrendingClient(
{ clientId: 'YOUR_CLIENT_ID', appName: 'my-app', appVersion: '1.0' },
data,
);
client.subscribeCatalogs(rows => rows.forEach(r => console.log(`${r.title}: ${r.items.length} items`)));
await client.start();
// process.on('SIGINT', () => { client.stop(); process.exit(0); }); // long-running servers
simkl-trending-urls.json with your app (or fetch + cache it once at startup) so the catalog is in scope when SimklTrendingClient runs. For Cloudflare Workers / Lambda / GitHub Actions etc., call await client.tick() from your scheduler instead of client.start() — see the “Background scheduling” comment at the bottom of the JavaScript tab.What you still need to do per-platform. The
SimklTrendingClient samples above wire up everything that’s host-agnostic — auto-pick combined, persistent cache, conditional revalidation, regen-lag backoff, jitter, retry with exponential backoff + timeout, in-flight dedup, cancellation, and subscribe/get. Two things are left to you because they depend on your platform:- Background scheduling that survives app lifecycle. In-process timers stop firing when an iOS app is backgrounded or after Android process death. Hook your platform’s periodic task system to
client.tick()—BGTaskScheduler(iOS),WorkManager.PeriodicWorkRequest(Android), theworkmanagerpackage (Flutter), ornode-cron/ Cloudflare Cron / AWS EventBridge for server-side runtimes. Each tab’s trailing comment shows the wireup. - Attribution back-link. Websites must link the rendered section title to the matching
simkl.com/<type>/best-*/most-watched/page — see Attribution required and Linking back.
Codable, Kotlin @Serializable, or Dart json_serializable.Catalogs — the rows on your home screen
Think of a TV-style discovery UI: rows of poster carousels stacked vertically, each row with a title at the top-left. The SDK exposes that asclient.catalogs() → [{id, title, items}, ...] and client.subscribeCatalogs(cb) — available in all 5 languages above. Each row has a stable id, an attribution-compliant title, and an items array. You iterate and render.
Three places rows can come from, in priority order:
cfg.catalogsin your SDK config — explicit, wins if set. Use this to mix trending + derived rows with customtransformclosures (see below).catalogs[]in the Copy-paste JSON above — the default list of rows. Already populated with one row per type (movies_today,tv_today,anime_today,dvd). Add, remove, or reorder entries in the JSON to change defaults across your whole app — no code change.- Auto-built fallback — if neither of the above is set, the SDK builds one row per
supportedType×timeframes[0]from your CONFIG.
new SimklTrendingClient() immediately gives you 4 ready-to-render rows — no extra config needed. Override only when you want custom rows.
Adding derived rows in code — when you want filter/sort logic that’s not in the JSON (e.g. “Hidden Gems”), pass cfg.catalogs with a transform function on each entry:
import { SimklTrendingClient, SimklTrendingRecipes as R } from './simkl-trending-client.js';
const client = new SimklTrendingClient({
supportedTypes: ['movies', 'tv', 'anime'],
size: 500, // recommended once you derive rows
catalogs: [
{ id: 'movies_today', source: 'movies' }, // trending (JSON title)
{ id: 'tv_today', source: 'tv' },
{ id: 'anime_today', source: 'anime' },
{ id: 'movies_anticipated', source: 'movies', title: 'Most Watchlisted Movies on Simkl', transform: R.mostWatchlisted },
{ id: 'hidden_gems_anime', source: 'anime', title: 'Hidden Gem Anime on Simkl', transform: items => R.hiddenGems(items, { minRank: 3000, source: 'mal' }) },
{ id: 'best_of_hbo', source: 'tv', title: 'Best of HBO — on Simkl', transform: items => R.bestOfNetwork(items, 'HBO') },
],
});
client.subscribeCatalogs(rows => { for (const r of rows) renderRow(r.title, r.items); });
await client.start();
client = SimklTrendingClient({
'supported_types': ['movies', 'tv', 'anime'],
'size': 500,
'catalogs': [
{'id': 'movies_today', 'source': 'movies'}, # trending (JSON title)
{'id': 'tv_today', 'source': 'tv'},
{'id': 'anime_today', 'source': 'anime'},
{'id': 'movies_anticipated', 'source': 'movies', 'title': 'Most Watchlisted Movies on Simkl',
'transform': lambda items: sorted(items, key=lambda i: -i.get('plan_to_watch', 0))},
{'id': 'hidden_gems_anime', 'source': 'anime', 'title': 'Hidden Gem Anime on Simkl',
'transform': lambda items: [i for i in items
if i.get('ratings',{}).get('simkl',{}).get('rating',0) >= 8.5
and i.get('watched',999) < 100]},
{'id': 'best_of_hbo', 'source': 'tv', 'title': 'Best of HBO — on Simkl',
'transform': lambda items: sorted([i for i in items if i.get('network') == 'HBO'
and i.get('status') == 'ended'
and i.get('ratings',{}).get('simkl',{}).get('rating',0) >= 8],
key=lambda i: -i['ratings']['simkl']['rating'])},
],
})
client.subscribe_catalogs(lambda rows: [renderRow(r['title'], r['items']) for r in rows])
client.start()
let cfg = SimklTrendingConfig(
supportedTypes: ["movies", "tv", "anime"],
size: 500,
catalogs: [
.init(id: "movies_today", source: "movies"), // trending (JSON title)
.init(id: "tv_today", source: "tv"),
.init(id: "anime_today", source: "anime"),
.init(id: "movies_anticipated", source: "movies", title: "Most Watchlisted Movies on Simkl",
transform: { items in items.sorted {
($0["plan_to_watch"] as? Int ?? 0) > ($1["plan_to_watch"] as? Int ?? 0)
} }),
.init(id: "hidden_gems_anime", source: "anime", title: "Hidden Gem Anime on Simkl",
transform: { items in items.filter {
let r = (($0["ratings"] as? [String: Any])?["simkl"] as? [String: Any])?["rating"] as? Double ?? 0
return r >= 8.5 && (($0["watched"] as? Int) ?? 999) < 100
} }),
.init(id: "best_of_hbo", source: "tv", title: "Best of HBO — on Simkl",
transform: { items in items.filter {
($0["network"] as? String) == "HBO" && ($0["status"] as? String) == "ended"
} }),
]
)
Task {
let client = SimklTrendingClient(config: cfg)
_ = await client.subscribeCatalogs { rows in
for r in rows { renderRow(r.title, r.items) }
}
await client.start()
}
val cfg = SimklTrendingConfig(
supportedTypes = listOf("movies", "tv", "anime"),
size = 500,
catalogs = listOf(
SimklTrendingCatalogConfig(id = "movies_today", source = "movies"), // trending (JSON title)
SimklTrendingCatalogConfig(id = "tv_today", source = "tv"),
SimklTrendingCatalogConfig(id = "anime_today", source = "anime"),
SimklTrendingCatalogConfig(id = "movies_anticipated", source = "movies", title = "Most Watchlisted Movies on Simkl",
transform = { items -> items.sortedByDescending { it.optInt("plan_to_watch") } }),
SimklTrendingCatalogConfig(id = "hidden_gems_anime", source = "anime", title = "Hidden Gem Anime on Simkl",
transform = { items -> items.filter {
it.optJSONObject("ratings")?.optJSONObject("simkl")?.optDouble("rating", 0.0) ?: 0.0 >= 8.5
&& it.optInt("watched", 999) < 100
} }),
SimklTrendingCatalogConfig(id = "best_of_hbo", source = "tv", title = "Best of HBO — on Simkl",
transform = { items -> items.filter {
it.optString("network") == "HBO" && it.optString("status") == "ended"
} }),
),
)
runBlocking {
val client = SimklTrendingClient(cfg)
client.subscribeCatalogs { rows -> for (r in rows) renderRow(r.title, r.items) }
client.start()
}
final cfg = SimklTrendingConfig(
supportedTypes: ['movies', 'tv', 'anime'],
size: 500,
catalogs: [
SimklTrendingCatalogConfig(id: 'movies_today', source: 'movies'), // trending (JSON title)
SimklTrendingCatalogConfig(id: 'tv_today', source: 'tv'),
SimklTrendingCatalogConfig(id: 'anime_today', source: 'anime'),
SimklTrendingCatalogConfig(id: 'movies_anticipated', source: 'movies', title: 'Most Watchlisted Movies on Simkl',
transform: (items) => [...items]..sort((a, b) => (b['plan_to_watch'] ?? 0).compareTo(a['plan_to_watch'] ?? 0))),
SimklTrendingCatalogConfig(id: 'hidden_gems_anime', source: 'anime', title: 'Hidden Gem Anime on Simkl',
transform: (items) => items.where((i) =>
((i['ratings']?['simkl']?['rating'] as num?) ?? 0) >= 8.5 && (i['watched'] as int? ?? 999) < 100).toList()),
SimklTrendingCatalogConfig(id: 'best_of_hbo', source: 'tv', title: 'Best of HBO — on Simkl',
transform: (items) => items.where((i) => i['network'] == 'HBO' && i['status'] == 'ended').toList()),
],
);
final client = SimklTrendingClient(config: cfg);
client.subscribeCatalogs((rows) { for (final r in rows) renderRow(r.title, r.items); });
await client.start();
client.catalogs() returns the resolved list on demand. Same {id, title, items} shape in every language.
Available recipe transforms (drop into transform: to derive a row). Each has tunable opts as the 2nd arg — hiddenGems(items, { minRating, minVotes, minRank, source }), etc. Dial them to your taste without forking.
| Group | Recipe names |
|---|---|
| Sorts | mostWatchlisted, highestRated, stickiest, newest, bestBoxOffice |
| Filters | byGenre, byGenreAll, byCountry, byNetwork, byStatus, byAnimeType, withTrailer, quickWatches, marathonWorthy, miniSeries, justReleased, inTheatersNow, recentlyOnDvd, ongoing, recentlyEnded, justPremiered, byYear, byDecade, currentYear, lastYear |
| Composed | hiddenGems, criticallyAcclaimed, crossPlatformConsensus, massiveAnticipation, bestOfNetwork, bestOfGenre, bestOfDecade |
| Groupings | groupByGenre, groupByCountry, groupByNetwork, groupByDecade (these return an object of buckets — convert to rows with Object.entries(R.groupByGenre(items)).map(([genre, items]) => ({ id: 'genre_' + genre, title: 'Trending ' + genre + ' on Simkl', items }))) |
Use
size: 500 once you render derived rows. The top-100 file gives ~10-30 items per derived row at best (e.g. hiddenGems off today_100 movies typically yields 10-15 candidates). The top-500 file gives ~50-200 per row — enough to scroll horizontally. Same fetch schedule, ~3-5x larger file.Adding / reordering rows costs nothing. Catalog rows are resolved at read time off the cached items. Changing
cfg.catalogs and recomputing produces a new row list in microseconds. The SDK refreshes the underlying cache on its own cadence (~1h for today, 1d for week/month), independent of how many rows you render.hiddenGems 67 movies / 156 TV / 204 anime · criticallyAcclaimed (default 8.5+ rating, 1k+ votes) 15 movies · marathonWorthy 75 TV · crossPlatformConsensus 66 movies · bestBoxOffice 449 movies parseable · quickWatches 38 movies under 90 min · byNetwork(tv, 'Netflix') 64 · byCountry(anime, 'jp') 361. Defaults are intentionally conservative — relax them if you want fewer empty rows.
Catalog ideas — recipe logic at a glance
TheSimklTrendingRecipes namespace is shipped Node-only. If you’re porting to another language (the SDK class scaffolding above already covers Python / Swift / Kotlin / Dart), here’s the logic for each recipe — drop it into your transform: / client.get(type) pipeline and you have the row.
| Catalog row | Logic |
|---|---|
| Most Watchlisted | sort plan_to_watch desc — items that appear on the most user watchlists. The name “Most Anticipated” misled here: this fires for already-released titles too (huge backlog appeal — Game of Thrones, Avatar, One Piece all rank high). Use this for the universal “what’s on people’s minds” row. |
| Highest Rated (Simkl) | sort ratings.simkl.rating desc, gate on ratings.simkl.votes >= 500 to drop noise |
| Highest Rated (IMDb / MAL) | same, on ratings.imdb.rating (movies/TV) or ratings.mal.rating (anime) |
| Stickiest | sort parseFloat(drop_rate) asc — viewers who started, finished, and didn’t bail |
| Newest releases | sort by parsed release_date desc — release_date is MM/DD/YYYY (e.g. 03/26/2026), which new Date(...) / Python datetime.strptime("%m/%d/%Y") / Swift DateFormatter("MM/dd/yyyy") all accept |
| Box-office hits (movies) | extract $NNNM from metadata, sort desc |
| Quick watches (movies) | parse runtime to minutes, filter ≤ 90, sort by rating |
| Marathon-worthy (TV/anime) | filter status === 'ended' AND 20 ≤ total_episodes ≤ 100, sort by rating |
| Mini-series (TV/anime) | filter total_episodes ≤ 6 |
| By genre / country / network / status / anime type | items.filter(...) on the matching field |
| In theaters now (movies) | filter theater date within last 30 days |
| Just released on DVD (movies) | filter dvd_date within last 14-30 days |
| Just premiered (TV/anime/movies) | filter status === 'premiere' — flags items whose first episode JUST aired (typically within ~30 days). Not an “upcoming premieres” filter; the data has no field for that. |
| By year (e.g. Movies of 2026) | parse the 4-digit year out of release_date (any format), filter equality |
| By decade (e.g. Best of the 90s) | parse year, filter decade ≤ y < decade + 10 |
| Current year / last year | same as By year, comparing against new Date().getFullYear() (or your language’s equivalent) — re-evaluated per call so the row stays current without re-shipping the JSON |
| Best of decade | combine By decade with bestOfNetwork’s shape — rating-sort + votes >= 100 floor inside the decade window |
| Has trailer | items.filter(i => i.trailer) — for trailer-first carousel UIs |
| Hidden Gems | filter rank > 2000 AND rating >= 7.5 AND votes >= 500, then sort by rating desc — well-loved titles OUTSIDE Simkl’s all-time top 2000 by popularity. Don’t filter by watched on trending feeds: that field is last-24h viewers, not lifetime, so a 150-cap surfaces mainstream titles like The Office or Interstellar. Rank is the right popularity gate; mainstream lives in rank 50-500, gems live in 2000+. Tune minRank up (3000, 5000) for stricter obscurity. |
| Critically acclaimed, widely seen | rating >= 8.5 AND votes >= 1000 — high bar on both axes |
| Massive anticipation | plan_to_watch / (watched + 1) >= 300 — pre-release / newly-announced (median ratio is ~200; 300 is upper quartile) |
| Cross-platform consensus | Simkl rating ≥ 8 AND IMDb (or MAL) rating ≥ 8 — both crowds agree |
| Best of | filter network match + status === 'ended', sort by rating |
| Grouped by genre / country / network / decade | bucket items into { "Action": [...], "Drama": [...] } for N rows from one file |
Tuning a recipe per catalog row. Every recipe accepts an options object as its second arg. In the JSON, set them via Same pattern works for any recipe —
recipe_opts. For example, to make tv_hidden_gems stricter — only titles outside Simkl’s top 3000 with rating ≥ 8 instead of the default 2000/7.5 — override the knobs:{
"id": "tv_hidden_gems",
"source": "tv",
"title": "Hidden Gem TV on Simkl",
"recipe": "hiddenGems",
"recipe_opts": { "minRank": 3000, "minRating": 8 }
}
bestOfDecade with { "decade": 1980 }, bestOfNetwork with { "network": "AMC" }, highestRated with { "minVotes": 2000 }, etc. Check today’s live data before bumping thresholds aggressively though; over-strict opts can produce empty rows on smaller trending feeds.Attribution still applies to derived rows. A row labeled “Most Watchlisted Movies” or “Hidden Gems” still needs to include “Simkl” or “Most Watched” to satisfy the attribution requirement — e.g. “Most Watchlisted on Simkl”, “Simkl Hidden Gems”, “Trending in Japan — Powered by Simkl”.
What’s in each item
Per-title arrays (movies, tv, anime) all share the same base shape, with a few type-specific fields.
Sample item
{
"title": "The Drama",
"alt_titles": [
{ "name": "La Comedia", "lang": 8, "type": "official" },
{ "name": "Дорама", "lang": 22, "type": "official" }
],
"url": "/movie/2533163/the-drama",
"poster": "19/19702715dfee84bd7c",
"fanart": "19/19840547ea18093f08",
"ids": {
"simkl_id": 2533163,
"slug": "the-drama",
"imdb": "tt33071426",
"tmdb": "1148540"
},
"release_date": "03/26/2026",
"rank": 3311,
"drop_rate": "1.6%",
"watched": 96,
"plan_to_watch": 820,
"ratings": {
"simkl": { "rating": 7.37, "votes": 136 },
"imdb": { "rating": 7.4, "votes": 42928 }
},
"country": "us",
"original_language": "en",
"runtime": "1h 45m",
"status": "ended",
"genres": ["Action", "Comedy", "Drama", "Romance"],
"trailer": "0ZDzsH3XGFA",
"overview": "A happily engaged couple is put to the test...",
"metadata": "March 26, 2026 • Budget $28M • Box office $121M",
"dvd_date": "05/05/2026",
"theater": "03/26/2026"
}
Field reference
string
Display title.
object[]
Localized and alternate titles. Always present on every catalog, but often an empty array — check length before reading. Popular titles can carry dozens of entries (60+ on the most widely-released films), so slice before rendering.Each entry has three keys:
name(string) — the alternate title.type(string) —officialfor a localized release title,originalfor the title in the production’s own language.lang(integer) — Simkl-internal language code. This is not an ISO 639 code and there is currently no public mapping table, so treat it as an opaque grouping key rather than something to render.
title and ignore this field entirely.string
Path on simkl.com (e.g.
/movie/2533163/the-drama). Prepend https://simkl.com to deep-link.string
Image path fragment. Combine with the prefixes in Image conventions — for example
https://wsrv.nl/?url=https://simkl.in/posters/{poster}_m.webp&q=90. When null, fall back to https://simkl.in/poster_no_pic.png (see fallbacks).string
Image path fragment for fanart. Same prefixing rules as
poster. When null, hide the fanart element — there’s no dedicated fanart placeholder.object
External and Simkl IDs. Always carries
simkl_id + slug. tmdb is near-universal; imdb appears on TV / movies, mal / anidb / anilist / kitsu on anime. Additional third-party slug and ID variants (letterslug, traktmslug, tvdbslug, trakttvslug, mdlslug, jwtv, …) may appear on items with those platform links — the response is permissive, so read the keys you need and ignore the rest.integer
Simkl service rank for this media type.
integer
Number of users who watched this title in the timeframe.
integer
Number of users with this title on their Plan-to-Watch watchlist.
string
Percentage of users who started and dropped, formatted as a string (e.g.
"1.6%").object
Aggregate ratings — Simkl and any external sources available for this title (IMDB for movies/TV, MAL for anime, etc.). Each entry is
{ rating: number, votes: number }.string
Original release date in
MM/DD/YYYY format.string
ISO-style country code (e.g.
us, jp). Nullable on anime (~28% of the anime feed); always populated on movies, TV, and DVD.string | null
Lowercase two-letter language code for the production’s original audio (
en, ja, zh, ko). Nullable on anime (~33% of the anime feed); always populated on movies, TV, and DVD. Distinct from country, which is the country of origin — a Japanese-language title can be country: "us" on a co-production.string
Human-readable duration (e.g.
1h 45m, 25m).string
One of
ended (production wrapped), ongoing (TV/anime still airing new episodes), or premiere (very-recently-released, still in the rollout window). The wider Simkl catalog also uses tba for not-yet-dated titles, but those don’t surface in trending.string[]
Array of genre tags.
string
YouTube video ID.
string
Synopsis.
string
Pre-formatted human-readable summary line (release year, budget, box office, network, etc.).
string
Movies only. DVD release date (
MM/DD/YYYY).string
Movies only. Theatrical release date (
MM/DD/YYYY).string | null
Anime only. Romanized Japanese title (e.g.
Mushoku Tensei III: Isekai Ittara Honki Dasu), for apps that display romaji alongside or instead of the English title. null on roughly a quarter of the anime feed — fall back to title.string
Anime only. One of
tv, movie, special, ova, ona, music video.integer
TV / anime. Episode count.
string
TV / anime. Broadcasting network.
Without the SDK — raw URL and curl
TheSimklTrendingClient above is the recommended way to consume trending data — it handles caching, conditional revalidation, retry, dedup, persistence, and the auto-pick combined endpoint logic. The patterns below are the bare-bones alternative for ad-hoc curl probes, server-side cron jobs that handle their own caching, or runtimes where you don’t want to drop a class file.
URL pattern and quick curl / JS / Python / Go examples
URL pattern and quick curl / JS / Python / Go examples
URL pattern
If you’d rather construct the URL in code:https://data.simkl.in/discover/trending/[<category>/]<timeframe>_<size>.json
<category>— omit for Combined (movies + tv + anime in one response); usemovies,tv, oranimefor type-specific.<timeframe>—today,week, ormonth.<size>—100or500.
https://data.simkl.in/discover/dvd/releases_<size>.json
Fetch it
curl -H 'User-Agent: my-app-name/1.0' \
"https://data.simkl.in/discover/trending/today_100.json?client_id=YOUR_CLIENT_ID&app-name=my-app-name&app-version=1.0"
const params = new URLSearchParams({
client_id: 'YOUR_CLIENT_ID',
'app-name': 'my-app-name',
'app-version': '1.0',
});
const r = await fetch(
`https://data.simkl.in/discover/trending/movies/today_100.json?${params}`,
{ headers: { 'User-Agent': 'my-app-name/1.0' } },
);
const items = await r.json();
console.log(items[0].title, items[0].watched);
import requests
r = requests.get(
'https://data.simkl.in/discover/trending/movies/today_100.json',
params={
'client_id': 'YOUR_CLIENT_ID',
'app-name': 'my-app-name',
'app-version': '1.0',
},
headers={'User-Agent': 'my-app-name/1.0'},
)
items = r.json()
print(items[0]['title'], items[0]['watched'])
package main
import (
"encoding/json"
"fmt"
"net/http"
)
type Item struct {
Title string `json:"title"`
Rank int `json:"rank"`
Watched int `json:"watched"`
}
func main() {
req, _ := http.NewRequest("GET",
"https://data.simkl.in/discover/trending/movies/today_100.json", nil)
q := req.URL.Query()
q.Set("client_id", "YOUR_CLIENT_ID")
q.Set("app-name", "my-app-name")
q.Set("app-version", "1.0")
req.URL.RawQuery = q.Encode()
req.Header.Set("User-Agent", "my-app-name/1.0")
resp, _ := http.DefaultClient.Do(req)
defer resp.Body.Close()
var items []Item
json.NewDecoder(resp.Body).Decode(&items)
fmt.Println(items[0].Title, items[0].Watched)
}
Always send a
User-Agent with your app name and version (e.g. myapp/1.0) to avoid accidental blocking.