Pre-built JSON for Simkl’s Most Watched lists — Today, Week, Month — for Movies, TV, and Anime, plus this month’s most watched in every genre. No API key required.
No auth required. Trending data is public — send the standard required URL parameters (client_id, app-name, app-version) and a User-Agent header, but no user Authorization token.Free to use. These files don’t count against any rate limit — not your app’s, and not a user’s AUTH V2 daily allowance. Wherever they cover what you show, use them instead of live API calls such as the /genres endpoints, which do count.
Which IDs can I send/expect? All accepted input identifiers and the keys you’ll see echoed back in responses are listed at Standard media objects → Supported ID keys. Send every ID you have on writes — Simkl picks the first that resolves and ignores the rest. Reminder: slug is response-only (never send it on a request).
Attribution required. When you display trending data in your app or website, the section title must include Simkl alongside Trending (or an equivalent like Most Watched / Popular). Any sensible combination works — feel free to invent your own wording, as long as both ideas appear together. A few examples to get you started:
Simkl Trending Movies
Trending Movies on Simkl
What's Trending on Simkl
Simkl Trending TV Shows
Trending TV on Simkl
Now Trending — Simkl
Simkl Trending Anime
Trending Anime on Simkl
Trending Now · Powered by Simkl
Simkl Trending Today
Trending Today on Simkl
Today on Simkl
Simkl Trending This Week
Trending This Week on Simkl
This Week's Top Picks — Simkl
Simkl Most Watched
Most Watched on Simkl
Popular on Simkl
Simkl Top 100 Movies
Top 100 on Simkl
Hot Right Now — Simkl
Simkl Top Charts
Top Charts on Simkl
Charting on Simkl
Simkl Trending Horror Movies
Trending Horror Movies on Simkl
Most Watched Isekai Anime on Simkl
For commercial use without attribution, contact us — we’re happy to discuss licensing.
Linking back (websites only)
If your client can render hyperlinks — websites, browser extensions, web apps — link the title to the matching Simkl Most Watched page. TV apps, consoles, CLIs, and other contexts that can’t open external URLs are exempt; the title alone is enough.
Most Watched Movies
simkl.com/movies/best-movies/most-watched
Most Watched TV
simkl.com/tv/best-shows/most-watched
Most Watched Anime
simkl.com/anime/best-anime/most-watched
A row built from a genre file links to the same page for its type.Simkl provides pre-built JSON files with trending data ranked by the number of watchers. These are the same rankings displayed on Simkl’s Most Watched pages for Movies, TV Shows, and Anime.Each file is available in two sizes: top 100 (_100) or top 500 (_500) items. Titles with the most watchers are returned first.
At a glance
Sizes
Top 100 (_100) and Top 500 (_500) per file.
By genre
This month’s most watched in every genre — one genre file each, month only, up to 500 items.
Last-Modified
Each file’s Last-Modified response header tells you exactly when it was generated.
Update frequency
Data
Description
Update frequency
Today
Most watched titles over the last 24 hours
Every hour
Week
Most watched titles over the last 7 days
Once a day
Month
Most watched titles over the last 30 days
Once a day
Genre
Most watched titles in one genre over the last 30 days
Once a day
DVD releases
Latest popular DVD releases (Movies only)
Once a day
Regeneration is best-effort, not strictly precise. The cadences above are targets — the actual job can take a few extra minutes (and occasionally longer) to finish. If your scheduled refresh hits a 304 Not Modified right at the expected time, the new file just hasn’t reached you yet; back off and retry in a few minutes rather than waiting another full cycle. See SimklTrendingClient — drop-in SDK for the recommended pattern.The CDN adds up to an hour. Cloudflare keeps each file for up to 1 hour (Cache-Control: max-age=3600) and is not cleared when a file is regenerated, so a new version can take up to an hour to replace the old one. An hourly today file can therefore be up to about two hours old when you read it. Last-Modified always tells you which version you have.
The file URLs ignore all query strings. Don’t add ?random=... or ?nocache=... — the CDN treats every variant as the same resource, so client-side cache-busting won’t deliver newer data. It only ever returns the copy Cloudflare is holding.
All trending data files
Append ?client_id=YOUR_CLIENT_ID&app-name=my-app-name&app-version=1.0 to every URL below — the SimklTrendingClient does this for you from CONFIG; if you’re rolling your own, see Without the SDK.
Combined
Movies + TV + anime in a single response. Top-level keys: movies, tv, anime. Best when you want all categories at once and don’t want three round-trips.
One category per file. Top-level response is the array directly (no wrapper object). Best when you only render one category, or want to fetch them independently.
This month’s most watched titles in every genre — one file per genre, per type. Same CDN, same required parameters and User-Agent, same caching and attribution rules as every other file on this page, and just as free: no user token, and no rate limit or AUTH V2 allowance spent. For this month’s most watched in a genre, use these instead of the /genres endpoints, where every call counts.Start from the genre list. Each type has an index.json listing the genres that have a file, sorted A–Z by name:
<type> is movies, tv or anime; <slug> comes from that type’s index.json.
Read the slugs from index.json — don’t hardcode them or build them from genre names. The list can change. The same idea can also have a different slug per type: movies and TV use science-fiction, anime uses sci-fi. Show name from index.json as the label.
The slugs are the same as the {genre} values of GET /movies/genres, GET /tv/genres and GET /anime/genres, with two exceptions: TV special-interest has no file, and anime hentai has a file but is not in the /anime/genres list.
The genre lists include adult genres: movies erotica, TV erotica and anime hentai. General-audience apps, and any app distributed through an app store, should skip these slugs.
How genre files differ from the files above:
This month only. No today or week, no _100 size, and no combined file.
_500 means up to 500. Small genres hold fewer — some under 10, such as TV video-game-play and anime yaoi. count in index.json tells you how many before you fetch.
Sorted by watched — this month’s watchers — highest first. Every item has at least 2 watchers.
Same items. Each item has exactly the fields of an item in <type>/month_500.json, so What’s in each item applies as-is.
Each genre file is its own ranking, not a slice of the main file. Filtering the main 500 file by genre — including the SDK’s byGenre and bestOfGenre recipes — only finds titles that made the overall top 500.
Don’t re-check membership against an item’s genres array. In rare cases the file’s genre is missing from it: TV korean-drama items don’t list “Korean Drama”. The file you fetched is the membership.
Updated once a day, in the morning US Eastern time. Treat it like the other daily files: refresh_seconds: 86400, with the same back-off when a refresh returns 304 (Update frequency).
Fetch one genre — read the list, pick a slug, fetch that file:
# 1. The genre list for one typecurl -H 'User-Agent: my-app-name/1.0' \ "https://data.simkl.in/discover/trending/anime/genre/index.json?client_id=YOUR_CLIENT_ID&app-name=my-app-name&app-version=1.0"# 2. One genre, using a slug from that listcurl -H 'User-Agent: my-app-name/1.0' \ "https://data.simkl.in/discover/trending/anime/genre/isekai_month_500.json?client_id=YOUR_CLIENT_ID&app-name=my-app-name&app-version=1.0"
Drop this object into your app as a constant. Replace YOUR_CLIENT_ID with your registered app’s client_id and my-app-name / 1.0 with your app’s name and version — both the URL params (required_query) AND the matching User-Agent header (user_agent_header) are required on every request, free or paid plan.Each entry has a title (attribution-compliant display label), a url (bare path — append ? + required_query when you fetch), and a refresh_seconds (how often the file is regenerated server-side — set your refresh timer to this value). The structure is category → timeframe → size so you can filter or iterate by any axis. DVD has no timeframe.
{ "required_query": "client_id=YOUR_CLIENT_ID&app-name=my-app-name&app-version=1.0", "user_agent_header": "my-app-name/1.0", "urls": { "combined": { "today": { "top_100": { "title": "Trending on Simkl — Today (Top 100)", "url": "https://data.simkl.in/discover/trending/today_100.json", "refresh_seconds": 3600 }, "top_500": { "title": "Trending on Simkl — Today (Top 500)", "url": "https://data.simkl.in/discover/trending/today_500.json", "refresh_seconds": 3600 } }, "week": { "top_100": { "title": "Trending on Simkl — This Week (Top 100)", "url": "https://data.simkl.in/discover/trending/week_100.json", "refresh_seconds": 86400 }, "top_500": { "title": "Trending on Simkl — This Week (Top 500)", "url": "https://data.simkl.in/discover/trending/week_500.json", "refresh_seconds": 86400 } }, "month": { "top_100": { "title": "Trending on Simkl — This Month (Top 100)", "url": "https://data.simkl.in/discover/trending/month_100.json", "refresh_seconds": 86400 }, "top_500": { "title": "Trending on Simkl — This Month (Top 500)", "url": "https://data.simkl.in/discover/trending/month_500.json", "refresh_seconds": 86400 } } }, "movies": { "today": { "top_100": { "title": "Trending Movies on Simkl — Today (Top 100)", "url": "https://data.simkl.in/discover/trending/movies/today_100.json", "refresh_seconds": 3600 }, "top_500": { "title": "Trending Movies on Simkl — Today (Top 500)", "url": "https://data.simkl.in/discover/trending/movies/today_500.json", "refresh_seconds": 3600 } }, "week": { "top_100": { "title": "Trending Movies on Simkl — This Week (Top 100)", "url": "https://data.simkl.in/discover/trending/movies/week_100.json", "refresh_seconds": 86400 }, "top_500": { "title": "Trending Movies on Simkl — This Week (Top 500)", "url": "https://data.simkl.in/discover/trending/movies/week_500.json", "refresh_seconds": 86400 } }, "month": { "top_100": { "title": "Trending Movies on Simkl — This Month (Top 100)", "url": "https://data.simkl.in/discover/trending/movies/month_100.json", "refresh_seconds": 86400 }, "top_500": { "title": "Trending Movies on Simkl — This Month (Top 500)", "url": "https://data.simkl.in/discover/trending/movies/month_500.json", "refresh_seconds": 86400 } } }, "tv": { "today": { "top_100": { "title": "Trending TV Shows on Simkl — Today (Top 100)", "url": "https://data.simkl.in/discover/trending/tv/today_100.json", "refresh_seconds": 3600 }, "top_500": { "title": "Trending TV Shows on Simkl — Today (Top 500)", "url": "https://data.simkl.in/discover/trending/tv/today_500.json", "refresh_seconds": 3600 } }, "week": { "top_100": { "title": "Trending TV Shows on Simkl — This Week (Top 100)", "url": "https://data.simkl.in/discover/trending/tv/week_100.json", "refresh_seconds": 86400 }, "top_500": { "title": "Trending TV Shows on Simkl — This Week (Top 500)", "url": "https://data.simkl.in/discover/trending/tv/week_500.json", "refresh_seconds": 86400 } }, "month": { "top_100": { "title": "Trending TV Shows on Simkl — This Month (Top 100)", "url": "https://data.simkl.in/discover/trending/tv/month_100.json", "refresh_seconds": 86400 }, "top_500": { "title": "Trending TV Shows on Simkl — This Month (Top 500)", "url": "https://data.simkl.in/discover/trending/tv/month_500.json", "refresh_seconds": 86400 } } }, "anime": { "today": { "top_100": { "title": "Trending Anime on Simkl — Today (Top 100)", "url": "https://data.simkl.in/discover/trending/anime/today_100.json", "refresh_seconds": 3600 }, "top_500": { "title": "Trending Anime on Simkl — Today (Top 500)", "url": "https://data.simkl.in/discover/trending/anime/today_500.json", "refresh_seconds": 3600 } }, "week": { "top_100": { "title": "Trending Anime on Simkl — This Week (Top 100)", "url": "https://data.simkl.in/discover/trending/anime/week_100.json", "refresh_seconds": 86400 }, "top_500": { "title": "Trending Anime on Simkl — This Week (Top 500)", "url": "https://data.simkl.in/discover/trending/anime/week_500.json", "refresh_seconds": 86400 } }, "month": { "top_100": { "title": "Trending Anime on Simkl — This Month (Top 100)", "url": "https://data.simkl.in/discover/trending/anime/month_100.json", "refresh_seconds": 86400 }, "top_500": { "title": "Trending Anime on Simkl — This Month (Top 500)", "url": "https://data.simkl.in/discover/trending/anime/month_500.json", "refresh_seconds": 86400 } } }, "dvd": { "top_100": { "title": "Latest DVD Releases on Simkl (Top 100)", "url": "https://data.simkl.in/discover/dvd/releases_100.json", "refresh_seconds": 86400 }, "top_500": { "title": "Latest DVD Releases on Simkl (Top 500)", "url": "https://data.simkl.in/discover/dvd/releases_500.json", "refresh_seconds": 86400 } } }, "catalogs": [ { "id": "movies_today", "source": "movies", "timeframe": "today" }, { "id": "tv_today", "source": "tv", "timeframe": "today" }, { "id": "anime_today", "source": "anime", "timeframe": "today" }, { "id": "dvd", "source": "dvd" }, { "id": "tv_premieres", "source": "tv", "title": "Trending TV Premieres on Simkl (Last 30 Days)", "recipe": "justReleased", "recipe_opts": { "withinDays": 30 } }, { "id": "movies_premieres", "source": "movies", "title": "Trending Movie Premieres on Simkl (Last 30 Days)", "recipe": "justReleased", "recipe_opts": { "withinDays": 30 } }, { "id": "tv_ongoing", "source": "tv", "title": "Currently Airing TV on Simkl", "recipe": "ongoing" }, { "id": "movies_this_year", "source": "movies", "title": "Movies This Year on Simkl", "recipe": "currentYear" }, { "id": "tv_this_year", "source": "tv", "title": "TV This Year on Simkl", "recipe": "currentYear" }, { "id": "anime_this_year", "source": "anime", "title": "Anime This Year on Simkl", "recipe": "currentYear" }, { "id": "movies_last_year", "source": "movies", "title": "Movies of Last Year on Simkl", "recipe": "lastYear" }, { "id": "tv_last_year", "source": "tv", "title": "TV of Last Year on Simkl", "recipe": "lastYear" }, { "id": "movies_anticipated", "source": "movies", "title": "Most Watchlisted Movies on Simkl", "recipe": "mostWatchlisted" }, { "id": "tv_anticipated", "source": "tv", "title": "Most Watchlisted TV on Simkl", "recipe": "mostWatchlisted" }, { "id": "anime_anticipated", "source": "anime", "title": "Most Watchlisted Anime on Simkl", "recipe": "mostWatchlisted" }, { "id": "movies_top_rated", "source": "movies", "title": "Top Rated Movies on Simkl", "recipe": "highestRated" }, { "id": "tv_top_rated", "source": "tv", "title": "Top Rated TV on Simkl", "recipe": "highestRated" }, { "id": "anime_top_rated", "source": "anime", "title": "Top Rated Anime on Simkl", "recipe": "highestRated", "recipe_opts": { "source": "mal" } }, { "id": "movies_by_rank", "source": "movies", "title": "All-Time Best Movies on Simkl", "recipe": "byRank" }, { "id": "tv_by_rank", "source": "tv", "title": "All-Time Best TV on Simkl", "recipe": "byRank" }, { "id": "anime_by_rank", "source": "anime", "title": "All-Time Best Anime on Simkl", "recipe": "byRank" }, { "id": "movies_box_office", "source": "movies", "title": "Top Box Office Movies on Simkl", "recipe": "bestBoxOffice" }, { "id": "movies_well_rated", "source": "movies", "title": "Well-rated movies on Simkl", "recipe": "wellRated" }, { "id": "tv_well_rated", "source": "tv", "title": "Well-rated TV on Simkl", "recipe": "wellRated" }, { "id": "anime_well_rated", "source": "anime", "title": "Well-rated anime on Simkl", "recipe": "wellRated", "recipe_opts": { "source": "mal" } }, { "id": "movies_underseen", "source": "movies", "title": "Underseen movies on Simkl", "recipe": "underseenGems" }, { "id": "tv_underseen", "source": "tv", "title": "Underseen TV on Simkl", "recipe": "underseenGems" }, { "id": "anime_underseen", "source": "anime", "title": "Underseen anime on Simkl", "recipe": "underseenGems" }, { "id": "tv_marathon", "source": "tv", "title": "Marathon-Worthy TV on Simkl", "recipe": "marathonWorthy" }, { "id": "movies_quick", "source": "movies", "title": "Quick Watches on Simkl (≤ 90 min)", "recipe": "quickWatches" }, { "id": "tv_best_netflix", "source": "tv", "title": "Best of Netflix on Simkl", "recipe": "bestOfNetwork", "recipe_opts": { "network": "Netflix" } }, { "id": "tv_best_hbo", "source": "tv", "title": "Best of HBO on Simkl", "recipe": "bestOfNetwork", "recipe_opts": { "network": "HBO" } }, { "id": "tv_best_disney", "source": "tv", "title": "Best of Disney+ on Simkl", "recipe": "bestOfNetwork", "recipe_opts": { "network": "Disney+" } }, { "id": "tv_best_prime", "source": "tv", "title": "Best of Prime Video on Simkl", "recipe": "bestOfNetwork", "recipe_opts": { "network": "Prime Video" } }, { "id": "tv_best_apple", "source": "tv", "title": "Best of Apple TV on Simkl", "recipe": "bestOfNetwork", "recipe_opts": { "network": "Apple TV" } }, { "id": "movies_best_action", "source": "movies", "title": "Best Action Movies on Simkl", "recipe": "bestOfGenre", "recipe_opts": { "genre": "Action" } }, { "id": "movies_best_drama", "source": "movies", "title": "Best Drama Movies on Simkl", "recipe": "bestOfGenre", "recipe_opts": { "genre": "Drama" } }, { "id": "movies_best_comedy", "source": "movies", "title": "Best Comedy Movies on Simkl", "recipe": "bestOfGenre", "recipe_opts": { "genre": "Comedy" } }, { "id": "movies_best_scifi", "source": "movies", "title": "Best Sci-Fi Movies on Simkl", "recipe": "bestOfGenre", "recipe_opts": { "genre": "Science Fiction" } }, { "id": "movies_best_thriller", "source": "movies", "title": "Best Thriller Movies on Simkl", "recipe": "bestOfGenre", "recipe_opts": { "genre": "Thriller" } }, { "id": "movies_best_horror", "source": "movies", "title": "Best Horror Movies on Simkl", "recipe": "bestOfGenre", "recipe_opts": { "genre": "Horror" } }, { "id": "tv_best_drama", "source": "tv", "title": "Best Drama TV on Simkl", "recipe": "bestOfGenre", "recipe_opts": { "genre": "Drama" } }, { "id": "tv_best_comedy", "source": "tv", "title": "Best Comedy TV on Simkl", "recipe": "bestOfGenre", "recipe_opts": { "genre": "Comedy" } }, { "id": "tv_best_crime", "source": "tv", "title": "Best Crime TV on Simkl", "recipe": "bestOfGenre", "recipe_opts": { "genre": "Crime" } }, { "id": "tv_best_scifi", "source": "tv", "title": "Best Sci-Fi TV on Simkl", "recipe": "bestOfGenre", "recipe_opts": { "genre": "Science-Fiction" } }, { "id": "anime_best_shounen", "source": "anime", "title": "Best Shounen Anime on Simkl", "recipe": "bestOfGenre", "recipe_opts": { "genre": "Shounen", "source": "mal" } }, { "id": "anime_best_isekai", "source": "anime", "title": "Best Isekai Anime on Simkl", "recipe": "bestOfGenre", "recipe_opts": { "genre": "Isekai", "source": "mal" } }, { "id": "anime_best_romance", "source": "anime", "title": "Best Romance Anime on Simkl", "recipe": "bestOfGenre", "recipe_opts": { "genre": "Romance", "source": "mal" } }, { "id": "movies_best_2020s", "source": "movies", "title": "Best Movies of the 2020s on Simkl", "recipe": "bestOfDecade", "recipe_opts": { "decade": 2020 } }, { "id": "movies_best_2010s", "source": "movies", "title": "Best Movies of the 2010s on Simkl", "recipe": "bestOfDecade", "recipe_opts": { "decade": 2010 } }, { "id": "movies_best_2000s", "source": "movies", "title": "Best Movies of the 2000s on Simkl", "recipe": "bestOfDecade", "recipe_opts": { "decade": 2000 } }, { "id": "movies_best_90s", "source": "movies", "title": "Best Movies of the 90s on Simkl", "recipe": "bestOfDecade", "recipe_opts": { "decade": 1990 } }, { "id": "tv_best_2020s", "source": "tv", "title": "Best TV of the 2020s on Simkl", "recipe": "bestOfDecade", "recipe_opts": { "decade": 2020 } }, { "id": "tv_best_2010s", "source": "tv", "title": "Best TV of the 2010s on Simkl", "recipe": "bestOfDecade", "recipe_opts": { "decade": 2010 } }, { "id": "anime_best_2020s", "source": "anime", "title": "Best Anime of the 2020s on Simkl", "recipe": "bestOfDecade", "recipe_opts": { "decade": 2020, "source": "mal" } }, { "id": "anime_best_2010s", "source": "anime", "title": "Best Anime of the 2010s on Simkl", "recipe": "bestOfDecade", "recipe_opts": { "decade": 2010, "source": "mal" } }, { "id": "anime_movies", "source": "anime", "title": "Trending Anime Movies on Simkl", "recipe": "byAnimeType", "recipe_opts": { "animeType": "movie" } }, { "id": "anime_ovas", "source": "anime", "title": "Trending Anime OVAs on Simkl", "recipe": "byAnimeType", "recipe_opts": { "animeType": "ova" } } ]}
Titles are pre-composed to satisfy the attribution requirement (every label includes both “Trending” / “Most Watched” and “Simkl”). Use them as-is or adapt the wording — just keep both halves.
catalogs is the default row list the SDK renders on a home screen — 58 rows out of the box, all titled and ready to display. Mix of: trending (movies/tv/anime today + DVD), premieres and currently airing, this year / last year, sort variants (mostWatchlisted, highestRated, byRank × 3 types), well-rated and underseen rows (wellRated × 3, underseenGems × 3), bestBoxOffice, marathonWorthy, quickWatches, Best of (Netflix / HBO / Disney+ / Prime Video / Apple TV), Best of for movies (Action / Drama / Comedy / Sci-Fi / Thriller / Horror), TV (Drama / Comedy / Crime / Sci-Fi) and anime (Shounen / Isekai / Romance), Best of , and Anime Movies / Anime OVAs. Edit the JSON to remove rows you don’t want, change titles, or swap recipe opts. Per-app override still works via cfg.catalogs in the SDK config. Every recipe ships in the JavaScript tab below. Python / Swift / Kotlin / Dart ship wellRated and underseenGems; a row that names any other recipe comes back empty in those languages until you port it (see Catalog ideas) or give the row a transform — never the unfiltered feed under a filtered row’s title.
refresh_seconds is 3600 (1 hour) for today files and 86400 (1 day) for everything else, matching the Update frequency table. This is the nominal regeneration cadence, not a guarantee — server regeneration can take a few minutes longer. For bandwidth-optimal refresh, pair this with the response’s Last-Modified header and send If-Modified-Since on subsequent requests — the server returns 304 Not Modified when nothing has changed. If a refresh scheduled for refresh_seconds returns 304, back off and retry rather than assuming the cache is final (see SimklTrendingClient — drop-in SDK).
Using the SDK below? Save this whole object as simkl-trending-urls.json in your project — the SDK class loads it back into a SIMKL_TRENDING_URLS symbol. Each language tab’s top comment shows the runtime-appropriate one-liner.
SimklTrendingClient — drop-in SDK
Two-step setup before pasting the SDK class below. First, save the Copy-paste JSON block above as simkl-trending-urls.json in your project (or as an asset your runtime can load). Second, load it back into a SIMKL_TRENDING_URLS symbol — each tab’s top comment shows the runtime-appropriate one-liner (import / require / fetch for JS, open() for Python, Bundle.main for Swift, context.assets for Android, rootBundle for Flutter). The constructor throws if neither this symbol nor an explicit urls argument is in scope.
A single class that wires up the entire data flow. Tune the CONFIG block at the top of each tab — everything else is plumbing you should not need to edit. What’s built in:
Auto-picks the Combined endpoint when you ask for 2+ of movies/tv/anime — one HTTP call per timeframe instead of two or three.
Persistent cache keyed on the URL — relaunch can paint the UI instantly from the last persisted snapshot before any network call fires. JS auto-detects the host (browser → localStorage[storageKey], Node → file at persistPath); other languages use platform-native paths. Cache entries older than persistMaxAgeSeconds are dropped on restore. client.clearCache() wipes both in-memory and persistent state (and notifies subscribers so the UI repaints empty). For server-side runtimes with many processes, swap the _persist / _restorePersisted / clearCache trio for a Redis / KV adapter.
Conditional revalidation with If-Modified-Since so refreshes hit the CDN with a cheap 304 when nothing has changed.
Regen-lag backoff — when the server hasn’t regenerated by the time refresh_seconds elapses, the client backs off via a [5min, 15min, 30min] ladder instead of hammering on the nominal cadence.
Jitter — ±5% random jitter on every refresh + ±10% on retry delays, so all clients don’t hit the CDN at exactly :00 and don’t retry-storm in lockstep.
Retry with exponential backoff + per-request timeout + Retry-After honored for network blips and 429s.
Permanent-error classification — 404 / 400 / 401 / 403 are not retried (they’d waste the backoff schedule on a request that will never succeed); the entry’s loop halts and you see the error in the log.
Process-wide minIntervalMs floor — N entries refreshing at the same tick are serialized through a 250 ms (configurable) gate so you never burst N parallel requests at the CDN.
Concurrent fetch dedup — two callers asking for the same URL fold into one in-flight request.
Cancellation — client.stop() aborts every in-flight request and clears every timer.
Subscribe / get — your UI subscribes to a type and receives the current cached items immediately, plus a callback on every successful refresh. get(type) is synchronous — paginate the returned array locally for UI pages instead of refetching.
client.tick() for background schedulers — call from a BGTaskScheduler / WorkManager / workmanager / cron periodic task to perform a single revalidate-and-persist pass.
Namespace-safe — every public symbol is prefixed Simkl… (SimklTrendingClient, SimklTrendingConfig, SimklTrendingError, SIMKL_TRENDING_URLS, etc.) so dropping the class into an existing app doesn’t collide with your own Config / Client / Entry types.
The only thing left to wire up per-platform is background scheduling (the OS-level periodic task that keeps the cache warm while your app is suspended) — see the comment at the bottom of each tab.The SDK doesn’t load the genre files yet — they aren’t in the Copy-paste JSON or the catalogs, so fetch them directly for now.
Try it live
A self-contained vanilla-JS demo of the SDK runs in CodePen below, with two live fetches against data.simkl.in: trending/today_500.json (combined movies/TV/anime in one round-trip) and dvd/releases_500.json. Click Result to see the rendered home screen; JS / HTML / CSS tabs show the wiring.
Open & fork in CodePen
Edit the catalog config, try other recipes, swap the test client_id for your own.
<!-- Save this whole file as `index.html` next to two siblings: simkl-trending-urls.json ← the "Copy-paste JSON" block above simkl-trending-client.js ← the "JavaScript (save as simkl-trending-client.js)" tab → Serve over http:// (ES modules don't load over file://). For local dev: `python -m http.server`. No build step, no framework — what you see is what you get.--><!doctype html><html lang="en"><head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1"> <title>Simkl Trending</title> <style> :root { color-scheme: dark; } body { margin: 0; background: #0b0d12; color: #e6e8ee; font-family: system-ui, -apple-system, Segoe UI, Roboto, Helvetica, Arial, sans-serif; } header { padding: 22px 26px 14px; border-bottom: 1px solid #1d2330; } header h1 { margin: 0 0 4px; font-size: 18px; letter-spacing: .02em; } header .sub { color: #8a93a6; font-size: 13px; } main { padding: 18px 0 60px; } .row { padding: 18px 0 6px; } .row h2 { margin: 0 0 10px; padding: 0 26px; font-size: 15px; font-weight: 600; color: #d6dbe8; } .row h2 .count { color: #6b7280; font-weight: 400; font-size: 12px; margin-left: 8px; } .scroller { display: flex; gap: 12px; overflow-x: auto; padding: 4px 26px 14px; scrollbar-width: thin; } .scroller::-webkit-scrollbar { height: 8px; } .scroller::-webkit-scrollbar-thumb { background: #2a3142; border-radius: 4px; } .card { flex: 0 0 auto; width: 132px; cursor: pointer; } .card .poster { width: 132px; height: 198px; background: #1d2330 center/cover no-repeat; border-radius: 6px; overflow: hidden; position: relative; } .card .poster .rank { position: absolute; top: 6px; left: 6px; background: rgba(0,0,0,.7); color: #fff; font-size: 11px; padding: 2px 6px; border-radius: 3px; } .card .poster .rating { position: absolute; bottom: 6px; right: 6px; background: rgba(0,0,0,.7); color: #f7c948; font-size: 11px; padding: 2px 6px; border-radius: 3px; } .card .title { margin-top: 6px; font-size: 12px; line-height: 1.3; color: #d6dbe8; max-height: 32px; overflow: hidden; } .card .meta { font-size: 11px; color: #8a93a6; margin-top: 2px; } .empty { padding: 4px 26px 14px; color: #6b7280; font-size: 12px; font-style: italic; } .toolbar { padding: 14px 26px; color: #8a93a6; font-size: 12px; border-top: 1px solid #1d2330; background: #0e1117; position: sticky; bottom: 0; } .toolbar button { background: #1d2330; color: #d6dbe8; border: 1px solid #2a3142; padding: 6px 10px; border-radius: 4px; cursor: pointer; font: inherit; } .toolbar button:hover { background: #232a3a; } a, a:visited { color: #6ea8fe; } </style></head><body> <header> <h1>Trending on Simkl</h1> <div class="sub">Loads <code>simkl-trending-urls.json</code>, wires up <code>SimklTrendingClient</code>, subscribes to all configured catalogs, renders one carousel per row.</div> </header> <main id="rows"></main> <footer class="toolbar"> <span id="status">booting…</span> · <button id="refresh">Refresh now</button> · <button id="clearcache">Clear cache</button> · Cache backend: <span id="cacheBackend"></span> </footer> <script type="module"> import { SimklTrendingClient } from './simkl-trending-client.js'; // ─── Render config — tweak here, not down inside render() ────────────────── const MAX_ITEMS_PER_ROW = 50; // posters per carousel row (each row can hold hundreds; this just caps DOM cost) const $rows = document.getElementById('rows'); const $status = document.getElementById('status'); document.getElementById('cacheBackend').textContent = (typeof localStorage !== 'undefined') ? 'localStorage' : '(none)'; // 1. Load the URL catalog (JSON catalog + 58 default rows). const data = await (await fetch('./simkl-trending-urls.json')).json(); // 2. Pull CONFIG out of the JSON so one file drives both URLs and client_id / app-name / app-version. const cfg = Object.fromEntries(new URLSearchParams(data.required_query)); // 3. Instantiate. Defaults pull supportedTypes from SDK CONFIG, catalogs from JSON's catalogs[] array. const client = new SimklTrendingClient({ clientId: cfg.client_id, appName: cfg['app-name'], appVersion: cfg['app-version'], userAgent: data.user_agent_header, // browsers ignore this anyway — set for non-browser runtimes size: 500, }, data); // 4. Render — diff-by-id so unchanged rows aren't rebuilt on every refresh. function render(rows) { const loaded = rows.some(r => r.items.length > 0); // a file has arrived — an empty row now just has no matches for (const row of rows) { let el = document.getElementById(`row-${row.id}`); if (!el) { el = document.createElement('section'); el.id = `row-${row.id}`; el.className = 'row'; el.innerHTML = `<h2>${escapeHtml(row.title)} <span class="count"></span></h2><div class="scroller"></div>`; $rows.appendChild(el); } el.querySelector('.count').textContent = `(${row.items.length})`; const $sc = el.querySelector('.scroller'); if (row.items.length === 0) { // empty is valid for a filtered row — never pad it $sc.innerHTML = ''; let m = el.querySelector('.empty'); if (!m) { m = document.createElement('div'); m.className = 'empty'; el.appendChild(m); } m.textContent = loaded ? 'nothing in the current file matches this row' : 'no items yet — waiting for first fetch'; } else { el.querySelector('.empty')?.remove(); $sc.innerHTML = row.items.slice(0, MAX_ITEMS_PER_ROW).map(itemHtml).join(''); } } $status.textContent = `${rows.length} rows · ${rows.reduce((s, r) => s + r.items.length, 0)} items total · last update ${new Date().toLocaleTimeString()}`; } function itemHtml(item) { const posterUrl = item.poster ? `https://wsrv.nl/?url=https://simkl.in/posters/${item.poster}_m.webp&q=90` : 'https://simkl.in/poster_no_pic.png'; const ratingTxt = item.ratings?.simkl?.rating ? `★ ${item.ratings.simkl.rating.toFixed(1)}` : ''; return ` <a class="card" href="https://simkl.com${item.url ?? ''}" target="_blank" rel="noopener"> <div class="poster" style="background-image:url('${posterUrl}')"> ${item.rank ? `<span class="rank">#${item.rank}</span>` : ''} ${ratingTxt ? `<span class="rating">${ratingTxt}</span>` : ''} </div> <div class="title">${escapeHtml(item.title || '')}</div> <div class="meta">${escapeHtml(formatMeta(item))}</div> </a> `; } function formatMeta(item) { const year = (item.release_date || '').match(/(\d{4})/)?.[1]; const bits = []; if (year) bits.push(year); if (item.runtime) bits.push(item.runtime); if (item.network) bits.push(item.network); if (item.country && !item.network) bits.push(item.country.toUpperCase()); return bits.join(' · '); } function escapeHtml(s) { return String(s).replace(/[&<>"']/g, c => ({ '&':'&','<':'<','>':'>','"':'"',"'":''' }[c])); } // 5. Subscribe — fires immediately from cache (if any), then on every refresh. client.subscribeCatalogs(render); // 6. Start. Boots fetches in the background. $status.textContent = 'starting…'; await client.start(); $status.textContent = 'running — refreshing on cadence'; // 7. Wire toolbar. document.getElementById('refresh').addEventListener('click', () => { $status.textContent = 'refreshing…'; client.tick() .then(() => { $status.textContent = 'refreshed via tick()'; }) .catch(e => { $status.textContent = 'refresh failed: ' + e.message; }); }); document.getElementById('clearcache').addEventListener('click', async () => { $status.textContent = 'clearing cache…'; await client.clearCache(); $status.textContent = 'cache cleared — re-fetching…'; client.tick() .then(() => { $status.textContent = 'cache cleared + re-fetched via tick()'; }) .catch(e => { $status.textContent = 're-fetch failed: ' + e.message; }); }); window.simklTrending = client; // exposed for console poking </script></body></html>
// === Universal ES module — works in any modern browser AND Node 22+ ===// This is the single SDK file. Save the entire block below as `simkl-trending-client.js`.// The "Browser (vanilla JS)" tab above imports from this exact file.// For Node: also add `"type": "module"` to your package.json (or save as `.mjs`).//// Then pass it the URL catalog object — three ways to load that:// Browser / Deno / CF Workers: const data = await (await fetch('./simkl-trending-urls.json')).json();// Node 22+ (ESM): import data from './simkl-trending-urls.json' with { type: 'json' };// Node (CommonJS .cjs) const data = require('./simkl-trending-urls.json');//// Then: `new SimklTrendingClient(config, data)` or set `globalThis.SIMKL_TRENDING_URLS = data` first.// === SIMKL_TRENDING_DEFAULTS — tune to match your app ===export const SIMKL_TRENDING_DEFAULTS = { // What to load supportedTypes: ['movies', 'tv', 'anime', 'dvd'], // movies / tv / anime / dvd — drop ones your app doesn't surface timeframes: ['today'], // today / week / month (ignored for dvd) size: 100, // 100 or 500 — use 500 if you render derived catalogs (see "Build your own catalogs" below) // HTTP — replace with YOUR values clientId: 'YOUR_CLIENT_ID', appName: 'my-app-name', appVersion: '1.0', userAgent: 'my-app-name/1.0', requestTimeoutMs: 8_000, minIntervalMs: 250, // process-wide floor between consecutive requests fetchRetry: { maxAttempts: 3, baseBackoffMs: 1_000, maxBackoffMs: 16_000, retryStatuses: [408, 425, 429, 500, 502, 503, 504], // never retry other 4xx — they're permanent }, // Refresh regenRetrySeconds: [300, 900, 1_800], // 5 / 15 / 30 min after a 304 (server not regenerated yet) jitterPercent: 0.05, // ±5% so all clients don't hit the CDN at exactly :00 // Persistent cache. Browser: localStorage[storageKey]. Node: file at persistPath. // Set BOTH to null to disable. For server-side runtimes with many processes, swap // this for a Redis / KV adapter (see _persist / _restorePersisted). storageKey: 'simkl-trending-cache', persistPath: './.simkl-trending-cache.json', persistMaxAgeSeconds: 7 * 86_400, // discard cache older than 7d // Display — applied when items are read for display; the cached files stay exactly as served. excludeVideoGamePlay: true, // hide titles classified "Video Game Play" (gameplay / Let's Play videos)};export class SimklTrendingError extends Error { constructor(message, { status, retryAfterMs, permanent } = {}) { super(message); this.name = 'SimklTrendingError'; this.status = status ?? null; this.retryAfterMs = retryAfterMs ?? null; this.permanent = Boolean(permanent); }}export class SimklTrendingClient { constructor(config = {}, data = globalThis.SIMKL_TRENDING_URLS) { if (!data) throw new SimklTrendingError('Pass the URL catalog (SIMKL_TRENDING_URLS) as the second arg or set it globally.'); this.cfg = { ...SIMKL_TRENDING_DEFAULTS, ...config }; this.urls = data.urls ?? data; // unwrap if { urls: {...}, catalogs: [...] } shape this.defaultCatalogs = data.catalogs ?? null; // pre-defined rows from the JSON this._cache = new Map(); this._inflight = new Map(); this._subs = new Map(); this._timers = new Set(); this._abort = new AbortController(); this._stopped = false; this._lastRequestAt = 0; this._warned = new Set(); } // === Public API === async start() { await this._restorePersisted(); for (const t of new Set(this._subs.keys())) this._notify(t); // paint immediately from restored cache for (const entry of this._plan()) this._loop(entry); } stop() { this._stopped = true; this._abort.abort(); for (const t of this._timers) clearTimeout(t); this._timers.clear(); } // Items to display for a type. Display filters (excludeVideoGamePlay) apply here, on read, so // charts, derived rows and a cache restored offline all show the same selection. get(type) { for (const v of this._cache.values()) { const items = v.itemsByType[type]; if (items) return this.cfg.excludeVideoGamePlay ? SimklTrendingRecipes.excludeVideoGamePlay(items) : items; } return []; } subscribe(type, cb) { if (!this._subs.has(type)) this._subs.set(type, new Set()); this._subs.get(type).add(cb); const items = this.get(type); if (items.length) cb(items); // immediate paint from cache return () => this._subs.get(type)?.delete(cb); } // === Catalogs — rows on the home screen === catalogs() { return this._catalogConfigs().map(c => this._buildCatalog(c)); } subscribeCatalogs(cb) { const sources = new Set(this._catalogConfigs().map(c => c.source)); const unsubs = []; let primed = false; // Coalesce: a combined fetch notifies 3 types (movies/tv/anime) in one tick; // cache-restore fires per subscribed source. queueMicrotask flushes once per tick. let queued = false; const fire = () => { if (queued) return; queued = true; queueMicrotask(() => { queued = false; cb(this.catalogs()); }); }; for (const s of sources) unsubs.push(this.subscribe(s, () => { primed = true; fire(); })); if (!primed) fire(); // empty cache — skeleton paint return () => unsubs.forEach(u => u()); } _catalogConfigs() { if (Array.isArray(this.cfg.catalogs)) return this.cfg.catalogs; // explicit override wins if (Array.isArray(this.defaultCatalogs)) return this.defaultCatalogs; // pulled from JSON's catalogs[] (recommended path) const tf0 = this.cfg.timeframes?.[0] ?? 'today'; // last resort: auto-build return this.cfg.supportedTypes.map(t => t === 'dvd' ? { id: 'dvd', source: 'dvd' } : { id: `${t}_${tf0}`, source: t, timeframe: tf0 }); } _fileTitleFor(source, timeframe) { const sizeKey = `top_${this.cfg.size}`; const file = source === 'dvd' ? this.urls.dvd?.[sizeKey] : this.urls?.[source]?.[timeframe]?.[sizeKey]; return file?.title ?? `Trending ${source} on Simkl`; } _buildCatalog(c) { if (!c.id || !c.source) throw new SimklTrendingError('catalogs[] entries need `id` and `source`'); const items = this.get(c.source); let out = items; if (typeof c.transform === 'function') { // inline closure (per-app overrides) out = c.transform(items); } else if (typeof c.recipe === 'string') { // JSON catalogs reference by name const fn = SimklTrendingRecipes[c.recipe]; if (typeof fn === 'function') out = fn(items, { type: c.source, ...(c.recipe_opts || {}) }); else { // never fall back to the unfiltered feed out = []; const msg = `SimklTrendingClient: unknown recipe "${c.recipe}" for catalog "${c.id}" — row left empty.`; if (!this._warned.has(msg)) { this._warned.add(msg); console.warn(msg); } } } return { id: c.id, title: c.title ?? this._fileTitleFor(c.source, c.timeframe ?? this.cfg.timeframes?.[0] ?? 'today'), items: out }; } // Call from a background scheduler (node-cron, Cloudflare Cron, AWS EventBridge) for a single pass. async tick() { await this._restorePersisted(); for (const entry of this._plan()) { const cached = this._cache.get(entry.url); const r = await this._fetch(entry, cached?.lastModified); if (!r.notModified) this._storeAndNotify(entry, r); } await this._persist(); } // === Internals === _requiredQuery() { const { clientId, appName, appVersion } = this.cfg; return `client_id=${clientId}&app-name=${encodeURIComponent(appName)}&app-version=${encodeURIComponent(appVersion)}`; } _plan() { const { supportedTypes, timeframes, size } = this.cfg; if (size !== 100 && size !== 500) throw new SimklTrendingError('size must be 100 or 500'); const key = `top_${size}`; const tvma = supportedTypes.filter(t => ['movies','tv','anime'].includes(t)); const useCombined = tvma.length >= 2; // 1 round-trip < 2-3 per timeframe const plan = []; if (tvma.length) { for (const tf of timeframes) { const file = useCombined ? this.urls.combined[tf][key] : this.urls[tvma[0]][tf][key]; plan.push({ ...file, types: tvma }); } } if (supportedTypes.includes('dvd')) plan.push({ ...this.urls.dvd[key], types: ['dvd'] }); return plan; } // Process-wide rate-limit floor (so N entries refreshing at once don't fire N simultaneous requests). async _throttle() { const gap = Date.now() - this._lastRequestAt; if (gap < this.cfg.minIntervalMs) await this._sleep((this.cfg.minIntervalMs - gap) / 1_000); this._lastRequestAt = Date.now(); } static _parseRetryAfter(value) { if (!value) return null; const secs = Number(value); if (Number.isFinite(secs)) return Math.max(0, secs * 1_000); const dateMs = Date.parse(value); return Number.isFinite(dateMs) ? Math.max(0, dateMs - Date.now()) : null; } // _fetch — dedup + retry + timeout + conditional revalidation + Retry-After. Normalizes array vs object response into itemsByType. async _fetch(entry, lastModified) { const sep = entry.url.includes('?') ? '&' : '?'; const url = entry.url + sep + this._requiredQuery(); if (this._inflight.has(url)) return this._inflight.get(url); // In Node/Deno/Bun/CF Workers, set If-Modified-Since manually for conditional revalidation. // In BROWSERS, the manual header would trigger a CORS preflight (If-Modified-Since isn't // CORS-safelisted), and most CDNs respond to OPTIONS with 405 → preflight fails → fetch // fails. Instead we use `cache: 'no-cache'` so the browser handles revalidation via its // own HTTP cache (browser-added If-None-Match / If-Modified-Since don't count as author // headers and skip preflight). User-Agent is dropped in browsers too — it's a forbidden // header per Fetch spec and silently ignored. const isBrowser = typeof window !== 'undefined'; const headers = { 'Accept': 'application/json' }; if (!isBrowser) { headers['User-Agent'] = this.cfg.userAgent; if (lastModified) headers['If-Modified-Since'] = lastModified; } const { fetchRetry, requestTimeoutMs } = this.cfg; const run = async () => { let lastErr; for (let attempt = 0; attempt < fetchRetry.maxAttempts; attempt++) { await this._throttle(); const ctl = new AbortController(); const time = setTimeout(() => ctl.abort(), requestTimeoutMs); const off = () => ctl.abort(); this._abort.signal.addEventListener('abort', off); try { const fetchOpts = { headers, signal: ctl.signal }; if (isBrowser) fetchOpts.cache = 'no-cache'; const res = await fetch(url, fetchOpts); if (res.status === 304) return { itemsByType: null, lastModified, notModified: true }; if (!res.ok) { const retryAfterMs = SimklTrendingClient._parseRetryAfter(res.headers.get('Retry-After')); const permanent = !fetchRetry.retryStatuses.includes(res.status); throw new SimklTrendingError(`${res.status} ${res.statusText} — ${entry.title}`, { status: res.status, retryAfterMs, permanent }); } // In browsers with cache:'no-cache', fetch resolves to 200 + cached body when the // server returned 304 to the browser's conditional request. Detect that via // Last-Modified equality so we skip a no-op re-render. const newLastModified = res.headers.get('Last-Modified'); if (isBrowser && lastModified && newLastModified === lastModified) { return { itemsByType: null, lastModified, notModified: true }; } const raw = await res.json(); // Combined endpoint returns {movies, tv, anime}. Keep ALL keys (not just wanted types) so // a dev who later adds 'anime' to supportedTypes can serve it from the same cached response. const itemsByType = Array.isArray(raw) ? { [entry.types[0]]: raw } // per-type / dvd → array : raw; // combined → store full object return { itemsByType, lastModified: newLastModified, notModified: false }; } catch (e) { if (this._stopped) throw e; if (e instanceof SimklTrendingError && e.permanent) throw e; // don't waste backoff on 404/400/etc lastErr = e; const honored = (e instanceof SimklTrendingError && e.retryAfterMs != null) ? e.retryAfterMs : null; const backoff = honored ?? Math.min(fetchRetry.baseBackoffMs * (2 ** attempt), fetchRetry.maxBackoffMs); const jitter = backoff * (Math.random() * 0.2 - 0.1); // ±10% jitter on retry delay await this._sleep(Math.max(0, backoff + jitter) / 1_000); } finally { clearTimeout(time); this._abort.signal.removeEventListener('abort', off); } } throw lastErr; }; const p = run().finally(() => this._inflight.delete(url)); this._inflight.set(url, p); return p; } async _loop(entry) { let lastModified = this._cache.get(entry.url)?.lastModified ?? null; let retry = 0; while (!this._stopped) { let r; try { r = await this._fetch(entry, lastModified); } catch (e) { if (this._stopped) return; if (e instanceof SimklTrendingError && e.permanent) { // 404 / 401 / etc — halt this entry's loop console.error(`${entry.title}: ${e.message} (permanent, halting loop)`); return; } console.error(`${entry.title}:`, e.message); await this._sleep(30); continue; } let wait; if (r.notModified) { wait = this.cfg.regenRetrySeconds[Math.min(retry++, this.cfg.regenRetrySeconds.length - 1)]; } else { this._storeAndNotify(entry, r); lastModified = r.lastModified; retry = 0; await this._persist(); const j = this.cfg.jitterPercent; wait = Math.round(entry.refresh_seconds * (1 + (Math.random() - 0.5) * 2 * j)); } await this._sleep(wait); } } _storeAndNotify(entry, r) { this._cache.set(entry.url, { ...r, types: entry.types, fetchedAt: Date.now() }); for (const t of entry.types) this._notify(t); } _notify(type) { const items = this.get(type); for (const cb of (this._subs.get(type) ?? [])) cb(items); } _sleep(seconds) { return new Promise(resolve => { const t = setTimeout(() => { this._timers.delete(t); resolve(); }, seconds * 1_000); this._timers.add(t); }); } // Wipe in-memory + persistent cache and notify subscribers (so UI repaints empty). // Call client.tick() after this if you want to immediately re-fetch. async clearCache() { this._cache.clear(); const ls = this._localStorage(); if (ls && this.cfg.storageKey) { try { ls.removeItem(this.cfg.storageKey); } catch { /* ignore */ } } if (this.cfg.persistPath && !ls) { try { const fs = await import('node:fs/promises'); await fs.unlink(this.cfg.persistPath); } catch { /* no file — fine */ } } for (const t of this._subs.keys()) this._notify(t); } // Persistence — auto-detects environment. Browser → localStorage. Node → file. // Swap these two methods (plus clearCache) for a Redis / KV adapter in server-side runtimes. _localStorage() { try { return globalThis.localStorage ?? null; } catch { return null; } } async _persist() { const data = JSON.stringify(Object.fromEntries(this._cache)); const ls = this._localStorage(); if (ls && this.cfg.storageKey) { try { ls.setItem(this.cfg.storageKey, data); } catch { /* quota etc */ } return; } if (!this.cfg.persistPath) return; try { const fs = await import('node:fs/promises'); await fs.writeFile(this.cfg.persistPath, data); } catch { /* best-effort */ } } async _restorePersisted() { const cutoff = Date.now() - this.cfg.persistMaxAgeSeconds * 1_000; let raw = null; const ls = this._localStorage(); if (ls && this.cfg.storageKey) { try { const s = ls.getItem(this.cfg.storageKey); if (s) raw = JSON.parse(s); } catch { /* corrupt — ignore */ } } if (!raw && this.cfg.persistPath && !ls) { try { const fs = await import('node:fs/promises'); raw = JSON.parse(await fs.readFile(this.cfg.persistPath, 'utf8')); } catch { /* no cache yet — fine */ } } if (!raw) return; for (const [url, val] of Object.entries(raw)) { if (val.fetchedAt && val.fetchedAt >= cutoff) this._cache.set(url, val); } }}// ─────────────────────────────────────────────────────────────────────────────// SimklTrendingRecipes — the catalog transforms referenced by name from the// JSON's `catalogs[]` array. Pure functions: `(items, opts?) => items`. They// never modify the items you pass in, and a filter can return a short or// empty list — that is a valid result, not a cue to fill the row.// JSON rows also receive the row's `source` as `opts.type`.// ─────────────────────────────────────────────────────────────────────────────const SIMKL_DAY_MS = 86_400_000;const simklNormGenre = g => String(g).toLowerCase().replace(/[\s_-]+/g, ' ').trim();const simklCount = v => (Number.isInteger(v) && v >= 0 ? v : null); // valid non-negative integer, else nullconst simklNumber = v => (typeof v === 'number' && Number.isFinite(v) ? v : null);// `release_date` is documented as MM/DD/YYYY. Parse it explicitly, in UTC; anything else is null.export function simklParseReleaseDate(s) { const m = typeof s === 'string' ? /^(\d{2})\/(\d{2})\/(\d{4})$/.exec(s) : null; if (!m) return null; const [mo, d, y] = [Number(m[1]), Number(m[2]), Number(m[3])]; const t = Date.UTC(y, mo - 1, d); const u = new Date(t); return u.getUTCFullYear() === y && u.getUTCMonth() === mo - 1 && u.getUTCDate() === d ? t : null; // rejects 02/30 etc.}// The "Video Game Play" classification — matched exactly after normalising case, spaces, `-` and `_`// (so the genre-file slug `video-game-play` matches too). Title words, networks and other// genres such as anime "Strategy Game" are never matched.export function simklIsVideoGamePlay(entry) { if (Array.isArray(entry?.genres) && entry.genres.some(g => simklNormGenre(g) === 'video game play')) return true; return [entry?.slug, entry?.name].some(v => typeof v === 'string' && simklNormGenre(v) === 'video game play'); // genre-list entry}// Defaults for underseenGems, per type, then for every type. Override any of them with opts / `recipe_opts`.export const SIMKL_UNDERSEEN_DEFAULTS = { movies: { source: 'imdb', minRating: 7.0, minVotes: 500, maxVotes: 25_000 }, tv: { source: 'imdb', minRating: 7.5, minVotes: 500, maxVotes: 10_000, excludeGenres: ['Sport', 'Sports', 'Reality', 'Talk', 'Talk Show', 'News'] }, anime: { source: 'mal', minRating: 7.5, minVotes: 500, maxVotes: 10_000 },};export const SIMKL_UNDERSEEN_COMMON = { maxSimklVotes: 500, // Simkl rating count must be known and at most this minAgeDays: 365, // released at least this many days ago maxTrackedAudience: null, // optional extra guard on watching + completed — off by default excludeGenres: [],};export const SimklTrendingRecipes = { // Sorts byRank(items) { return [...items].sort((a, b) => (a.rank ?? Infinity) - (b.rank ?? Infinity)); }, // Sorts by plan_to_watch desc — i.e. how many users have added the item to their watchlist. // Despite the historical "mostAnticipated" name, this fires for already-released titles too // (Game of Thrones, Avatar, One Piece — all rank high because they have huge backlog appeal). // `mostWatchlisted` is the accurate name; `mostAnticipated` is kept as an alias for back-compat. mostWatchlisted(items) { return [...items].sort((a, b) => (b.plan_to_watch ?? 0) - (a.plan_to_watch ?? 0)); }, mostAnticipated(items) { return [...items].sort((a, b) => (b.plan_to_watch ?? 0) - (a.plan_to_watch ?? 0)); }, // alias of mostWatchlisted newest(items) { return [...items].filter(i => i.release_date).sort((a, b) => +new Date(b.release_date) - +new Date(a.release_date)); }, stickiest(items) { return [...items].sort((a, b) => parseFloat(a.drop_rate ?? 'Infinity') - parseFloat(b.drop_rate ?? 'Infinity')); }, highestRated(items, { source = 'simkl', minVotes = 500 } = {}) { return items.filter(i => (i.ratings?.[source]?.votes ?? 0) >= minVotes) .sort((a, b) => (b.ratings?.[source]?.rating ?? 0) - (a.ratings?.[source]?.rating ?? 0)); }, bestBoxOffice(items) { // movies only const bo = i => { const m = (i.metadata || '').match(/Box office \$([\d.]+)([MBK])/); return m ? parseFloat(m[1]) * { K: 1e3, M: 1e6, B: 1e9 }[m[2]] : null; }; return items.map(i => ({ i, v: bo(i) })).filter(x => x.v != null).sort((a, b) => b.v - a.v).map(x => x.i); }, // Composed (multiple signals) // Well-rated titles outside the top `minRank` of Simkl's rank, sorted by rating. `rank` is a // rating-based order (1 = best), not a count of viewers, so this row still holds widely seen // titles — use underseenGems for smaller-audience picks. wellRated(items, { minRating = 7.5, minVotes = 500, minRank = 2000, source = 'simkl' } = {}) { return items .filter(i => (i.rank ?? 0) > minRank && (i.ratings?.[source]?.rating ?? 0) >= minRating && (i.ratings?.[source]?.votes ?? 0) >= minVotes) .sort((a, b) => (b.ratings?.[source]?.rating ?? 0) - (a.ratings?.[source]?.rating ?? 0)); }, // DEPRECATED — the old name of wellRated, kept so saved configs keep working. Same selection. hiddenGems(items, opts) { return SimklTrendingRecipes.wellRated(items, opts); }, // Well-rated titles with a smaller audience, by external rating count. A heuristic for a // smaller audience — not proof that a title is obscure worldwide. Needs `type` // ('movies' | 'tv' | 'anime'); JSON rows pass their `source`. Defaults: SIMKL_UNDERSEEN_DEFAULTS. underseenGems(items, opts = {}) { if (!Object.hasOwn(SIMKL_UNDERSEEN_DEFAULTS, opts.type)) return []; const o = { ...SIMKL_UNDERSEEN_COMMON, ...SIMKL_UNDERSEEN_DEFAULTS[opts.type], ...opts }; const limits = [o.minRating, o.minVotes, o.maxVotes, o.maxSimklVotes, o.minAgeDays]; if (!limits.every(Number.isFinite) || (o.maxTrackedAudience != null && !Number.isFinite(o.maxTrackedAudience))) return []; const latest = Number(o.now ?? Date.now()) - o.minAgeDays * SIMKL_DAY_MS; // released on or before this if (!Number.isFinite(latest)) return []; const skip = new Set((o.excludeGenres ?? []).map(simklNormGenre)); const picked = []; for (const i of items) { if (!i || i.status === 'premiere') continue; const released = simklParseReleaseDate(i.release_date); if (released == null || released > latest) continue; // missing, invalid, future or too recent const rating = simklNumber(i.ratings?.[o.source]?.rating); const votes = simklCount(i.ratings?.[o.source]?.votes); if (rating == null || votes == null) continue; if (rating < o.minRating || votes < o.minVotes || votes > o.maxVotes) continue; const simklVotes = simklCount(i.ratings?.simkl?.votes); if (simklVotes == null || simklVotes > o.maxSimklVotes) continue; if (skip.size && Array.isArray(i.genres) && i.genres.some(g => skip.has(simklNormGenre(g)))) continue; if (o.maxTrackedAudience != null) { // optional: watching + completed const watching = simklCount(i.watching), completed = simklCount(i.completed); if (watching == null || completed == null) continue; // missing is not zero if (watching + completed > o.maxTrackedAudience) continue; } picked.push({ i, rating, votes }); } return picked.sort((a, b) => b.rating - a.rating || a.votes - b.votes).map(p => p.i); }, // Drops titles classified "Video Game Play" — and the `video-game-play` entry from a genre list. // The client applies this to everything it displays when `excludeVideoGamePlay` is on. excludeVideoGamePlay(items) { return items.filter(i => !simklIsVideoGamePlay(i)); }, criticallyAcclaimed(items, { minRating = 8.5, minVotes = 1000, source = 'simkl' } = {}) { return items.filter(i => (i.ratings?.[source]?.rating ?? 0) >= minRating && (i.ratings?.[source]?.votes ?? 0) >= minVotes) .sort((a, b) => (b.ratings?.[source]?.rating ?? 0) - (a.ratings?.[source]?.rating ?? 0)); }, marathonWorthy(items, { minEpisodes = 20, maxEpisodes = 100, minRating = 8, source = 'simkl' } = {}) { return items.filter(i => i.status === 'ended' && (i.total_episodes ?? 0) >= minEpisodes && (i.total_episodes ?? 0) <= maxEpisodes && (i.ratings?.[source]?.rating ?? 0) >= minRating); }, crossPlatformConsensus(items, { simklMin = 8, otherMin = 8 } = {}) { return items.filter(i => (i.ratings?.simkl?.rating ?? 0) >= simklMin && ((i.ratings?.imdb?.rating ?? i.ratings?.mal?.rating) ?? 0) >= otherMin); }, massiveAnticipation(items, { minRatio = 300 } = {}) { return items.filter(i => (i.plan_to_watch ?? 0) / ((i.watched ?? 0) + 1) >= minRatio) .sort((a, b) => (b.plan_to_watch / (b.watched + 1)) - (a.plan_to_watch / (a.watched + 1))); }, bestOfNetwork(items, { network, minRating = 7.5, requireEnded = false, source = 'simkl' } = {}) { return items.filter(i => i.network === network && (!requireEnded || i.status === 'ended') && (i.ratings?.[source]?.rating ?? 0) >= minRating) .sort((a, b) => (b.ratings?.[source]?.rating ?? 0) - (a.ratings?.[source]?.rating ?? 0)); }, bestOfGenre(items, { genre, minRating = 7.5, minVotes = 300, source = 'simkl' } = {}) { return items.filter(i => i.genres?.includes(genre) && (i.ratings?.[source]?.rating ?? 0) >= minRating && (i.ratings?.[source]?.votes ?? 0) >= minVotes) .sort((a, b) => (b.ratings?.[source]?.rating ?? 0) - (a.ratings?.[source]?.rating ?? 0)); }, // Filters (all use { ...opts } so they're JSON-describable) byGenre (items, { genre } = {}) { return items.filter(i => i.genres?.includes(genre)); }, byCountry (items, { country } = {}) { const c = String(country || '').toLowerCase(); return items.filter(i => i.country === c); }, byNetwork (items, { network } = {}) { return items.filter(i => i.network === network); }, byStatus (items, { status } = {}) { return items.filter(i => i.status === status); }, byAnimeType(items, { animeType } = {}) { return items.filter(i => i.anime_type === animeType); }, withTrailer(items) { return items.filter(i => i.trailer); }, ongoing(items) { return items.filter(i => i.status === 'ongoing'); }, recentlyEnded(items) { return items.filter(i => i.status === 'ended'); }, // `status === 'premiere'` flags items whose first episode JUST aired (typically within ~30 days). // It is NOT an "upcoming premieres" filter — those items have status 'ongoing' or no status set. justPremiered(items) { return items.filter(i => i.status === 'premiere'); }, premiering(items) { return items.filter(i => i.status === 'premiere'); }, // alias of justPremiered (kept for back-compat) miniSeries(items, { maxEpisodes = 6 } = {}) { return items.filter(i => (i.total_episodes ?? Infinity) <= maxEpisodes); }, justReleased(items, { withinDays = 30 } = {}) { const cutoff = Date.now() - withinDays * 86_400_000; return items.filter(i => i.release_date && +new Date(i.release_date) >= cutoff); }, inTheatersNow(items, { withinDays = 30 } = {}) { // movies only const cutoff = Date.now() - withinDays * 86_400_000; return items.filter(i => i.theater && +new Date(i.theater) >= cutoff); }, recentlyOnDvd(items, { withinDays = 30 } = {}) { // movies only const cutoff = Date.now() - withinDays * 86_400_000; return items.filter(i => i.dvd_date && +new Date(i.dvd_date) >= cutoff); }, // Era filters — release_date in trending feeds is "MM/DD/YYYY"; the regex grabs the 4-digit year either way. byYear (items, { year } = {}) { return items.filter(i => +String(i.release_date||'').match(/\d{4}/)?.[0] === year); }, byDecade (items, { decade } = {}) { return items.filter(i => { const y = +String(i.release_date||'').match(/\d{4}/)?.[0]; return y >= decade && y < decade + 10; }); }, currentYear(items) { const y = new Date().getFullYear(); return items.filter(i => +String(i.release_date||'').match(/\d{4}/)?.[0] === y); }, lastYear (items) { const y = new Date().getFullYear()-1; return items.filter(i => +String(i.release_date||'').match(/\d{4}/)?.[0] === y); }, bestOfDecade(items, { decade, source = 'simkl', minVotes = 100 } = {}) { // decade + rating sort, mirrors bestOfNetwork/bestOfGenre return items.filter(i => { const y = +String(i.release_date||'').match(/\d{4}/)?.[0]; return y >= decade && y < decade + 10 && (i.ratings?.[source]?.votes ?? 0) >= minVotes; }) .sort((a, b) => (b.ratings?.[source]?.rating ?? 0) - (a.ratings?.[source]?.rating ?? 0)); }, quickWatches(items, { maxMinutes = 90 } = {}) { // movies only const m = s => { if (!s) return Infinity; const x = s.match(/^(?:(\d+)h)?\s*(?:(\d+)m)?$/); return x ? ((parseInt(x[1]||'0')*60) + parseInt(x[2]||'0')) || Infinity : Infinity; }; return items.filter(i => m(i.runtime) <= maxMinutes); },};/* === Sample usage (put this in a SEPARATE file — app.js / main.mjs / etc.) ===import data from './simkl-trending-urls.json' with { type: 'json' };import { SimklTrendingClient } from './simkl-trending-client.js';const client = new SimklTrendingClient({ clientId: 'YOUR_CLIENT_ID', appName: 'my-app', appVersion: '1.0' }, data);client.subscribeCatalogs(rows => { for (const row of rows) console.log(`${row.title}: ${row.items.length} items`);});await client.start();// process.on('SIGINT', () => { client.stop(); process.exit(0); });=== Background scheduling (server-side runtimes) ===For long-running servers, the in-process loop is enough. For serverless / cron-triggeredruntimes (Cloudflare Workers, AWS Lambda, GitHub Actions), wire `await client.tick()` toyour scheduler instead of calling `start()`. For browser usage, see the Browser tab above. */
import datetime, email.utils, json, math, os, random, re, threading, time, urllib.parseimport requests# === Load the URL catalog ===# 1. Save the "Copy-paste JSON" block above as `simkl-trending-urls.json` in your project.# 2. Load it into SIMKL_TRENDING_URLS:# with open('simkl-trending-urls.json') as f: SIMKL_TRENDING_URLS = json.load(f)# Or for a packaged module:# import importlib.resources, json# SIMKL_TRENDING_URLS = json.loads(importlib.resources.files('your_pkg').joinpath('simkl-trending-urls.json').read_text())# You can also pass it explicitly: SimklTrendingClient(urls=urls_dict)# SimklTrendingClient raises SimklTrendingError if it can't find the catalog in either place.# === SIMKL_TRENDING_DEFAULTS — tune to match your app ===SIMKL_TRENDING_DEFAULTS = { # What to load 'supported_types': ['movies', 'tv', 'anime', 'dvd'], # movies / tv / anime / dvd — drop ones your app doesn't surface 'timeframes': ['today'], # today / week / month (ignored for dvd) 'size': 100, # 100 or 500 — use 500 if you render derived catalogs # HTTP — replace with YOUR values 'client_id': 'YOUR_CLIENT_ID', 'app_name': 'my-app-name', 'app_version': '1.0', 'user_agent': 'my-app-name/1.0', 'request_timeout_s': 8, 'min_interval_s': 0.25, # process-wide floor between consecutive requests 'fetch_retry': { 'max_attempts': 3, 'base_backoff_s': 1, 'max_backoff_s': 16, 'retry_statuses': [408, 425, 429, 500, 502, 503, 504], # never retry other 4xx — they're permanent }, # Refresh 'regen_retry_s': [300, 900, 1800], # 5 / 15 / 30 min after a 304 (server not regenerated yet) 'jitter_percent': 0.05, # ±5% so all clients don't hit the CDN at exactly :00 # Persistent cache — set persist_path to None to disable. For server-side runtimes, # swap _persist / _restore_persisted for a Redis / KV adapter. 'persist_path': './.simkl-trending-cache.json', 'persist_max_age_s': 7 * 86400, # discard cache older than 7d # Display — applied when items are read for display; the cached files stay exactly as served. 'exclude_video_game_play': True, # hide titles classified "Video Game Play" (gameplay / Let's Play videos) # Catalogs — the rows on your home screen. Each becomes a {'id', 'title', 'items'} dict # returned by client.catalogs() / pushed to client.subscribe_catalogs(cb). If None, # the JSON's catalogs[] (or one row per supported_type with the JSON's pre-composed title). # { 'id', 'source', 'timeframe'?, 'title'?, 'transform'?, 'recipe'?, 'recipe_opts'? } 'catalogs': None,}class SimklTrendingError(Exception): def __init__(self, message, status=None, retry_after_s=None, permanent=False): super().__init__(message) self.status = status self.retry_after_s = retry_after_s self.permanent = bool(permanent)# === Recipes — named by the JSON's catalogs[]. This port ships wellRated (plus its deprecated# alias hiddenGems) and underseenGems. A row naming any other recipe stays EMPTY until you port it# (see "Catalog ideas" below) or give the row a `transform` — never the unfiltered feed.# Recipes never modify what they're given; a short or empty result is valid.def _simkl_obj(v): return v if isinstance(v, dict) else {}def _simkl_norm_genre(g): return re.sub(r'[\s_-]+', ' ', str(g)).strip().lower()def _simkl_number(v): return float(v) if type(v) in (int, float) and math.isfinite(v) else None # bool is not a number heredef _simkl_count(v): # valid non-negative integer, else None n = _simkl_number(v) return int(n) if n is not None and n >= 0 and n == int(n) else Nonedef simkl_parse_release_date(s): """`release_date` is documented as MM/DD/YYYY — parse it explicitly, in UTC. Anything else → None.""" if not isinstance(s, str) or not re.fullmatch(r'[0-9]{2}/[0-9]{2}/[0-9]{4}', s): return None try: return datetime.datetime.strptime(s, '%m/%d/%Y').replace(tzinfo=datetime.timezone.utc) except ValueError: return None # e.g. 02/30/2024def simkl_is_video_game_play(entry): """The "Video Game Play" classification, matched exactly after normalising case, spaces, - and _. Also matches a genre-list entry (slug `video-game-play`). Never matches title words or networks.""" if not isinstance(entry, dict): return False genres = entry.get('genres') if isinstance(genres, list) and any(_simkl_norm_genre(g) == 'video game play' for g in genres): return True return any(isinstance(v, str) and _simkl_norm_genre(v) == 'video game play' for v in (entry.get('slug'), entry.get('name')))def simkl_exclude_video_game_play(items): return [i for i in items if not simkl_is_video_game_play(i)]def simkl_well_rated(items, opts=None): """Well-rated titles outside the top `minRank` of Simkl's rank (a rating-based order, not a viewer count) — so this still holds widely seen titles. Formerly hiddenGems; same selection.""" o = {'minRating': 7.5, 'minVotes': 500, 'minRank': 2000, 'source': 'simkl', **(opts or {})} r = lambda i: _simkl_obj(_simkl_obj(i.get('ratings')).get(o['source'])) picked = [i for i in items if (i.get('rank') or 0) > o['minRank'] and (r(i).get('rating') or 0) >= o['minRating'] and (r(i).get('votes') or 0) >= o['minVotes']] return sorted(picked, key=lambda i: -(r(i).get('rating') or 0))SIMKL_UNDERSEEN_DEFAULTS = { # per type; override with opts / recipe_opts 'movies': {'source': 'imdb', 'minRating': 7.0, 'minVotes': 500, 'maxVotes': 25000}, 'tv': {'source': 'imdb', 'minRating': 7.5, 'minVotes': 500, 'maxVotes': 10000, 'excludeGenres': ['Sport', 'Sports', 'Reality', 'Talk', 'Talk Show', 'News']}, 'anime': {'source': 'mal', 'minRating': 7.5, 'minVotes': 500, 'maxVotes': 10000},}SIMKL_UNDERSEEN_COMMON = {'maxSimklVotes': 500, 'minAgeDays': 365, 'maxTrackedAudience': None, 'excludeGenres': []}def simkl_underseen_gems(items, opts=None): """Well-rated titles with a smaller audience, by external rating count — a heuristic, not proof a title is obscure worldwide. Needs opts['type'] ('movies' | 'tv' | 'anime'); JSON rows pass their source.""" opts = opts or {} if opts.get('type') not in SIMKL_UNDERSEEN_DEFAULTS: return [] o = {**SIMKL_UNDERSEEN_COMMON, **SIMKL_UNDERSEEN_DEFAULTS[opts['type']], **opts} cap = o['maxTrackedAudience'] limits = (o['minRating'], o['minVotes'], o['maxVotes'], o['maxSimklVotes'], o['minAgeDays']) if any(_simkl_number(x) is None for x in limits) or (cap is not None and _simkl_number(cap) is None): return [] now = o.get('now') or datetime.datetime.now(datetime.timezone.utc) latest = now - datetime.timedelta(days=o['minAgeDays']) # released on or before this skip = {_simkl_norm_genre(g) for g in (o['excludeGenres'] or [])} picked = [] for i in items: if not isinstance(i, dict) or i.get('status') == 'premiere': continue released = simkl_parse_release_date(i.get('release_date')) if released is None or released > latest: continue # missing, invalid, future or too recent ext = _simkl_obj(_simkl_obj(i.get('ratings')).get(o['source'])) rating, votes = _simkl_number(ext.get('rating')), _simkl_count(ext.get('votes')) if rating is None or votes is None: continue if rating < o['minRating'] or votes < o['minVotes'] or votes > o['maxVotes']: continue simkl_votes = _simkl_count(_simkl_obj(_simkl_obj(i.get('ratings')).get('simkl')).get('votes')) if simkl_votes is None or simkl_votes > o['maxSimklVotes']: continue if skip and isinstance(i.get('genres'), list) and any(_simkl_norm_genre(g) in skip for g in i['genres']): continue if cap is not None: # optional: watching + completed watching, completed = _simkl_count(i.get('watching')), _simkl_count(i.get('completed')) if watching is None or completed is None: continue # missing is not zero if watching + completed > cap: continue picked.append((i, rating, votes)) picked.sort(key=lambda p: (-p[1], p[2])) # rating desc, then rating count asc return [p[0] for p in picked]SIMKL_TRENDING_RECIPES = { 'wellRated': simkl_well_rated, 'hiddenGems': simkl_well_rated, # DEPRECATED — old name of wellRated 'underseenGems': simkl_underseen_gems,}class SimklTrendingClient: def __init__(self, config=None, data=None): data = data or globals().get('SIMKL_TRENDING_URLS') if not data: raise SimklTrendingError('Pass the URL catalog (SIMKL_TRENDING_URLS) as the second arg or define it globally.') self.cfg = {**SIMKL_TRENDING_DEFAULTS, **(config or {})} self.urls = data.get('urls', data) # unwrap {urls: {...}, catalogs: [...]} shape self.default_catalogs = data.get('catalogs') # pre-defined rows from JSON self._cache = {} self._inflight = {} self._subs = {} self._lock = threading.RLock() self._stopped = threading.Event() self._threads = [] self._warned = set() self._last_request_at = 0.0 # === Public API === def start(self): self._restore_persisted() with self._lock: for t in list(self._subs): self._notify(t) # paint immediately from restored cache for entry in self._plan(): th = threading.Thread(target=self._loop, args=(entry,), daemon=True) th.start() self._threads.append(th) def stop(self): self._stopped.set() # Wipe in-memory + persistent cache and notify subscribers (so UI repaints empty). # Call client.tick() after this to immediately re-fetch. def clear_cache(self): with self._lock: self._cache.clear() subs_snapshot = {t: list(cbs) for t, cbs in self._subs.items()} if self.cfg.get('persist_path'): try: os.remove(self.cfg['persist_path']) except OSError: pass # no file — fine for t, cbs in subs_snapshot.items(): for cb in cbs: cb([]) # Items to display for a type. Display filters (exclude_video_game_play) apply here, on read, # so charts, derived rows and a cache restored offline all show the same selection. def get(self, type_): with self._lock: for v in self._cache.values(): if type_ in v['items_by_type']: items = v['items_by_type'][type_] return simkl_exclude_video_game_play(items) if self.cfg['exclude_video_game_play'] else items return [] def subscribe(self, type_, cb): with self._lock: self._subs.setdefault(type_, []).append(cb) items = self.get(type_) if items: cb(items) # immediate paint from cache def unsub(): with self._lock: if cb in self._subs.get(type_, []): self._subs[type_].remove(cb) return unsub # Resolve all configured catalogs to [{'id','title','items'}, ...] — TV-home-screen rows. def catalogs(self): return [self._build_catalog(c) for c in self._catalog_configs()] # Subscribe to ALL catalog rows at once. cb([{...}, ...]) fires on every refresh. def subscribe_catalogs(self, cb): sources = {c['source'] for c in self._catalog_configs()} unsubs = [] def fire(_): cb(self.catalogs()) for s in sources: unsubs.append(self.subscribe(s, fire)) if not any(self.get(s) for s in sources): cb(self.catalogs()) # empty cache — fire once for skeleton paint def unsub(): for u in unsubs: u() return unsub # Call from a background scheduler (cron, Celery beat, APScheduler) for a single pass. def tick(self): self._restore_persisted() for entry in self._plan(): with self._lock: cached = self._cache.get(entry['url']) r = self._fetch(entry, cached and cached['last_modified']) if not r['not_modified']: self._store_and_notify(entry, r) self._persist() # === Internals === def _required_query(self): c = self.cfg return (f"client_id={c['client_id']}" f"&app-name={urllib.parse.quote(c['app_name'])}" f"&app-version={urllib.parse.quote(c['app_version'])}") def _plan(self): c = self.cfg if c['size'] not in (100, 500): raise SimklTrendingError('size must be 100 or 500') key = f"top_{c['size']}" tvma = [t for t in c['supported_types'] if t in ('movies', 'tv', 'anime')] use_combined = len(tvma) >= 2 # 1 round-trip < 2-3 per timeframe plan = [] if tvma: for tf in c['timeframes']: f = (self.urls['combined'][tf][key] if use_combined else self.urls[tvma[0]][tf][key]) plan.append({**f, 'types': tvma}) if 'dvd' in c['supported_types']: plan.append({**self.urls['dvd'][key], 'types': ['dvd']}) return plan # Process-wide rate-limit floor (so N entries refreshing at once don't fire N simultaneous requests). def _throttle(self): with self._lock: gap = time.monotonic() - self._last_request_at if gap < self.cfg['min_interval_s']: if self._stopped.wait(self.cfg['min_interval_s'] - gap): return False self._last_request_at = time.monotonic() return True @staticmethod def _parse_retry_after(value): if not value: return None try: return max(0.0, float(value)) except (TypeError, ValueError): pass try: dt = email.utils.parsedate_to_datetime(value) return max(0.0, dt.timestamp() - time.time()) except (TypeError, ValueError): return None # _fetch — dedup + retry + timeout + If-Modified-Since + Retry-After. Normalizes list vs dict into items_by_type. def _fetch(self, entry, last_modified): sep = '&' if '?' in entry['url'] else '?' url = entry['url'] + sep + self._required_query() with self._lock: inflight = self._inflight.get(url) if inflight: inflight['done'].wait() if inflight.get('error'): raise inflight['error'] return inflight['result'] done = threading.Event() slot = {'done': done, 'result': None, 'error': None} with self._lock: self._inflight[url] = slot try: headers = {'User-Agent': self.cfg['user_agent'], 'Accept': 'application/json'} if last_modified: headers['If-Modified-Since'] = last_modified retry_cfg = self.cfg['fetch_retry'] last_err = None for attempt in range(retry_cfg['max_attempts']): self._throttle() try: res = requests.get(url, headers=headers, timeout=self.cfg['request_timeout_s']) if res.status_code == 304: slot['result'] = {'items_by_type': None, 'last_modified': last_modified, 'not_modified': True} return slot['result'] if not res.ok: permanent = res.status_code not in retry_cfg['retry_statuses'] retry_after_s = self._parse_retry_after(res.headers.get('Retry-After')) raise SimklTrendingError(f"{res.status_code} {res.reason} — {entry['title']}", status=res.status_code, retry_after_s=retry_after_s, permanent=permanent) raw = res.json() items_by_type = ({entry['types'][0]: raw} if isinstance(raw, list) # per-type / dvd → list else {t: raw.get(t, []) for t in entry['types']}) # combined → dict, filter to wanted types slot['result'] = {'items_by_type': items_by_type, 'last_modified': res.headers.get('Last-Modified'), 'not_modified': False} return slot['result'] except SimklTrendingError as e: if e.permanent: slot['error'] = e; raise # don't waste backoff on permanent errors last_err = e honored = e.retry_after_s if e.retry_after_s is not None else None backoff = honored if honored is not None else min(retry_cfg['base_backoff_s'] * (2 ** attempt), retry_cfg['max_backoff_s']) backoff += backoff * (random.random() * 0.2 - 0.1) # ±10% jitter if self._stopped.wait(max(0, backoff)): raise RuntimeError('stopped') except Exception as e: if self._stopped.is_set(): raise last_err = e backoff = min(retry_cfg['base_backoff_s'] * (2 ** attempt), retry_cfg['max_backoff_s']) if self._stopped.wait(backoff): raise RuntimeError('stopped') slot['error'] = last_err raise last_err finally: done.set() with self._lock: self._inflight.pop(url, None) def _loop(self, entry): with self._lock: c = self._cache.get(entry['url']) last_modified = c['last_modified'] if c else None retry = 0 while not self._stopped.is_set(): try: r = self._fetch(entry, last_modified) except SimklTrendingError as e: if self._stopped.is_set(): return if e.permanent: # 404 / 401 / etc — halt this entry's loop print(f"{entry['title']}: {e} (permanent, halting loop)") return print(f"{entry['title']}: {e}") if self._stopped.wait(30): return continue except Exception as e: if self._stopped.is_set(): return print(f"{entry['title']}: {e}") if self._stopped.wait(30): return continue if r['not_modified']: wait = self.cfg['regen_retry_s'][min(retry, len(self.cfg['regen_retry_s']) - 1)] retry += 1 else: self._store_and_notify(entry, r) last_modified = r['last_modified'] retry = 0 self._persist() j = self.cfg['jitter_percent'] wait = round(entry['refresh_seconds'] * (1 + (random.random() - 0.5) * 2 * j)) if self._stopped.wait(wait): return def _store_and_notify(self, entry, r): with self._lock: self._cache[entry['url']] = {**r, 'types': entry['types'], 'fetched_at': time.time()} for t in entry['types']: self._notify(t) def _notify(self, type_): with self._lock: cbs = list(self._subs.get(type_, [])) items = self.get(type_) for cb in cbs: cb(items) # === Catalog helpers === def _catalog_configs(self): if self.cfg.get('catalogs'): return self.cfg['catalogs'] # explicit override wins if self.default_catalogs: return self.default_catalogs # JSON's catalogs (recommended path) tf0 = (self.cfg.get('timeframes') or ['today'])[0] # last resort: auto-build out = [] for t in self.cfg['supported_types']: if t == 'dvd': out.append({'id': 'dvd', 'source': 'dvd'}) else: out.append({'id': f'{t}_{tf0}', 'source': t, 'timeframe': tf0}) return out def _file_title_for(self, source, timeframe=None): size_key = f"top_{self.cfg['size']}" if source == 'dvd': file = (self.urls.get('dvd') or {}).get(size_key) else: file = ((self.urls.get(source) or {}).get(timeframe) or {}).get(size_key) return (file or {}).get('title', f'Trending {source} on Simkl') def _build_catalog(self, c): if not c.get('id') or not c.get('source'): raise SimklTrendingError("catalogs[] entries need 'id' and 'source'") items = self.get(c['source']) transform = c.get('transform') recipe = c.get('recipe') if callable(transform): out = transform(items) elif recipe: # JSON rows name a recipe fn = SIMKL_TRENDING_RECIPES.get(recipe) if fn: out = fn(items, {'type': c['source'], **(c.get('recipe_opts') or {})}) else: # never fall back to the unfiltered feed out = [] msg = f"SimklTrendingClient: recipe {recipe!r} (catalog {c['id']!r}) isn't in this port — row left empty." if msg not in self._warned: self._warned.add(msg); print(msg) else: out = items title = c.get('title') or self._file_title_for(c['source'], c.get('timeframe') or (self.cfg.get('timeframes') or ['today'])[0]) return {'id': c['id'], 'title': title, 'items': out} # Persistence — swap these two methods for a Redis / KV adapter in server-side runtimes. def _persist(self): if not self.cfg['persist_path']: return try: with self._lock: dump = json.dumps(self._cache) tmp = self.cfg['persist_path'] + '.tmp' with open(tmp, 'w') as f: f.write(dump) os.replace(tmp, self.cfg['persist_path']) except Exception: pass # best-effort def _restore_persisted(self): if not self.cfg['persist_path'] or not os.path.exists(self.cfg['persist_path']): return try: with open(self.cfg['persist_path']) as f: raw = json.load(f) cutoff = time.time() - self.cfg['persist_max_age_s'] with self._lock: for url, val in raw.items(): if val.get('fetched_at', 0) >= cutoff: self._cache[url] = val except Exception: pass # no cache yet — fine# === Run ===client = SimklTrendingClient()client.subscribe_catalogs(lambda rows: [print(f"{r['title']}: {len(r['items'])} items") for r in rows])client.start()# import signal; signal.signal(signal.SIGINT, lambda *_: (client.stop(), exit(0)))# === Background scheduling (server-side) ===# For long-running processes, the in-process threads are enough. For cron-style runtimes# (Celery beat, APScheduler, GitHub Actions), wire `client.tick()` to your scheduler instead of start().
import Foundation// === Load the URL catalog ===// 1. Save the "Copy-paste JSON" block above as `simkl-trending-urls.json` in your app bundle (e.g.// drag it into Xcode and tick "Copy items if needed" + your app target's "Copy Bundle Resources").// 2. Load it into a global `SIMKL_TRENDING_URLS: [String: Any]` (and pass it to the constructor, or// let the actor pick it up from the global on first use):// let url = Bundle.main.url(forResource: "simkl-trending-urls", withExtension: "json")!// let data = try Data(contentsOf: url)// let SIMKL_TRENDING_URLS = try JSONSerialization.jsonObject(with: data) as! [String: Any]// SimklTrendingClient(config:, trending: SIMKL_TRENDING_URLS) — fatalErrors if neither global nor arg is set.// === CONFIG — tune to match your app ===struct SimklTrendingConfig { // What to load var supportedTypes: [String] = ["movies", "tv", "anime", "dvd"] // movies / tv / anime / dvd — drop ones your app doesn't surface var timeframes: [String] = ["today"] // today / week / month (ignored for dvd) var size: Int = 100 // 100 or 500 — use 500 if you render derived catalogs (see "Build your own catalogs" below) // HTTP — replace with YOUR values var clientId: String = "YOUR_CLIENT_ID" var appName: String = "my-app-name" var appVersion: String = "1.0" var userAgent: String = "my-app-name/1.0" var requestTimeout: TimeInterval = 8 var minIntervalMs: Int = 250 // process-wide floor between consecutive requests var fetchMaxAttempts: Int = 3 var fetchBaseBackoff: TimeInterval = 1 var fetchMaxBackoff: TimeInterval = 16 var retryStatuses: Set<Int> = [408, 425, 429, 500, 502, 503, 504] // never retry other 4xx — they're permanent // Refresh var regenRetrySeconds: [Int] = [300, 900, 1800] // 5 / 15 / 30 min after a 304 var jitterPercent: Double = 0.05 // ±5% // Persistent cache — set persistFile to nil to disable var persistFile: URL? = FileManager.default .urls(for: .cachesDirectory, in: .userDomainMask).first?.appendingPathComponent("simkl-trending-cache.json") var persistMaxAgeSeconds: TimeInterval = 7 * 86_400 // Display — applied when items are read for display; the cached files stay exactly as served. var excludeVideoGamePlay: Bool = true // hide titles classified "Video Game Play" (gameplay / Let's Play videos) // Catalogs — the rows on your home screen. Nil = the JSON's catalogs[] (or one row per supportedType). var catalogs: [SimklTrendingCatalogConfig]? = nil}struct SimklTrendingCatalogConfig { let id: String // unique row key let source: String // 'movies' | 'tv' | 'anime' | 'dvd' var timeframe: String? = nil // defaults to cfg.timeframes[0] var title: String? = nil // defaults to JSON's pre-composed title var transform: (([[String: Any]]) -> [[String: Any]])? = nil var recipe: String? = nil // a SimklTrendingRecipes name (JSON rows) var recipeOpts: [String: Any]? = nil}struct SimklTrendingCatalog { let id: String let title: String let items: [[String: Any]]}struct SimklTrendingError: Error { let message: String let status: Int? let retryAfter: TimeInterval? let permanent: Bool var localizedDescription: String { message }}struct SimklTrendingEntry: Codable { let title: String, url: String let refreshSeconds: Int let types: [String]}// === Recipes — named by the JSON's catalogs[]. This port ships wellRated (plus its deprecated// alias hiddenGems) and underseenGems. A row naming any other recipe stays EMPTY until you port it// (see "Catalog ideas" below) or give the row a `transform` — never the unfiltered feed.// Recipes never modify what they're given; a short or empty result is valid.enum SimklTrendingRecipes { typealias Item = [String: Any] static func run(_ name: String, _ items: [Item], _ opts: [String: Any]) -> [Item]? { switch name { case "wellRated", "hiddenGems": return wellRated(items, opts) // hiddenGems: DEPRECATED old name case "underseenGems": return underseenGems(items, opts) default: return nil } } // Well-rated titles outside the top `minRank` of Simkl's rank (a rating-based order, not a viewer // count) — so this still holds widely seen titles. Formerly hiddenGems; same selection. static func wellRated(_ items: [Item], _ opts: [String: Any] = [:]) -> [Item] { let minRating = number(opts["minRating"]) ?? 7.5, minVotes = number(opts["minVotes"]) ?? 500 let minRank = number(opts["minRank"]) ?? 2000 let source = opts["source"] as? String ?? "simkl" func rating(_ i: Item) -> Double { number(ext(i, source)?["rating"]) ?? 0 } let picked = items.filter { i in (number(i["rank"]) ?? 0) > minRank && rating(i) >= minRating && (number(ext(i, source)?["votes"]) ?? 0) >= minVotes } return picked.enumerated().sorted { a, b in // stable: ties keep feed order rating(a.element) != rating(b.element) ? rating(a.element) > rating(b.element) : a.offset < b.offset }.map { $0.element } } // Per-type defaults for underseenGems; override any of them with opts / recipe_opts. static let underseenDefaults: [String: [String: Any]] = [ "movies": ["source": "imdb", "minRating": 7.0, "minVotes": 500, "maxVotes": 25_000], "tv": ["source": "imdb", "minRating": 7.5, "minVotes": 500, "maxVotes": 10_000, "excludeGenres": ["Sport", "Sports", "Reality", "Talk", "Talk Show", "News"]], "anime": ["source": "mal", "minRating": 7.5, "minVotes": 500, "maxVotes": 10_000], ] static let underseenCommon: [String: Any] = ["maxSimklVotes": 500, "minAgeDays": 365] // maxTrackedAudience: off unless set // Well-rated titles with a smaller audience, by external rating count — a heuristic, not proof a // title is obscure worldwide. Needs opts["type"] ("movies" | "tv" | "anime"); JSON rows pass their source. static func underseenGems(_ items: [Item], _ opts: [String: Any] = [:], now: Date = Date()) -> [Item] { guard let type = opts["type"] as? String, let base = underseenDefaults[type] else { return [] } let o = underseenCommon.merging(base) { $1 }.merging(opts) { $1 } guard let minRating = number(o["minRating"]), let minVotes = number(o["minVotes"]), let maxVotes = number(o["maxVotes"]), let maxSimklVotes = number(o["maxSimklVotes"]), let minAgeDays = number(o["minAgeDays"]), let source = o["source"] as? String else { return [] } let capRaw = o["maxTrackedAudience"] let cap = (capRaw == nil || capRaw is NSNull) ? nil : number(capRaw) if capRaw != nil && !(capRaw is NSNull) && cap == nil { return [] } let latest = now.addingTimeInterval(-minAgeDays * 86_400) // released on or before this let skip = Set(((o["excludeGenres"] as? [String]) ?? []).map(normGenre)) var picked: [(item: Item, rating: Double, votes: Int, order: Int)] = [] for (n, i) in items.enumerated() { if (i["status"] as? String) == "premiere" { continue } guard let released = parseReleaseDate(i["release_date"]), released <= latest else { continue } // missing, invalid, future or too recent guard let rating = number(ext(i, source)?["rating"]), let votes = count(ext(i, source)?["votes"]) else { continue } if rating < minRating || Double(votes) < minVotes || Double(votes) > maxVotes { continue } guard let simklVotes = count(ext(i, "simkl")?["votes"]), Double(simklVotes) <= maxSimklVotes else { continue } if !skip.isEmpty, let genres = i["genres"] as? [Any], genres.contains(where: { skip.contains(normGenre("\($0)")) }) { continue } if let cap = cap { // optional: watching + completed guard let watching = count(i["watching"]), let completed = count(i["completed"]) else { continue } // missing is not zero if Double(watching + completed) > cap { continue } } picked.append((i, rating, votes, n)) } return picked.sorted { a, b in if a.rating != b.rating { return a.rating > b.rating } // rating desc if a.votes != b.votes { return a.votes < b.votes } // then rating count asc return a.order < b.order }.map { $0.item } } // The "Video Game Play" classification, matched exactly after normalising case, spaces, - and _. // Also matches a genre-list entry (slug "video-game-play"). Never matches title words or networks. static func isVideoGamePlay(_ e: Item) -> Bool { if let genres = e["genres"] as? [Any], genres.contains(where: { normGenre("\($0)") == "video game play" }) { return true } return [e["slug"], e["name"]].contains { ($0 as? String).map(normGenre) == "video game play" } } static func excludeVideoGamePlay(_ items: [Item]) -> [Item] { items.filter { !isVideoGamePlay($0) } } // `release_date` is documented as MM/DD/YYYY — parse it explicitly, in UTC. Anything else → nil. static func parseReleaseDate(_ v: Any?) -> Date? { guard let s = v as? String else { return nil } let p = s.split(separator: "/", omittingEmptySubsequences: false) guard p.count == 3, p[0].count == 2, p[1].count == 2, p[2].count == 4, p.allSatisfy({ $0.allSatisfy { $0 >= "0" && $0 <= "9" } }), let mo = Int(p[0]), let d = Int(p[1]), let y = Int(p[2]) else { return nil } var cal = Calendar(identifier: .gregorian) cal.timeZone = TimeZone(identifier: "UTC")! guard let date = cal.date(from: DateComponents(year: y, month: mo, day: d)) else { return nil } let back = cal.dateComponents([.year, .month, .day], from: date) return (back.year == y && back.month == mo && back.day == d) ? date : nil // rejects 02/30 etc. } static func normGenre(_ g: String) -> String { g.lowercased().split(whereSeparator: { $0 == " " || $0 == "-" || $0 == "_" || $0.isWhitespace }).joined(separator: " ") } static func ext(_ i: Item, _ source: String) -> [String: Any]? { (i["ratings"] as? [String: Any])?[source] as? [String: Any] } static func number(_ v: Any?) -> Double? { if let i = v as? Int { return Double(i) } if let d = v as? Double, d.isFinite { return d } return nil } static func count(_ v: Any?) -> Int? { // valid non-negative integer, else nil guard let d = number(v), d >= 0, d == d.rounded(), d < 9e15 else { return nil } return Int(d) }}actor SimklTrendingClient { private var cfg: SimklTrendingConfig private let trending: [String: Any] private var cache: [String: _SimklCached] = [:] private var inflight: [String: Task<SimklTrendingFetchResult, Error>] = [:] private var subs: [String: [UUID: ([[String: Any]]) -> Void]] = [:] private var loops: [Task<Void, Never>] = [] private var stopped: Bool = false private var lastRequestAt: Date? = nil // for minIntervalMs floor private var warned: Set<String> = [] struct _SimklCached { var itemsByType: [String: [[String: Any]]]; var lastModified: String?; var types: [String]; var fetchedAt: Date } struct SimklTrendingFetchResult { let itemsByType: [String: [[String: Any]]]?; let lastModified: String?; let notModified: Bool } private var defaultCatalogs: [[String: Any]]? = nil init(config: SimklTrendingConfig = SimklTrendingConfig(), trending: [String: Any]? = nil) { self.cfg = config let data = trending ?? (SIMKL_TRENDING_URLS as [String: Any]) // Accept both shapes: { urls: {...}, catalogs: [...] } from JSON OR the legacy unwrapped { combined, movies, ... }. self.trending = (data["urls"] != nil) ? data : ["urls": data] self.defaultCatalogs = data["catalogs"] as? [[String: Any]] } // === Public API === func start() async { restorePersisted() for t in subs.keys { notify(type: t) } // paint immediately from restored cache for entry in plan() { let task = Task { await loop(entry) } loops.append(task) } } func stop() { stopped = true for task in loops { task.cancel() } for (_, t) in inflight { t.cancel() } loops.removeAll(); inflight.removeAll() } // Wipe in-memory + persistent cache and notify subscribers (so UI repaints empty). // Call client.tick() after this to immediately re-fetch. func clearCache() { cache.removeAll() if let url = cfg.persistFile { try? FileManager.default.removeItem(at: url) } for (t, _) in subs { notify(type: t) } } // Items to display for a type. Display filters (excludeVideoGamePlay) apply here, on read, // so charts, derived rows and a cache restored offline all show the same selection. func get(_ type: String) -> [[String: Any]] { for v in cache.values { if let arr = v.itemsByType[type] { return cfg.excludeVideoGamePlay ? SimklTrendingRecipes.excludeVideoGamePlay(arr) : arr } } return [] } func subscribe(_ type: String, _ cb: @escaping ([[String: Any]]) -> Void) -> UUID { let id = UUID() subs[type, default: [:]][id] = cb let items = get(type); if !items.isEmpty { cb(items) } // immediate paint from cache return id } func unsubscribe(_ type: String, _ id: UUID) { subs[type]?.removeValue(forKey: id) } // === Catalogs — rows on the home screen === func catalogs() -> [SimklTrendingCatalog] { return catalogConfigs().map { buildCatalog($0) } } func subscribeCatalogs(_ cb: @escaping ([SimklTrendingCatalog]) -> Void) -> [UUID] { let sources = Set(catalogConfigs().map { $0.source }) var ids: [UUID] = [] for source in sources { ids.append(subscribe(source) { _ in cb(self.catalogs()) }) } if sources.allSatisfy({ get($0).isEmpty }) { cb(catalogs()) } // empty cache — skeleton paint return ids } private func catalogConfigs() -> [SimklTrendingCatalogConfig] { if let cs = cfg.catalogs { return cs } // explicit override wins if let raw = defaultCatalogs { // JSON's catalogs[] (recommended) return raw.compactMap { d in guard let id = d["id"] as? String, let src = d["source"] as? String else { return nil } return SimklTrendingCatalogConfig(id: id, source: src, timeframe: d["timeframe"] as? String, title: d["title"] as? String, transform: nil, recipe: d["recipe"] as? String, recipeOpts: d["recipe_opts"] as? [String: Any]) } } let tf0 = cfg.timeframes.first ?? "today" // last resort: auto-build return cfg.supportedTypes.map { t in t == "dvd" ? SimklTrendingCatalogConfig(id: "dvd", source: "dvd") : SimklTrendingCatalogConfig(id: "\(t)_\(tf0)", source: t, timeframe: tf0) } } private func fileTitleFor(_ source: String, _ timeframe: String?) -> String { let sizeKey = "top_\(cfg.size)" let urls = trending["urls"] as? [String: Any] ?? [:] let file: [String: Any]? if source == "dvd" { file = (urls["dvd"] as? [String: Any])?[sizeKey] as? [String: Any] } else { let bucket = (urls[source] as? [String: Any])?[timeframe ?? "today"] as? [String: Any] file = bucket?[sizeKey] as? [String: Any] } return (file?["title"] as? String) ?? "Trending \(source) on Simkl" } private func buildCatalog(_ c: SimklTrendingCatalogConfig) -> SimklTrendingCatalog { let items = get(c.source) var out = items if let transform = c.transform { out = transform(items) } else if let recipe = c.recipe { // JSON rows name a recipe var opts = c.recipeOpts ?? [:] if opts["type"] == nil { opts["type"] = c.source } if let rows = SimklTrendingRecipes.run(recipe, items, opts) { out = rows } else { // never fall back to the unfiltered feed out = [] let msg = "SimklTrendingClient: recipe \"\(recipe)\" (catalog \"\(c.id)\") isn't in this port — row left empty." if !warned.contains(msg) { warned.insert(msg); print(msg) } } } let title = c.title ?? fileTitleFor(c.source, c.timeframe ?? cfg.timeframes.first) return SimklTrendingCatalog(id: c.id, title: title, items: out) } // Call from BGTaskScheduler for a single revalidate-and-persist pass. func tick() async { restorePersisted() for entry in plan() { let lm = cache[entry.url]?.lastModified if let r = try? await fetchDedupd(entry, lastModified: lm), !r.notModified, r.itemsByType != nil { storeAndNotify(entry: entry, result: r) } } persist() } // === Internals === private func requiredQuery() -> String { let an = cfg.appName.addingPercentEncoding(withAllowedCharacters: .urlQueryAllowed) ?? cfg.appName let av = cfg.appVersion.addingPercentEncoding(withAllowedCharacters: .urlQueryAllowed) ?? cfg.appVersion return "client_id=\(cfg.clientId)&app-name=\(an)&app-version=\(av)" } private func plan() -> [SimklTrendingEntry] { precondition(cfg.size == 100 || cfg.size == 500, "size must be 100 or 500") let key = "top_\(cfg.size)" let urls = trending["urls"] as! [String: Any] let tvma = cfg.supportedTypes.filter { ["movies","tv","anime"].contains($0) } let useCombined = tvma.count >= 2 // 1 round-trip < 2-3 per timeframe func make(_ raw: Any, types: [String]) -> SimklTrendingEntry { let d = raw as! [String: Any] return SimklTrendingEntry(title: d["title"] as! String, url: d["url"] as! String, refreshSeconds: d["refresh_seconds"] as! Int, types: types) } var out: [SimklTrendingEntry] = [] if !tvma.isEmpty { for tf in cfg.timeframes { let bucket = (useCombined ? urls["combined"] : urls[tvma[0]]) as! [String: Any] out.append(make((bucket[tf] as! [String: Any])[key]!, types: tvma)) } } if cfg.supportedTypes.contains("dvd") { out.append(make((urls["dvd"] as! [String: Any])[key]!, types: ["dvd"])) } return out } // In-flight dedup: actor-serialized lookup of inflight tasks. private func fetchDedupd(_ entry: SimklTrendingEntry, lastModified: String?) async throws -> SimklTrendingFetchResult { let sep = entry.url.contains("?") ? "&" : "?" let url = entry.url + sep + requiredQuery() if let t = inflight[url] { return try await t.value } let task = Task<SimklTrendingFetchResult, Error> { try await self.runFetch(url: url, entry: entry, lastModified: lastModified) } inflight[url] = task defer { inflight[url] = nil } return try await task.value } // Process-wide rate-limit floor (so N entries refreshing at once don't fire N simultaneous requests). private func throttle() async { if let last = lastRequestAt { let gapMs = Date().timeIntervalSince(last) * 1_000 let floor = Double(cfg.minIntervalMs) if gapMs < floor { try? await Task.sleep(nanoseconds: UInt64((floor - gapMs) * 1_000_000)) } } lastRequestAt = Date() } private static func parseRetryAfter(_ value: String?) -> TimeInterval? { guard let v = value else { return nil } if let s = TimeInterval(v) { return max(0, s) } let f = DateFormatter(); f.dateFormat = "EEE, dd MMM yyyy HH:mm:ss zzz"; f.locale = Locale(identifier: "en_US_POSIX") if let d = f.date(from: v) { return max(0, d.timeIntervalSinceNow) } return nil } // runFetch — throttle + retry with exponential backoff + Retry-After + permanent-error classify. Normalizes both response shapes. private func runFetch(url: String, entry: SimklTrendingEntry, lastModified: String?) async throws -> SimklTrendingFetchResult { var lastErr: Error? for attempt in 0..<cfg.fetchMaxAttempts { await throttle() var req = URLRequest(url: URL(string: url)!, timeoutInterval: cfg.requestTimeout) req.setValue(cfg.userAgent, forHTTPHeaderField: "User-Agent") req.setValue("application/json", forHTTPHeaderField: "Accept") if let lm = lastModified { req.setValue(lm, forHTTPHeaderField: "If-Modified-Since") } do { let (data, response) = try await URLSession.shared.data(for: req) let http = response as! HTTPURLResponse if http.statusCode == 304 { return SimklTrendingFetchResult(itemsByType: nil, lastModified: lastModified, notModified: true) } guard 200..<300 ~= http.statusCode else { let retryAfter = Self.parseRetryAfter(http.value(forHTTPHeaderField: "Retry-After")) let permanent = !cfg.retryStatuses.contains(http.statusCode) throw SimklTrendingError(message: "\(http.statusCode) — \(entry.title)", status: http.statusCode, retryAfter: retryAfter, permanent: permanent) } let raw = try JSONSerialization.jsonObject(with: data) var ibt: [String: [[String: Any]]] = [:] if let arr = raw as? [[String: Any]] { ibt[entry.types[0]] = arr } // per-type / dvd → array else if let obj = raw as? [String: Any] { // combined → object, filter to wanted types for t in entry.types { ibt[t] = (obj[t] as? [[String: Any]]) ?? [] } } return SimklTrendingFetchResult(itemsByType: ibt, lastModified: http.value(forHTTPHeaderField: "Last-Modified"), notModified: false) } catch let e as SimklTrendingError where e.permanent { throw e // don't waste backoff on permanent errors } catch { if Task.isCancelled || stopped { throw error } lastErr = error let honored = (error as? SimklTrendingError)?.retryAfter let backoff = honored ?? min(cfg.fetchBaseBackoff * pow(2, Double(attempt)), cfg.fetchMaxBackoff) let jitter = backoff * (Double.random(in: 0..<1) * 0.2 - 0.1) // ±10% jitter try? await Task.sleep(nanoseconds: UInt64(max(0, backoff + jitter) * 1_000_000_000)) } } throw lastErr ?? SimklTrendingError(message: "fetch failed", status: nil, retryAfter: nil, permanent: false) } private func loop(_ entry: SimklTrendingEntry) async { var lastModified = cache[entry.url]?.lastModified var retry = 0 while !stopped && !Task.isCancelled { let r: SimklTrendingFetchResult do { r = try await fetchDedupd(entry, lastModified: lastModified) } catch let e as SimklTrendingError where e.permanent { // 404 / 401 / etc — halt this entry's loop print("\(entry.title): \(e.message) (permanent, halting loop)"); return } catch { if stopped { return } try? await Task.sleep(nanoseconds: 30 * 1_000_000_000); continue } let wait: Int if r.notModified { wait = cfg.regenRetrySeconds[min(retry, cfg.regenRetrySeconds.count - 1)]; retry += 1 } else if r.itemsByType != nil { storeAndNotify(entry: entry, result: r) lastModified = r.lastModified; retry = 0 persist() wait = Int(Double(entry.refreshSeconds) * (1 + (Double.random(in: 0..<1) - 0.5) * 2 * cfg.jitterPercent)) } else { wait = 30 } try? await Task.sleep(nanoseconds: UInt64(wait) * 1_000_000_000) } } private func storeAndNotify(entry: SimklTrendingEntry, result r: SimklTrendingFetchResult) { cache[entry.url] = _SimklCached(itemsByType: r.itemsByType ?? [:], lastModified: r.lastModified, types: entry.types, fetchedAt: Date()) for t in entry.types { notify(type: t) } } private func notify(type: String) { let items = get(type) for cb in (subs[type] ?? [:]).values { cb(items) } } private func persist() { guard let url = cfg.persistFile else { return } var dump: [String: [String: Any]] = [:] for (k, v) in cache { dump[k] = ["itemsByType": v.itemsByType, "lastModified": v.lastModified as Any, "types": v.types, "fetchedAt": v.fetchedAt.timeIntervalSince1970] } if let data = try? JSONSerialization.data(withJSONObject: dump) { try? data.write(to: url, options: .atomic) } } private func restorePersisted() { guard let url = cfg.persistFile, let data = try? Data(contentsOf: url), let raw = (try? JSONSerialization.jsonObject(with: data)) as? [String: [String: Any]] else { return } let cutoff = Date().addingTimeInterval(-cfg.persistMaxAgeSeconds) for (k, v) in raw { guard let fa = v["fetchedAt"] as? TimeInterval, Date(timeIntervalSince1970: fa) >= cutoff, let ibt = v["itemsByType"] as? [String: [[String: Any]]], let types = v["types"] as? [String] else { continue } cache[k] = _SimklCached(itemsByType: ibt, lastModified: v["lastModified"] as? String, types: types, fetchedAt: Date(timeIntervalSince1970: fa)) } }}// === Run ===Task { let client = SimklTrendingClient() _ = await client.subscribeCatalogs { rows in for row in rows { print("\(row.title): \(row.items.count) items") } } await client.start()}// === Background scheduling (iOS) ===// Register a BGAppRefreshTask in your AppDelegate. In the handler call `await client.tick()`// to perform a single revalidate-and-persist pass — Task.sleep timers do not fire while the// app is suspended, so BGTaskScheduler is what keeps the cache warm in the background.
import kotlinx.coroutines.*import kotlinx.coroutines.sync.Muteximport kotlinx.coroutines.sync.withLockimport okhttp3.OkHttpClientimport okhttp3.Requestimport org.json.JSONArrayimport org.json.JSONObjectimport java.io.Fileimport java.net.URLEncoderimport java.util.concurrent.TimeUnitimport kotlin.random.Randomimport kotlin.math.pow// === Load the URL catalog ===// 1. Save the "Copy-paste JSON" block above as `simkl-trending-urls.json` in `app/src/main/assets/`.// 2. Load it into a global `SIMKL_TRENDING_URLS: JSONObject` once at app startup// (e.g. in your Application.onCreate or DI module):// val SIMKL_TRENDING_URLS = JSONObject(// context.assets.open("simkl-trending-urls.json").bufferedReader().use { it.readText() }// )// You can also skip the global and pass it directly: `SimklTrendingClient(cfg, trending = urls)`.// The constructor throws if neither the global nor the `trending` arg is provided.// === CONFIG — tune to match your app ===data class SimklTrendingConfig( // What to load val supportedTypes: List<String> = listOf("movies", "tv", "anime", "dvd"), // movies / tv / anime / dvd — drop ones your app doesn't surface val timeframes: List<String> = listOf("today"), // today / week / month (ignored for dvd) val size: Int = 100, // 100 or 500 — use 500 if you render derived catalogs (see "Build your own catalogs" below) // HTTP — replace with YOUR values val clientId: String = "YOUR_CLIENT_ID", val appName: String = "my-app-name", val appVersion: String = "1.0", val userAgent: String = "my-app-name/1.0", val requestTimeoutMs: Long = 8_000, val minIntervalMs: Long = 250, // process-wide floor between consecutive requests val fetchMaxAttempts: Int = 3, val fetchBaseBackoffMs: Long = 1_000, val fetchMaxBackoffMs: Long = 16_000, val retryStatuses: Set<Int> = setOf(408, 425, 429, 500, 502, 503, 504), // never retry other 4xx — they're permanent // Refresh val regenRetrySeconds: List<Int> = listOf(300, 900, 1_800), // 5 / 15 / 30 min after a 304 val jitterPercent: Double = 0.05, // ±5% // Persistent cache — set persistFile to null to disable. In Android pass context.cacheDir. val persistFile: File? = null, // e.g. File(context.cacheDir, "simkl-trending-cache.json") val persistMaxAgeMs: Long = 7L * 86_400 * 1_000, // Display — applied when items are read for display; the cached files stay exactly as served. val excludeVideoGamePlay: Boolean = true, // hide titles classified "Video Game Play" (gameplay / Let's Play videos) // Catalogs — the rows on your home screen. Null = the JSON's catalogs[] (or one row per supportedType). val catalogs: List<SimklTrendingCatalogConfig>? = null,)data class SimklTrendingCatalogConfig( val id: String, val source: String, // "movies" | "tv" | "anime" | "dvd" val timeframe: String? = null, // defaults to cfg.timeframes[0] val title: String? = null, // defaults to JSON's pre-composed title val transform: ((List<JSONObject>) -> List<JSONObject>)? = null, val recipe: String? = null, // a SimklTrendingRecipes name (JSON rows) val recipeOpts: Map<String, Any?>? = null,)data class SimklTrendingCatalog(val id: String, val title: String, val items: List<JSONObject>)data class SimklTrendingEntry(val title: String, val url: String, val refreshSeconds: Int, val types: List<String>)class SimklTrendingError(message: String, val status: Int? = null, val retryAfterMs: Long? = null, val permanent: Boolean = false) : RuntimeException(message)// === Recipes — named by the JSON's catalogs[]. This port ships wellRated (plus its deprecated// alias hiddenGems) and underseenGems. A row naming any other recipe stays EMPTY until you port it// (see "Catalog ideas" below) or give the row a `transform` — never the unfiltered feed.// Recipes never modify what they're given; a short or empty result is valid.object SimklTrendingRecipes { fun run(name: String, items: List<JSONObject>, opts: Map<String, Any?>): List<JSONObject>? = when (name) { "wellRated", "hiddenGems" -> wellRated(items, opts) // hiddenGems: DEPRECATED old name "underseenGems" -> underseenGems(items, opts) else -> null } // Well-rated titles outside the top `minRank` of Simkl's rank (a rating-based order, not a viewer // count) — so this still holds widely seen titles. Formerly hiddenGems; same selection. fun wellRated(items: List<JSONObject>, opts: Map<String, Any?> = emptyMap()): List<JSONObject> { val minRating = number(opts["minRating"]) ?: 7.5 val minVotes = number(opts["minVotes"]) ?: 500.0 val minRank = number(opts["minRank"]) ?: 2000.0 val source = opts["source"] as? String ?: "simkl" fun rating(i: JSONObject) = number(ext(i, source)?.opt("rating")) ?: 0.0 return items.filter { (number(it.opt("rank")) ?: 0.0) > minRank && rating(it) >= minRating && (number(ext(it, source)?.opt("votes")) ?: 0.0) >= minVotes }.sortedByDescending { rating(it) } // stable: ties keep feed order } // Per-type defaults for underseenGems; override any of them with opts / recipe_opts. val UNDERSEEN_DEFAULTS: Map<String, Map<String, Any?>> = mapOf( "movies" to mapOf("source" to "imdb", "minRating" to 7.0, "minVotes" to 500, "maxVotes" to 25_000), "tv" to mapOf("source" to "imdb", "minRating" to 7.5, "minVotes" to 500, "maxVotes" to 10_000, "excludeGenres" to listOf("Sport", "Sports", "Reality", "Talk", "Talk Show", "News")), "anime" to mapOf("source" to "mal", "minRating" to 7.5, "minVotes" to 500, "maxVotes" to 10_000), ) val UNDERSEEN_COMMON: Map<String, Any?> = mapOf( "maxSimklVotes" to 500, "minAgeDays" to 365, "maxTrackedAudience" to null, "excludeGenres" to emptyList<String>()) // Well-rated titles with a smaller audience, by external rating count — a heuristic, not proof a // title is obscure worldwide. Needs opts["type"] ("movies" | "tv" | "anime"); JSON rows pass their source. fun underseenGems(items: List<JSONObject>, opts: Map<String, Any?>, nowMs: Long = System.currentTimeMillis()): List<JSONObject> { val base = (opts["type"] as? String)?.let { UNDERSEEN_DEFAULTS[it] } ?: return emptyList() val o = UNDERSEEN_COMMON + base + opts val minRating = number(o["minRating"]) ?: return emptyList() val minVotes = number(o["minVotes"]) ?: return emptyList() val maxVotes = number(o["maxVotes"]) ?: return emptyList() val maxSimklVotes = number(o["maxSimklVotes"]) ?: return emptyList() val minAgeDays = number(o["minAgeDays"]) ?: return emptyList() val source = o["source"] as? String ?: return emptyList() val capRaw = o["maxTrackedAudience"] val cap = if (capRaw == null) null else (number(capRaw) ?: return emptyList()) val latest = nowMs - (minAgeDays * 86_400_000).toLong() // released on or before this val skip = ((o["excludeGenres"] as? List<*>) ?: emptyList<Any>()).map { normGenre(it.toString()) }.toSet() data class Pick(val item: JSONObject, val rating: Double, val votes: Long) val picked = mutableListOf<Pick>() for (i in items) { if (i.optString("status") == "premiere") continue val released = parseReleaseDate(i.opt("release_date")) ?: continue // missing or invalid if (released > latest) continue // future or too recent val rating = number(ext(i, source)?.opt("rating")) ?: continue val votes = count(ext(i, source)?.opt("votes")) ?: continue if (rating < minRating || votes < minVotes || votes > maxVotes) continue val simklVotes = count(ext(i, "simkl")?.opt("votes")) ?: continue if (simklVotes > maxSimklVotes) continue val genres = i.optJSONArray("genres") if (skip.isNotEmpty() && genres != null && (0 until genres.length()).any { normGenre(genres.opt(it).toString()) in skip }) continue if (cap != null) { // optional: watching + completed val watching = count(i.opt("watching")) ?: continue // missing is not zero val completed = count(i.opt("completed")) ?: continue if (watching + completed > cap) continue } picked.add(Pick(i, rating, votes)) } return picked.sortedWith(compareByDescending<Pick> { it.rating }.thenBy { it.votes }).map { it.item } } // The "Video Game Play" classification, matched exactly after normalising case, spaces, - and _. // Also matches a genre-list entry (slug "video-game-play"). Never matches title words or networks. fun isVideoGamePlay(e: JSONObject): Boolean { val g = e.optJSONArray("genres") if (g != null && (0 until g.length()).any { normGenre(g.opt(it).toString()) == "video game play" }) return true return listOf(e.opt("slug"), e.opt("name")).any { it is String && normGenre(it) == "video game play" } } fun excludeVideoGamePlay(items: List<JSONObject>): List<JSONObject> = items.filter { !isVideoGamePlay(it) } // `release_date` is documented as MM/DD/YYYY — parse it explicitly, in UTC (epoch ms). Anything else → null. fun parseReleaseDate(v: Any?): Long? { val s = v as? String ?: return null if (!Regex("[0-9]{2}/[0-9]{2}/[0-9]{4}").matches(s)) return null return runCatching { java.time.LocalDate.of(s.substring(6, 10).toInt(), s.substring(0, 2).toInt(), s.substring(3, 5).toInt()) .atStartOfDay(java.time.ZoneOffset.UTC).toInstant().toEpochMilli() }.getOrNull() // rejects 02/30 etc. } // JSON recipe_opts → Map (works with Android's built-in org.json, which has no toMap()). fun optsOf(o: JSONObject?): Map<String, Any?> = o?.keys()?.asSequence()?.associateWith { k -> when (val v = o.opt(k)) { JSONObject.NULL -> null; is JSONArray -> (0 until v.length()).map { v.opt(it) }; else -> v } } ?: emptyMap() fun normGenre(g: String) = g.lowercase().split(Regex("[\\s_-]+")).filter { it.isNotEmpty() }.joinToString(" ") private fun ext(i: JSONObject, source: String): JSONObject? = i.optJSONObject("ratings")?.optJSONObject(source) private fun number(v: Any?): Double? = (v as? Number)?.toDouble()?.takeIf { it.isFinite() } private fun count(v: Any?): Long? = number(v)?.takeIf { it >= 0 && it == Math.floor(it) && it < 9e15 }?.toLong()}class SimklTrendingClient( private val cfg: SimklTrendingConfig = SimklTrendingConfig(), data: JSONObject = SIMKL_TRENDING_URLS,) { // Accept both JSON shapes: { urls: {...}, catalogs: [...] } OR the legacy unwrapped { combined, movies, ... }. private val trending: JSONObject = if (data.has("urls")) data else JSONObject().put("urls", data) private val defaultCatalogs: JSONArray? = data.optJSONArray("catalogs") private data class _SimklCached(val itemsByType: Map<String, JSONArray>, val lastModified: String?, val types: List<String>, val fetchedAt: Long) data class SimklTrendingFetchResult(val itemsByType: Map<String, JSONArray>?, val lastModified: String?, val notModified: Boolean) private val http = OkHttpClient.Builder() .callTimeout(cfg.requestTimeoutMs, TimeUnit.MILLISECONDS).build() private val cache = mutableMapOf<String, _SimklCached>() private val inflight = mutableMapOf<String, Deferred<SimklTrendingFetchResult>>() private val subs = mutableMapOf<String, MutableMap<Int, (List<JSONObject>) -> Unit>>() private val warned = java.util.Collections.synchronizedSet(mutableSetOf<String>()) private var subIdSeq = 0 private val mu = Mutex() private val scope = CoroutineScope(SupervisorJob() + Dispatchers.IO) private var lastRequestAt = 0L // for minIntervalMs floor // === Public API === suspend fun start() { restorePersisted() mu.withLock { subs.keys.toList() }.forEach { notify(it) } // paint immediately from restored cache plan().forEach { entry -> scope.launch { loop(entry) } } } fun stop() { scope.cancel() } /** Wipe in-memory + persistent cache and notify subscribers (UI repaints empty). Call client.tick() after to re-fetch. */ suspend fun clearCache() { val snapshot = mu.withLock { cache.clear() subs.mapValues { it.value.values.toList() } } cfg.persistFile?.takeIf { it.exists() }?.delete() snapshot.forEach { (_, cbs) -> cbs.forEach { it(emptyList()) } } } // Items to display for a type. Display filters (excludeVideoGamePlay) apply here, on read, // so charts, derived rows and a cache restored offline all show the same selection. suspend fun get(type: String): List<JSONObject> = mu.withLock { for (v in cache.values) v.itemsByType[type]?.let { arr -> val items = (0 until arr.length()).mapNotNull { arr.optJSONObject(it) } return@withLock if (cfg.excludeVideoGamePlay) SimklTrendingRecipes.excludeVideoGamePlay(items) else items } emptyList() } suspend fun subscribe(type: String, cb: (List<JSONObject>) -> Unit): Int { val id = mu.withLock { val i = ++subIdSeq subs.getOrPut(type) { mutableMapOf() }[i] = cb i } val items = get(type); if (items.isNotEmpty()) cb(items) // immediate paint from cache return id } suspend fun unsubscribe(type: String, id: Int) { mu.withLock { subs[type]?.remove(id) } } // === Catalogs — rows on the home screen === suspend fun catalogs(): List<SimklTrendingCatalog> = catalogConfigs().map { buildCatalog(it) } suspend fun subscribeCatalogs(cb: (List<SimklTrendingCatalog>) -> Unit): List<Int> { val sources = catalogConfigs().map { it.source }.toSet() val ids = mutableListOf<Int>() for (s in sources) { ids.add(subscribe(s) { _ -> scope.launch { cb(catalogs()) } }) } if (sources.all { get(it).isEmpty() }) cb(catalogs()) // empty cache — skeleton paint return ids } private fun catalogConfigs(): List<SimklTrendingCatalogConfig> { cfg.catalogs?.let { return it } // explicit override wins defaultCatalogs?.let { arr -> // JSON's catalogs[] return (0 until arr.length()).mapNotNull { i -> val o = arr.optJSONObject(i) ?: return@mapNotNull null val id = o.optString("id"); val src = o.optString("source") if (id.isEmpty() || src.isEmpty()) null else SimklTrendingCatalogConfig(id = id, source = src, timeframe = if (o.isNull("timeframe")) null else o.optString("timeframe"), title = if (o.isNull("title")) null else o.optString("title"), transform = null, recipe = if (o.isNull("recipe")) null else o.optString("recipe"), recipeOpts = SimklTrendingRecipes.optsOf(o.optJSONObject("recipe_opts"))) } } val tf0 = cfg.timeframes.firstOrNull() ?: "today" // last resort: auto-build return cfg.supportedTypes.map { t -> if (t == "dvd") SimklTrendingCatalogConfig(id = "dvd", source = "dvd") else SimklTrendingCatalogConfig(id = "${t}_${tf0}", source = t, timeframe = tf0) } } private fun fileTitleFor(source: String, timeframe: String?): String { val sizeKey = "top_${cfg.size}" val urls = trending.optJSONObject("urls") ?: return "Trending $source on Simkl" val file = if (source == "dvd") urls.optJSONObject("dvd")?.optJSONObject(sizeKey) else urls.optJSONObject(source)?.optJSONObject(timeframe ?: "today")?.optJSONObject(sizeKey) return file?.optString("title") ?: "Trending $source on Simkl" } private suspend fun buildCatalog(c: SimklTrendingCatalogConfig): SimklTrendingCatalog { val items = get(c.source) val transform = c.transform val recipe = c.recipe val out = when { transform != null -> transform(items) recipe != null -> // JSON rows name a recipe SimklTrendingRecipes.run(recipe, items, mapOf<String, Any?>("type" to c.source) + (c.recipeOpts ?: emptyMap())) ?: emptyList<JSONObject>().also { // never fall back to the unfiltered feed val msg = "SimklTrendingClient: recipe \"$recipe\" (catalog \"${c.id}\") isn't in this port — row left empty." if (warned.add(msg)) println(msg) } else -> items } val title = c.title ?: fileTitleFor(c.source, c.timeframe ?: cfg.timeframes.firstOrNull()) return SimklTrendingCatalog(c.id, title, out) } // Call from a WorkManager PeriodicWorker for a single revalidate-and-persist pass. suspend fun tick() { restorePersisted() plan().forEach { entry -> val lm = mu.withLock { cache[entry.url]?.lastModified } runCatching { fetchDedupd(entry, lm) }.getOrNull()?.let { r -> if (!r.notModified && r.itemsByType != null) storeAndNotify(entry, r) } } persist() } // === Internals === private fun requiredQuery(): String = "client_id=${cfg.clientId}" + "&app-name=${URLEncoder.encode(cfg.appName, "UTF-8")}" + "&app-version=${URLEncoder.encode(cfg.appVersion, "UTF-8")}" private fun plan(): List<SimklTrendingEntry> { require(cfg.size == 100 || cfg.size == 500) { "size must be 100 or 500" } val key = "top_${cfg.size}" val urls = trending.getJSONObject("urls") val tvma = cfg.supportedTypes.filter { it in listOf("movies", "tv", "anime") } val useCombined = tvma.size >= 2 // 1 round-trip < 2-3 per timeframe fun make(o: JSONObject, types: List<String>) = SimklTrendingEntry(o.getString("title"), o.getString("url"), o.getInt("refresh_seconds"), types) val out = mutableListOf<SimklTrendingEntry>() if (tvma.isNotEmpty()) cfg.timeframes.forEach { tf -> val bucket = if (useCombined) urls.getJSONObject("combined") else urls.getJSONObject(tvma[0]) out.add(make(bucket.getJSONObject(tf).getJSONObject(key), tvma)) } if ("dvd" in cfg.supportedTypes) out.add(make(urls.getJSONObject("dvd").getJSONObject(key), listOf("dvd"))) return out } // In-flight dedup: only one outstanding network call per URL. private suspend fun fetchDedupd(entry: SimklTrendingEntry, lastModified: String?): SimklTrendingFetchResult { val sep = if (entry.url.contains('?')) "&" else "?" val url = entry.url + sep + requiredQuery() val existing = mu.withLock { inflight[url] } if (existing != null) return existing.await() val task = scope.async { runFetch(url, entry, lastModified) } mu.withLock { inflight[url] = task } try { return task.await() } finally { mu.withLock { inflight.remove(url) } } } // Process-wide rate-limit floor (so N entries refreshing at once don't fire N simultaneous requests). private suspend fun throttle() { val gap = System.currentTimeMillis() - lastRequestAt if (gap < cfg.minIntervalMs) delay(cfg.minIntervalMs - gap) lastRequestAt = System.currentTimeMillis() } private fun parseRetryAfter(value: String?): Long? { if (value == null) return null value.toLongOrNull()?.let { return maxOf(0L, it * 1000L) } return runCatching { val fmt = java.time.format.DateTimeFormatter.RFC_1123_DATE_TIME val dt = java.time.ZonedDateTime.parse(value, fmt) maxOf(0L, dt.toInstant().toEpochMilli() - System.currentTimeMillis()) }.getOrNull() } // runFetch — throttle + retry with exponential backoff + Retry-After + permanent-error classify. Normalizes both response shapes. private suspend fun runFetch(url: String, entry: SimklTrendingEntry, lastModified: String?): SimklTrendingFetchResult { var lastErr: Exception? = null for (attempt in 0 until cfg.fetchMaxAttempts) { throttle() try { val req = Request.Builder().url(url) .header("User-Agent", cfg.userAgent) .header("Accept", "application/json") .apply { lastModified?.let { header("If-Modified-Since", it) } }.build() http.newCall(req).execute().use { res -> if (res.code == 304) return SimklTrendingFetchResult(null, lastModified, notModified = true) if (!res.isSuccessful) { val retryAfter = parseRetryAfter(res.header("Retry-After")) val permanent = res.code !in cfg.retryStatuses throw SimklTrendingError("${res.code} ${res.message} — ${entry.title}", status = res.code, retryAfterMs = retryAfter, permanent = permanent) } val body = res.body!!.string().trim() val ibt = mutableMapOf<String, JSONArray>() if (body.startsWith("[")) ibt[entry.types[0]] = JSONArray(body) // per-type / dvd → array else { // combined → object, filter to wanted types val obj = JSONObject(body) entry.types.forEach { ibt[it] = obj.optJSONArray(it) ?: JSONArray() } } return SimklTrendingFetchResult(ibt, res.header("Last-Modified"), notModified = false) } } catch (e: CancellationException) { throw e } catch (e: SimklTrendingError) { if (e.permanent) throw e // don't waste backoff on permanent errors lastErr = e val honored = e.retryAfterMs val backoff = honored ?: (cfg.fetchBaseBackoffMs * 2.0.pow(attempt.toDouble())).toLong().coerceAtMost(cfg.fetchMaxBackoffMs) val jitter = (backoff * (Random.nextDouble() * 0.2 - 0.1)).toLong() // ±10% jitter delay(maxOf(0L, backoff + jitter)) } catch (e: Exception) { lastErr = e val backoff = (cfg.fetchBaseBackoffMs * 2.0.pow(attempt.toDouble())).toLong().coerceAtMost(cfg.fetchMaxBackoffMs) delay(backoff) } } throw lastErr ?: SimklTrendingError("fetch failed") } private suspend fun loop(entry: SimklTrendingEntry) { var lastModified = mu.withLock { cache[entry.url]?.lastModified } var retry = 0 while (scope.isActive) { val r = try { fetchDedupd(entry, lastModified) } catch (e: CancellationException) { return } catch (e: SimklTrendingError) { if (e.permanent) { println("${entry.title}: ${e.message} (permanent, halting loop)"); return } delay(30_000L); continue } catch (e: Exception) { delay(30_000L); continue } val wait = if (r.notModified) { cfg.regenRetrySeconds[minOf(retry, cfg.regenRetrySeconds.lastIndex)].also { retry++ } } else { storeAndNotify(entry, r) lastModified = r.lastModified; retry = 0 persist() (entry.refreshSeconds * (1.0 + (Random.nextDouble() - 0.5) * 2 * cfg.jitterPercent)).toInt() } delay(wait * 1_000L) } } private suspend fun storeAndNotify(entry: SimklTrendingEntry, r: SimklTrendingFetchResult) { mu.withLock { cache[entry.url] = _SimklCached(r.itemsByType ?: emptyMap(), r.lastModified, entry.types, System.currentTimeMillis()) } entry.types.forEach { notify(it) } } private suspend fun notify(type: String) { val items = get(type) val cbs = mu.withLock { subs[type]?.values?.toList().orEmpty() } cbs.forEach { it(items) } } private suspend fun persist() { val f = cfg.persistFile ?: return val dump = JSONObject() mu.withLock { for ((url, v) in cache) { val o = JSONObject() val ibt = JSONObject(); v.itemsByType.forEach { (k, arr) -> ibt.put(k, arr) } o.put("itemsByType", ibt) o.put("lastModified", v.lastModified ?: JSONObject.NULL) o.put("types", JSONArray(v.types)); o.put("fetchedAt", v.fetchedAt) dump.put(url, o) } } runCatching { val tmp = File(f.parentFile, f.name + ".tmp") tmp.writeText(dump.toString()); tmp.renameTo(f) } } private suspend fun restorePersisted() { val f = cfg.persistFile ?: return if (!f.exists()) return runCatching { val raw = JSONObject(f.readText()) val cutoff = System.currentTimeMillis() - cfg.persistMaxAgeMs mu.withLock { raw.keys().forEach { url -> val o = raw.getJSONObject(url) val fa = o.optLong("fetchedAt", 0L); if (fa < cutoff) return@forEach val ibtO = o.getJSONObject("itemsByType") val ibt = mutableMapOf<String, JSONArray>() ibtO.keys().forEach { t -> ibt[t] = ibtO.getJSONArray(t) } val types = (0 until o.getJSONArray("types").length()).map { o.getJSONArray("types").getString(it) } cache[url] = _SimklCached(ibt, if (o.isNull("lastModified")) null else o.getString("lastModified"), types, fa) } } } }}// === Run ===runBlocking { val client = SimklTrendingClient() client.subscribeCatalogs { rows -> for (row in rows) println("${row.title}: ${row.items.size} items") } client.start()}// === Background scheduling (Android) ===// In-process coroutines stop when the app process is killed. Schedule a// WorkManager.PeriodicWorkRequest whose Worker calls `client.tick()` to keep the cache warm// in the background. Use cfg.persistFile = File(context.cacheDir, "simkl-trending-cache.json")// so the next foreground launch can paint the UI from disk instantly.
import 'dart:async';import 'dart:convert';import 'dart:io';import 'dart:math';import 'package:http/http.dart' as http;// import 'package:path_provider/path_provider.dart'; // for persistFile in Flutter — see Run block// === Load the URL catalog ===// 1. Save the "Copy-paste JSON" block above as `assets/simkl-trending-urls.json`, then declare// it under `flutter: assets:` in pubspec.yaml.// 2. Load it into a global `SIMKL_TRENDING_URLS` once at app startup (after WidgetsFlutterBinding.ensureInitialized):// final SIMKL_TRENDING_URLS = jsonDecode(// await rootBundle.loadString('assets/simkl-trending-urls.json')// ) as Map<String, dynamic>;// For pure-Dart (server / CLI / tests), use:// final SIMKL_TRENDING_URLS = jsonDecode(await File('simkl-trending-urls.json').readAsString()) as Map<String, dynamic>;// You can also skip the global and pass it explicitly: `SimklTrendingClient(trending: urlsMap)`.// === CONFIG — tune to match your app ===class SimklTrendingConfig { // What to load final List<String> supportedTypes; // movies / tv / anime / dvd final List<String> timeframes; // today / week / month (ignored for dvd) final int size; // 100 or 500 — use 500 if you render derived catalogs (see "Build your own catalogs" below) // HTTP — replace with YOUR values final String clientId, appName, appVersion, userAgent; final Duration requestTimeout; final Duration minInterval; // process-wide floor between consecutive requests final int fetchMaxAttempts; final Duration fetchBaseBackoff, fetchMaxBackoff; final Set<int> retryStatuses; // never retry other 4xx — they're permanent // Refresh final List<int> regenRetrySeconds; // 5 / 15 / 30 min after a 304 final double jitterPercent; // ±5% // Persistent cache — set persistFile to null to disable. final File? persistFile; final Duration persistMaxAge; // Display — applied when items are read for display; the cached files stay exactly as served. final bool excludeVideoGamePlay; // hide titles classified "Video Game Play" (gameplay / Let's Play videos) // Catalogs — the rows on your home screen. Null = the JSON's catalogs[] (or one row per supportedType). final List<SimklTrendingCatalogConfig>? catalogs; const SimklTrendingConfig({ this.supportedTypes = const ['movies', 'tv', 'anime', 'dvd'], this.timeframes = const ['today'], this.size = 100, this.clientId = 'YOUR_CLIENT_ID', this.appName = 'my-app-name', this.appVersion = '1.0', this.userAgent = 'my-app-name/1.0', this.requestTimeout = const Duration(seconds: 8), this.minInterval = const Duration(milliseconds: 250), this.fetchMaxAttempts = 3, this.fetchBaseBackoff = const Duration(seconds: 1), this.fetchMaxBackoff = const Duration(seconds: 16), this.retryStatuses = const {408, 425, 429, 500, 502, 503, 504}, this.regenRetrySeconds = const [300, 900, 1800], this.jitterPercent = 0.05, this.persistFile, this.persistMaxAge = const Duration(days: 7), this.excludeVideoGamePlay = true, this.catalogs, });}class SimklTrendingCatalogConfig { final String id; final String source; // 'movies' | 'tv' | 'anime' | 'dvd' final String? timeframe; // defaults to cfg.timeframes.first final String? title; // defaults to JSON's pre-composed title final List<dynamic> Function(List<dynamic>)? transform; final String? recipe; // a SimklTrendingRecipes name (JSON rows) final Map<String, dynamic>? recipeOpts; SimklTrendingCatalogConfig({required this.id, required this.source, this.timeframe, this.title, this.transform, this.recipe, this.recipeOpts});}// === Recipes — named by the JSON's catalogs[]. This port ships wellRated (plus its deprecated// alias hiddenGems) and underseenGems. A row naming any other recipe stays EMPTY until you port it// (see "Catalog ideas" below) or give the row a `transform` — never the unfiltered feed.// Recipes never modify what they're given; a short or empty result is valid.class SimklTrendingRecipes { static List<dynamic>? run(String name, List<dynamic> items, Map<String, dynamic> opts) { switch (name) { case 'wellRated': case 'hiddenGems': return wellRated(items, opts); // hiddenGems: DEPRECATED old name case 'underseenGems': return underseenGems(items, opts); default: return null; } } // Well-rated titles outside the top `minRank` of Simkl's rank (a rating-based order, not a viewer // count) — so this still holds widely seen titles. Formerly hiddenGems; same selection. static List<dynamic> wellRated(List<dynamic> items, [Map<String, dynamic> opts = const {}]) { final minRating = _number(opts['minRating']) ?? 7.5, minVotes = _number(opts['minVotes']) ?? 500; final minRank = _number(opts['minRank']) ?? 2000; final source = opts['source'] as String? ?? 'simkl'; double rating(dynamic i) => _number(_ext(i, source)?['rating']) ?? 0; final picked = items.where((i) => i is Map && (_number(i['rank']) ?? 0) > minRank && rating(i) >= minRating && (_number(_ext(i, source)?['votes']) ?? 0) >= minVotes).toList(); final order = {for (var n = 0; n < picked.length; n++) picked[n]: n}; return picked..sort((a, b) { // stable: ties keep feed order final c = rating(b).compareTo(rating(a)); return c != 0 ? c : order[a]!.compareTo(order[b]!); }); } // Per-type defaults for underseenGems; override any of them with opts / recipe_opts. static const underseenDefaults = <String, Map<String, Object>>{ 'movies': {'source': 'imdb', 'minRating': 7.0, 'minVotes': 500, 'maxVotes': 25000}, 'tv': {'source': 'imdb', 'minRating': 7.5, 'minVotes': 500, 'maxVotes': 10000, 'excludeGenres': ['Sport', 'Sports', 'Reality', 'Talk', 'Talk Show', 'News']}, 'anime': {'source': 'mal', 'minRating': 7.5, 'minVotes': 500, 'maxVotes': 10000}, }; static const underseenCommon = <String, Object?>{ 'maxSimklVotes': 500, 'minAgeDays': 365, 'maxTrackedAudience': null, 'excludeGenres': <String>[]}; // Well-rated titles with a smaller audience, by external rating count — a heuristic, not proof a // title is obscure worldwide. Needs opts['type'] ('movies' | 'tv' | 'anime'); JSON rows pass their source. static List<dynamic> underseenGems(List<dynamic> items, Map<String, dynamic> opts, {DateTime? now}) { final base = underseenDefaults[opts['type']]; if (base == null) return []; final o = <String, dynamic>{...underseenCommon, ...base, ...opts}; final minRating = _number(o['minRating']), minVotes = _number(o['minVotes']), maxVotes = _number(o['maxVotes']); final maxSimklVotes = _number(o['maxSimklVotes']), minAgeDays = _number(o['minAgeDays']); final source = o['source'], capRaw = o['maxTrackedAudience']; final cap = capRaw == null ? null : _number(capRaw); if (minRating == null || minVotes == null || maxVotes == null || maxSimklVotes == null || minAgeDays == null || source is! String || (capRaw != null && cap == null)) return []; final latest = (now ?? DateTime.now().toUtc()) .subtract(Duration(milliseconds: (minAgeDays * 86400000).round())); // released on or before this final skip = ((o['excludeGenres'] as List?) ?? const []).map((g) => _normGenre('$g')).toSet(); final picked = <List<dynamic>>[]; // [item, rating, votes, order] for (var n = 0; n < items.length; n++) { final i = items[n]; if (i is! Map || i['status'] == 'premiere') continue; final released = parseReleaseDate(i['release_date']); if (released == null || released.isAfter(latest)) continue; // missing, invalid, future or too recent final rating = _number(_ext(i, source)?['rating']), votes = _count(_ext(i, source)?['votes']); if (rating == null || votes == null) continue; if (rating < minRating || votes < minVotes || votes > maxVotes) continue; final simklVotes = _count(_ext(i, 'simkl')?['votes']); if (simklVotes == null || simklVotes > maxSimklVotes) continue; final genres = i['genres']; if (skip.isNotEmpty && genres is List && genres.any((g) => skip.contains(_normGenre('$g')))) continue; if (cap != null) { // optional: watching + completed final watching = _count(i['watching']), completed = _count(i['completed']); if (watching == null || completed == null) continue; // missing is not zero if (watching + completed > cap) continue; } picked.add([i, rating, votes, n]); } picked.sort((a, b) { final r = (b[1] as double).compareTo(a[1] as double); // rating desc if (r != 0) return r; final v = (a[2] as int).compareTo(b[2] as int); // then rating count asc return v != 0 ? v : (a[3] as int).compareTo(b[3] as int); }); return picked.map((p) => p[0]).toList(); } // The "Video Game Play" classification, matched exactly after normalising case, spaces, - and _. // Also matches a genre-list entry (slug 'video-game-play'). Never matches title words or networks. static bool isVideoGamePlay(dynamic e) { if (e is! Map) return false; final genres = e['genres']; if (genres is List && genres.any((g) => _normGenre('$g') == 'video game play')) return true; return [e['slug'], e['name']].any((v) => v is String && _normGenre(v) == 'video game play'); } static List<dynamic> excludeVideoGamePlay(List<dynamic> items) => items.where((i) => !isVideoGamePlay(i)).toList(); // `release_date` is documented as MM/DD/YYYY — parse it explicitly, in UTC. Anything else → null. static DateTime? parseReleaseDate(dynamic v) { if (v is! String || !RegExp(r'^[0-9]{2}/[0-9]{2}/[0-9]{4}$').hasMatch(v)) return null; final mo = int.parse(v.substring(0, 2)), d = int.parse(v.substring(3, 5)), y = int.parse(v.substring(6, 10)); final t = DateTime.utc(y, mo, d); return (t.year == y && t.month == mo && t.day == d) ? t : null; // rejects 02/30 etc. } static String _normGenre(String g) => g.toLowerCase().split(RegExp(r'[\s_-]+')).where((s) => s.isNotEmpty).join(' '); static Map? _ext(dynamic i, String source) { final r = i is Map ? i['ratings'] : null; final s = r is Map ? r[source] : null; return s is Map ? s : null; } static double? _number(dynamic v) => (v is num && v.isFinite) ? v.toDouble() : null; static int? _count(dynamic v) { final n = _number(v); return (n != null && n >= 0 && n == n.truncateToDouble() && n < 9e15) ? n.toInt() : null; }}class SimklTrendingCatalog { final String id, title; final List<dynamic> items; SimklTrendingCatalog(this.id, this.title, this.items);}class SimklTrendingError implements Exception { final String message; final int? status; final Duration? retryAfter; final bool permanent; SimklTrendingError(this.message, {this.status, this.retryAfter, this.permanent = false}); @override String toString() => 'SimklTrendingError($status): $message';}class SimklTrendingEntry { final String title, url; final int refreshSeconds; final List<String> types; SimklTrendingEntry(this.title, this.url, this.refreshSeconds, this.types);}class _SimklCached { Map<String, List<dynamic>> itemsByType; String? lastModified; List<String> types; int fetchedAt; _SimklCached(this.itemsByType, this.lastModified, this.types, this.fetchedAt);}class SimklTrendingFetchResult { final Map<String, List<dynamic>>? itemsByType; final String? lastModified; final bool notModified; SimklTrendingFetchResult({this.itemsByType, this.lastModified, this.notModified = false});}class SimklTrendingClient { final SimklTrendingConfig cfg; final Map<String, dynamic> trending; final _cache = <String, _SimklCached>{}; final _inflight = <String, Future<SimklTrendingFetchResult>>{}; final _subs = <String, Map<int, void Function(List<dynamic>)>>{}; final _client = http.Client(); int _subIdSeq = 0; bool _stopped = false; int _lastRequestAt = 0; // for minInterval floor final _loops = <Future<void>>[]; final _warned = <String>{}; final List<dynamic>? defaultCatalogs; // Accept both JSON shapes: { urls: {...}, catalogs: [...] } OR the legacy unwrapped { combined, movies, ... }. factory SimklTrendingClient({SimklTrendingConfig? config, Map<String, dynamic>? trending}) { final raw = trending ?? (SIMKL_TRENDING_URLS as Map<String, dynamic>); final urlsMap = raw['urls'] != null ? raw : { 'urls': raw }; return SimklTrendingClient._internal(config ?? const SimklTrendingConfig(), urlsMap as Map<String, dynamic>, raw['catalogs'] as List<dynamic>?); } SimklTrendingClient._internal(this.cfg, this.trending, this.defaultCatalogs); // === Public API === Future<void> start() async { await _restorePersisted(); for (final t in _subs.keys.toList()) _notify(t); // paint immediately from restored cache for (final entry in _plan()) _loops.add(_loop(entry)); } void stop() { _stopped = true; _client.close(); } /// Wipe in-memory + persistent cache and notify subscribers (UI repaints empty). Call client.tick() after to re-fetch. Future<void> clearCache() async { _cache.clear(); final f = cfg.persistFile; if (f != null && await f.exists()) { try { await f.delete(); } catch (_) {/* ignore */} } for (final t in _subs.keys.toList()) _notify(t); } // Items to display for a type. Display filters (excludeVideoGamePlay) apply here, on read, // so charts, derived rows and a cache restored offline all show the same selection. List<dynamic> get(String type) { for (final v in _cache.values) { final items = v.itemsByType[type]; if (items != null) return cfg.excludeVideoGamePlay ? SimklTrendingRecipes.excludeVideoGamePlay(items) : items; } return const []; } int subscribe(String type, void Function(List<dynamic>) cb) { final id = ++_subIdSeq; (_subs[type] ??= {})[id] = cb; final items = get(type); if (items.isNotEmpty) cb(items); // immediate paint from cache return id; } void unsubscribe(String type, int id) { _subs[type]?.remove(id); } // === Catalogs — rows on the home screen === List<SimklTrendingCatalog> catalogs() => _catalogConfigs().map(_buildCatalog).toList(); List<int> subscribeCatalogs(void Function(List<SimklTrendingCatalog>) cb) { final sources = _catalogConfigs().map((c) => c.source).toSet(); final ids = <int>[]; for (final s in sources) { ids.add(subscribe(s, (_) => cb(catalogs()))); } if (sources.every((s) => get(s).isEmpty)) cb(catalogs()); // empty cache — skeleton paint return ids; } List<SimklTrendingCatalogConfig> _catalogConfigs() { if (cfg.catalogs != null) return cfg.catalogs!; // explicit override wins if (defaultCatalogs != null) { // JSON's catalogs[] return defaultCatalogs!.whereType<Map>().map((d) => SimklTrendingCatalogConfig( id: d['id'] as String, source: d['source'] as String, timeframe: d['timeframe'] as String?, title: d['title'] as String?, recipe: d['recipe'] as String?, recipeOpts: (d['recipe_opts'] as Map?)?.cast<String, dynamic>(), )).toList(); } final tf0 = cfg.timeframes.isNotEmpty ? cfg.timeframes.first : 'today'; // last resort: auto-build return cfg.supportedTypes.map((t) { if (t == 'dvd') return SimklTrendingCatalogConfig(id: 'dvd', source: 'dvd'); return SimklTrendingCatalogConfig(id: '${t}_$tf0', source: t, timeframe: tf0); }).toList(); } String _fileTitleFor(String source, String? timeframe) { final sizeKey = 'top_${cfg.size}'; final urls = trending['urls'] as Map?; if (urls == null) return 'Trending $source on Simkl'; final file = source == 'dvd' ? ((urls['dvd'] as Map?)?[sizeKey]) : (((urls[source] as Map?)?[timeframe ?? 'today'] as Map?)?[sizeKey]); return (file as Map?)?['title'] as String? ?? 'Trending $source on Simkl'; } SimklTrendingCatalog _buildCatalog(SimklTrendingCatalogConfig c) { final items = get(c.source); var out = items; if (c.transform != null) { out = c.transform!(items); } else if (c.recipe != null) { // JSON rows name a recipe final rows = SimklTrendingRecipes.run(c.recipe!, items, {'type': c.source, ...?c.recipeOpts}); if (rows != null) { out = rows; } else { // never fall back to the unfiltered feed out = const []; final msg = 'SimklTrendingClient: recipe "${c.recipe}" (catalog "${c.id}") isn\'t in this port — row left empty.'; if (_warned.add(msg)) print(msg); } } final title = c.title ?? _fileTitleFor(c.source, c.timeframe ?? (cfg.timeframes.isNotEmpty ? cfg.timeframes.first : 'today')); return SimklTrendingCatalog(c.id, title, out); } // Call from a `workmanager` periodic task for a single revalidate-and-persist pass. Future<void> tick() async { await _restorePersisted(); for (final entry in _plan()) { final lm = _cache[entry.url]?.lastModified; try { final r = await _fetchDedupd(entry, lm); if (!r.notModified && r.itemsByType != null) _storeAndNotify(entry, r); } catch (_) { /* swallow — best-effort */ } } await _persist(); } // === Internals === String _requiredQuery() => 'client_id=${cfg.clientId}' '&app-name=${Uri.encodeQueryComponent(cfg.appName)}' '&app-version=${Uri.encodeQueryComponent(cfg.appVersion)}'; List<SimklTrendingEntry> _plan() { if (cfg.size != 100 && cfg.size != 500) throw ArgumentError('size must be 100 or 500'); final key = 'top_${cfg.size}'; final urls = trending['urls'] as Map; final tvma = cfg.supportedTypes.where((t) => ['movies','tv','anime'].contains(t)).toList(); final useCombined = tvma.length >= 2; // 1 round-trip < 2-3 per timeframe SimklTrendingEntry make(Map e, List<String> types) => SimklTrendingEntry(e['title'], e['url'], e['refresh_seconds'], types); final out = <SimklTrendingEntry>[]; if (tvma.isNotEmpty) for (final tf in cfg.timeframes) { final bucket = (useCombined ? urls['combined'] : urls[tvma[0]]) as Map; out.add(make((bucket[tf] as Map)[key], tvma)); } if (cfg.supportedTypes.contains('dvd')) out.add(make((urls['dvd'] as Map)[key], ['dvd'])); return out; } // In-flight dedup: only one outstanding network call per URL. Future<SimklTrendingFetchResult> _fetchDedupd(SimklTrendingEntry entry, String? lastModified) { final sep = entry.url.contains('?') ? '&' : '?'; final url = entry.url + sep + _requiredQuery(); final existing = _inflight[url]; if (existing != null) return existing; final task = _runFetch(url, entry, lastModified).whenComplete(() { _inflight.remove(url); }); _inflight[url] = task; return task; } // Process-wide rate-limit floor (so N entries refreshing at once don't fire N simultaneous requests). Future<void> _throttle() async { final gap = DateTime.now().millisecondsSinceEpoch - _lastRequestAt; if (gap < cfg.minInterval.inMilliseconds) { await Future.delayed(Duration(milliseconds: cfg.minInterval.inMilliseconds - gap)); } _lastRequestAt = DateTime.now().millisecondsSinceEpoch; } static Duration? _parseRetryAfter(String? value) { if (value == null) return null; final secs = num.tryParse(value); if (secs != null) return Duration(milliseconds: (secs.toDouble() * 1000).round().clamp(0, 1 << 31)); try { final dt = HttpDate.parse(value); return Duration(milliseconds: (dt.millisecondsSinceEpoch - DateTime.now().millisecondsSinceEpoch).clamp(0, 1 << 31)); } catch (_) { return null; } } // _runFetch — throttle + retry with exponential backoff + Retry-After + permanent-error classify. Normalizes both response shapes. Future<SimklTrendingFetchResult> _runFetch(String url, SimklTrendingEntry entry, String? lastModified) async { Object? lastErr; for (var attempt = 0; attempt < cfg.fetchMaxAttempts; attempt++) { await _throttle(); try { final headers = {'User-Agent': cfg.userAgent, 'Accept': 'application/json'}; if (lastModified != null) headers['If-Modified-Since'] = lastModified; final res = await _client.get(Uri.parse(url), headers: headers).timeout(cfg.requestTimeout); if (res.statusCode == 304) return SimklTrendingFetchResult(itemsByType: null, lastModified: lastModified, notModified: true); if (res.statusCode != 200) { final retryAfter = _parseRetryAfter(res.headers['retry-after']); final permanent = !cfg.retryStatuses.contains(res.statusCode); throw SimklTrendingError('${res.statusCode} ${res.reasonPhrase} — ${entry.title}', status: res.statusCode, retryAfter: retryAfter, permanent: permanent); } final raw = jsonDecode(res.body); final ibt = <String, List<dynamic>>{}; if (raw is List) ibt[entry.types[0]] = raw; // per-type / dvd → array else if (raw is Map) for (final t in entry.types) ibt[t] = (raw[t] as List?) ?? []; // combined → object, filter to wanted types return SimklTrendingFetchResult(itemsByType: ibt, lastModified: res.headers['last-modified']); } on SimklTrendingError catch (e) { if (e.permanent) rethrow; // don't waste backoff on permanent errors lastErr = e; final honoredMs = e.retryAfter?.inMilliseconds; final backoffMs = honoredMs ?? min(cfg.fetchBaseBackoff.inMilliseconds * pow(2, attempt).toInt(), cfg.fetchMaxBackoff.inMilliseconds); final jitterMs = (backoffMs * (Random().nextDouble() * 0.2 - 0.1)).round(); // ±10% jitter await Future.delayed(Duration(milliseconds: max(0, backoffMs + jitterMs))); } catch (e) { if (_stopped) rethrow; lastErr = e; final backoffMs = min(cfg.fetchBaseBackoff.inMilliseconds * pow(2, attempt).toInt(), cfg.fetchMaxBackoff.inMilliseconds); await Future.delayed(Duration(milliseconds: backoffMs)); } } throw lastErr ?? SimklTrendingError('fetch failed'); } Future<void> _loop(SimklTrendingEntry entry) async { var lastModified = _cache[entry.url]?.lastModified; var retry = 0; while (!_stopped) { late SimklTrendingFetchResult r; try { r = await _fetchDedupd(entry, lastModified); } on SimklTrendingError catch (e) { if (_stopped) return; if (e.permanent) { print('${entry.title}: ${e.message} (permanent, halting loop)'); return; } await Future.delayed(const Duration(seconds: 30)); continue; } catch (e) { if (_stopped) return; await Future.delayed(const Duration(seconds: 30)); continue; } int wait; if (r.notModified) { wait = cfg.regenRetrySeconds[min(retry, cfg.regenRetrySeconds.length - 1)]; retry++; } else { _storeAndNotify(entry, r); lastModified = r.lastModified; retry = 0; await _persist(); wait = (entry.refreshSeconds * (1 + (Random().nextDouble() - 0.5) * 2 * cfg.jitterPercent)).round(); } await Future.delayed(Duration(seconds: wait)); } } void _storeAndNotify(SimklTrendingEntry entry, SimklTrendingFetchResult r) { _cache[entry.url] = _SimklCached(r.itemsByType ?? {}, r.lastModified, entry.types, DateTime.now().millisecondsSinceEpoch); for (final t in entry.types) _notify(t); } void _notify(String type) { final items = get(type); for (final cb in (_subs[type]?.values ?? const Iterable.empty())) cb(items); } Future<void> _persist() async { final f = cfg.persistFile; if (f == null) return; try { final dump = { for (final e in _cache.entries) e.key: { 'itemsByType': e.value.itemsByType, 'lastModified': e.value.lastModified, 'types': e.value.types, 'fetchedAt': e.value.fetchedAt, }, }; final tmp = File('${f.path}.tmp'); await tmp.writeAsString(jsonEncode(dump)); await tmp.rename(f.path); } catch (_) { /* best-effort */ } } Future<void> _restorePersisted() async { final f = cfg.persistFile; if (f == null || !await f.exists()) return; try { final raw = jsonDecode(await f.readAsString()) as Map<String, dynamic>; final cutoff = DateTime.now().millisecondsSinceEpoch - cfg.persistMaxAge.inMilliseconds; for (final entry in raw.entries) { final v = entry.value as Map; final fa = (v['fetchedAt'] as num?)?.toInt() ?? 0; if (fa < cutoff) continue; final ibt = (v['itemsByType'] as Map).map((k, val) => MapEntry(k as String, List<dynamic>.from(val as List))); _cache[entry.key] = _SimklCached(ibt, v['lastModified'] as String?, List<String>.from(v['types'] as List), fa); } } catch (_) { /* no cache yet — fine */ } }}// === Run ===Future<void> main() async { // For Flutter, set persistFile via path_provider: // final dir = await getApplicationCacheDirectory(); // final cfg = SimklTrendingConfig(persistFile: File('${dir.path}/simkl-trending-cache.json')); final client = SimklTrendingClient(); client.subscribeCatalogs((rows) { for (final row in rows) print('${row.title}: ${row.items.length} items'); }); await client.start();}// === Background scheduling (Flutter) ===// Future.delayed stops when the app is killed. Schedule a `workmanager` package periodic task// whose callback calls `await client.tick()` to keep the cache warm in the background. Use the// path_provider snippet above for persistFile so the next foreground launch paints from disk.
Node 22+, Deno, Bun, React Native, Cloudflare Workers, and other fetch-based JS runtimes — the JavaScript tab above (the SDK file) is universal: save it verbatim as simkl-trending-client.js, then wire it up from your app entry point in ~6 lines. The Browser tab right above it is the HTML+DOM wrapper you’d skip in a headless runtime; here’s the non-browser equivalent:
app.mjs (or index.js App.tsx)
// Pick the loader your runtime prefers:import data from './simkl-trending-urls.json' with { type: 'json' }; // Node 22+ ESM// const data = require('./simkl-trending-urls.json'); // Node CommonJS// const data = await (await fetch('/simkl-trending-urls.json')).json(); // Deno / Bun / RN / CF Workersimport { SimklTrendingClient } from './simkl-trending-client.js';const client = new SimklTrendingClient( { clientId: 'YOUR_CLIENT_ID', appName: 'my-app', appVersion: '1.0' }, data,);client.subscribeCatalogs(rows => rows.forEach(r => console.log(`${r.title}: ${r.items.length} items`)));await client.start();// process.on('SIGINT', () => { client.stop(); process.exit(0); }); // long-running servers
Bundle simkl-trending-urls.json with your app (or fetch + cache it once at startup) so the catalog is in scope when SimklTrendingClient runs. For Cloudflare Workers / Lambda / GitHub Actions etc., call await client.tick() from your scheduler instead of client.start() — see the “Background scheduling” comment at the bottom of the JavaScript tab.
What you still need to do per-platform. The SimklTrendingClient samples above wire up everything that’s host-agnostic — auto-pick combined, persistent cache, conditional revalidation, regen-lag backoff, jitter, retry with exponential backoff + timeout, in-flight dedup, cancellation, and subscribe/get. Two things are left to you because they depend on your platform:
Background scheduling that survives app lifecycle. In-process timers stop firing when an iOS app is backgrounded or after Android process death. Hook your platform’s periodic task system to client.tick() — BGTaskScheduler (iOS), WorkManager.PeriodicWorkRequest (Android), the workmanager package (Flutter), or node-cron / Cloudflare Cron / AWS EventBridge for server-side runtimes. Each tab’s trailing comment shows the wireup.
Attribution back-link. Websites must link the rendered section title to the matching simkl.com/<type>/best-*/most-watched/ page — see Attribution required and Linking back.
If you want stronger typing on the per-item shape, the Field reference below shows the JSON, ready to bind to a TypeScript interface, Swift Codable, Kotlin @Serializable, or Dart json_serializable.
Catalogs — the rows on your home screen
Think of a TV-style discovery UI: rows of poster carousels stacked vertically, each row with a title at the top-left. The SDK exposes that as client.catalogs() → [{id, title, items}, ...] and client.subscribeCatalogs(cb) — available in all 5 languages above. Each row has a stable id, an attribution-compliant title, and an items array. You iterate and render.Three places rows can come from, in priority order:
cfg.catalogs in your SDK config — explicit, wins if set. Use this to mix trending + derived rows with custom transform closures (see below).
catalogs[] in the Copy-paste JSON above — the default list of rows. Already populated with 58 rows: the four trending rows (movies_today, tv_today, anime_today, dvd) and 54 derived ones. Add, remove, or reorder entries in the JSON to change defaults across your whole app — no code change.
Auto-built fallback — if neither of the above is set, the SDK builds one row per supportedType × timeframes[0] from your CONFIG.
So with the JSON catalogs already populated, new SimklTrendingClient() immediately gives you all 58 rows — no extra config needed. Override only when you want custom rows.Adding derived rows in code — when you want filter/sort logic that’s not in the JSON (a stricter underseen row, say), pass cfg.catalogs with a transform function on each entry:
import { SimklTrendingClient, SimklTrendingRecipes as R } from './simkl-trending-client.js';const client = new SimklTrendingClient({ supportedTypes: ['movies', 'tv', 'anime'], size: 500, // recommended once you derive rows catalogs: [ { id: 'movies_today', source: 'movies' }, // trending (JSON title) { id: 'tv_today', source: 'tv' }, { id: 'anime_today', source: 'anime' }, { id: 'movies_anticipated', source: 'movies', title: 'Most Watchlisted Movies on Simkl', transform: R.mostWatchlisted }, { id: 'underseen_anime', source: 'anime', title: 'Underseen anime on Simkl', transform: items => R.underseenGems(items, { type: 'anime', maxTrackedAudience: 2000 }) }, // optional extra guard { id: 'best_of_hbo', source: 'tv', title: 'Best of HBO — on Simkl', transform: items => R.bestOfNetwork(items, { network: 'HBO' }) }, ],});client.subscribeCatalogs(rows => { for (const r of rows) renderRow(r.title, r.items); });await client.start();
client = SimklTrendingClient({ 'supported_types': ['movies', 'tv', 'anime'], 'size': 500, 'catalogs': [ {'id': 'movies_today', 'source': 'movies'}, # trending (JSON title) {'id': 'tv_today', 'source': 'tv'}, {'id': 'anime_today', 'source': 'anime'}, {'id': 'movies_anticipated', 'source': 'movies', 'title': 'Most Watchlisted Movies on Simkl', 'transform': lambda items: sorted(items, key=lambda i: -i.get('plan_to_watch', 0))}, {'id': 'underseen_anime', 'source': 'anime', 'title': 'Underseen anime on Simkl', # optional extra guard ↓ 'transform': lambda items: simkl_underseen_gems(items, {'type': 'anime', 'maxTrackedAudience': 2000})}, {'id': 'best_of_hbo', 'source': 'tv', 'title': 'Best of HBO — on Simkl', 'transform': lambda items: sorted([i for i in items if i.get('network') == 'HBO' and i.get('status') == 'ended' and i.get('ratings',{}).get('simkl',{}).get('rating',0) >= 8], key=lambda i: -i['ratings']['simkl']['rating'])}, ],})client.subscribe_catalogs(lambda rows: [renderRow(r['title'], r['items']) for r in rows])client.start()
let cfg = SimklTrendingConfig( supportedTypes: ["movies", "tv", "anime"], size: 500, catalogs: [ .init(id: "movies_today", source: "movies"), // trending (JSON title) .init(id: "tv_today", source: "tv"), .init(id: "anime_today", source: "anime"), .init(id: "movies_anticipated", source: "movies", title: "Most Watchlisted Movies on Simkl", transform: { items in items.sorted { ($0["plan_to_watch"] as? Int ?? 0) > ($1["plan_to_watch"] as? Int ?? 0) } }), .init(id: "underseen_anime", source: "anime", title: "Underseen anime on Simkl", // optional extra guard ↓ transform: { items in SimklTrendingRecipes.underseenGems(items, ["type": "anime", "maxTrackedAudience": 2000]) }), .init(id: "best_of_hbo", source: "tv", title: "Best of HBO — on Simkl", transform: { items in items.filter { ($0["network"] as? String) == "HBO" && ($0["status"] as? String) == "ended" } }), ])Task { let client = SimklTrendingClient(config: cfg) _ = await client.subscribeCatalogs { rows in for r in rows { renderRow(r.title, r.items) } } await client.start()}
val cfg = SimklTrendingConfig( supportedTypes = listOf("movies", "tv", "anime"), size = 500, catalogs = listOf( SimklTrendingCatalogConfig(id = "movies_today", source = "movies"), // trending (JSON title) SimklTrendingCatalogConfig(id = "tv_today", source = "tv"), SimklTrendingCatalogConfig(id = "anime_today", source = "anime"), SimklTrendingCatalogConfig(id = "movies_anticipated", source = "movies", title = "Most Watchlisted Movies on Simkl", transform = { items -> items.sortedByDescending { it.optInt("plan_to_watch") } }), SimklTrendingCatalogConfig(id = "underseen_anime", source = "anime", title = "Underseen anime on Simkl", // optional extra guard ↓ transform = { items -> SimklTrendingRecipes.underseenGems(items, mapOf("type" to "anime", "maxTrackedAudience" to 2000)) }), SimklTrendingCatalogConfig(id = "best_of_hbo", source = "tv", title = "Best of HBO — on Simkl", transform = { items -> items.filter { it.optString("network") == "HBO" && it.optString("status") == "ended" } }), ),)runBlocking { val client = SimklTrendingClient(cfg) client.subscribeCatalogs { rows -> for (r in rows) renderRow(r.title, r.items) } client.start()}
final cfg = SimklTrendingConfig( supportedTypes: ['movies', 'tv', 'anime'], size: 500, catalogs: [ SimklTrendingCatalogConfig(id: 'movies_today', source: 'movies'), // trending (JSON title) SimklTrendingCatalogConfig(id: 'tv_today', source: 'tv'), SimklTrendingCatalogConfig(id: 'anime_today', source: 'anime'), SimklTrendingCatalogConfig(id: 'movies_anticipated', source: 'movies', title: 'Most Watchlisted Movies on Simkl', transform: (items) => [...items]..sort((a, b) => (b['plan_to_watch'] ?? 0).compareTo(a['plan_to_watch'] ?? 0))), SimklTrendingCatalogConfig(id: 'underseen_anime', source: 'anime', title: 'Underseen anime on Simkl', // optional extra guard ↓ transform: (items) => SimklTrendingRecipes.underseenGems(items, {'type': 'anime', 'maxTrackedAudience': 2000})), SimklTrendingCatalogConfig(id: 'best_of_hbo', source: 'tv', title: 'Best of HBO — on Simkl', transform: (items) => items.where((i) => i['network'] == 'HBO' && i['status'] == 'ended').toList()), ],);final client = SimklTrendingClient(config: cfg);client.subscribeCatalogs((rows) { for (final r in rows) renderRow(r.title, r.items); });await client.start();
Need the rows synchronously (e.g. inside a render function)? client.catalogs() returns the resolved list on demand. Same {id, title, items} shape in every language.Available recipe transforms (drop into transform: to derive a row). Each has tunable opts as the 2nd arg — underseenGems(items, { type, minRating, maxVotes, … }), bestOfNetwork(items, { network }), etc. Dial them to your taste without forking.
hiddenGems (now wellRated), mostAnticipated (now mostWatchlisted), premiering (now justPremiered) — same results, kept so saved configs keep working
A derived row is a filtered subset of one trending file, so a short or empty row is a valid result. How many titles a row gets depends on its filter and on what is trending right now — some rows hold a handful, some hold none. Show a short row as it is, or hide an empty one, but never pad it with titles from the unfiltered feed. Load size: 500 once you render derived rows: every filter then has the top 500 to choose from instead of the top 100, on the same fetch schedule.
Adding / reordering rows costs nothing. Catalog rows are resolved at read time off the cached items. Changing cfg.catalogs and recomputing produces a new row list in microseconds. The SDK refreshes the underlying cache on its own cadence (~1h for today, 1d for week/month), independent of how many rows you render.
Catalog ideas — recipe logic at a glance
Every recipe ships in the JavaScript tab. The Python / Swift / Kotlin / Dart tabs ship wellRated and underseenGems; for any other recipe, here’s the logic — drop it into your transform: / client.get(type) pipeline and you have the row.
Catalog row
Logic
Most Watchlisted
sort plan_to_watch desc — items that appear on the most user watchlists. The name “Most Anticipated” misled here: this fires for already-released titles too (huge backlog appeal — Game of Thrones, Avatar, One Piece all rank high). Use this for the universal “what’s on people’s minds” row.
Highest Rated (Simkl)
sort ratings.simkl.rating desc, gate on ratings.simkl.votes >= 500 to drop noise
Highest Rated (IMDb / MAL)
same, on ratings.imdb.rating (movies/TV) or ratings.mal.rating (anime)
Stickiest
sort parseFloat(drop_rate) asc — viewers who started, finished, and didn’t bail
Newest releases
sort by parsed release_date desc — release_date is MM/DD/YYYY (e.g. 03/26/2026), which new Date(...) / Python datetime.strptime("%m/%d/%Y") / Swift DateFormatter("MM/dd/yyyy") all accept
Box-office hits(movies)
extract $NNNM from metadata, sort desc
Quick watches(movies)
parse runtime to minutes, filter ≤ 90, sort by rating
Marathon-worthy(TV/anime)
filter status === 'ended' AND 20 ≤ total_episodes ≤ 100, sort by rating
Mini-series(TV/anime)
filter total_episodes ≤ 6
By genre / country / network / status / anime type
items.filter(...) on the matching field
In theaters now(movies)
filter theater date within last 30 days
Just released on DVD(movies)
filter dvd_date within last 14-30 days
Just premiered(TV/anime/movies)
filter status === 'premiere' — flags items whose first episode JUST aired (typically within ~30 days). Not an “upcoming premieres” filter; the data has no field for that.
By year (e.g. Movies of 2026)
parse the 4-digit year out of release_date (any format), filter equality
By decade (e.g. Best of the 90s)
parse year, filter decade ≤ y < decade + 10
Current year / last year
same as By year, comparing against new Date().getFullYear() (or your language’s equivalent) — re-evaluated per call so the row stays current without re-shipping the JSON
Best of decade
combine By decade with bestOfNetwork’s shape — rating-sort + votes >= 100 floor inside the decade window
Has trailer
items.filter(i => i.trailer) — for trailer-first carousel UIs
Well-rated (wellRated, formerly hiddenGems)
filter rank > 2000 AND rating >= 7.5 AND votes >= 500, then sort by rating desc. rank is a rating-based order, not an audience measure, so this row keeps widely seen titles
Underseen (underseenGems)
IMDb rating ≥ 7.0 from 500–25,000 IMDb ratings (movies), IMDb ≥ 7.5 from 500–10,000 (TV), MAL ≥ 7.5 from 500–10,000 (anime); at most 500 Simkl ratings; released at least 365 days ago and not premiere; TV skips Sport, Reality, Talk Show and News. Sort by rating desc, then fewer ratings first. Optional maxTrackedAudience caps watching + completed
Critically acclaimed, widely seen
rating >= 8.5 AND votes >= 1000 — high bar on both axes
Massive anticipation
plan_to_watch / (watched + 1) >= 300 — pre-release / newly-announced. watched follows the file’s timeframe, so the same title scores differently in the today, week and month files
Cross-platform consensus
Simkl rating ≥ 8 AND IMDb (or MAL) rating ≥ 8 — both crowds agree
Best of
filter network match + status === 'ended', sort by rating
Grouped by genre / country / network / decade
bucket items into { "Action": [...], "Drama": [...] } for N rows from one file
Tuning a recipe per catalog row. Every recipe accepts an options object as its second arg. In the JSON, set them via recipe_opts. For example, to make tv_underseen stricter — an IMDb rating of at least 8 from no more than 5,000 IMDb ratings, instead of the default 7.5 and 10,000 — override those two knobs:
{ "id": "tv_underseen", "source": "tv", "title": "Underseen TV on Simkl", "recipe": "underseenGems", "recipe_opts": { "minRating": 8, "maxVotes": 5000 }}
Same pattern works for any recipe — bestOfDecade with "decade": 1980, bestOfNetwork with "network": "AMC", highestRated with "minVotes": 2000, etc. Stricter options mean shorter rows, and an empty row is a valid result — not something to fill.
Attribution still applies to derived rows. A row labeled “Most Watchlisted Movies” or “Underseen TV” still needs to include “Simkl” or “Most Watched” to satisfy the attribution requirement — e.g. “Most Watchlisted on Simkl”, “Underseen TV on Simkl”, “Trending in Japan — Powered by Simkl”.
Gameplay videos (excludeVideoGamePlay)
Some TV titles are recorded gameplay — Let’s Play videos and playthroughs — and carry the Video Game Play genre. The SDK hides them by default with excludeVideoGamePlay: true (exclude_video_game_play in Python), the same choice Simkl’s app makes.
Exact match only. A title is hidden when one of its genres is Video Game Play, compared after normalising case, spaces, - and _. Words in a title, whole networks and other genres — anime’s Strategy Game included — never match, so adaptations such as A Minecraft Movie and Mortal Kombat stay.
Everywhere you display. The client applies it whenever it reads items for display, so trending charts, derived rows and a cache restored at launch or offline all show the same selection.
The data isn’t changed. The cached and persisted files stay exactly as served. Set the option to false and the titles come back without a refetch.
Genre lists too. The SDK doesn’t load the genre files yet. If you list genres from index.json yourself, skip the video-game-play entry — SimklTrendingRecipes.excludeVideoGamePlay(genres) drops it in JavaScript.
What’s in each item
Per-title arrays (movies, tv, anime) all share the same base shape, with a few type-specific fields.
Localized and alternate titles. Always present on every catalog, but often an empty array — check length before reading. Popular titles can carry dozens of entries (60+ on the most widely-released films), so slice before rendering.Each entry has three keys:
name(string) — the alternate title.
type(string) — official for a localized release title, original for the title in the production’s own language.
lang(integer) — Simkl-internal language code. This is not an ISO 639 code and there is currently no public mapping table, so treat it as an opaque grouping key rather than something to render.
If you only need one display title per item, use title and ignore this field entirely.
string
Path on simkl.com (e.g. /movie/2533163/the-drama). Prepend https://simkl.com to deep-link.
string
Image path fragment. Combine with the prefixes in Image conventions — for example https://wsrv.nl/?url=https://simkl.in/posters/{poster}_m.webp&q=90. When null, fall back to https://simkl.in/poster_no_pic.png (see fallbacks).
string
Image path fragment for fanart. Same prefixing rules as poster. When null, hide the fanart element — there’s no dedicated fanart placeholder.
object
External and Simkl IDs. Always carries simkl_id + slug. tmdb is near-universal; imdb appears on TV / movies, mal / anidb / anilist / kitsu on anime. Additional IDs (tvdb, tvdbslug, traktslug, letterboxd, mdlslug, …) appear on items with those platform links, named as everywhere else in the API — the response is permissive, so read the keys you need and ignore the rest.
Show properties
integer
Simkl ID — primary key for /movies/{id}, /tv/{id}, /anime/{id}.
string
URL-safe slug.
string
IMDB ID, e.g. tt0944947.
string
TMDB ID.
string
TVDB ID (TV / anime).
string
MyAnimeList ID (anime).
string
AniDB ID (anime).
string
AniList ID (anime).
string
Kitsu ID (anime).
integer | null
The title’s place in Simkl’s rank for its type — the order behind the All-Time Best rows, where 1 is the best-rated. It is based on ratings, weighted by how many votes a title has. It is not a count of viewers and doesn’t reliably tell mainstream titles from obscure ones. null when the title has no rank yet.
integer
Number of users who watched this title in the file’s timeframe — today, this week or this month. On the DVD release files, where there is no timeframe, it is instead watching + completed.
integer
Number of users with this title on their Plan-to-Watch watchlist.
integer
Number of users with this title in Watching. All time — not limited to the file’s timeframe.
integer
Number of users with this title in Completed. All time — not limited to the file’s timeframe.
watched is the only count that follows the timeframe. One movie, read from the three files on the same day:
File
watched
watching
completed
today_500
12
66
5,691
week_500
85
66
5,691
month_500
462
66
5,691
Use watched to ask “what is popular right now?” and watching + completed to ask “how many Simkl users have this title in Watching or Completed?”.
string
Percentage of users who started and dropped, formatted as a string (e.g. "1.6%").
object
Aggregate ratings — Simkl and any external sources available for this title (IMDB for movies/TV, MAL for anime, etc.). Each entry is { rating: number, votes: number }, where votes is how many ratings that source has.
string
Original release date in MM/DD/YYYY format.
string
ISO-style country code (e.g. us, jp). Nullable on anime (~28% of the anime feed); always populated on movies, TV, and DVD.
string | null
Lowercase two-letter language code for the production’s original audio (en, ja, zh, ko). Nullable on anime (~33% of the anime feed); always populated on movies, TV, and DVD. Distinct from country, which is the country of origin — a Japanese-language title can be country: "us" on a co-production.
string
Human-readable duration (e.g. 1h 45m, 25m).
string
One of ended (production wrapped), ongoing (TV/anime still airing new episodes), or premiere (very-recently-released, still in the rollout window). The wider Simkl catalog also uses tba for not-yet-dated titles, but those don’t surface in trending.
string[]
Array of genre tags. A TV title tagged Video Game Play is recorded gameplay; the SDK hides those by default — see Gameplay videos.
Movies only. Theatrical release date (MM/DD/YYYY). Set on every movie with a theatrical release on file — past releases included, so compare it with today before calling a film “in theaters”. null when none is on file.
string | null
Movies only. Director’s name; several are joined with commas. null when none is on file.
string | null
Content rating as the rating body issues it — PG-13, R, TV-MA, TV-14, NR and so on. On movies, TV and anime. null when no rating is on file (common on TV).
object[] | null
Anime only. Animation studios, each { "id": 736, "name": "GoHands" } — id is a Simkl company ID. null when none is on file.
string | null
Anime only. The anime season it premiered in — Winter, Spring, Summer or Fall plus the year, e.g. Fall 2026. null when no premiere date is on file.
string | null
Anime only. Romanized Japanese title (e.g. Mushoku Tensei III: Isekai Ittara Honki Dasu), for apps that display romaji alongside or instead of the English title. null on roughly a quarter of the anime feed — fall back to title.
string
Anime only. One of tv, movie, special, ova, ona, music video.
integer
TV / anime. Episode count.
string
TV / anime. Broadcasting network.
Without the SDK — raw URL and curl
The SimklTrendingClient above is the recommended way to consume trending data — it handles caching, conditional revalidation, retry, dedup, persistence, and the auto-pick combined endpoint logic. The patterns below are the bare-bones alternative for ad-hoc curl probes, server-side cron jobs that handle their own caching, or runtimes where you don’t want to drop a class file.
URL pattern and quick curl / JS / Python / Go examples