harry/RSS-Feed
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add manual subreddit/YouTube channel configuration (manage.php)

Browse Source 
Browser-based feed selection layered on top of config/feeds.php:

- manage.php (password + CSRF gated, session based): add subreddits
  (name / r/name / URL) and YouTube channels (@handle / UC id / URL,
  handle resolved to channel_id once, youtube.com only); enable/disable
  toggles; remove manual feeds; config feeds are hidden, not deleted.
- New feeds are validated with one immediate fetch (bypassing the
  per-refresh host cap): HTTP 404 rejects the entry, 429 keeps it for
  a later refresh. YouTube labels adopt the real channel title.
- feeds table gains origin (config|manual) and enabled columns, with
  a guarded ALTER migration for existing databases.
- Only enabled feeds are refreshed and served: getItems/countItems
  filter by feed_url in enabled feeds, so deselected feeds' cached
  stories are hidden, not deleted; re-adding a URL re-enables it.
- Dashboard header gets a Manage link; manage page styled with the
  existing design tokens; README documents the feature.
This commit is contained in:
Harwinder Sekhon
2026-10-05 18:58:38 -04:00
parent 403fd2c604
commit 41f07b6ad8
14 changed files with 1123 additions and 28 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+74 -11
View File
@@ -15,6 +15,9 @@ No framework, no build step. PHP + SQLite for caching, Bootstrap 5 for the UI.
`fetched_at`), newest first.
- Filters by **source** (`youtube` / `reddit` / `rt`) and **time range**
(all time, 1h, 2h, 6h, 24h, 7d).
- Lets you **manually configure the subreddit and YouTube channel
selection** from the browser (`manage.php`, password protected): add
channels by `@handle`, disable or remove feeds — no config editing needed.
- Paginates with a **"Load more stories"** button (AJAX, 20 per page).
- Caches everything in **SQLite** so page loads never wait on the network; feeds
are refreshed on demand when stale (default TTL 30 min).
@@ -80,7 +83,9 @@ rm composer-setup.php
### 3. Configure
- **Feeds** — edit `config/feeds.php` (see *Configuration* for the format and the
YouTube `@handle` → `channel_id` note).
YouTube `@handle` → `channel_id` note), or skip this entirely and manage the
selection from the browser on `manage.php` (see *Managing feeds*; set its
password in `config/app.php`).
- **Runtime** — optionally tune `config/app.php` (page size, TTLs, ranges,
timeouts, user agent).
@@ -223,6 +228,8 @@ button when `has_more` is false.
### Caching & refresh semantics
- `items` and `feeds` live in `data/feeds.sqlite` (WAL mode, outside the web root).
- Only **enabled** feeds are refreshed and served — disabling one from the
manage page stops both without deleting its cached items.
- A feed is **stale** when `now - last_attempt_at >= cache_ttl`, or, if its last
attempt errored, when `now - last_attempt_at >= error_retry_ttl + jitter`
(jitter derived from `crc32(url)` so rate-limited feeds don't all retry at once).
@@ -239,23 +246,29 @@ button when `has_more` is false.
```
.
├── index.php Page entry point: refresh, filter, render page 1
├── manage.php Manual feed configuration (password protected)
├── api/items.php JSON endpoint for "Load more" (returns rendered HTML)
├── config/
│ ├── app.php Runtime settings (see Configuration)
│ └── feeds.php The feed list: [{source, label, url}, ...]
│ └── feeds.php The default feed list: [{source, label, url}, ...]
├── src/
│ ├── bootstrap.php Wires config + classes together, returns the container
│ ├── Database.php PDO SQLite connection + schema migration
│ ├── FeedFetcher.php HTTP + parse + normalize a single feed
│ ├── FeedRepository.php All SQL: upsert/query/count, staleness, attempt state
│ ├── FeedRepository.php All SQL: upsert/query/count, staleness, feed management
│ ├── FeedService.php Orchestration: refreshStale(), page(), sources()
│ ├── FeedManager.php Manual selection: addReddit()/addYouTube()/remove()/setEnabled()
│ ├── ChannelResolver.php YouTube @handle / channel URL → channel_id → Atom feed URL
│ ├── ManageAuth.php manage.php gate: password, CSRF, flash messages
│ └── View.php Escaping + render() + labels/colors/time formatting
├── templates/
│ ├── layout.php Full page shell, filters, grid, empty state, Load more
│ ├── manage.php Feed management page (login, add forms, feed list)
│ └── cards.php One feed card partial (used by page + API)
├── assets/
│ ├── app.js Load-more, relative timestamps, filter propagation
│ └── style.css Design tokens + card/pill/header styling
│ ├── manage.js Confirm dialogs on the manage page
│ └── style.css Design tokens + card/pill/header/manage styling
├── data/ SQLite database (gitignored, created at runtime)
└── vendor/ Composer deps (gitignored)
```
@@ -266,10 +279,13 @@ button when `has_more` is false.
|---|---|---|
| `App\Database` | Creates the SQLite file/dir, applies schema, sets WAL + busy timeout | `connect()` |
| `App\FeedFetcher` | Fetch one feed (with host throttle + 5xx retry), parse with laminas, extract media fields, normalize to a plain array | `fetch()` |
| `App\FeedRepository` | Every SQL statement; returns arrays/DTO-less rows | `syncFeeds()`, `staleFeeds()`, `upsertItems()`, `recordAttempt()`, `getItems()`, `countItems()`, `feedsStatus()` |
| `App\FeedRepository` | Every SQL statement; returns arrays/DTO-less rows | `syncFeeds()`, `enabledFeeds()`, `staleFeeds()`, `upsertItems()`, `recordAttempt()`, `getItems()`, `countItems()`, `addFeed()`, `setFeedEnabled()`, `deleteFeed()`, `feedsStatus()` |
| `App\FeedService` | Business flow: refresh policy + paginated reads | `refreshStale()`, `page()`, `sources()`, `feedsStatus()` |
| `App\FeedManager` | Manual selection changes; validates new feeds with one immediate fetch (404 rejects the entry) | `addReddit()`, `addYouTube()`, `remove()`, `setEnabled()` |
| `App\ChannelResolver` | YouTube input (`@handle`, `UC…` ID or channel URL) → Atom feed URL; resolves handles once via the channel page (youtube.com only) | `resolve()` |
| `App\ManageAuth` | Session gate for `manage.php`: password login, CSRF tokens, flash messages | `login()`, `authed()`, `csrfToken()`, `checkCsrf()` |
| `App\View` | Output escaping, template rendering, source labels/colors, ranges, `timeAgo()` | `e()`, `render()`, `sourceLabel()`, `sourceColor()`, `rangeLabel()`, `timeAgo()` |
| `src/bootstrap.php` | Builds and returns `['config','feeds','pdo','repository','fetcher','service']` | — |
| `src/bootstrap.php` | Builds and returns `['config','feeds','pdo','repository','fetcher','resolver','manager','service']` | — |
`FeedFetcher` notes: laminas does not expose the YouTube `media:` extension, so
`extractExtras()` does a second DOM/XPath pass over the raw XML to pull
@@ -298,14 +314,19 @@ escapes again on output).
Indexes: `idx_items_published(published_at DESC)`,
`idx_items_source(source, published_at DESC)`.
`feeds` — one row per configured feed + health state:
`feeds` — one row per feed (config or manually added) + selection/health state:
| column | notes |
|---|---|
| `url` | primary key |
| `source`, `label` | from `config/feeds.php` |
| `source`, `label` | from `config/feeds.php` or the manage page |
| `last_attempt_at`, `last_success_at` | ISO-8601 UTC |
| `last_error` | last failure message, `NULL` on success |
| `origin` | `config` (from `config/feeds.php`) or `manual` (added via manage.php) |
| `enabled` | `1` = selected; disabled feeds are neither refreshed nor shown |
Only items whose `feed_url` belongs to an enabled feed are served — disabling or
removing a feed hides its cached stories without deleting them.
---
@@ -325,6 +346,7 @@ Indexes: `idx_items_published(published_at DESC)`,
| `ranges` | `1h,2h,6h,24h,7d` | time-filter windows (key → seconds) |
| `request_timeout` / `connect_timeout` | `15` / `10` | Guzzle timeouts |
| `user_agent` | `RSSFeed/1.0 (...)` | sent to providers (matters for Reddit) |
| `manage_password` | `rssfeed-admin` | password for `manage.php`; plain or `password_hash()` value, empty = management disabled |
`config/feeds.php` returns a list of feeds:
@@ -343,12 +365,52 @@ Resolve a handle once (fetch `https://www.youtube.com/@handle` and read
---
## Managing feeds (manual configuration)
`manage.php` (linked as **Manage** in the dashboard header) configures the
subreddit/YouTube selection from the browser. It is password protected:
- Set `manage_password` in `config/app.php`. It accepts a plain password (fine
for a LAN/dev box) or a `password_hash()` value for better hygiene:
```bash
php -r 'echo password_hash("your secret", PASSWORD_DEFAULT), PHP_EOL;'
```
- An empty value disables the page (it shows a setup notice instead).
What you can do there:
| Action | Input | Behavior |
|---|---|---|
| Add subreddit | `homelab`, `r/homelab`, or any `reddit.com/r/…` URL | Normalized to `https://www.reddit.com/r/<name>/.rss`, labeled `r/<name>` |
| Add YouTube channel | `@handle`, `UC…` channel ID, or any channel URL | Handle resolved to `channel_id` once, stored as the Atom feed URL; the label becomes the real channel title |
| Disable / enable | toggle per feed | Disabled feeds are neither refreshed nor shown (their cached stories are kept and reappear when re-enabled) |
| Remove | manual feeds | Deleted from the selection; re-add anytime |
| Remove | config feeds | Hidden (`enabled = 0`) — `config/feeds.php` re-registers them on every sync, so they cannot be deleted from the UI |
Semantics worth knowing:
- Adding a feed **fetches it immediately** (bypassing the per-refresh Reddit
host cap): an `HTTP 404` rejects the entry ("not found, nothing was added"),
while a rate-limited (`429`) add is kept and fills in on a later refresh.
- Re-adding an existing feed URL **re-enables** it.
- Handle resolution only ever requests `youtube.com` pages — user input can
never point the resolver at arbitrary URLs.
- All POST actions are CSRF-protected (session token) and use POST-then-redirect.
- `config/feeds.php` remains the default set and keeps working as before; the
manage page layers manual feeds and on/off state on top of it (persisted in
the `feeds` table, so they survive restarts).
---
## Routes & query parameters
| Route | Params | Returns |
|---|---|---|
| `index.php` (`/RSSFeed/`) | `source` (optional), `range` (optional) | full HTML page, page 1 |
| `api/items.php` | `page` (default 1), `source`, `range` | JSON `{ html, page, page_size, count, total, has_more, range, refresh }` |
| `manage.php` | — (POST: `login`, `logout`, `add_reddit`, `add_youtube`, `toggle`, `remove`) | management page (password gated) |
- `source` must match a `source` in `config/feeds.php`, else ignored.
- `range` must be a key in `ranges`, else ignored.
@@ -358,9 +420,10 @@ Resolve a handle once (fetch `https://www.youtube.com/@handle` and read
## Adding feeds and sources
**New feed (existing source):** add an entry to `config/feeds.php`. `syncFeeds()`
registers it automatically on the next request; it will be fetched on the next
refresh.
**New feed (existing source):** either add it on the **manage page** (no code
editing, see *Managing feeds*) or add an entry to `config/feeds.php`.
`syncFeeds()` registers config entries automatically on the next request; they
will be fetched on the next refresh.
**New source type** (e.g. `hackernews`): add feeds with `'source' => 'hackernews'`,
then teach the UI its identity in `src/View.php`: