diff --git a/README.md b/README.md index c990706..ef6ea35 100644 --- a/README.md +++ b/README.md @@ -192,6 +192,48 @@ indexed. The search box still matches without regard to case. 6. Open the site, create an account, and watch. +## Database Schema + +CourseHub uses MySQL 8.0+ or MariaDB 10.5+. The schema is applied with: + +```bash +mysql coursehub < schema.sql +``` + +Each table's purpose and columns: + +| Table | Purpose | +| --- | --- | +| **users** | User accounts: username, password hash, registration timestamp | +| **folders** | Directory hierarchy: every folder under VIDEOS_ROOT with parent, depth, sort order, whether it's a course, and which course owns it (nullable for categories) | +| **videos** | MP4 files found by the scanner: folder they live in, name, path, filesize, duration (filled in by the player on first play), sort order, and whether they're still on disk | +| **quizzes** | Quiz files (`*_quiz_solved.txt`): folder they live in, filename, path, question count (parsed from the file), sort order, and whether they're still on disk | +| **catalog_courses** | The vendor's course catalogue (e.g. INE): vendor uuid, name, slug, folder key (used to match library folders), difficulty, tags, descriptions (short and full), learning paths, categories, duration, and the vendor's page URL. Applied via `seeds/ine_catalog.sql`, not written by the scanner | +| **course_meta** | What's known about each library course beyond its folder name: source (sidecar or catalogue), which catalogue id matched it, and all the metadata (title, difficulty, tags, descriptions, paths, categories, duration). Rewritten by every scan from sidecars and the catalogue | +| **video_progress** | Per-user playback state: position (seconds) and completion flag (sticky at ≥95% watched), with an updated timestamp for cleanup | +| **quiz_progress** | Per-user quiz results: best score, last score, total questions, attempt count, and a completion flag (sticky when all questions answered correctly in one attempt) | +| **login_attempts** | Failed sign-in attempts (username + IP): timestamp and count. Used by the login throttle (5 failures per username or 20 per IP in 15 minutes = 429 response). Rows older than an hour are purged; created lazily on first failure | +| **advisor_requests** | Questions put to the AI course advisor: user, timestamp, whether it succeeded, the model name, and token counts (prompt, cache hit, completion). Created lazily on first use. Kept to track the per-user daily question limit and usage cost | +| **app_meta** | App-wide key/value store: currently just `last_scan_at` (the timestamp of the most recent library scan). Created lazily on first scan; reads tolerate its absence | + +### Key relationships and constraints + +- **Folders** reference themselves (`parent_id`) and have a `course_id` pointing to their nearest course ancestor (NULL for top-level categories). +- **Videos** and **quizzes** reference the folder they sit in (`folder_id`); deleted folders cascade-delete their contents. +- **course_meta** rows reference the folder they describe (`folder_id`); a course has exactly one row or none. +- **catalog_courses** is never touched by the scanner; it's applied separately via `seeds/ine_catalog.sql` and matched to library folders by `folder_key`. +- **Progress tables** (video_progress, quiz_progress, advisor_requests, login_attempts) cascade-delete when a user is deleted. +- **Paths are case-sensitive** (`utf8mb4_bin` collation) to match the filesystem; the rest of the table uses case-insensitive collation. The `UNIQUE` key on `path` ensures no duplicates. + +### Indices + +- `folders`: parent lookup, course_id grouping for rollups, `is_course` + `is_missing` filtering. +- `videos` and `quizzes`: folder lookup, implicit on foreign key constraints. +- `login_attempts`: username+time and ip+time for throttling queries. +- `advisor_requests`: user+time for daily-limit checks. +- `video_progress`: user+time for the periodic progress save beacon. +- `catalog_courses`: folder_key for matching against library folders. + ## Quizzes Course folders may contain quiz text files next to the videos. Files named @@ -310,10 +352,19 @@ question are described in any depth. The model replies in JSON; the server drops any course id it made up before the browser sees it, and links are built only from ids that exist. That applies to the prose as well — an id the model writes inline as `#412` is resolved against the same check and becomes a link -to that course, and anything that does not resolve is left as plain text. Each question (including one that fails) counts toward the -per-user daily limit, kept in `advisor_requests` together with the token -counts, so you can see what it costs. A conversation lives in the browser tab -and is resent with each turn, so follow-ups ("I already know MPLS") work. +to that course, and anything that does not resolve is left as plain text. + +Each question (including one that fails) counts toward the per-user daily limit. +The `advisor_requests` table records: the user, when the question was asked, +whether it succeeded, the model name used, and the token counts (prompt, cache +hit, and completion). This lets you audit the advisor's cost and load. The +daily limit resets in a rolling window: a question asked at midnight counts +toward the limit until midnight tomorrow. + +A conversation lives in the browser tab (in `sessionStorage` under the key +`coursehub.advisor.chat`), survives navigating between pages, and is resent +with each turn so follow-ups ("I already know MPLS") work. The conversation +is cleared when the user signs out or closes the tab. Any OpenAI-format chat endpoint can stand in for DeepSeek — point `AI_BASE_URL` and `AI_MODEL` at it; a local model server keeps even the advisor offline. If @@ -328,11 +379,16 @@ apply — leave them at a minute or more. ## Search The top-bar box searches as you type (from two characters, debounced) and also -on Enter. It matches folder, video and quiz **names** and their relative -**paths**, so `Course 1/Intro` lists everything under that section. Results are -grouped into courses/sections, videos and quizzes, with your progress shown on +on Enter. It matches against: + +- **Folder names and paths** — `folders.name` and `folders.path` (case-insensitive despite the `utf8mb4_bin` collation on path, because the search query is lowercased). +- **Video names and paths** — `videos.name` and `videos.path`. +- **Quiz names and paths** — `quizzes.name` and `quizzes.path`. +- **Course metadata** — once the catalogue is seeded, the `course_meta` table's `title`, `tags` (JSON array, searched as text), and `description` fields are indexed. + +Results are grouped into courses/sections, videos and quizzes, with your progress (from `video_progress` and `quiz_progress` tables) shown on each row. Searches live at `#/search/`, so a result page can be -bookmarked or shared. +bookmarked or shared. Only items with `is_missing = 0` are shown (files that still exist on disk). ## Resetting progress @@ -392,14 +448,14 @@ next to it, and grids drop to two columns. `#/settings` shows what the server actually knows and what this browser remembers: -- **Library folder** — `VIDEOS_ROOT`, whether it is readable, indexed counts, - total running time, the watched threshold, and when the last scan ran. -- **Preferences**, stored per browser: autoplay next video, open the library - in list view. -- **Reset progress** — the account-wide reset, with a count of how many courses - currently hold progress. +- **Library folder** — `VIDEOS_ROOT`, whether it is readable, indexed counts (from `folders`, `videos`, `quizzes` tables with `is_missing = 0`), + total running time (summed from `videos.duration_seconds`), the watched threshold (`COMPLETE_THRESHOLD`, defaults to 95%), and when the last scan ran (from `app_meta.last_scan_at`). +- **Course metadata** — how many courses have descriptions (from `course_meta` table), how many came from a sidecar JSON (`source = 'sidecar'`), and how many from the catalogue (`source = 'catalog'`). +- **Course advisor** — enabled or off, the model name if on, how many questions you've asked today (count from `advisor_requests` in the rolling 24-hour window), and the daily limit. +- **Preferences**, stored per browser (in `localStorage`): autoplay next video, open the library in list view, and whether to show the advisor button. +- **Reset progress** — the account-wide reset, with a count of how many courses currently hold progress (from `video_progress` and `quiz_progress` grouped by course). -Scan time lives in an `app_meta` key/value table. Reads tolerate the table not +The scan timestamp lives in the `app_meta` key/value table under the key `last_scan_at`. Reads tolerate the table not existing, so an install created before it was added keeps working until its next scan creates it. @@ -408,18 +464,27 @@ next scan creates it. - The player POSTs `{video_id, position, duration}` to `api/progress.php` every 5 s while playing, plus on pause/seek, via `navigator.sendBeacon` when the tab is hidden or closed, and when navigating to another view - in the SPA. -- A video is marked **completed** at ≥ 95 % watched (`COMPLETE_THRESHOLD`). - Completion is sticky — rewatching doesn't un-complete it. + in the SPA. These updates write to the `video_progress` table, one row per + user per video, with the position (seconds), reported duration, completion flag, + and an updated timestamp. +- A video is marked **completed** (in `video_progress.completed`) at ≥ 95 % watched + (`COMPLETE_THRESHOLD`). Completion is sticky — rewatching doesn't un-complete it. + The same logic applies to quizzes: once `quiz_progress.completed` is true + (every question answered correctly in one attempt), it stays true. - Folder progress is computed over the folder's entire subtree using a path - prefix match, with partial credit for videos in progress. -- Every course is rolled up in **two queries for the whole library**, grouped by - `folders.course_id`, and a folder's children in two more — not two per card. - A library of a few hundred courses would otherwise spend a page load on - several hundred prefix-matched queries. + prefix match against `folders.path`, with partial credit for videos in progress. + This calculation spans both `video_progress` and `quiz_progress` tables. +- **Querying for progress is optimized:** every course is rolled up in + **two queries for the whole library** (one for videos, one for quizzes), + grouped by `folders.course_id`, and a folder's children in two more — not + two per card. A library of a few hundred courses would otherwise spend a + page load on several hundred prefix-matched queries. See `courseRollups()` and + `advisorProgress()` in `includes/functions.php`. - Video duration is read at scan time by parsing the mp4's `mvhd` header in - pure PHP, so listings show durations before anyone plays the video. If a file - can't be parsed, the browser reports its duration on first play instead. + pure PHP, so listings show durations before anyone plays the video. The duration + is stored in `videos.duration_seconds`. If a file can't be parsed at scan time, + the `duration_seconds` is NULL and the browser reports its duration on first play, + updating `video_progress.duration_seconds` for that user. ## When something breaks