harry/coursehub
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Document database schema and add comprehensive table information

Browse Source 
- Add new 'Database Schema' section with detailed table descriptions
- Document each table's purpose, columns, and relationships
- Explain key constraints, foreign keys, and cascade-delete behavior
- List indices and their purposes for performance optimization
- Enhance Course Advisor section with advisor_requests table details
- Explain the rolling 24-hour daily limit window and token tracking
- Clarify where conversation data is stored (sessionStorage)
- Improve Settings section with database references
- Enhance Search section with specific table and field information
- Expand 'How progress works' with database table references
- Document the optimization of progress queries for performance
- Reference specific functions (courseRollups, advisorProgress)
- Clarify when tables are created lazily (login_attempts, app_meta, advisor_requests)

This makes the database structure transparent for users, operators, and developers.
This commit is contained in:
Harwinder Sekhon
2026-09-10 07:04:46 -04:00
parent 76d8f259b2
commit 07b7aa758c
1 changed files with 90 additions and 25 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+90 -25
View File
@@ -192,6 +192,48 @@ indexed. The search box still matches without regard to case.
6. Open the site, create an account, and watch.
## Database Schema
CourseHub uses MySQL 8.0+ or MariaDB 10.5+. The schema is applied with:
```bash
mysql coursehub < schema.sql
```
Each table's purpose and columns:
| Table | Purpose |
| --- | --- |
| **users** | User accounts: username, password hash, registration timestamp |
| **folders** | Directory hierarchy: every folder under VIDEOS_ROOT with parent, depth, sort order, whether it's a course, and which course owns it (nullable for categories) |
| **videos** | MP4 files found by the scanner: folder they live in, name, path, filesize, duration (filled in by the player on first play), sort order, and whether they're still on disk |
| **quizzes** | Quiz files (`*_quiz_solved.txt`): folder they live in, filename, path, question count (parsed from the file), sort order, and whether they're still on disk |
| **catalog_courses** | The vendor's course catalogue (e.g. INE): vendor uuid, name, slug, folder key (used to match library folders), difficulty, tags, descriptions (short and full), learning paths, categories, duration, and the vendor's page URL. Applied via `seeds/ine_catalog.sql`, not written by the scanner |
| **course_meta** | What's known about each library course beyond its folder name: source (sidecar or catalogue), which catalogue id matched it, and all the metadata (title, difficulty, tags, descriptions, paths, categories, duration). Rewritten by every scan from sidecars and the catalogue |
| **video_progress** | Per-user playback state: position (seconds) and completion flag (sticky at ≥95% watched), with an updated timestamp for cleanup |
| **quiz_progress** | Per-user quiz results: best score, last score, total questions, attempt count, and a completion flag (sticky when all questions answered correctly in one attempt) |
| **login_attempts** | Failed sign-in attempts (username + IP): timestamp and count. Used by the login throttle (5 failures per username or 20 per IP in 15 minutes = 429 response). Rows older than an hour are purged; created lazily on first failure |
| **advisor_requests** | Questions put to the AI course advisor: user, timestamp, whether it succeeded, the model name, and token counts (prompt, cache hit, completion). Created lazily on first use. Kept to track the per-user daily question limit and usage cost |
| **app_meta** | App-wide key/value store: currently just `last_scan_at` (the timestamp of the most recent library scan). Created lazily on first scan; reads tolerate its absence |
### Key relationships and constraints
- **Folders** reference themselves (`parent_id`) and have a `course_id` pointing to their nearest course ancestor (NULL for top-level categories).
- **Videos** and **quizzes** reference the folder they sit in (`folder_id`); deleted folders cascade-delete their contents.
- **course_meta** rows reference the folder they describe (`folder_id`); a course has exactly one row or none.
- **catalog_courses** is never touched by the scanner; it's applied separately via `seeds/ine_catalog.sql` and matched to library folders by `folder_key`.
- **Progress tables** (video_progress, quiz_progress, advisor_requests, login_attempts) cascade-delete when a user is deleted.
- **Paths are case-sensitive** (`utf8mb4_bin` collation) to match the filesystem; the rest of the table uses case-insensitive collation. The `UNIQUE` key on `path` ensures no duplicates.
### Indices
- `folders`: parent lookup, course_id grouping for rollups, `is_course` + `is_missing` filtering.
- `videos` and `quizzes`: folder lookup, implicit on foreign key constraints.
- `login_attempts`: username+time and ip+time for throttling queries.
- `advisor_requests`: user+time for daily-limit checks.
- `video_progress`: user+time for the periodic progress save beacon.
- `catalog_courses`: folder_key for matching against library folders.
## Quizzes
Course folders may contain quiz text files next to the videos. Files named
@@ -310,10 +352,19 @@ question are described in any depth. The model replies in JSON; the server
drops any course id it made up before the browser sees it, and links are built
only from ids that exist. That applies to the prose as well — an id the model
writes inline as `#412` is resolved against the same check and becomes a link
to that course, and anything that does not resolve is left as plain text. Each question (including one that fails) counts toward the
per-user daily limit, kept in `advisor_requests` together with the token
counts, so you can see what it costs. A conversation lives in the browser tab
and is resent with each turn, so follow-ups ("I already know MPLS") work.
to that course, and anything that does not resolve is left as plain text.
Each question (including one that fails) counts toward the per-user daily limit.
The `advisor_requests` table records: the user, when the question was asked,
whether it succeeded, the model name used, and the token counts (prompt, cache
hit, and completion). This lets you audit the advisor's cost and load. The
daily limit resets in a rolling window: a question asked at midnight counts
toward the limit until midnight tomorrow.
A conversation lives in the browser tab (in `sessionStorage` under the key
`coursehub.advisor.chat`), survives navigating between pages, and is resent
with each turn so follow-ups ("I already know MPLS") work. The conversation
is cleared when the user signs out or closes the tab.
Any OpenAI-format chat endpoint can stand in for DeepSeek — point `AI_BASE_URL`
and `AI_MODEL` at it; a local model server keeps even the advisor offline. If
@@ -328,11 +379,16 @@ apply — leave them at a minute or more.
## Search
The top-bar box searches as you type (from two characters, debounced) and also
on Enter. It matches folder, video and quiz **names** and their relative
**paths**, so `Course 1/Intro` lists everything under that section. Results are
grouped into courses/sections, videos and quizzes, with your progress shown on
on Enter. It matches against:
- **Folder names and paths** — `folders.name` and `folders.path` (case-insensitive despite the `utf8mb4_bin` collation on path, because the search query is lowercased).
- **Video names and paths** — `videos.name` and `videos.path`.
- **Quiz names and paths** — `quizzes.name` and `quizzes.path`.
- **Course metadata** — once the catalogue is seeded, the `course_meta` table's `title`, `tags` (JSON array, searched as text), and `description` fields are indexed.
Results are grouped into courses/sections, videos and quizzes, with your progress (from `video_progress` and `quiz_progress` tables) shown on
each row. Searches live at `#/search/<query>`, so a result page can be
bookmarked or shared.
bookmarked or shared. Only items with `is_missing = 0` are shown (files that still exist on disk).
## Resetting progress
@@ -392,14 +448,14 @@ next to it, and grids drop to two columns.
`#/settings` shows what the server actually knows and what this browser
remembers:
- **Library folder** — `VIDEOS_ROOT`, whether it is readable, indexed counts,
total running time, the watched threshold, and when the last scan ran.
- **Preferences**, stored per browser: autoplay next video, open the library
in list view.
- **Reset progress** — the account-wide reset, with a count of how many courses
currently hold progress.
- **Library folder** — `VIDEOS_ROOT`, whether it is readable, indexed counts (from `folders`, `videos`, `quizzes` tables with `is_missing = 0`),
total running time (summed from `videos.duration_seconds`), the watched threshold (`COMPLETE_THRESHOLD`, defaults to 95%), and when the last scan ran (from `app_meta.last_scan_at`).
- **Course metadata** — how many courses have descriptions (from `course_meta` table), how many came from a sidecar JSON (`source = 'sidecar'`), and how many from the catalogue (`source = 'catalog'`).
- **Course advisor** — enabled or off, the model name if on, how many questions you've asked today (count from `advisor_requests` in the rolling 24-hour window), and the daily limit.
- **Preferences**, stored per browser (in `localStorage`): autoplay next video, open the library in list view, and whether to show the advisor button.
- **Reset progress** — the account-wide reset, with a count of how many courses currently hold progress (from `video_progress` and `quiz_progress` grouped by course).
Scan time lives in an `app_meta` key/value table. Reads tolerate the table not
The scan timestamp lives in the `app_meta` key/value table under the key `last_scan_at`. Reads tolerate the table not
existing, so an install created before it was added keeps working until its
next scan creates it.
@@ -408,18 +464,27 @@ next scan creates it.
- The player POSTs `{video_id, position, duration}` to `api/progress.php`
every 5 s while playing, plus on pause/seek, via `navigator.sendBeacon`
when the tab is hidden or closed, and when navigating to another view
in the SPA.
- A video is marked **completed** at ≥ 95 % watched (`COMPLETE_THRESHOLD`).
Completion is sticky — rewatching doesn't un-complete it.
in the SPA. These updates write to the `video_progress` table, one row per
user per video, with the position (seconds), reported duration, completion flag,
and an updated timestamp.
- A video is marked **completed** (in `video_progress.completed`) at ≥ 95 % watched
(`COMPLETE_THRESHOLD`). Completion is sticky — rewatching doesn't un-complete it.
The same logic applies to quizzes: once `quiz_progress.completed` is true
(every question answered correctly in one attempt), it stays true.
- Folder progress is computed over the folder's entire subtree using a path
prefix match, with partial credit for videos in progress.
- Every course is rolled up in **two queries for the whole library**, grouped by
`folders.course_id`, and a folder's children in two more — not two per card.
A library of a few hundred courses would otherwise spend a page load on
several hundred prefix-matched queries.
prefix match against `folders.path`, with partial credit for videos in progress.
This calculation spans both `video_progress` and `quiz_progress` tables.
- **Querying for progress is optimized:** every course is rolled up in
**two queries for the whole library** (one for videos, one for quizzes),
grouped by `folders.course_id`, and a folder's children in two more — not
two per card. A library of a few hundred courses would otherwise spend a
page load on several hundred prefix-matched queries. See `courseRollups()` and
`advisorProgress()` in `includes/functions.php`.
- Video duration is read at scan time by parsing the mp4's `mvhd` header in
pure PHP, so listings show durations before anyone plays the video. If a file
can't be parsed, the browser reports its duration on first play instead.
pure PHP, so listings show durations before anyone plays the video. The duration
is stored in `videos.duration_seconds`. If a file can't be parsed at scan time,
the `duration_seconds` is NULL and the browser reports its duration on first play,
updating `video_progress.duration_seconds` for that user.
## When something breaks