Seed the vendor catalogue into SQL and add an optional AI course advisor

Course metadata: a scan now gives every course a course_meta row from the
downloader's sidecar JSON when the folder has one, else from catalog_courses
matched by the folder name the downloader would create (folderKeyFor, a port
of ine.py's sanitize_name). tools/build-catalog-seed.php turns the INE export
into seeds/ine_catalog.sql, an idempotent patch that creates its own table and
upserts by id, so it can be applied to the live server before or after the
code and re-run freely. Course pages show the blurb, difficulty, vendor and
tags; search matches tags and blurbs; Settings reports how each course got its
details. Against the live library 163 of 179 courses match.

Course advisor, off by default: api/advisor.php answers "which course should I
take to learn X / what comes before Y" with courses from the library, in
order, plus vendor courses not yet downloaded. includes/ai_client.php is a
small curl wrapper for any OpenAI-format chat endpoint (DeepSeek V4 Flash by
default, no composer); the prompt is a byte-stable catalogue of the library so
the provider's prefix cache covers it. Every course id the model returns is
validated server-side, a bad JSON reply gets one retry then a plain-text
fallback, and a per-user daily cap with token counts lives in
advisor_requests. With AI_ENABLED false the app makes no outbound requests
and the box does not appear; users can also hide it per browser.

Claude-Session: https://claude.ai/code/session_012enquL2RZdjy3D5AcKP5yx
This commit is contained in:
Harwinder Sekhon
2026-09-08 18:36:43 -04:00
parent 04314327db
commit 59fab56359
14 changed files with 1548 additions and 16 deletions
+103 -3
View File
@@ -36,7 +36,14 @@ plain HTTP resources.
- **Subtitles**: a video with a sibling `.srt` file (`Lesson.mp4` +
`Lesson.srt`) gets a CC button in the player — the file is converted to
WebVTT on the fly by `captions.php`, addressed by DB id like the stream.
- **Search** courses, sections, videos and quizzes from the top bar.
- **Search** courses, sections, videos and quizzes from the top bar — and a
course's catalogue tags and blurb, once the catalogue is seeded.
- **Course details** (difficulty, tags, a blurb, the learning paths a course
belongs to) from the vendor's catalogue or the downloader's sidecar file —
see *Course catalogue* below.
- An optional **AI course advisor** on the home page: "I want to learn SD-WAN,
where do I start?" answered with courses from your own library, in order,
with links. Off unless you switch it on — see *Course advisor* below.
- **Reset progress** for one video, one quiz, a whole course/section, or the
entire account.
- Course and section listings switch between **grid and list** layout, and the
@@ -178,7 +185,11 @@ indexed. The search box still matches without regard to case.
re-derives which folders are courses, and brings an older install's tables up
to date first, so there is no separate migration step.
5. Open the site, create an account, and watch.
5. **Seed the course catalogue** (optional, once) — see *Course catalogue*
below. It gives courses a description, difficulty and tags, and it is what
the advisor reasons over.
6. Open the site, create an account, and watch.
## Quizzes
@@ -217,6 +228,93 @@ correct set. Best score, last score, and attempt count are stored per user;
a quiz is *completed* once every question is answered correctly in one
attempt, and quiz scores count toward folder progress alongside videos.
## Course catalogue
Folder names are all the scanner can see. Two things add more:
- **A sidecar file.** The INE downloader (`ine.py`) drops a `<slug>.json` into a
course folder once every file of the course has been fetched. It carries the
course's title, difficulty, tags, description and learning paths, and the
scanner reads it.
- **The catalogue seed.** The downloader can also export the whole catalogue
you have access to (`all_courses_with_access.json`). Turn it into an SQL
patch and apply it to any install, dev or live, before or after a code
deploy — it creates its own table and upserts by id, so re-running it is
harmless:
```bash
php tools/build-catalog-seed.php ~/ine-dl/all_courses_with_access.json > seeds/ine_catalog.sql
mysql coursehub < seeds/ine_catalog.sql # or: sudo mysql coursehub < …
php scan.php # matches the library to it
```
`seeds/` is git-ignored (the file holds the vendor's course descriptions);
ship it alongside the code. Rebuild it whenever the export is refreshed.
Every scan then resolves one `course_meta` row per course: the sidecar when
there is one, otherwise the catalogue entry whose **folder key** matches the
course folder's name. The key is the folder name the downloader would have
created — spaces and hyphens become underscores, other punctuation is dropped
(`folderKeyFor()` in `includes/functions.php`) — so `Cisco SD-WAN Use Design,
Cloud & Advanced Features` finds `Cisco_SD_WAN_Use_Design_Cloud_Advanced_Features`,
and a folder renamed by hand still matches as long as the words are the same.
Courses from elsewhere (CBT Nuggets, your own notes) simply have no row and
look as they always did. The scan summary and the Settings page say how many
courses got their details from a sidecar, from the catalogue, or from neither.
Any other `.json` under the library (the downloader also writes one per lab
lesson) is ignored.
## Course advisor
An optional box on the home page that answers questions like *"I want to
learn SD-WAN, where do I start?"* or *"what should I know before Core MPLS?"*
with courses from your own library, in a suggested order, each linking to its
course page with your progress on it. When something you need is not on the
server but is in the seeded catalogue, it says so and links to the vendor's
page; topics covered nowhere are listed too.
It is **off by default** and the app never depends on it. To turn it on, set
in `config.php`:
```php
define('AI_ENABLED', true);
define('AI_API_KEY', 'sk-…'); // https://platform.deepseek.com/
define('AI_BASE_URL', 'https://api.deepseek.com'); // any OpenAI-format endpoint
define('AI_MODEL', 'deepseek-v4-flash');
define('AI_DAILY_LIMIT', 50); // questions per user per day
```
With `AI_ENABLED` false (or no key) the box does not appear, `api/advisor.php`
answers 503, and the app makes **no outbound requests at all**. The one extra
requirement when it is on is PHP's curl extension. Each user can also hide the
box for their browser from Settings.
What leaves the server when it is on: the names of your courses and sections
with their sizes, the catalogue details (difficulty, tags, blurb, learning
paths), the catalogue titles of courses you have access to but have not
downloaded, your progress percentage per course, and the conversation you
type. Never video files, subtitles or quiz contents.
How it works: the whole library is rendered as one stable block of text and
sent with every question, and the provider caches that prefix, so a question
costs a fraction of a cent. The model replies in JSON; the server drops any
course id it made up before the browser sees it, and links are built only from
ids that exist. Each question (including one that fails) counts toward the
per-user daily limit, kept in `advisor_requests` together with the token
counts, so you can see what it costs. A conversation lives in the browser tab
and is resent with each turn, so follow-ups ("I already know MPLS") work.
Any OpenAI-format chat endpoint can stand in for DeepSeek — point `AI_BASE_URL`
and `AI_MODEL` at it; a local model server keeps even the advisor offline. If
the server rejects DeepSeek's `thinking` / `reasoning_effort` fields, add
`define('AI_REASONING_PARAM', false);` to leave them out.
Timeouts: a reply takes a few seconds to tens of seconds. PHP's own limit is
raised for the call, but PHP-FPM's `request_terminate_timeout`, Nginx's
`fastcgi_read_timeout` (60 s by default) and Cloudflare's ~100 s cap still
apply — leave them at a minute or more.
## Search
The top-bar box searches as you type (from two characters, debounced) and also
@@ -352,11 +450,13 @@ returned as JSON, with detail gated behind `DEBUG`.
if the origin is unreachable except through the proxy; otherwise the header
can be spoofed to dodge the per-IP limit.
- The shipped `.htaccess` (Apache) denies direct access to `videos/`,
`includes/`, `config.php` and `schema.sql` — without it, video URLs would
`includes/`, `seeds/`, `tools/`, `config.php` and `schema.sql` — without it, video URLs would
bypass login entirely. The rules match those folder names anywhere in the
URL path, so they hold whether the app lives at `/` or under a prefix such as
`/coursehub/`. Nginx setups need the equivalent `location` blocks, or keep
the videos and includes outside the docroot (the safest choice either way).
- The server makes no outbound requests unless the AI advisor is switched on,
and then only to `AI_BASE_URL` (see *Course advisor*).
- Only `.mp4` files ever reach the browser, and only through `stream.php`.
A quiz's exhibit image is the one thing loaded from elsewhere — its URL is
whatever the quiz file says, sent with `referrerpolicy="no-referrer"`.