News-ML-Decoder/WORKFLOW_AND_USAGE.md

XAUUSD Sentiment Workflow and Usage Guide

Purpose

This model answers one question:

Does a news headline imply bullish, neutral, or bearish sentiment for XAUUSD?

• Bullish means the news supports a higher XAUUSD price.
• Bearish means the news supports a lower XAUUSD price.
• Neutral means the direction is unclear, weak, or conflicting.

The model runs locally with scikit-learn. It does not require an API call and can be placed before an existing LLM news-analysis step to reduce latency and LLM usage.

Files

Path	Purpose
news_sentiment.py	Training, scoring, and Python API
newsdata.db	SQLite training data
models/xauusd_sentiment.joblib	Trained model artifact
models/xauusd_sentiment.metrics.json	Training and evaluation metrics
requirements.txt	Runtime dependencies

Recommended Workflow

flowchart LR
    A[New article] --> B[Validate headline and description]
    B --> C[XAUUSD local model]
    C --> D{Prediction confident?}
    D -->|Yes| E[Use local prediction]
    D -->|No| F[Call existing LLM]
    E --> G[Store result and source]
    F --> G
    G --> H[Retain reviewed or LLM labels]
    H --> I[Periodic model retraining]

Start in shadow mode: run the local model and LLM together, but continue using the LLM result. Log both outputs for comparison. After validating the model on live news, enable the confidence-based LLM fallback workflow.

Environment Setup

From this repository:

python3 -m venv .venv
. .venv/bin/activate
python -m pip install -r requirements.txt

The saved model was built with the exact versions in requirements.txt.

If the existing Python project is in another directory, make this repository importable when starting it:

export PYTHONPATH="/home/harwinder/Development/ML-NEWS:$PYTHONPATH"
export XAUUSD_MODEL_PATH="/home/harwinder/Development/ML-NEWS/models/xauusd_sentiment.joblib"
python /path/to/existing/project/main.py

Add the same environment variables to the service manager, Docker container, or deployment configuration used in production.

Command-Line Usage

Score one headline:

.venv/bin/python news_sentiment.py score \
  --headline "Gold gains as the dollar weakens"

Score a headline and description:

.venv/bin/python news_sentiment.py score \
  --headline "Gold slides as Treasury yields rise" \
  --description "A stronger US dollar adds pressure to bullion."

The output has this structure:

{
  "target": "XAUUSD",
  "label": "bearish",
  "score": -0.82,
  "confidence": 0.87,
  "probabilities": {
    "bearish": 0.87,
    "neutral": 0.08,
    "bullish": 0.05
  }
}

The score is calculated as:

P(XAUUSD bullish) - P(XAUUSD bearish)

Use label as the primary classification. A neutral prediction can still have a positive or negative score when the model sees a secondary directional lean.

Batch JSONL Usage

Input records must be newline-delimited JSON. Each record can contain headline, title, or text, plus an optional description:

{"id": 1, "headline": "Gold rises as the dollar falls"}
{"id": 2, "headline": "Gold holds steady ahead of CPI"}
{"id": 3, "title": "Gold falls as Treasury yields climb"}

Score the file:

.venv/bin/python news_sentiment.py score --input articles.jsonl > scored.jsonl

Each output record preserves the input fields and adds a sentiment object.

Python Integration

Load the model once when the application process starts. Do not load it for every article.

import os
from pathlib import Path

from news_sentiment import load_model, score_news


MODEL_PATH = Path(
    os.environ.get(
        "XAUUSD_MODEL_PATH",
        "/home/harwinder/Development/ML-NEWS/models/xauusd_sentiment.joblib",
    )
)

XAUUSD_MODEL = load_model(MODEL_PATH)


def analyze_xauusd_news(news: dict) -> dict:
    prediction = score_news(
        XAUUSD_MODEL,
        headline=news["headline"],
        description=news.get("description"),
    )

    return {
        "target": prediction["target"],
        "sentiment": prediction["label"],
        "score": prediction["score"],
        "confidence": prediction["confidence"],
        "probabilities": prediction["probabilities"],
        "score_source": "xauusd_sklearn_v3",
    }

Example:

news = {
    "headline": "Gold rallies as the US dollar weakens",
    "description": "Falling Treasury yields provided additional support.",
}

result = analyze_xauusd_news(news)
print(result)

Connecting to the Existing LLM Workflow

Assume the existing project currently calls:

result = analyze_news_with_llm(headline, description)

For a full local replacement:

result = analyze_xauusd_news(
    {
        "headline": headline,
        "description": description,
    }
)

For the recommended hybrid workflow, call the LLM only when the local result is uncertain:

def analyze_news(news: dict) -> dict:
    try:
        ml_result = analyze_xauusd_news(news)
    except Exception:
        llm_result = analyze_news_with_llm(
            news["headline"], news.get("description")
        )
        llm_result["score_source"] = "llm_model_error_fallback"
        return llm_result

    ordered_probabilities = sorted(
        ml_result["probabilities"].values(), reverse=True
    )
    probability_margin = ordered_probabilities[0] - ordered_probabilities[1]

    uncertain = (
        ml_result["confidence"] < 0.70
        or probability_margin < 0.20
    )

    if not uncertain:
        return ml_result

    llm_result = analyze_news_with_llm(
        news["headline"], news.get("description")
    )
    llm_result["score_source"] = "llm_low_confidence_fallback"
    return llm_result

The 0.70 confidence and 0.20 margin are initial operating thresholds, not universal constants. Tune them using results from the project's live news stream. Higher thresholds make more LLM calls but reduce reliance on uncertain local predictions.

Input Guidance

Use English RSS-style input matching the training data:

• Provide the original headline.
• Provide a short description or summary when available.
• Avoid adding an LLM-generated interpretation to the input.
• Avoid passing an entire long article as the description.
• Preserve terms such as XAUUSD, gold, USD, Fed, and Treasury yields.
• Reject records where both headline and description are empty.

Persisting Predictions

Always record how a score was produced. Do not silently mix model and LLM scores without a source field.

Recommended fields are:

article_hash
target
label
score
confidence
probabilities_json
score_source
model_version
predicted_at

If the existing headlines table must be used, a model result can be mapped as follows:

import json
import sqlite3


prediction = analyze_xauusd_news(news)

with sqlite3.connect("newsdata.db") as database:
    database.execute(
        """
        UPDATE headlines
        SET score = ?,
            score_by_currency_json = ?,
            score_source = ?
        WHERE hash = ? AND score IS NULL
        """,
        (
            prediction["score"],
            json.dumps({"XAU": prediction["score"]}),
            prediction["score_source"],
            news["hash"],
        ),
    )

The score IS NULL condition prevents overwriting existing LLM training labels. A separate predictions table is preferable when confidence, probabilities, prediction history, or multiple model versions must be retained.

Retraining Workflow

The trainer opens the SQLite database read-only and uses rows with:

score_source = "llm"
label = xau_score, else score_by_currency_json["XAU"], else --extra-labels
source is not a test feed (feed.example, good.example)

Retrain after adding a meaningful amount of new labeled data:

.venv/bin/python news_sentiment.py train --db newsdata.db \
  --extra-labels eval/results/backfill_labels.jsonl

Use --scope gold to train only on headlines that mention gold (the pre-2026-09-22 data), and --model models/candidates/<name>.joblib to train a candidate without replacing the deployed model. Grade candidates with eval/eval_golden.py --model ... and eval/eval_archive.py --db ../newsdata.db --extra-labels results/backfill_labels.jsonl.

Training performs the following steps:

1. Reads XAU sentiment labels (xau_score, JSON XAU, extra labels).
2. Normalizes and deduplicates article text.
3. Maps negative, zero, and positive XAU scores to classes.
4. Evaluates a stratified random holdout.
5. Evaluates a chronological holdout on the newest articles.
6. Fits the final model on all eligible examples.
7. Writes the model and metrics report.

Before deploying a retrained model, inspect:

python -m json.tool models/xauusd_sentiment.metrics.json

Prioritize the chronological holdout result because it better represents future news. Do not deploy solely because random holdout accuracy improved.

Production Rollout

1. Run the model in shadow mode alongside the LLM.
2. Log disagreements, confidence, latency, and final accepted output.
3. Review obvious directional cases and low-confidence predictions.
4. Enable local predictions only above conservative thresholds.
5. Keep the LLM as a fallback for uncertain or failed predictions.
6. Add reviewed LLM results to the training set.
7. Retrain and compare chronological metrics periodically.

Load one model per application process. A process restart is the simplest way to load a newly deployed model artifact. Only load trusted .joblib files, because joblib model files can execute code during deserialization.

Monitoring

Track at least:

• Number of articles scored
• Bullish, neutral, and bearish distribution
• Average confidence and probability margin
• Percentage routed to the LLM
• ML versus LLM agreement rate
• Errors and model-loading failures
• Results by publisher and news source
• Current model version

Sudden changes in class distribution or confidence can indicate data drift, changed feed formatting, or a new market regime.

Current Limitations

• Training contains 2,620 deduplicated XAU-labeled examples, all DeepSeek labels.
• The data covers only 15 days (2026-09-08 to 09-22) of one regime.
• The model estimates news sentiment, not future returns or trade profitability.
• Confidence values are useful for routing but are not guaranteed to be fully calibrated probabilities.
• Unusual wording and economic conditions absent from training can reduce accuracy.

The current chronological holdout accuracy is 66.6% (macro-F1 0.659). Continue using the LLM fallback until live shadow-mode results justify broader local-model usage.

Troubleshooting

Model not found

Use an absolute model path or set XAUUSD_MODEL_PATH:

export XAUUSD_MODEL_PATH="/home/harwinder/Development/ML-NEWS/models/xauusd_sentiment.joblib"

Cannot import news_sentiment

Add the repository to PYTHONPATH before starting the existing project:

export PYTHONPATH="/home/harwinder/Development/ML-NEWS:$PYTHONPATH"

scikit-learn version warning

Install the pinned dependencies from this repository:

python -m pip install -r /home/harwinder/Development/ML-NEWS/requirements.txt

Empty article error

At least one of headline or description must contain text.

Predictions look unrelated to XAUUSD

Confirm that the loaded artifact is models/xauusd_sentiment.joblib and that the result contains "target": "XAUUSD".