00 transfer learning, made tangible
Three frontier abilities, zero training, one laptop.
Sentiment, zero-shot classification and fill-mask — three different NLP jobs, done by
three pretrained models, called through three pipeline() one-liners. Not one
gradient step was taken here. Everything below is captured output from a real run.
01 sentiment
A model fine-tuned on movie reviews, pointed at anything.
DistilBERT with an SST-2 classification head. It has exactly two labels, so every sentence it ever sees has to be one or the other. The bar is the model's confidence.
The last card is the interesting one. “Cautiously optimistic” is a hedge said out loud — and the model does not hedge back. With two labels available it cannot answer “mixed”, so it commits, and it commits at almost the same confidence as the sandwich. Confidence here is not a measure of doubt.
02 zero-shot
Labels invented at runtime, on a model that never saw them.
These four categories were typed into a list a second before the run. No dataset was collected, no head was trained, no weights were touched.
No fine-tuning happened. The model reasons about what the labels mean — each one is turned into a sentence to test, and the winner is whichever the text most implies.
03 fill-mask
The pretraining objective itself, run in the open.
Cover one word. Guess it. That is the entire exercise — and DistilBERT's base checkpoint has never done anything else.
Bars are absolute probability, not rank-normalised — these five do not add up to 1, because they are the top five of a thirty-thousand-word vocabulary.
This guessing game, played over the whole internet, is pretraining. It is where the other two abilities came from.
04 the point
Somebody else paid the pretraining bill.
None of these three models was trained here; all three were downloaded, and the only compute this laptop spent was on inference.
Two of them began life doing exactly the exercise in section 03 — cover a word, guess it, adjust, repeat, across a corpus far larger than any team could ever label by hand.
That single objective is enough to force a model to internalise grammar, topic, tone, entailment and a great deal of ordinary world knowledge, because you cannot guess the missing word without them.
Fine-tuning then spends a few thousand labelled examples redirecting that competence at one narrow task — which is why the sentiment model needed movie reviews, not the internet.
And zero-shot skips even that: bart-large-mnli was taught to judge whether one sentence entails another, so any label you invent becomes a sentence to test, and a classifier you never trained already exists.
05 the script
All of it, in one file.
Three pipeline() calls, section-level fault tolerance so one missing model
cannot take the rest down, and a REPL at the end.
#!/usr/bin/env python3
"""
MOOD RING - three frontier NLP abilities, zero training, one laptop.
Three pretrained models do three different jobs here, and none of them were
trained by us. That is the whole point: transfer learning means somebody else
paid the pretraining bill and we get the abilities for the price of a download.
1. SENTIMENT distilbert-base-uncased-finetuned-sst-2-english (~255 MB)
2. ZERO-SHOT facebook/bart-large-mnli (~1.6 GB)
3. FILL-MASK distilbert-base-uncased (~265 MB)
HONESTY ABOUT THE FIRST RUN
The first run DOWNLOADS these weights - roughly 2.1 GB in total, from
~255 MB for the smallest to ~1.6 GB for bart-large-mnli, which is by far
the big one. Expect the zero-shot section to pause for a while on a cold
cache. Files land in ~/.cache/huggingface/hub (override with HF_HOME).
Every run after that is fully offline - nothing is uploaded, ever, and no
API key is involved. The sections are ordered smallest-model-first so you
see real output before the 1.6 GB download starts.
USAGE
python mood_ring.py run the three demos, then an interactive REPL
python mood_ring.py --json run the three demos, write results.json, exit
Each section is independently fault tolerant: if a model fails to load (say
you go offline halfway through), that section prints a one-line skip and the
rest of the program carries on.
"""
from __future__ import annotations
import argparse
import datetime as _dt
import json
import os
import pathlib
import sys
try:
import transformers
from transformers import pipeline
except ImportError:
sys.exit(
"mood_ring needs the transformers library (and a torch backend).\n"
" install: pip install transformers torch\n"
" then run: python mood_ring.py"
)
transformers.logging.set_verbosity_error()
os.environ.setdefault("TOKENIZERS_PARALLELISM", "false")
HERE = pathlib.Path(__file__).resolve().parent
SENTIMENT_MODEL = "distilbert-base-uncased-finetuned-sst-2-english"
ZERO_SHOT_MODEL = "facebook/bart-large-mnli"
FILL_MASK_MODEL = "distilbert-base-uncased"
BAR_WIDTH = 24
SENTIMENT_TEXTS = [
# clearly positive - the easy case, and the model should be near-certain
"This is hands down the best sandwich I have ever eaten.",
# dry negative - no insults, no swearing, just resignation
"Sure, the meeting could have been an email.",
# upset - unambiguous, high intensity
"I waited two hours and nobody even told me the flight was cancelled.",
# genuinely ambivalent - this is the interesting one. The model has exactly
# two labels available, so it CANNOT say "mixed"; forcing a side is the
# whole point of including it. Note what the score does NOT do: it does not
# drop to reflect the hedge. Confidence here is not calibrated doubt.
"Cautiously optimistic about the new release.",
]
ZERO_SHOT_LABELS = ["cooking", "technology", "sports", "finance"]
ZERO_SHOT_TEXTS = [
"Sear the steak in a hot pan, then finish it in the oven with butter and thyme.",
"He tore his ACL in the third quarter and is out for the season.",
"Interest rates on the thirty-year mortgage climbed again this quarter.",
]
FILL_MASK_PROMPT = "The best way to learn machine learning is to [MASK] things."
# ---------------------------------------------------------------- formatting
def bar(score: float, width: int = BAR_WIDTH) -> str:
"""A █ bar over a ░ track, so 0.02 still looks like something."""
filled = int(round(max(0.0, min(1.0, score)) * width))
return "█" * filled + "░" * (width - filled)
def pct(score: float) -> str:
return f"{score * 100:5.1f}%"
def rule(title: str) -> None:
print(f"\n\033[1m{title}\033[0m")
print("─" * 72)
def loading(model: str, size: str) -> None:
print(f" loading {model} ({size})")
print(" first run downloads it to ~/.cache/huggingface; after that it is cached.")
sys.stdout.flush()
def skip(section: str, exc: Exception) -> None:
print(f" [skipped] {section}: {type(exc).__name__}: {exc}".replace("\n", " ")[:200])
# ------------------------------------------------------------------ sections
def run_sentiment() -> tuple[object | None, list[dict]]:
"""Smallest model, so it goes first: real output before the big download."""
rule("1 · SENTIMENT — a model fine-tuned on movie reviews, pointed at anything")
loading(SENTIMENT_MODEL, "~255 MB")
try:
clf = pipeline("sentiment-analysis", model=SENTIMENT_MODEL)
except Exception as exc: # offline, corrupt cache, no backend...
skip("sentiment", exc)
return None, []
captured: list[dict] = []
try:
print()
for text in SENTIMENT_TEXTS:
result = clf(text)[0]
label, score = result["label"], float(result["score"])
print(f" [{label:<8}] {bar(score)} {pct(score)} “{text}”")
captured.append({"text": text, "label": label, "score": score})
print()
print(" Only two labels exist, so ambivalence has nowhere to go. The last")
print(" sentence hedges out loud and the model still commits, hard. Forcing a")
print(" side is the interesting part — the confidence is not a measure of doubt.")
except Exception as exc:
skip("sentiment demo", exc)
return clf, captured
def run_zero_shot() -> list[dict]:
rule("2 · ZERO-SHOT — labels invented at runtime, no training of any kind")
loading(ZERO_SHOT_MODEL, "~1.6 GB — the big one; this is the long pause")
try:
clf = pipeline("zero-shot-classification", model=ZERO_SHOT_MODEL)
except Exception as exc:
skip("zero-shot", exc)
return []
captured: list[dict] = []
try:
print(f"\n labels: {', '.join(ZERO_SHOT_LABELS)}\n")
for text in ZERO_SHOT_TEXTS:
result = clf(text, candidate_labels=ZERO_SHOT_LABELS)
label = result["labels"][0]
score = float(result["scores"][0])
print(f" “{text}”")
print(f" → [{label:<10}] {bar(score)} {pct(score)}\n")
captured.append({"text": text, "top_label": label, "score": score})
print(" Nobody fine-tuned this model on cooking, sports or finance. The labels")
print(" were made up a second ago — it reasons about what the WORDS mean.")
except Exception as exc:
skip("zero-shot demo", exc)
return captured
def run_fill_mask() -> dict:
rule("3 · FILL-MASK — the pretraining objective itself, run in the open")
loading(FILL_MASK_MODEL, "~265 MB")
try:
unmasker = pipeline("fill-mask", model=FILL_MASK_MODEL, top_k=5)
except Exception as exc:
skip("fill-mask", exc)
return {}
captured: dict = {}
try:
print(f"\n “{FILL_MASK_PROMPT}”\n")
candidates = []
for guess in unmasker(FILL_MASK_PROMPT):
token = guess["token_str"].strip()
score = float(guess["score"])
print(f" {token:<12} {bar(score)} {pct(score)}")
candidates.append({"token": token, "score": score})
print()
print(" This guessing game, played over the whole internet, IS pretraining.")
print(" Grammar, topic, tone, world knowledge — the other two abilities you")
print(" just watched came out of nothing more exotic than this.")
captured = {"prompt": FILL_MASK_PROMPT, "candidates": candidates}
except Exception as exc:
skip("fill-mask demo", exc)
return captured
# ---------------------------------------------------------------------- repl
def repl(clf) -> None:
rule("REPL — type a sentence, get a reading. Blank line or Ctrl-D to leave.")
if clf is None:
print(" sentiment model unavailable, so there is nothing to type at. Bye.")
return
print()
while True:
try:
text = input("> ").strip()
except (EOFError, KeyboardInterrupt):
print("\n bye.")
return
if not text:
print(" bye.")
return
try:
result = clf(text)[0]
label, score = result["label"], float(result["score"])
face = ":)" if label.upper().startswith("POS") else ":("
print(f" {face} {label:<8} {bar(score)} {pct(score)}\n")
except Exception as exc:
skip("reading", exc)
# ---------------------------------------------------------------------- main
def main() -> None:
parser = argparse.ArgumentParser(
description="Three pretrained models, three NLP abilities, zero training."
)
parser.add_argument(
"--json",
action="store_true",
help="run the three demo sections, write results.json, skip the REPL",
)
args = parser.parse_args()
print("\n\033[1mMOOD RING\033[0m · three frontier abilities, zero training, one laptop")
print(f"transformers {transformers.__version__}")
print("First run downloads ~2.1 GB of weights to ~/.cache/huggingface. Offline after.")
clf, sentiment = run_sentiment()
zero_shot = run_zero_shot()
fill_mask = run_fill_mask()
if args.json:
payload = {
"run_date": _dt.date.today().isoformat(),
"transformers_version": transformers.__version__,
"models": {
"sentiment": SENTIMENT_MODEL,
"zero_shot": ZERO_SHOT_MODEL,
"fill_mask": FILL_MASK_MODEL,
},
"sentiment": sentiment,
"zero_shot": zero_shot,
"fill_mask": fill_mask,
}
out = HERE / "results.json"
out.write_text(json.dumps(payload, indent=2) + "\n", encoding="utf-8")
print(f"\nwrote {out}")
return
repl(clf)
if __name__ == "__main__":
main()
06 run it yourself
Two commands, and one honest download.
Install
CPU is fine. No API key, no account, nothing leaves your machine.
Run
The sections are ordered smallest-model-first, so you get real output on screen before the 1.6 GB download starts.
What the first run actually downloads
Weights are fetched once and cached in ~/.cache/huggingface/hub
(override with HF_HOME). Every run after the first is fully offline.