Notes

RLHF and DPO: Aligning AI to Human Preferences Both techniques address the same core problem: after pre-training on raw text, a language model needs to be steered toward responses that are helpful, safe, and aligned with what humans actually want. They’re two different approaches to the same goal. RLHF — Reinforcement Learning from Human Feedback The idea: Train a separate model to predict what humans prefer, then use that model as a reward signal to fine-tune the LLM via RL. ...

Instruction tuning is a fine-tuning technique where a pre-trained language model is further trained on a dataset of (instruction, response) pairs to make it better at following natural language instructions. How it works A base language model trained on raw text is good at predicting the next token, but not necessarily at being helpful. Instruction tuning bridges that gap by showing the model thousands to millions of examples like: Instruction: “Summarize this article in 3 bullet points.” Response: “• Point 1 …” The model learns to map user intent → useful output. ...

Perplexity in Language Models Perplexity measures how well a probability model predicts a sample of text. Intuitively, it captures how “surprised” or “perplexed” a model is when it encounters new text — a lower perplexity means the model found the text more predictable, i.e., it’s a better model. The Core Idea A language model assigns a probability to every sequence of words. Given a test sentence, the model predicts the probability of each next word given all preceding words: ...

Model quantization is the process of reducing the numerical precision of a neural network’s weights (and sometimes activations) to make models smaller and faster, with acceptable loss in accuracy. The core idea Neural networks store parameters as floating-point numbers — typically 32-bit floats (float32). Quantization maps these to lower-precision representations like 16-bit floats, 8-bit integers, or even 4-bit integers. Fewer bits per number means less memory and faster arithmetic. Common precision levels Format Bits Typical use float32 32 Training baseline bfloat16 / float16 16 Training & inference on GPUs int8 8 Efficient inference int4 / int3 / int2 4 or less Aggressive compression (LLMs) How it works Post-training quantization (PTQ) takes a trained model and converts its weights after the fact. It’s fast and simple but can hurt accuracy at very low bit depths. ...

To login g c l o u d a u t h l o g i n Application Default Login g c l o u d a u t h a p p l i c a t i o n - d e f a u l t l o g i n Project Set g c l o u d c o n f i g s e t p r o j e c t b u d d y h q - p r d GKE cluster context set dev: ...

Use this command: k u b e c t l p o r t - f o r w a r d s v c / t e m p o r a l - w e b 8 0 8 0 : 8 0 8 0 The first port is the localhost port, the second the service port.