Ivan Pavlov

Fine-tuned LLMs sometimes exhibit strikingly misaligned behavior — an unsettling phenomenon with no agreed mechanistic account. This project evaluates candidate explanations using linear probes, SAE features, and activation steering to determine what internal representations shift during fine-tuning and which shifts are causally responsible for misaligned outputs.

Reproduced the Tülu3 post-training pipeline on Llama 3.1 8B and adapted it to the SwissAI Apertus (8B/70B) models: full SFT run followed by DPO. Discovered that DPO degraded MMLU despite clean training metrics — a concrete example of optimization signal diverging from downstream capability. Extended the olmes evaluation harness with vLLM + tensor parallelism, seven standard benchmarks, and automated SLURM job generation; productionized the DPO trainer in the SwissAI codebase with full W&B logging.

Tests whether targeted external evidence helps a frozen video-native VLM (Qwen2-VL-7B) answer HD-EPIC multiple-choice questions under compute constraints. The model receives one of three conditions: native video alone, tool-selected evidence alone, or native video augmented with evidence. Six evidence-selection tools were evaluated (CLIP retrieval, motion+CLIP cascade, OCR crop, object tracking) alongside routing strategies ranging from keyword rules to TF-IDF classifiers. Key finding: a hand-crafted rule-based router achieved +3.9 pp over fixed-tool baselines, while learned classifiers consistently underperformed — pointing to a hard within-category gap of 16 pp that cheap visual features cannot close.

A 2D Lattice Boltzmann solver (D2Q9, BGK collision) for flow past one or two circular cylinders in a rectangular channel. The repository contains three solvers: a serial reference implementation, a distributed-memory MPI version, and a hand-written CUDA kernel — plus the profiling and scaling studies behind them. Covers domain decomposition, halo exchange, GPU memory layout, and strong/weak scaling analysis.

A modern interactive textbook covering combinatorics, graph theory, probability, logic, and algorithm complexity. 300+ auto-graded problems with Python examples. The Russian edition has the larger following.

Covers greedy algorithms, divide-and-conquer, and dynamic programming through 55 interactive interview-style problems with hints and detailed solutions. 91 lessons, 8 h/week pace.