Stargazer's Blog

Insights from AI/ML Research Presentations

StreamingLLM: Efficient Streaming Language Models

Exploring how attention sinks enable infinite context processing in large language models, based on my presentation of this innovative approach.

December 2024 • AI/ML Presentation • Infinite Context

AlphaGeometry: AI for Olympiad Geometry

My analysis of DeepMind's breakthrough system that combines neural networks with symbolic reasoning for complex geometry problems.

December 2024 • AI/ML Presentation • Mathematical Reasoning

Infini-attention: Infinite Context Transformers

Understanding Google's approach to achieving infinite context length through compressive memory mechanisms, from my presentation slides.

December 2024 • AI/ML Presentation • Transformer Architecture

GaLore: Memory-Efficient LLM Training

My presentation on this technique that reduces memory requirements for training large language models through gradient projection.

December 2024 • AI/ML Presentation • Training Optimization

WARP: Weight Averaged Rewarded Policies

Insights from my presentation on how weight averaging enhances reinforcement learning from human feedback for better AI alignment.

December 2024 • AI/ML Presentation • AI Alignment