source: kdnuggets: 5 fun papers that explain llms clearly
level: technical
the attention is all you need paper introduced the transformer architecture, which uses self-attention to let each token weigh the importance of others in a sequence. this design replaced older recurrent or convolutional models and now underpins most major llms like gpt, llama, and claude. key ideas include multi-head attention and positional encoding, which help models capture long-range context.
the gpt-3 paper showed that large language models can perform many tasks through in-context learning, where a model follows instructions or examples in a prompt without weight updates. this 175-billion-parameter model demonstrated that scaling up pretraining on diverse text enables few-shot generalization, reducing the need for task-specific fine-tuning and making prompting a central technique in nlp.
the scaling laws paper found that model performance improves predictably with more parameters, data, and compute. this work explains the industry's push toward larger models and training runs, and it informs current discussions on compute-optimal training and data quality. the instructgpt paper then detailed how supervised fine-tuning and reinforcement learning from human feedback turn a base model into a helpful, instruction-following assistant. finally, the rag paper introduced retrieval-augmented generation, where a model retrieves external documents to ground its answers, a method widely used in chatbots and enterprise search.
why it matters: these papers explain the core mechanisms behind modern llms, helping practitioners understand model behavior, improve prompting, and design retrieval-based applications.