
Build a Large Language Model
Sebastian Raschka · 2024
Programming and computer science › Lisp and Scheme
#1,720 most recommended
Hacker News
26
mentions by 26 people · top 13% of 15,732 books
Open Library
1.00★★★★★★★★★★
1 rating · 82 want to read
Wikipedia
—
No article yet
About the book
A hands-on guide to building a GPT-style large language model in PyTorch, from raw text to a working model. It covers data preparation and tokenization, the attention mechanism, the transformer architecture, pretraining on unlabeled text, and finetuning for classification and instruction following. It also shows how to load pretrained GPT-2 weights. Understanding LLMs requires building one, because each component, from attention to the training loop, is simple enough to implement and inspect step by step. The book holds that a model's general abilities come from pretraining on large unlabeled text, and that narrower behaviour is added afterward through finetuning. It presents the working code as the basis for understanding how these systems function internally.
Written by Claude from what it knows of this book.
What Hacker News says 46% recommend · 0% critical
In the commenters’ own words
This -- and obviously David Ng's article -- are absolutely fascinating pieces of work. I have a few (very naive) questions: There is a widespread intuition…
There are a couple of new books on the topic that are slated to drop any day now, IIRC. Of what's already published, a few I'm familiar with include…
What Open Library readers say 1.00 from 1 rating
- 50
- 40
- 30
- 20
- 11
- 82 want to read
- 7 reading
- 0 have read
Only 1 rating: treat the average as a hint.
What critics say, via Wikipedia
No Wikipedia article about this book yet, so no critics to quote.
Reddit communities — coming later
The threads it came up in
Details
- First published
- 2024
- ISBN
- 9781633437166
- Editions known
- 2 at Open Library
- Open Library
- OL38398010W
Read next: its neighbours on the mosaic
Deep LearningChristopher M. Bishop · 2023 · recommended by 3 people · loosely similar by subject
Hands-On Large Language ModelsJay Alammar · 2024 · recommended by 3 people · similar by subject and what people say about it
Beautiful RacketMatthew Butterick · 2017 · recommended by 44 people · similar by subject and what people say about it
Little LearnerDaniel P. Friedman · 2023 · recommended by 10 people · similar by subject and what people say about it
Dive into Deep LearningAston Zhang · 2023 · recommended by 30 people · similar by subject and what people say about it
Natural Language Processing with TransformersLewis Tunstall · 2022 · recommended by 2 people · similar by subject and what people say about it
Practical TLA+Hillel Wayne · 2018 · recommended by 15 people · similar by subject and what people say about it
Grokking Deep LearningAndrew Trask · 2019 · recommended by 7 people · similar by subject and what people say about it
Data science from scratchJoel Grus · 2015 · recommended by 22 people · similar by subject and what people say about it
Python machine learningSebastian Raschka · 2015 · recommended by 5 people · similar by subject and what people say about it
Domain Modeling Made FunctionalScott Wlaschin · 2018 · recommended by 43 people · similar by subject and what people say about it
ML for the working programmerLawrence C. Paulson · 1991 · recommended by 16 people · similar by subject and what people say about it
Building a Second BrainTiago Forte · 2022 · recommended by 16 people · similar by subject and what people say about it
Language Implementation PatternsTerence Parr · 2009 · recommended by 18 people · similar by subject and what people say about it
Understanding Machine LearningShai Shalev-Shwartz · 2014 · recommended by 12 people · similar by subject and what people say about it
Modern compiler implementation in MLAndrew W. Appel · 1997 · recommended by 24 people · similar by subject and what people say about it
The Model ThinkerScott E. Page · 2018 · recommended by 9 people · similar by subject and what people say about it
Lisp in Small PiecesChristian Queinnec · 1996 · recommended by 101 people · similar by subject and what people say about it
Build Your Own LispMr Daniel Holden · 2014 · recommended by 23 people · similar by subject and what people say about it
Realm of RacketForrest Bice · 2013 · recommended by 37 people · similar by subject and what people say about it
Parsing Techniques A Practical GuideCeriel Jacobs · 2010 · recommended by 7 people · similar by subject and what people say about it
Compiling with ContinuationsAndrew W. Appel · 1992 · recommended by 19 people · similar by subject and what people say about it
The Nature of CodeDaniel Shiffman · 2012 · recommended by 73 people · similar by subject and what people say about it
Grokking Deep Reinforcement LearningMiguel Morales · 2020 · recommended by 1 person · similar by subject and what people say about it