DeepRead
Cover of Build a Large Language Model

Build a Large Language Model

Sebastian Raschka · 2024

Programming and computer science › Lisp and Scheme

#1,720 most recommended

About the book

A hands-on guide to building a GPT-style large language model in PyTorch, from raw text to a working model. It covers data preparation and tokenization, the attention mechanism, the transformer architecture, pretraining on unlabeled text, and finetuning for classification and instruction following. It also shows how to load pretrained GPT-2 weights. Understanding LLMs requires building one, because each component, from attention to the training loop, is simple enough to implement and inspect step by step. The book holds that a model's general abilities come from pretraining on large unlabeled text, and that narrower behaviour is added afterward through finetuning. It presents the working code as the basis for understanding how these systems function internally.

Written by Claude from what it knows of this book.

What Hacker News says 46% recommend · 0% critical

010202024202520262026 peak · 11
This bookAverage book in Lisp and Scheme

In the commenters’ own words

What Open Library readers say 1.00 from 1 rating

  1. 50
  2. 40
  3. 30
  4. 20
  5. 11
  • 82 want to read
  • 7 reading
  • 0 have read

Only 1 rating: treat the average as a hint.

Open Library’s page for the book

What critics say, via Wikipedia

No Wikipedia article about this book yet, so no critics to quote.

Reddit communities — coming later

The threads it came up in

Details

First published
2024
ISBN
9781633437166
Editions known
2 at Open Library
Open Library
OL38398010W

Explore in the mosaic

Read next: its neighbours on the mosaic

  • Deep Learning
    Christopher M. Bishop · 2023 · recommended by 3 people · loosely similar by subject
  • Hands-On Large Language Models
    Jay Alammar · 2024 · recommended by 3 people · similar by subject and what people say about it
  • Beautiful Racket
    Matthew Butterick · 2017 · recommended by 44 people · similar by subject and what people say about it
  • Little Learner
    Daniel P. Friedman · 2023 · recommended by 10 people · similar by subject and what people say about it
  • Dive into Deep Learning
    Aston Zhang · 2023 · recommended by 30 people · similar by subject and what people say about it
  • Natural Language Processing with Transformers
    Lewis Tunstall · 2022 · recommended by 2 people · similar by subject and what people say about it
  • Practical TLA+
    Hillel Wayne · 2018 · recommended by 15 people · similar by subject and what people say about it
  • Grokking Deep Learning
    Andrew Trask · 2019 · recommended by 7 people · similar by subject and what people say about it
  • Data science from scratch
    Joel Grus · 2015 · recommended by 22 people · similar by subject and what people say about it
  • Python machine learning
    Sebastian Raschka · 2015 · recommended by 5 people · similar by subject and what people say about it
  • Domain Modeling Made Functional
    Scott Wlaschin · 2018 · recommended by 43 people · similar by subject and what people say about it
  • ML for the working programmer
    Lawrence C. Paulson · 1991 · recommended by 16 people · similar by subject and what people say about it
  • Building a Second Brain
    Tiago Forte · 2022 · recommended by 16 people · similar by subject and what people say about it
  • Language Implementation Patterns
    Terence Parr · 2009 · recommended by 18 people · similar by subject and what people say about it
  • Understanding Machine Learning
    Shai Shalev-Shwartz · 2014 · recommended by 12 people · similar by subject and what people say about it
  • Modern compiler implementation in ML
    Andrew W. Appel · 1997 · recommended by 24 people · similar by subject and what people say about it
  • The Model Thinker
    Scott E. Page · 2018 · recommended by 9 people · similar by subject and what people say about it
  • Lisp in Small Pieces
    Christian Queinnec · 1996 · recommended by 101 people · similar by subject and what people say about it
  • Build Your Own Lisp
    Mr Daniel Holden · 2014 · recommended by 23 people · similar by subject and what people say about it
  • Realm of Racket
    Forrest Bice · 2013 · recommended by 37 people · similar by subject and what people say about it
  • Parsing Techniques A Practical Guide
    Ceriel Jacobs · 2010 · recommended by 7 people · similar by subject and what people say about it
  • Compiling with Continuations
    Andrew W. Appel · 1992 · recommended by 19 people · similar by subject and what people say about it
  • The Nature of Code
    Daniel Shiffman · 2012 · recommended by 73 people · similar by subject and what people say about it
  • Grokking Deep Reinforcement Learning
    Miguel Morales · 2020 · recommended by 1 person · similar by subject and what people say about it