Price: $59.99 - $51.67
(as of Aug 28, 2025 10:27:27 UTC – Details)
Learn how to create, train, and tweak large language models (LLMs) by building one from the ground up!
In Build a Large Language Model (from Scratch) bestselling author Sebastian Raschka guides you step by step through creating your own LLM. Each stage is explained with clear text, diagrams, and examples. You’ll go from the initial design and creation, to pretraining on a general corpus, and on to fine-tuning for specific tasks.
Build a Large Language Model (from Scratch) teaches you how to:
• Plan and code all the parts of an LLM
• Prepare a dataset suitable for LLM training
• Fine-tune LLMs for text classification and with your own data
• Use human feedback to ensure your LLM follows instructions
• Load pretrained weights into an LLM
Build a Large Language Model (from Scratch) takes you inside the AI black box to tinker with the internal systems that power generative AI. As you work through each key stage of LLM creation, you’ll develop an in-depth understanding of how LLMs work, their limitations, and their customization methods. Your LLM can be developed on an ordinary laptop, and used as your own personal assistant.
Purchase of the print book includes a free eBook in PDF and ePub formats from Manning Publications.
About the technology
Physicist Richard P. Feynman reportedly said, “I don’t understand anything I can’t build.” Based on this same powerful principle, bestselling author Sebastian Raschka guides you step by step as you build a GPT-style LLM that you can run on your laptop. This is an engaging book that covers each stage of the process, from planning and coding to training and fine-tuning.
About the book
Build a Large Language Model (From Scratch) is a practical and eminently-satisfying hands-on journey into the foundations of generative AI. Without relying on any existing LLM libraries, you’ll code a base model, evolve it into a text classifier, and ultimately create a chatbot that can follow your conversational instructions. And you’ll really understand it because you built it yourself!
What’s inside
• Plan and code an LLM comparable to GPT-2
• Load pretrained weights
• Construct a complete training pipeline
• Fine-tune your LLM for text classification
• Develop LLMs that follow human instructions
About the reader
Readers need intermediate Python skills and some knowledge of machine learning. The LLM you create will run on any modern laptop and can optionally utilize GPUs.
About the author
Sebastian Raschka is a Staff Research Engineer at Lightning AI, where he works on LLM research and develops open-source software.
The technical editor on this book was David Caswell.
Table of Contents
1 Understanding large language models
2 Working with text data
3 Coding attention mechanisms
4 Implementing a GPT model from scratch to generate text
5 Pretraining on unlabeled data
6 Fine-tuning for classification
7 Fine-tuning to follow instructions
A Introduction to PyTorch
B References and further reading
C Exercise solutions
D Adding bells and whistles to the training loop
E Parameter-efficient fine-tuning with LoRA
From the Publisher
“The most understandable and comprehensive explanation of language models yet! Its unique and practical teaching style achieves a level of understanding you can’t get any other way.”
Cameron Wolfe, Senior Scientist, Netflix
“Sebastian combines deep knowledge with practical engineering skills and a knack for making complex ideas simple. This is the guide you need!”
Chip Huyen, author of Designing Machine Learning Systems and AI Engineering
“Definitive, up-to-date coverage. Highly recommended!”
Dr. Vahid Mirjalili, Senior Data Scientist, FM Global
why this book?
Build a Large Language Model (From Scratch) offers a practical, hands-on approach to understanding and constructing large language models (LLMs) from the ground up.
By guiding you through each stage—from data preparation and coding attention mechanisms to pretraining and fine-tuning—this book demystifies the inner workings of LLMs using Python and PyTorch.
Ideal for developers and machine learning enthusiasts, it empowers you to build a functional GPT-style model on a standard laptop, fostering a deeper comprehension of generative AI technologies.
about Manning
Manning helps developers and tech professionals stay ahead in a fast-moving industry with expert-led books, videos, and projects. Learning never stops, but it’s hard to keep up, so we focus on content that’s practical, clear, and trusted. As an independent publisher, we adapt quickly, from pioneering early-access books to offering DRM-free eBooks. Our series, like “In Action” and “In a Month of Lunches”, reflect a commitment to making complex topics accessible.
Add to Cart
Add to Cart
Add to Cart
Add to Cart
Add to Cart
Add to Cart
Customer Reviews
4.5 out of 5 stars 18
4.2 out of 5 stars 22
4.7 out of 5 stars 4
4.8 out of 5 stars 7
5.0 out of 5 stars 1
4.1 out of 5 stars 8
Price
$50.66$50.66 $45.51$45.51 $59.99$59.99 $57.13$57.13 $39.59$39.59 $51.86$51.86
Level of proficiency
Intermediate Intermediate Intermediate Intermediate Intermediate Advanced
About the reader
For data scientists and ML engineers. For intermediate Python programmers. For intermediate Python programmers. For developers, engineers, and product managers. For data scientists and data analysts. For data scientists and machine learning engineers.
Special features
Includes liveBook with out built-in AI assistant. Includes liveBook with out built-in AI assistant. Includes liveBook with out built-in AI assistant. Includes liveBook with out built-in AI assistant. Includes liveBook with out built-in AI assistant. Includes liveBook with out built-in AI assistant.
Pages
456 344 688 328 232 520
Publisher : Manning
Publication date : October 29, 2024
Language : English
Print length : 368 pages
ISBN-10 : 1633437167
ISBN-13 : 978-1633437166
Item Weight : 1.35 pounds
Dimensions : 7.38 x 0.7 x 9.25 inches
Best Sellers Rank: #5,438 in Books (See Top 100 in Books) #1 in Data Processing #2 in Python Programming #2 in Computer Neural Networks
Customer Reviews: 4.6 4.6 out of 5 stars 263 ratings var dpAcrHasRegisteredArcLinkClickAction; P.when(‘A’, ‘ready’).execute(function(A) { if (dpAcrHasRegisteredArcLinkClickAction !== true) { dpAcrHasRegisteredArcLinkClickAction = true; A.declarative( ‘acrLink-click-metrics’, ‘click’, { “allowLinkDefault”: true }, function (event) { if (window.ue) { ue.count(“acrLinkClickCount”, (ue.count(“acrLinkClickCount”) || 0) + 1); } } ); } }); P.when(‘A’, ‘cf’).execute(function(A) { A.declarative(‘acrStarsLink-click-metrics’, ‘click’, { “allowLinkDefault” : true }, function(event){ if(window.ue) { ue.count(“acrStarsLinkWithPopoverClickCount”, (ue.count(“acrStarsLinkWithPopoverClickCount”) || 0) + 1); } }); });
