Sign-In
Cart
Build a Large Language Model (from Scratch)
Paperback
Series: From Scratch
ISBN13: 9781633437166
Publisher: Manning Publications
Published: Oct 29 2024
Pages: 368
Weight: 1.35
Height: 0.90 Width: 7.40 Depth: 9.20
Language: English
In Build a Large Language Model (from Scratch) bestselling author Sebastian Raschka guides you step by step through creating your own LLM. Each stage is explained with clear text, diagrams, and examples. You'll go from the initial design and creation, to pretraining on a general corpus, and on to fine-tuning for specific tasks.
Build a Large Language Model (from Scratch) teaches you how to:
- Plan and code all the parts of an LLM
- Fine-tune LLMs for text classification and with your own data
- Use human feedback to ensure your LLM follows instructions
- Load pretrained weights into an LLM
Build a Large Language Model (from Scratch) takes you inside the AI black box to tinker with the internal systems that power generative AI. As you work through each key stage of LLM creation, you'll develop an in-depth understanding of how LLMs work, their limitations, and their customization methods. Your LLM can be developed on an ordinary laptop, and used as your own personal assistant.
Purchase of the print book includes a free eBook in PDF and ePub formats from Manning Publications.
About the technology
Physicist Richard P. Feynman reportedly said, I don't understand anything I can't build. Based on this same powerful principle, bestselling author Sebastian Raschka guides you step by step as you build a GPT-style LLM that you can run on your laptop. This is an engaging book that covers each stage of the process, from planning and coding to training and fine-tuning.
About the book
Build a Large Language Model (From Scratch) is a practical and eminently-satisfying hands-on journey into the foundations of generative AI. Without relying on any existing LLM libraries, you'll code a base model, evolve it into a text classifier, and ultimately create a chatbot that can follow your conversational instructions. And you'll really understand it because you built it yourself!
What's inside
- Plan and code an LLM comparable to GPT-2
- Load pretrained weights
- Construct a complete training pipeline
- Fine-tune your LLM for text classification
- Develop LLMs that follow human instructions
About the reader
Readers need intermediate Python skills and some knowledge of machine learning. The LLM you create will run on any modern laptop and can optionally utilize GPUs.
About the author
Sebastian Raschka is a Staff Research Engineer at Lightning AI, where he works on LLM research and develops open-source software.
The technical editor on this book was David Caswell.
Table of Contents
1 Understanding large language models
2 Working with text data
3 Coding attention mechanisms
4 Implementing a GPT model from scratch to generate text
5 Pretraining on unlabeled data
6 Fine-tuning for classification
7 Fine-tuning to follow instructions
A Introduction to PyTorch
B References and further reading
C Exercise solutions
D Adding bells and whistles to the training loop
E Parameter-efficient fine-tuning with LoRA
Also from
Raschka, Sebastian
Machine Learning with PyTorch and Scikit-Learn: Develop machine learning and deep learning models with Python
Liu, Yuxi (Hayden)
Raschka, Sebastian
Paperback
Machine Learning Q and AI: 30 Essential Questions and Answers on Machine Learning and AI
Raschka, Sebastian
Paperback
Python Machine Learning: Machine Learning and Deep Learning with Python, scikit-learn, and TensorFlow 2
Raschka, Sebastian
Mirjalili, Vahid
Paperback
Machine Learning with PyTorch and Scikit-Learn: Develop machine learning and deep learning models with Python
Raschka, Sebastian
Liu, Yuxi (Hayden)
Hardcover
Python Machine Learning - Second Edition: Machine Learning and Deep Learning with Python, scikit-learn, and TensorFlow
Mirjalili, Vahid
Raschka, Sebastian
Paperback
Python: Deeper Insights into Machine Learning: Leverage benefits of machine learning techniques using Python
Hearty, John
Julian, David
Raschka, Sebastian
Paperback
Python Machine Learning: Unlock deeper insights into Machine Leaning with this vital guide to cutting-edge predictive analytics
Raschka, Sebastian
Paperback
Also in
General Computers
This Program Is Brought to You by . . .: Distributing Television News Online
Braun, Joshua A.
Paperback
The Year in Tech, 2025: The Insights You Need from Harvard Business Review
Webb, Amy
Farri, Elisa
Review, Harvard Business
Paperback
The Technological Republic: Hard Power, Soft Belief, and the Future of the West
Karp, Alexander C.
Zamiska, Nicholas W.
Hardcover
AI Snake Oil: What Artificial Intelligence Can Do, What It Can't, and How to Tell the Difference
Narayanan, Arvind
Kapoor, Sayash
Hardcover
Hbr's 10 Must Reads on AI (with Bonus Article How to Win with Machine Learning by Ajay Agrawal, Joshua Gans, and AVI Goldfarb)
Iansiti, Marco
Review, Harvard Business
Davenport, Thomas H.
Paperback
Mindmasters: The Data-Driven Science of Predicting and Changing Human Behavior
Matz, Sandra
Hardcover
Generative Ai: The Insights You Need from Harvard Business Review
Mollick, Ethan
Cremer, David De
Review, Harvard Business
Paperback
The Coming Wave: Technology, Power, and the Twenty-First Century's Greatest Dilemma
Suleyman, Mustafa
Hardcover
AI for Educators: Learning Strategies, Teacher Efficiencies, and a Vision for an Artificial Intelligence Future
Miller, Matt
Paperback
More Human: How the Power of AI Can Transform the Way You Lead
Carter, Jacqueline
Hougaard, Rasmus
Hardcover
The AI Con: How to Fight Big Tech's Hype and Create the Future We Want
Bender, Emily M.
Hanna, Alex
Hardcover
Teaching with AI: A Practical Guide to a New Era of Human Learning
Bowen, José Antonio
Watson, C. Edward
Paperback
Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems
Kleppmann, Martin
Paperback
Brave New Words: How AI Will Revolutionize Education (and Why That's a Good Thing)
Khan, Salman
Hardcover
Python Crash Course, 3rd Edition: A Hands-On, Project-Based Introduction to Programming
Matthes, Eric
Paperback
AI Valley: Microsoft, Google, and the Trillion-Dollar Race to Cash in on Artificial Intelligence
Rivlin, Gary
Hardcover
Princeton Review AP Computer Science a Prep, 8th Edition: 5 Practice Tests + Complete Content Review + Strategies & Techniques
The Princeton Review
Paperback
Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence
Crawford, Kate
Paperback
Banking on (Artificial) Intelligence: Navigating the Realities of AI in Financial Services
Lau, Theodora
Paperback
The Black Swan: Second Edition: The Impact of the Highly Improbable: With a New Section: On Robustness and Fragility
Taleb, Nassim Nicholas
Paperback
Nexus: A Brief History of Information Networks from the Stone Age to AI (Large Print Edition)
Harari, Yuval Noah
Paperback
Digital Dharma: How AI Can Elevate Spiritual Intelligence and Personal Well-Being
Chopra, Deepak
Hardcover
AI for Life: 100+ Ways to Use Artificial Intelligence to Make Your Life Easier, More Productive...and More Fun!
Quillian, Celia
Paperback
RHCSA Red Hat Enterprise Linux 9: Training and Exam Preparation Guide (EX200), Third Edition
Ghori, Asghar
Paperback
The Death of Expertise: The Campaign Against Established Knowledge and Why It Matters
Nichols, Tom
Paperback
Minecraft: Exploded Builds: Medieval Fortress: An Official Mojang Book
Mojang Ab
The Official Minecraft Team
Paperback
Fans First: Change The Game, Break the Rules & Create an Unforgettable Experience
Cole, Jesse
Paperback
Hands-On Large Language Models: Language Understanding and Generation
Alammar, Jay
Grootendorst, Maarten
Paperback
Minecraft: Guide Collection 4-Book Boxed Set (Updated): Survival (Updated), Creative (Updated), Redstone (Updated), Combat
Mojang Ab
The Official Minecraft Team
Hardcover
The Thinking Machine: Jensen Huang, Nvidia, and the World's Most Coveted Microchip
Witt, Stephen
Hardcover
Embedded Systems with ARM Cortex-M Microcontrollers in Assembly Language and C: Fourth Edition
Zhu, Yifeng
Paperback
The Chaos Machine: The Inside Story of How Social Media Rewired Our Minds and Our World
Fisher, Max
Paperback
What Tech Calls Thinking: An Inquiry Into the Intellectual Bedrock of Silicon Valley
Daub, Adrian
Paperback
R for Data Science: Import, Tidy, Transform, Visualize, and Model Data
Wickham, Hadley
Grolemund, Garrett
Cetinkaya-Rundel, Mine
Paperback
Recoding America: Why Government Is Failing in the Digital Age and How We Can Do Better
Pahlka, Jennifer
Hardcover
Laptops for Seniors in Easy Steps, 9th Edition: Updated to Cover All Laptops with the Windows 11 2024 Update
Vandome, Nick
Paperback
Computer Science: An Illustrated History of the World's Smartest Machines (100 Ponderables)
Jackson, Tom
Hardcover
Practical Charts: The Essential Guide to Creating Clear, Compelling Charts for Reports and Presentations
Desbarats, Nicholas P.
Paperback
The Mechanic and the Luddite: A Ruthless Criticism of Technology and Capitalism
Sadowski, Jathan
Paperback
Exploring Windows 11 - 2024 Edition: The Illustrated, Practical Guide to Using Microsoft Windows
Wilson, Kevin
Paperback
Fundamentals of Data Engineering: Plan and Build Robust Data Systems
Reis, Joe
Housley, Matt
Paperback
Prompt Engineering for Generative AI: Future-Proof Inputs for Reliable AI Outputs
Taylor, Mike
Phoenix, James
Paperback
Algorithms to Live by: The Computer Science of Human Decisions
Christian, Brian
Griffiths, Tom
Paperback
Arduino: 101 Beginners Guide: How to get started with Your Arduino (Tips, Tricks, Projects and More!)
Savasgard, Erik
Paperback
Isc2 Cissp Certified Information Systems Security Professional Official Study Guide & Practice Tests Bundle
Chapple, Mike
Stewart, James Michael
Gibson, Darril
Paperback
Human + Machine, Updated and Expanded: Reimagining Work in the Age of AI
Daugherty, Paul R.
Wilson, H. James
Hardcover
Verified: How to Think Straight, Get Duped Less, and Make Better Decisions about What to Believe Online
Caulfield, Mike
Wineburg, Sam
Paperback
You Look Like a Thing and I Love You: How Artificial Intelligence Works and Why It's Making the World a Weirder Place
Shane, Janelle
Paperback
Exploring Apple Mac - Sequoia Edition: The Illustrated, Practical Guide to Using MacOS
Wilson, Kevin
Paperback
Crypto Confidential: Winning and Losing Millions in the New Frontier of Finance
Eliason, Nathaniel
Hardcover
Designing Machine Learning Systems: An Iterative Process for Production-Ready Applications
Huyen, Chip
Paperback
Building AI-Powered Products: The Essential Guide to AI and Genai Product Management
Nika, Marily
Paperback
Your Stone Age Brain in the Screen Age: Coping with Digital Distraction and Sensory Overload
Cytowic, Richard E.
Hardcover
How to Teach AI: Weaving Strategies and Activities Into Any Content Area
Poth, Rachelle Dené
Paperback
Automate the Boring Stuff with Python, 2nd Edition: Practical Programming for Total Beginners
Sweigart, Al
Paperback
Future Ready: The Four Pathways to Capturing Digital Value
Sebastian, Ina M.
Woerner, Stephanie L.
Weill, Peter
Hardcover
AP Computer Science a Premium, 12th Edition: Prep Book with 6 Practice Tests + Comprehensive Review + Online Practice
Barron's Educational Series
Teukolsky, Roselyn
Paperback
The Experimentation Machine: Finding Product-Market Fit in the Age of AI
Bussgang, Jeffrey J.
Hardcover
Irresistible: The Rise of Addictive Technology and the Business of Keeping Us Hooked
Alter, Adam
Paperback
Ocp Oracle Certified Professional Java Se 21 Developer Study Guide
Boyarsky, Jeanne
Selikoff, Scott
Paperback
The Algorithm: How AI Decides Who Gets Hired, Monitored, Promoted, and Fired and Why We Need to Fight Back Now
Schellmann, Hilke
Hardcover
Better Data Visualizations: A Guide for Scholars, Researchers, and Wonks
Schwabish, Jonathan
Paperback
ITIL(R) 4 Essentials: Your essential guide for the ITIL 4 Foundation exam and beyond
Agutter, Claire
Paperback