Dr V. Holiday 2023 Viewing Guide

Goal: To help prepare you for those difficult holiday conversations…

Like: How does ChatGPT work?

Karpathy: The busy person’s intro to LLMs

Karpathy LLM: YouTube Link

Reading List For Andrej Karpathy’s “Intro to Large Language Models” Video

Pay attention to the section on LLM security at the end of the talk.

Making LLMs “uncool” (Language Warning)

Making Large Language Models Uncool Again: Youtube

Uncool “takeaways”

• ~30b parameter models a missed opportunity
• We are “fine-tuning” wrong
• Uncertainty future directions for small (fine-tuned) vs large (API) models
• LLM architecture not the path to AG(S)I

Deep dive into understanding LLMs

What is ChatGPT doing…and why does it work? Youtube

AI Engineering

OpenAI Dev Day LLM Performance Talk

Techniques for increasing LLM Performance

Patterns for AI Engineering

Building Blocks for LLM Systems & Products: Eugene Yan

Patterns for Building LLM-based Systems & Products

RAG in Production

Building Production-Ready RAG Applications: Jerry Liu

The need for “Guardrails”

Trust, but Verify: Shreya Rajpal

A “firewall” for your LLM

Trust, but Verify: Shreya Rajpal

Nuggets for the week…

One Year of ChatGPT!

Chen, Hailin, Fangkai Jiao, Xingxuan Li, Chengwei Qin, Mathieu Ravaut, Ruochen Zhao, Caiming Xiong, and Shafiq Joty. 2023. “ChatGPT’s One-Year Anniversary: Are Open-Source Large Language Models Catching Up?” arXiv. http://arxiv.org/abs/2311.16989.

LLM Capabilities

Chen, Hailin, Fangkai Jiao, Xingxuan Li, Chengwei Qin, Mathieu Ravaut, Ruochen Zhao, Caiming Xiong, and Shafiq Joty. 2023. “ChatGPT’s One-Year Anniversary: Are Open-Source Large Language Models Catching Up?” arXiv. http://arxiv.org/abs/2311.16989.

Agent Capabilities

Chen, Hailin, Fangkai Jiao, Xingxuan Li, Chengwei Qin, Mathieu Ravaut, Ruochen Zhao, Caiming Xiong, and Shafiq Joty. 2023. “ChatGPT’s One-Year Anniversary: Are Open-Source Large Language Models Catching Up?” arXiv. http://arxiv.org/abs/2311.16989.

Model Consistency

Chen, Lingjiao, Matei Zaharia, and James Zou. 2023. “How Is ChatGPT’s Behavior Changing over Time?” arXiv. https://doi.org/10.48550/arXiv.2307.09009.

Model Consistency over time

Chen, Lingjiao, Matei Zaharia, and James Zou. 2023. “How Is ChatGPT’s Behavior Changing over Time?” arXiv. https://doi.org/10.48550/arXiv.2307.09009.

Mixture of Experts (MoE)

MoE

Agent Memory

Adding Long Term Memory to OpenGPTs

Long Term Memory

Adding Long Term Memory to OpenGPTs

Determinism vs Stochasticity?

• Determinism through RAG and Tool use
• Tool code created by LLM based Co-pilots or Agents
• What “Programming Language” should we use for the deterministic part?
• We don’t have an integrated paradigm for what “systems engineering” means in a age of AIs

For after Christmas…

The end of Programming?