Date: June 23, 2023 | Estimated Reading Time: 31 min | Author: Lilian Weng Prompt Engineering Prompt Engineering, also known as In-Context Prompting, refers to methods for how to …

2021年7月11日 · [Updated on 2021-09-19: Highly recommend this blog post on score-based generative modeling by Yang Song (author of several key papers in the references)]. [Updated …

2023年6月23日 · Building agents with LLM (large language model) as its core controller is a cool concept. Several proof-of-concepts demos, such as AutoGPT, GPT-Engineer and BabyAGI, …

2023年3月15日 · @article{weng2023prompt, title = "Prompt Engineering", author = "Weng, Lilian", journal = "lilianweng.github.io", year = "2023", month = "Mar", url = …

2023年1月27日 · Many new Transformer architecture improvements have been proposed since my last post on “The Transformer Family” about three years ago. Here I did a big refactoring …

2024年4月12日 · Diffusion models have demonstrated strong results on image synthesis in past years. Now the research community has started working on a harder task—using it for video …

2020年4月7日 · See my old post for other types of attention if interested.. Multi-Head Self-Attention#. The multi-head self-attention module is a key component in Transformer. Rather …

[Updated on 2021-02-01: Updated to version 2.0 with several work added and many typos fixed.] [Updated on 2021-05-26: Add P-tuning and Prompt Tuning in the “prompt design” section.] …

2024年11月28日 · Date: May 5, 2019 | Estimated Reading Time: 15 min | Author: Lilian Weng Implementing Deep Reinforcement Learning Models with Tensorflow + OpenAI Gym The full …

[Updated on 2023-01-24: add a small section on Distillation.] Large transformer models are mainstream nowadays, creating SoTA results for a variety of tasks. They are powerful but very …