-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
9 changed files
with
34 additions
and
24 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1 +1 @@ | ||
<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Chapter 8: Decoding Strategies on Deep Learning for Natural Language Processing (DL4NLP)</title><link>https://slds-lmu.github.io/dl4nlp/chapters/08_decoding/</link><description>Recent content in Chapter 8: Decoding Strategies on Deep Learning for Natural Language Processing (DL4NLP)</description><generator>Hugo</generator><language>en-us</language><atom:link href="https://slds-lmu.github.io/dl4nlp/chapters/08_decoding/index.xml" rel="self" type="application/rss+xml"/><item><title>Chapter 08.01: What is Decoding?</title><link>https://slds-lmu.github.io/dl4nlp/chapters/08_decoding/08_01_intro/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://slds-lmu.github.io/dl4nlp/chapters/08_decoding/08_01_intro/</guid><description/></item><item><title>Chapter 08.02: Greedy & Beam Search</title><link>https://slds-lmu.github.io/dl4nlp/chapters/08_decoding/08_02_determ/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://slds-lmu.github.io/dl4nlp/chapters/08_decoding/08_02_determ/</guid><description/></item><item><title>Chapter 08.03: Stochastic Decoding & CS/CD</title><link>https://slds-lmu.github.io/dl4nlp/chapters/08_decoding/08_03_sampling/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://slds-lmu.github.io/dl4nlp/chapters/08_decoding/08_03_sampling/</guid><description/></item><item><title>Chapter 08.04: Decoding Hyperparameters & Practical considerations</title><link>https://slds-lmu.github.io/dl4nlp/chapters/08_decoding/08_04_hyper_param/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://slds-lmu.github.io/dl4nlp/chapters/08_decoding/08_04_hyper_param/</guid><description/></item><item><title>Chapter 08.05: Decoding Hyperparameters & Practical considerations</title><link>https://slds-lmu.github.io/dl4nlp/chapters/08_decoding/08_05_eval_metrics/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://slds-lmu.github.io/dl4nlp/chapters/08_decoding/08_05_eval_metrics/</guid><description/></item></channel></rss> | ||
<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Chapter 8: Decoding Strategies on Deep Learning for Natural Language Processing (DL4NLP)</title><link>https://slds-lmu.github.io/dl4nlp/chapters/08_decoding/</link><description>Recent content in Chapter 8: Decoding Strategies on Deep Learning for Natural Language Processing (DL4NLP)</description><generator>Hugo</generator><language>en-us</language><atom:link href="https://slds-lmu.github.io/dl4nlp/chapters/08_decoding/index.xml" rel="self" type="application/rss+xml"/><item><title>Chapter 08.01: What is Decoding?</title><link>https://slds-lmu.github.io/dl4nlp/chapters/08_decoding/08_01_intro/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://slds-lmu.github.io/dl4nlp/chapters/08_decoding/08_01_intro/</guid><description><p>Here we introduce the concept of decoding. Given a prompt and a generative language model, how does it generate text? The model produces a probability distribution over all tokens in the vocabulary. The way the model uses that probability distribution to generate the next token is what is called a decoding strategy.</p></description></item><item><title>Chapter 08.02: Greedy & Beam Search</title><link>https://slds-lmu.github.io/dl4nlp/chapters/08_decoding/08_02_determ/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://slds-lmu.github.io/dl4nlp/chapters/08_decoding/08_02_determ/</guid><description><p>Here we introduce two deterministic decoding strategies, greedy &amp; beam search. Both methods are determenistic, which means there is no sampling involved when generating text. While greedy decoding always chooses the token with the highest probability, while beam search keeps track of multiple beams to generate the next token.</p></description></item><item><title>Chapter 08.03: Stochastic Decoding & CS/CD</title><link>https://slds-lmu.github.io/dl4nlp/chapters/08_decoding/08_03_sampling/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://slds-lmu.github.io/dl4nlp/chapters/08_decoding/08_03_sampling/</guid><description><p>In this chapter you will learn about more methods beyond simple deterministic decoding strategies. We introduce sampling with temperature, where you add a temperature parameter into the softmax formula, top-k [1] and top-p [2] sampling, where you sample from a set of top tokens and finally contrastive search [3] and contrastive decoding [4].</p></description></item><item><title>Chapter 08.04: Decoding Hyperparameters & Practical considerations</title><link>https://slds-lmu.github.io/dl4nlp/chapters/08_decoding/08_04_hyper_param/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://slds-lmu.github.io/dl4nlp/chapters/08_decoding/08_04_hyper_param/</guid><description><p>In this chapter you will learn how to use the different decoding strategies in practice. When using models from huggingface you can choose the decoding strategy by specifying the hyperparameters of the <code>generate</code> method of those models.</p></description></item><item><title>Chapter 08.05: Evaluation Metrics</title><link>https://slds-lmu.github.io/dl4nlp/chapters/08_decoding/08_05_eval_metrics/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://slds-lmu.github.io/dl4nlp/chapters/08_decoding/08_05_eval_metrics/</guid><description><p>Here we answer the question on how to evaluate the generated outputs in open ended text generation. We first explain <strong>BLEU</strong> [1] and <strong>ROUGE</strong> [2], which are metrics for tasks with a gold reference. Then we introduce <strong>diversity</strong>, <strong>coherence</strong> [3] and <strong>MAUVE</strong> [4], which are metrics for tasks without a gold reference such as open ended text generation. You will also learn about human evaluation.</p></description></item></channel></rss> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.