Week 5: Deep learning and Word2Vec

DATA 441/641 · Week 5

Notes

• 5.1 Introduction to Deep Learning(slides)
• 5.2 Distributed Representations(slides)
• 5.3 Word2Vec CBOW(slides)
• 5.4 Word2Vec Skip-gram(slides)
• 5.5 Word2Vec Practical Considerations(slides)

Labs

• Lab 5: Word2Vec on Text Messages(PDF)
• Lab 5 solutions (password)

Additional

• Embeddings(PDF)

Additional readings

• Practical NLP, Ch. 3 continued
• Mikolov et al., Efficient estimation of word representations

Discussion

• CBOW versus skip-gram: which is predicting which?
• What does a negative sample do in Word2Vec training?