Week 2: Data acquisition and preprocessing

DATA 441/641 · Week 2

Notes

• 2.1 Data Acquisition(slides)
• 2.2 Text Extraction and Cleanup(slides)
• 2.3 Preprocessing(slides)
• 2.4 Advanced Preprocessing(slides)

Labs

• Lab 2: Text Extraction and Cleanup(PDF)
• Lab 2 solutions (password)

Homework

• Homework 2(PDF)

Additional

• Preprocessing notes(PDF)
• Preprocessing pipeline(PDF)
• POS tagging(PDF)

Additional readings

• Practical NLP, Ch. 2 (pipeline)
• Unicode in Python

Discussion

• When is a public web scrape the wrong data source for a product classifier?
• Stemming or lemmatization: which would you pick for a search box, and why?