CS6200 Boston Schedule

CS6200 Information Retrieval, SUMMER-FULL 2020

about CS6200 home schedule grades piazza data resources

* Schedule and materials subject to change

Week / Topics	Lecture	Reading	Assignment
Week1 5/4 - 5/11 Retrieval Models / Intro IR intro Queries and documents Matching scores Cosine ElasticSearch demo	Slides Intro Slides Vector Space 1 Notes Retrieval Models	Background: Linear Algebra Background Programming: Java or Python [MRS] Boolean Retrieval [CMS[ Chaper 1 [MRS] Vector Space Models	HW 1 Due: Fri 5/22
Week 2 5/11 - 5/18 Retrieval Models / Vector Space ElasticSearch demo AP89 data Tf-Idf, BM25 Co-occurrence, bigrams trec_eval Metasearch	Slides Vector Space 2 Slides: Tf-Idf, BM25 Slides: Metasearch	[MRS[ Probabilistic Information Retrieval [CMS] Retrieval Models Slides [PAPER] Document Ranking and the Vector-Space Model. by Lee et all ,1997. [PAPER] Models for Metasearch, by Aslam et all.
Week3 5/18 - 5/25 Retrieval Models / Language Models Language generative Models Query Likelihood Model Divergence Smoothing Relevance Feedback	Slides: Language Models Slides: Pseudo Relevance Feedback Retrieval Model toy example	Background: Probabilities [MRS] Language Models for Information Retrieval [CMS] : Chapter 7, Language Models [PAPER] A Study of Smoothing Methods for Language Models Applied to Ad Hoc Information Retrieval by Chengxiang Zhai and John Lafferty
Week 4 5/25 - 6/1 Indexing / Index Construction Inverted Index Stopping, stemming Index Constrcution Ngrams, Skipgrams, Finding Blurbs Co-occurrance MON 5/25 Memorial Day, NO CLASS	Slides: Index Pre-Process Slides : Indexing 1 : Construction Finding blurbs (minimum spans) Notes: Indexing	[MRS] : Index construction, Index compression, Computing Scores [CMS] Chapter 5 Additional Slides on ElasticSearch, Indexing Zobel's Inverted files paper	HW 2 Due: Wed 6/10
Week 5 6/1 - 6/8 Indexing / Storage Compression Zipfs and Heaps laws Index Storage Distributed Indexes	Slides: Index Pre-Process Slides: Indexing 2 Slides: Text Stats Slides: Compression	A summary of the WSDM talk by Greg Linden Wikipedia : Delta Encoding; Elias Delta Encoding, Elias Gamma Encoding Notes : Lempel Ziv
Week 6 6/8 - 6/15 Crawling Crawling Basics HTTP Links Graph BFS recap Frontier/Queue Duplicates MON 6/8 NEU Reflection Day, NO CLASS	Slides: Crawling_1 Notes: Crawling Slides: intro to graphs	Web Crawler - Wikipedia Web Crawling Tutorial by Christopher Olston and Marc Najork inr-017.dvi	HW3 Due: 7/3
Week 7 6/15 - 6/22 Crawling / Merging Link Graph Freshness VS Coverage Vertical Search	Slides: Crawling 2 Note: Merging indexes in ES	Paper: Crawl Longevity Paper: IRLbot Paper: Near Duplicates Paper: Powerlaw Internet Topology
Week 8 6/22 - 6/29 Link Graph PageRank Topical PageRank HITS SALSA Intro: IR Evaluation	Notes : Webgraph Recap : Markov Chains Slides(A): PageRank PageRank Explained Slides(B): PageRank Degree Analysis Paper: HITS Paper: PageRank Paper: SALSA	PageRank Examples Paper: Graph Structure in the Web HITS Lecture from Cornell Wikipedia: SALSA paper: HITS vs SALSA HITS vs SALSA visual comparison tool, cdf file PageRank basic pseudocode	HW4 Due: Tue 7/13
Week 9 6/29 - 7/6 IR Evaluation/ Measures IR ranking performance Set measures: Precision, Recall, F1, Accuracy, ROC, confusion matrix Ranking measures: R-prec, AP, nDCG, Reciprocal Rank Significance tests	Slides: Evaluation_1 Notes : IR Evaluation Paper: Classification of IR measures as of year 2000	Paper: IR Metrics VS users Wikipedia: ROC	HW 5 Due: 7/23
Week 10 7/6 - 7/13 IR Evaluation / Assessments Relevance Assessments Diversity eval using subtopics Assessors, Crowdsourcing, Cost Assessor Interface Test Collection Construction	Slides: Evaluation_2	paper : IR metrics, tests (slides) code: Diversity measures slides: User Studies, TREC Algorithms: Diversifying Results Paper: Batch vs Users Evaluation Paper: Do friends make you smarter? Paper: Users vs models on measure efectiveness
Week 11 7/13 - 7/20 Machine Learning / Features Document understanding Features Extracting Query Features Similarity How to measure ML ML algorithms	Slides: Document Features Notes: Machine Learning for Text Online Class: Document Features (annotated pdf) Online Class: Linear Regression (annotated pdf)	Andrew Ng Coursera ML course wikipedia: Machine Learning LibLinear library	HW 6 Due: 8/3
Week 12 7/20 - 7/27 ML / Algorithms / Ranking Text Classification with unigrams Sparse matrix How/Why Learning Works What to expect Learning to Rank LambdaMart Pairwise Models	Slides: Brief intro to Machine Learning Notes: ML Decision Trees Notes: ML Linear Regression Notes: ML Logistic Regression Online ClassDecision Trees (annotated pdf) Cheng's note, sparse format, Learning Code for HW7 Bingyu's demo for HW7	Paper: From RankNet to LambdaRank to LambdaMart Wikipedia: Learning To Rank Paper: Yahoo Learning to Rank challenge Paper: AdaBoost and Rankboost Paper: RankNet Paper: LambdaMart, LambdaMart2 LSTM	HW7 Due: Mon 8/10
Week 13 7/27 - 8/3 ML/ Clustering / Topic Models Clustering LDA/PCA Topic Models	Slides: Topic Models Slides : Clustering (Mirek Riedewald)	Slides : Topic Models/LDA (Blei) * https://github.com/JohnLangford/vowpal_wabbit/wiki * http://scikit-learn.org/stable/auto_examples/applications/topics_extraction_with_nmf_lda.html * https://ariddell.org/lda.html * https://pypi.python.org/pypi/lda * https://cran.r-project.org/web/packages/lda/index.html * http://programminghistorian.org/lessons/topic-modeling-and-mallet	HW8 (no credit, but can make up 50 points of past hws) Due: Tue 8/15
Week 14 8/3 - 8/10 Adversarial IR, Spam Spam Link Farms	Slides:TrustRank Google Bombs