Skip to content
All projects
Data Science · AI & MLMay 2021

FindIT! UGM Data Analytics Competition

A competition entry for UGM's FindIT! data analytics contest: an Indonesian-language text-similarity pipeline with NLP preprocessing, vectorisation and cosine similarity.

Approach

  • Preprocessing: tokenising, removing punctuation and stopwords, and stemming Indonesian text with Sastrawi (and English text with NLTK)
  • Features: bag-of-words with CountVectorizer, normalised, with PCA for exploration
  • Matching: cosine similarity between documents to find the closest pairs