Vocab extractor for language learners using Stanza and frequency ranks
Vocab extractor for language learners using Stanza and frequency ranks
I'm building a Telegram bot to practice Dutch. GPT-4o-mini kept picking vocabulary words I already knew, so I built a classical NLP pipeline to do it instead. It takes a short text + learner level (A0–B1) and returns the best words to study, using Stanza for parsing and corpus frequency ranks (SUBTLEX-NL, srLex, SUBTLEX-US) for scoring. Wins at A1/A2, loses at A0 where the LLM picks more obvious words. I also tried adding multi-word phrases (ADJ+NOUN, VERB+NOUN, phrasal verbs) backed by NPMI-scored collocation whitelists. Couldn't beat GPT there because it just "knows" which phrases matter. For the phrase work I had to extract collocations from 100M+ OpenSubtitles lines. Published them as a free dataset: https://huggingface.co/datasets/vladvlasov256/opensubs-collo... There are 43K bigrams across English, Dutch, and Serbian. Source https://github.com/vladvlasov256/vocab-nlp
Share cardActual performance
Launch Intel predictions
Analyze your own launch →Incorrect prediction on native model
Similar products
WordAddict, personal lexicon for language learners
for intermediate-to-advanced foreign language learners
Memozora – A flashcard maker for language learners
Concept extractor
Nativish, an app to help language learners
Crossword puzzles for language learners
Crowdsourced foreign language materials for intermediate learners
Vibe Language – A messenger for language learners
Lingohackers – new online community for language learners
pzip- blazing fast concurrent zip archiver and extractor