A Comprehensive Guide to NLTK in Python: Volume 1

Tokenizing Text in Python for Natural Language Processing

Recent Course Review:

"Great course! The things that Mike taught are practical and can be applied in the real world immediately." -- Ricky Valencia

Welcome to A Comprehensive Guide to NLTK in Python: Volume 1

This is the very FIRST course in a series of courses that will focus on NLTK.

Natural Language ToolKit (NLTK) is a comprehensive Python library for natural language processing and text analytics.

Note: This isn't a modeling building course. This course is laser focused on a very specific part of natural language processing called tokenization.

This is the first part in a series of courses crafted to help you master NLP. This course will cover the basics of tokenizing text and using WordNet

Tokenization is a method of breaking up a piece of text into many pieces, such as sentences and words, and is an essential first step for recipes in the later courses. WordNet is a dictionary designed for programmatic access by natural language processing systems.

NLTK was originally created in 2001 as part of a computational linguistics course in the Department of Computer and Information Science at the University of Pennsylvania

We will take Natural Language Processing — or NLP for short — in a wide sense to cover any kind of computer manipulation of natural language. At one extreme, it could be as simple as counting word frequencies to compare different writing styles.

At the other extreme, NLP involves "understanding" complete human utterances, at least to the extent of being able to give useful responses to them.

Technologies based on NLP are becoming increasingly widespread. For example, phones and handheld computers support predictive text and handwriting recognition; web search engines give access to information locked up in unstructured text; machine translation allows us to retrieve texts written in Chinese and read them in Spanish; text analysis enables us to detect sentiment in tweets and blogs.

A Jupyter notebook is a web app that allows you to write and annotate Python code interactively. It's a great way to experiment, do research, and share what you are working on.

In this course all of the tutorials will be created using jupyter notebooks. In the preview lessons we install Python. Check them out. They are completely free.

By providing more natural human-machine interfaces, and more sophisticated access to stored information, language processing has come to play a central role in the multilingual information society.

Thanks for your interest in A Comprehensive Guide to NLTK in Python: Volume 1