Natural language processing (NLP) has experienced a rapid growth over the last few years and has become an important skill to build applications that range from social features to clinical and health solutions. In this tutorial, we will introduce PyTorch as a tool to build and experiment with various modern NLP techniques by building deep learning architectures based on convolutional neural networks (CNNs), recurrent neural networks (RNNs), and bidirectional long short-term memory networks (biLSTMs). We will cover a wide range of topics with the purpose of providing participants with enough fundamental knowledge and skills to be able to apply modern NLP to real-world problems using PyTorch. Some concepts and topics include but not limited to data loaders, vectorization, computation graphs, sentiment analysis, fine-grained emotion classification, and neural machine translation. Nowadays, it is just not enough to arbitrarily train a model and deploy it for production use without properly debugging it. This tutorial also aims to provide hands-on examples and well-organized exercises that teach students how to properly test, train and evaluate NLP models using best practices. Once models are properly trained and evaluated they will be efficiently transformed, stored, and then restored to obtain inferences from real-world, natural language data.