27. Text Classification in Python

By Bernd Klein. Last modified: 17 Feb 2022.

On this page ➤

Introduction

In the previous chapter, we have deduced the formula for calculating the probability that a document d belongs to a category or class c, denoted as P(c|d).

We have transformed the standard formular for P(c|d), as it is used in many treatises¹, into a numerically stable form.

We use a Naive Bayes classifier for our implementation in Python. The formal introduction into the Naive Bayes approach can be found in our previous chapter.

Bag of Words

Python is ideal for text classification, because of it's strong string class with powerful methods. Furthermore the regular expression module re of Python provides the user with tools, which are way beyond other programming languages.

The only downside might be that this Python implementation is not tuned for efficiency.

Live Python training

Enjoying this page? We offer live Python training courses covering the content of this site.

See our Python training courses

See our Machine Learning with Python training courses

Python Implementation of Previous Chapter

Document Representation

The document representation, which is based on the bag of word model, is illustrated in the following diagram:

Document Representation

Imports Needed

Our implementation needs the regular expression module re and the os module:

import re
import os

We will use in our implementation the function dict_merge_sum from the exercise 1 of our chapter on dictionaries:

def dict_merge_sum(d1, d2):
    """ Two dicionaries d1 and d2 with numerical values and
    possibly disjoint keys are merged and the values are added if
    the exist in both values, otherwise the missing value is taken to
    be 0"""
    
    return { k: d1.get(k, 0) + d2.get(k, 0) for k in set(d1) | set(d2) }

d1 = dict(a=4, b=5, d=8)
d2 = dict(a=1, d=10, e=9)

dict_merge_sum(d1, d2)

OUTPUT:

{'e': 9, 'd': 18, 'a': 5, 'b': 5}

BagOfWordsClass

class BagOfWords(object):
    """ Implementing a bag of words, words corresponding with their 
    frequency of usages in a "document" for usage by the 
    Document class, Category class and the Pool class."""
    
    def __init__(self):
        self.__number_of_words = 0
        self.__bag_of_words = {}
        
        
    def __add__(self, other):
        """ Overloading of the "+" operator to join two BagOfWords """
        
        erg = BagOfWords() 
        erg.__bag_of_words = dict_merge_sum(self.__bag_of_words, 
                                            other.__bag_of_words)
        return erg
        
    def add_word(self,word):
        """ A word is added in the dictionary __bag_of_words"""
        self.__number_of_words += 1
        if word in self.__bag_of_words:
            self.__bag_of_words[word] += 1
        else:
            self.__bag_of_words[word] = 1
    
    def len(self):
        """ Returning the number of different words of an object """
        return len(self.__bag_of_words)
    
    def Words(self):
        """ Returning a list of the words contained in the object """
        return self.__bag_of_words.keys()