Python Logging Module

Python Logging module is used to implement a flexible event-driven logging system, which serves as a convenient way of storing log events or messages for an application.

Python Logging Module – Loggers

The Logger object is the object of this module which we can manipulate to do all our required logging.

To instantiate a Logger object, we must always specify:

log_object = logging.getLogger(name)

Multiple calls to getLogger(name) with the same name always give a reference to the same object.

Now that we have our logger object, we can use multiple functions on it.

Writing messages to the console

Whenever there need to be events reported, we emit the contents of the logger objects so that the main running program gets notified of the status changes.

To do this, we have demarcation for the level of severity of a message to be emitted, called LEVEL.

This is used to write to the corresponding log file or to the console. The default level is WARNING, which means that only events of this level and above will be tracked (i.e, WARNING, ERROR, and CRITICAL will be tracked by default)

This allows the programmer to have control over how these status messages can be displayed, based on the severity level chosen.

Format: logging.info(message) will display the message onto the console/file.

The following example illustrates this method

import logging
# This message will be printed to the console
logging.warning('Warning message')

# Will not be printed to the console
logging.info('Works as expected')

Output

WARNING:root:Warning message

Logging into a file

We use logging.basicConfig() to create a logging file handler.

Format: logging.basicConfig(filename, level)

import logging

# Create the logfile
logging.basicConfig(filename='sample.log', level=logging.DEBUG)

logging.debug('Debug message')
logging.info('info message')
logging.warning('Warning message')

Output

root@Ubuntu # cat sample.log
DEBUG:root:Debug message
INFO:root:info message
WARNING:root:Warning message

NOTE: The call to basicConfig() must come before any call to debug(), info(), etc.

There is another parameter filemode, for the basicConfig() function, which specifies the mode of the logfile.

The below example makes sample.log have write-only mode, which means any messages written to it will overwrite the previous content of the file.

logging.basicConfig(filename='sample.log', filemode='w', level=logging.DEBUG)

Logging from multiple modules

Since the log-file object and the handlers provide the same context in multiple modules, we can use them directly in other modules.

An example is shown below

# main.py
import logging
import sample_module

def main():
    logging.basicConfig(filename='application.log', level=logging.INFO)
    logging.info('main module started')
    sample_module.sample_function()
    logging.info('completed')

main()

# sample_module.py
import logging

def sample_function():
    logging.info('Sample module doing work')

Here, the same logging object can be shared by multiple modules, thus making it a good fit for modularised code.

Format of the messages

By default, the output message has a message header containing the name of the node and the level of the message. To change the format of the displayed messages, a suitable format must be specified.

import logging
logging.basicConfig(format='%(levelname)s:%(message)s', level=logging.DEBUG)
logging.debug('sample message')

Output

DEBUG:sample message

Display Date and Time in the message

Add the %(asctime)s format specifier to signify the time in the message.

import logging
logging.basicConfig(format='%(asctime)s %(message)s', datefmt='%m/%d/%Y %I:%M:%S %p')
logging.warning('Sample message')

Output

12/26/2019 12:50:38 PM Sample message

Logger object names

The log message, by default, has the first part containing the level and the name of the logger object used. (For example: DEBUG:ROOT:sample message)

Usually, if the name argument is not specified, it defaults to ROOT, the name of the root node.

Otherwise, it is good practice to use the __name__ variable, because it is the name of the module in the Python package namespace.

import logging

logger = logging.getLogger(__name__)

Modifying the level of the message

Logger objects provide us a way of modifying the threshold level at which messages are displayed. The setLevel(level) method is used to set the level of the logger object.

Format: logger.setLevel(level)

import logging

logger = logging.getLogger(__name__)

# Set the log level as DEBUG
logger.setLevel(logging.DEBUG)

# The DEBUG level message is displayed
logger.debug('Debug message')

Output

No handlers could be found for logger "__main__"

This is not what we expected. Why is the message not being displayed, and what is a handler?