python logging

I have used logging on and off for a few years - but never really sat down and go it fully sorted out in my head.

This short tutorial is an attempt to fix that.

Module Logging

It is already there - and has been for ages - just you (and me) never knew about it.

How to I use it ?

import logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

Then at some point in your code ....

     logger.info("This is Info")
     logger.warning("Oh no Mr {0}",BadName)

I think you should get the drift from that point.

Altering the level of logging

When in development mode - it is quite useful to see things that you (hopefully) can ignore when in production mode. This is especially true when debugging something which is real-time, or near-time is involved.

To alter the logging level - do something like this

logging.basicConfig(level=logging.DEBUG)

If you only want Warnings (and worse .... Error, Critical)

logging.basicConfig(level=logging.WARNINGS)

Using multiple Modules

Of course you are using modules to break your code into smaller pieces - The temptation now is to put this logging.basicConfig in each module - but this is a mistake - becuase you can have different modules with different logging levels (yes you can use an external config file) - but I prefer not too.

Solution

When you create a module - pass in the logging object.

Like this example

import logging

def foo():
    logger = logging.getLogger(__name__)
    logger.info('Hi, foo')

class Bar(object):
    def __init__(self, logger=None):
        self.logger = logger or logging.getLogger(__name__)

    def bar(self):
        self.logger.info('Hi, bar')

The correct way to do it

I like this way to handle logging as I can control the output from a yaml file. Better still would be to have an opt_parse which allowed you to choose the correct yaml file.

Yaml based config

version: 1

formatters:
    simple:
        format: "%(name)s - %(lineno)d -  %(message)s"

    complex:
        format: "%(asctime)s - %(name)s - %(lineno)d -  %(message)s"


handlers:
    console:
        class: logging.StreamHandler
        level: DEBUG
        formatter: simple

    file:
        class: logging.handlers.TimedRotatingFileHandler
        when: midnight
        backupCount: 5
        level: DEBUG
        formatter: simple
        filename : Thrift.log

loggers:

    qsoWidget:
        level: DEBUG
        handlers: [console]
        propagate: yes

    ContestUi:    

THis config can be also writen like this

formatters:
  complex: {format: '%(asctime)s - %(name)s - %(lineno)d -  %(message)s'}
  simple: {format: '%(name)s - %(lineno)d -  %(message)s'}
handlers:
  console: {class: logging.StreamHandler, formatter: simple, level: DEBUG}
  file: {backupCount: 5, class: logging.handlers.TimedRotatingFileHandler, filename: fatrow.log,
    formatter: simple, level: DEBUG, when: midnight}
loggers:
  __main__:
    handlers: [console, file]
    level: DEBUG
    propagate: false
  fatrow:
    handlers: [console]
    level: DEBUG
    propagate: true
version: 1

Python - The main

The main module should look like this

import logging.config
import logging
with open('logging.yaml','rt') as f:
        config=yaml.safe_load(f.read())
        f.close()
logging.config.dictConfig(config)
logger=logging.getLogger(__name__)
logger.info("Contest is starting")

Sub Modules/Classes

These should start like this

import logging

class locator(object):
    def __init__(self):
        self.logger = logging.getLogger(__name__)
        self.logger.debug('{} initialized')