Logging modular

What is a journal ？

Logging is a way to track events that occur while some software is running . adopt log Analysis of , It is convenient for users to understand the system or software 、 The operation of the application .

Level of logging

logging Module log levels are DEBUG < INFO < WARNING < ERROR < CRITICAL Five kinds .

• DEBUG - Debug mode , The application scenario is problem diagnosis ;
• INFO - Only general information is recorded in the program , Used to confirm that everything is working properly ;
• WARNING - Print warning messages , The system is still working ;
• ERROR - Information recorded when an error causes some functions to fail to function properly ;
• CRITICAL - When there is a serious mistake , Information recorded when the application cannot continue running .

Part of the

• logger： Provide a method of logging .
• handler： Select the output place of the log （ One logger Add multiple handler）.
• filter： Provide users with more fine-grained output of the control log .
• format： The user formats the information of the output log .

Configuration method

1. Basic configuration

logging.basicConfig(filename="config.log", 
                    filemode="w", 
                    format="%(asctime)s-%(name)s-%(levelname)s-%(message)s", 
                    level=logging.INFO)

2. How to use a profile

fileConfig(filename,defaults=None,disable_existing_loggers=Ture )

2. Use a dictionary to write configuration information

 Use dictConfig(dict,defaults=None, disable_existing_loggers=Ture ) function 

Log output

• StreamHandler

logging.StreamHandler: Log output to stream , It can be sys.stderr,sys.stdout Or documents 

• FileHandler

logging.FileHandler: Log output to file 

• BaseRotatingHandler

logging.handlers.BaseRotatingHandler: Basic log rollback mode 

• RotatingHandler

logging.handlers.RotatingHandler: Log rollback mode , Support the maximum number of log files and log file rollback 

Explanation of some terms

Logging.Formatter： This class configures the format of the log , Customize the date and time in it , When outputting, it will be displayed in the set format .
Logging.Logger：Logger yes Logging The body of the module , To carry out the following three tasks ：
    1.  Provides the program with an interface for logging 
    2.  Determine the level of the log , And determine whether to filter 
    3.  Distribute this log to different handler

Encapsulation of log

​ In actual development , We will logging Write a module separately and encapsulate it into a class , Then in other functional modules, such as communication protocol module , Database module , The business layer ,API Call modules and so on , Call in the .

​ In practice , As mentioned earlier logging.getLogger(" ") Passing in no name is suitable for only a small range of system use , Once multiple modules are involved , Situations requiring system division , Make a proposal to class Set an incoming parameter , And pass this parameter to logging.getLogger(" ") , To standardize the division of logs .

import logging
import logging.handlers
import os
import time


class logs(object):
    def __init__(self):
        #  Get module name , When testing, you can directly control the module , However, in actual use, you need to name different modules that need to write logs 
        #  Columns such as ： Communication protocol module , Test module , Database module , Business layer module ,API Call module 
        #  You can consider  __init__(self,model_name)  It's introduced like this , And then another one list Specify the module name 
        self.logger = logging.getLogger("")
        #  Set the level of output 
        LEVELS = {
    'NOSET': logging.NOTSET,
                  'DEBUG': logging.DEBUG,
                  'INFO': logging.INFO,
                  'WARNING': logging.WARNING,
                  'ERROR': logging.ERROR,
                  'CRITICAL': logging.CRITICAL}
        #  Create file directory 
        logs_dir = "logs"
        if os.path.exists(logs_dir) and os.path.isdir(logs_dir):
            pass
        else:
            os.mkdir(logs_dir)
        #  modify log Save the location 
        timestamp = time.strftime("%Y-%m-%d", time.localtime())
        logfilename = '%s.txt' % timestamp
        logfilepath = os.path.join(logs_dir, logfilename)
        rotatingFileHandler = logging.handlers.RotatingFileHandler(filename=logfilepath,
                                                                   maxBytes=1024 * 1024 * 50,
                                                                   backupCount=5)
        #  Set the output format 
        formatter = logging.Formatter('[%(asctime)s] [%(levelname)s] %(message)s', '%Y-%m-%d %H:%M:%S')
        rotatingFileHandler.setFormatter(formatter)
        #  Console handle 
        console = logging.StreamHandler()
        console.setLevel(logging.NOTSET)
        console.setFormatter(formatter)
        #  Add content to the log handle 
        self.logger.addHandler(rotatingFileHandler)
        self.logger.addHandler(console)
        self.logger.setLevel(logging.NOTSET)

    def info(self, message):
        self.logger.info(message)

    def debug(self, message):
        self.logger.debug(message)

    def warning(self, message):
        self.logger.warning(message)

    def error(self, message):
        self.logger.error(message)

Demo

1. Log output - Console

   import logging  #  introduce logging modular 
   logging.basicConfig(level=logging.DEBUG,
                       format='%(asctime)s - %(filename)s[line:%(lineno)d] - %(levelname)s: %(message)s')  # logging.basicConfig Function to configure the output format and mode of the log 
   
   #  Since the level in the basic log configuration is set to DEBUG, So the print information will be displayed on the console 
   logging.info('this is a loggging info message')
   logging.debug('this is a loggging debug message')
   logging.warning('this is loggging a warning message')
   logging.error('this is an loggging error message')
   logging.critical('this is a loggging critical message')
   

2. Log output - file

import logging
import os.path
import time


#  First step , Create a logger
logger = logging.getLogger()
logger.setLevel(logging.INFO)  #  Set up Log Grade 

#  The second step , Create a handler, For writing log files 
rq = time.strftime('%Y%m%d%H%M', time.localtime(time.time()))
log_path = os.path.dirname(os.getcwd()) + '/Logs/'
log_name = log_path + rq + '.log'
logfile = log_name

fh = logging.FileHandler(logfile, mode='w')
fh.setLevel(logging.DEBUG)  #  Output to file Of log Level switch 

#  The third step , Definition handler The output format of 
formatter = logging.Formatter("%(asctime)s - %(filename)s[line:%(lineno)d] - %(levelname)s: %(message)s")
fh.setFormatter(formatter)

#  Step four , take logger Add to handler Inside 
logger.addHandler(fh)

#  journal 
logger.debug('this is a logger debug message')
logger.info('this is a logger info message')
logger.warning('this is a logger warning message')
logger.error('this is a logger error message')
logger.critical('this is a logger critical message')

3. Log output - Console and files

ch = logging.StreamHandler()	#  Just insert a handler Output to console 
ch.setLevel(logging.WARNING)  #  Output to console Of log Level switch 
#  Add the following code in step 4 and step 5 respectively 
ch.setFormatter(formatter)
logger.addHandler(ch)
6、format Common format description 
    %(levelno)s:  Print log level values 
    %(levelname)s:  Print log level name 
    %(pathname)s:  Print the path of the currently executing program , In fact, that is sys.argv[0]
    %(filename)s:  Print the name of the currently executing program 
    %(funcName)s:  Print the current function of the log 
    %(lineno)d:  Print the current line number of the log 
    %(asctime)s:  Time to print the log 
    %(thread)d:  Print thread ID
    %(threadName)s:  Print thread name 
    %(process)d:  Printing process ID
    %(message)s:  Print log information 

    
import os.path
import time
import logging
#  Create a logger
logger = logging.getLogger()
logger.setLevel(logging.INFO)  # Log Grade master switch 

#  Create a handler, For writing log files 
rq = time.strftime('%Y%m%d%H%M', time.localtime(time.time()))
log_path = os.path.dirname(os.getcwd()) + '/Logs/'
log_name = log_path + rq + '.log'
logfile = log_name
fh = logging.FileHandler(logfile, mode='w')
fh.setLevel(logging.DEBUG)  #  Output to file Of log Level switch 

#  Definition handler The output format of 
formatter = logging.Formatter("%(asctime)s - %(filename)s[line:%(lineno)d] - %(levelname)s: %(message)s")
fh.setFormatter(formatter)
logger.addHandler(fh)
#  Use logger.XX To record errors , there "error" It can be modified according to the required level 
try:
    open('/path/to/does/not/exist', 'rb')
except (SystemExit, KeyboardInterrupt):
    raise
except Exception, e:
    logger.error('Failed to open file', exc_info=True)

4. Log scrolling and expired deletion （ By time ）

#  Field explanation 
filename： Log file name prefix;

when： Is a string , The basic unit used to describe the rolling period , The values and meanings of a string are as follows ： 
“S”: Seconds 
“M”: Minutes 
“H”: Hours 
“D”: Days 
“W”: Week day (0=Monday) 
“midnight”: Roll over at midnight

interval:  Rolling cycle , The unit has when Appoint , such as ：when=’D’,interval=1, A log file is generated every day 
backupCount:  Indicates the number of reserved log files 

# coding:utf-8
import logging
import time
import re
from logging.handlers import TimedRotatingFileHandler
from logging.handlers import RotatingFileHandler


def backroll():
    # Log print format 
    log_fmt = '%(asctime)s\tFile \"%(filename)s\",line %(lineno)s\t%(levelname)s: %(message)s'
    formatter = logging.Formatter(log_fmt)
    # establish TimedRotatingFileHandler object 
    log_file_handler = TimedRotatingFileHandler(filename="ds_update", when="M", interval=2, backupCount=2)
    #log_file_handler.suffix = "%Y-%m-%d_%H-%M.log"
    #log_file_handler.extMatch = re.compile(r"^\d{4}-\d{2}-\d{2}_\d{2}-\d{2}.log$")
    log_file_handler.setFormatter(formatter)
    logging.basicConfig(level=logging.INFO)
    log = logging.getLogger()
    log.addHandler(log_file_handler)
    # Circular printing log 
    log_content = "test log"
    count = 0
    while count < 30:
        log.error(log_content)
        time.sleep(20)
        count = count + 1
    log.removeHandler(log_file_handler)


if __name__ == "__main__":
    backroll()