Logging, messaging, warnings and exceptions
I am opening this issue to propose a solution for how logging, messaging, warnings and exceptions are handled, so @all let me know if you would agree on this, since it is related to a few other issues, such as #35 (closed) and #77 (closed).
My suggestion here would be to have a root logger, using the standard python logging library, that will be displaying information at 3 different levels: INFO
, WARNING
and ERROR
. Some more can added if needed.
The configuration of the root logger will be fixed by in terms of i) the format of loggings, and ii) the logging channels, and all messages, warnings and error will be logged in a default rotating file as well as in the console. This root logger will allow each module to have a child instance that inherits all properties from the root (format, channels etc.).
The root logger will be instantiated in the loggers.py
file:
import logging
def getLogger(name='aaad'):
logger = logging.getLogger(name)
# Read a logger config file and set the format, channels etc.
logger.setLevel(...)
logger.addHandler(...)
return logger
Within each module, a child logger will be instantiated from the root. For example in the dataset.py:
from aaad.sampler.strategies import Strategy
import loggers
logger = loggers.getLogger().getChild('dataset')
dp_long = constants.design_par_long
pa_long = constants.perf_attribute_long
dr_long = constants.design_rep_long
class Dataset:
"""
This class manages the Dataset. The data, model checkpoints and other logging information resides in the respective folder/file structure:
- :code:`{self.datapath}/checkpoints/`
the logger aaad.dataset will be instantiated, as child of the root, and taking care of the logs within the module. Therefore, instead of simply printing messages like this:
print("* No files exist for this dataset with name {}!".format(filename_dataset))
one would have to send the information to the logger aaad.dataset
as follows:
logger.info("* No files exist for this dataset with name {}!".format(filename_dataset))
This turn will be streamed to the console and the logging file in the following format:
21/10/2023 11:46:36 AM - [aaad.dataset] - [INFO] - No files exist for this dataset with name ...
which can be customized.
The benefits of such an approach are:
- It gives the flexibility to filter logs based on i) custom verbosity level and ii) the hierarchy on the logging tree ('aaad', 'aaad.dataset', 'aaad.dataset.load' etc).
- Minimal changes are required in the existing modules for the implementation of such a logger
- The level of logs to be displayed can be specified from the user by changing a global variable
- Warnings and exceptions, although not in place yet, can be also logged by built-in functions.