diff --git a/README.md b/README.md index 08bca9c..6b3e655 100644 --- a/README.md +++ b/README.md @@ -14,12 +14,10 @@ - [Console Logging with Details](#console-logging-with-details) - [File Logging with Rotation](#file-logging-with-rotation) - [File Logging with Compression and JSON Format](#file-logging-with-compression-and-json-format) - - [Graylog Integration](#graylog-integration) - - [AWS CloudWatch Integration](#aws-cloudwatch-integration) - [Mixing it all together](#mixing-it-all-together) ## Introduction -APV emerged from a simple observation: despite the abundance of logging solutions, there's a glaring lack of standardization in application logging. As a developer deeply entrenched in Elasticsearch, AWS, and Graylog ecosystems, I found myself repeatedly grappling with inconsistent log formats and cumbersome integrations. APV is my response to this challenge – a logging library that doesn't aim to revolutionize the field, but rather to streamline it. +APV emerged from a simple observation: despite the abundance of logging solutions, there's a glaring lack of standardization in application logging. APV is my response to this challenge – a logging library that doesn't aim to revolutionize the field, but rather to streamline it. ## Requirements - Python 3.10+ @@ -28,17 +26,7 @@ APV emerged from a simple observation: despite the abundance of logging solution ### From PyPI ```bash -# Basic installation pip install apv - -# With CloudWatch support -pip install apv[cloudwatch] - -# With ECS logging support -pip install apv[ecs] - -# With all optional dependencies -pip install "apv[cloudwatch,ecs]" ``` ### From Source @@ -53,34 +41,23 @@ pip install . - **File Logging**: Write logs to files with support for log rotation based on size and number of backups. - **Log Compression**: Automatically compress old log files using gzip to save disk space. - **JSON Logging**: Output logs in JSON format for better structure and integration with log management systems. -- **ECS Logging**: Output logs in ECS format for better integration with [Elasticsearch](https://www.elastic.co/elasticsearch/) - **Detailed Log Messages**: Option to include module name, function name, and line number in log messages. -- **Graylog Integration**: Send logs to a [Graylog](https://www.graylog.org/) server using GELF over UDP. -- **AWS CloudWatch Integration**: Send logs to [AWS CloudWatch Logs](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/WhatIsCloudWatchLogs.html). -- **Customizable Logging Levels**: Set the logging level to control verbosity. ## Configuration Options The `setup_logging` function accepts the following keyword arguments to customize logging behavior: -| Name | Default | Description | -|--------------------------|--------------------------|--------------------------------------------------------------------------------------| -| `level` | `INFO` | The logging level. *(`DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`)* | -| `date_format` | `'%Y-%m-%d %H:%M:%S'` | The date format for log messages. | -| `log_to_disk` | `False` | Whether to log to disk. | -| `max_log_size` | `10*1024*1024` *(10 MB)* | The maximum size of log files before rotation *(in bytes)*. | -| `max_backups` | `7` | The maximum number of backup log files to keep. | -| `log_file_name` | `'app'` | The base name of the log file. | -| `json_log` | `False` | Whether to log in JSON format. | -| `ecs_log` | `False` | Whether to log in ECS format. | -| `show_details` | `False` | Whether to include module name, function name, & line number in log messages. | -| `compress_backups` | `False` | Whether to compress old log files using gzip. | -| `enable_graylog` | `False` | Whether to enable logging to a Graylog server. | -| `graylog_host` | `None` | The Graylog server host. *(Required if `enable_graylog` is `True`)* | -| `graylog_port` | `None` | The Graylog server port. *(Required if `enable_graylog` is `True`)* | -| `enable_cloudwatch` | `False` | Whether to enable logging to AWS CloudWatch Logs. | -| `cloudwatch_group_name` | `None` | The name of the CloudWatch log group. *(Required if `enable_cloudwatch` is `True`)* | -| `cloudwatch_stream_name` | `None` | The name of the CloudWatch log stream. *(Required if `enable_cloudwatch` is `True`)* | +| Name | Default | Description | +|-------------------|--------------------------|-------------------------------------------------------------------------------| +| `level` | `INFO` | The logging level. *(`DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`)* | +| `date_format` | `'%Y-%m-%d %H:%M:%S'` | The date format for log messages. | +| `log_to_disk` | `False` | Whether to log to disk. | +| `max_log_size` | `10*1024*1024` *(10 MB)* | The maximum size of log files before rotation *(in bytes)*. | +| `max_backups` | `7` | The maximum number of backup log files to keep. | +| `log_file_name` | `'app'` | The base name of the log file. | +| `json_log` | `False` | Whether to log in JSON format. | +| `show_details` | `False` | Whether to include module name, function name, & line number in log messages. | +| `compress_backups`| `False` | Whether to compress old log files using gzip. | ## Usage @@ -147,53 +124,6 @@ apv.setup_logging( logging.debug('This is a debug message in JSON format.') ``` -### Graylog Integration - -```python -import logging -import apv - -# Set up logging to Graylog server -apv.setup_logging( - level='INFO', - enable_graylog=True, - graylog_host='graylog.example.com', - graylog_port=12201 -) - -logging.info('This message will be sent to Graylog.') -``` - -### AWS CloudWatch Integration - -```python -import logging -import apv - -# Set up logging to AWS CloudWatch Logs -apv.setup_logging( - level='INFO', - enable_cloudwatch=True, - cloudwatch_group_name='my_log_group', - cloudwatch_stream_name='my_log_stream' -) - -logging.info('This message will be sent to AWS CloudWatch.') -``` - -### ECS Logging - -```python -import logging -import apv - -# Set up ECS logging -apv.setup_logging( - level='INFO', - ecs_log=True -) -``` - ### Mixing it all together ```python @@ -209,12 +139,6 @@ apv.setup_logging( log_file_name='app', json_log=True, compress_backups=True, - enable_graylog=True, - graylog_host='graylog.example.com', - graylog_port=12201, - enable_cloudwatch=True, - cloudwatch_group_name='my_log_group', - cloudwatch_stream_name='my_log_stream', show_details=True ) ``` diff --git a/apv/__init__.py b/apv/__init__.py index 3c91ec6..628377d 100644 --- a/apv/__init__.py +++ b/apv/__init__.py @@ -1,4 +1,3 @@ -from .apv import * - -__version__ = '1.0.4' -__author__ = 'acidvegas' +#!/usr/bin/env python3 +# Advanced Python Logging - Developed by acidvegas in Python (https://git.acid.vegas/apv) +# apv/__init__.py \ No newline at end of file diff --git a/apv/apv.py b/apv/apv.py index 72e0693..49b1e1b 100644 --- a/apv/apv.py +++ b/apv/apv.py @@ -2,143 +2,216 @@ # Advanced Python Logging - Developed by acidvegas in Python (https://git.acid.vegas/apv) # apv.py +import gzip +import json import logging import logging.handlers +import os +import socket import sys -sys.stdout.reconfigure(encoding='utf-8') + +class LogColors: + '''ANSI color codes for log messages''' + + NOTSET = '\033[97m' # White text + DEBUG = '\033[96m' # Cyan + INFO = '\033[92m' # Green + WARNING = '\033[93m' # Yellow + ERROR = '\033[91m' # Red + CRITICAL = '\033[97m\033[41m' # White on Red + FATAL = '\033[97m\033[41m' # Same as CRITICAL + DATE = '\033[90m' # Dark Grey + MODULE = '\033[95m' # Pink + FUNCTION = '\033[94m' # Blue + LINE = '\033[33m' # Orange + RESET = '\033[0m' + SEPARATOR = '\033[90m' # Dark Grey + + +class ConsoleFormatter(logging.Formatter): + '''A formatter for the consolethat supports colored output''' + + def __init__(self, datefmt: str = None, details: bool = False): + super().__init__(datefmt=datefmt) + self.details = details + + + def format(self, record: logging.LogRecord) -> str: + ''' + Format a log record for the console + + :param record: The log record to format + ''' + + # Get the color for the log level + color = getattr(LogColors, record.levelname, LogColors.RESET) + + # Format the log level + log_level = f'{color}{record.levelname:<8}{LogColors.RESET}' + + # Get the log message + message = record.getMessage() + + # Format the timestamp + asctime = f'{LogColors.DATE}{self.formatTime(record, self.datefmt)}' + + # Get the separator + separator = f'{LogColors.SEPARATOR} ┃ {LogColors.RESET}' + details = f'{LogColors.MODULE}{record.module}{separator}{LogColors.FUNCTION}{record.funcName}{separator}{LogColors.LINE}{record.lineno}{separator}' if self.details else '' + + return f'{asctime}{separator}{log_level}{separator}{details}{message}' + + +class JsonFormatter(logging.Formatter): + '''Formatter for JSON output''' + + def __init__(self, datefmt: str = None): + super().__init__(datefmt=datefmt) + + + def format(self, record: logging.LogRecord) -> str: + ''' + Format a log record for JSON output + + :param record: The log record to format + ''' + + # Create a dictionary to store the log record + log_dict = { + '@timestamp' : self.formatTime(record, self.datefmt), + 'level' : record.levelname, + 'message' : record.getMessage(), + 'process_id' : record.process, + 'process_name' : record.processName, + 'thread_id' : record.thread, + 'thread_name' : record.threadName, + 'logger_name' : record.name, + 'filename' : record.filename, + 'line_number' : record.lineno, + 'function' : record.funcName, + 'module' : record.module, + 'hostname' : socket.gethostname() + } + + # Add the exception if it exists + if record.exc_info: + log_dict['exception'] = self.formatException(record.exc_info) + + # Add any custom attributes that start with an underscore + custom_attrs = {k: v for k, v in record.__dict__.items() if k.startswith('_') and not k.startswith('__')} + log_dict.update(custom_attrs) + + return json.dumps(log_dict) + + +class GZipRotatingFileHandler(logging.handlers.RotatingFileHandler): + '''RotatingFileHandler that compresses rotated log files''' + + def rotation_filename(self, default_name: str) -> str: + return default_name + '.gz' + + def rotate(self, source: str, dest: str): + with open(source, 'rb') as src, gzip.open(dest, 'wb') as dst: + dst.write(src.read()) + class LoggerSetup: - def __init__(self, level='INFO', date_format='%Y-%m-%d %H:%M:%S', - log_to_disk=False, max_log_size=10*1024*1024, - max_backups=7, log_file_name='app', json_log=False, - ecs_log=False, show_details=False, compress_backups=False, - enable_graylog=False, graylog_host=None, graylog_port=None, - enable_cloudwatch=False, cloudwatch_group_name=None, cloudwatch_stream_name=None): + def __init__(self, level: str = 'INFO', date_format: str = '%Y-%m-%d %H:%M:%S', log_to_disk: bool = False, max_log_size: int = 10*1024*1024, max_backups: int = 7, log_file_name: str = 'app', json_log: bool = False, show_details: bool = False, compress_backups: bool = False): ''' - Initialize the LoggerSetup with provided parameters. + Initialize the LoggerSetup with provided parameters - :param level: The logging level (e.g., 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL'). - :param date_format: The date format for log messages. - :param log_to_disk: Whether to log to disk. - :param max_log_size: The maximum size of log files before rotation. - :param max_backups: The maximum number of backup log files to keep. - :param log_file_name: The base name of the log file. - :param json_log: Whether to log in JSON format. - :param show_details: Whether to show detailed log messages. - :param compress_backups: Whether to compress old log files using gzip. - :param enable_graylog: Whether to enable Graylog logging. - :param graylog_host: The Graylog host. - :param graylog_port: The Graylog port. - :param enable_cloudwatch: Whether to enable CloudWatch logging. - :param cloudwatch_group_name: The CloudWatch log group name. - :param cloudwatch_stream_name: The CloudWatch log stream name. + :param level: The logging level (e.g., 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL') + :param date_format: The date format for log messages + :param log_to_disk: Whether to log to disk + :param max_log_size: The maximum size of log files before rotation + :param max_backups: The maximum number of backup log files to keep + :param log_file_name: The base name of the log file + :param json_log: Whether to log in JSON format + :param show_details: Whether to show detailed log messages + :param compress_backups: Whether to compress old log files using gzip ''' - self.level = level - self.date_format = date_format - self.log_to_disk = log_to_disk - self.max_log_size = max_log_size - self.max_backups = max_backups - self.log_file_name = log_file_name - self.json_log = json_log - self.ecs_log = ecs_log - self.show_details = show_details - self.compress_backups = compress_backups - self.enable_graylog = enable_graylog - self.graylog_host = graylog_host - self.graylog_port = graylog_port - self.enable_cloudwatch = enable_cloudwatch - self.cloudwatch_group_name = cloudwatch_group_name - self.cloudwatch_stream_name = cloudwatch_stream_name + self.level = level + self.date_format = date_format + self.log_to_disk = log_to_disk + self.max_log_size = max_log_size + self.max_backups = max_backups + self.log_file_name = log_file_name + self.json_log = json_log + self.show_details = show_details + self.compress_backups = compress_backups def setup(self): - '''Set up logging with various handlers and options.''' + '''Set up logging with various handlers and options''' # Clear existing handlers logging.getLogger().handlers.clear() - logging.getLogger().setLevel(logging.DEBUG) # Capture all logs at the root level + logging.getLogger().setLevel(logging.DEBUG) # Convert the level string to a logging level object level_num = getattr(logging, self.level.upper(), logging.INFO) + # Setup console handler self.setup_console_handler(level_num) + # Setup file handler if enabled if self.log_to_disk: self.setup_file_handler(level_num) - if self.enable_graylog: - self.setup_graylog_handler(level_num) - - if self.enable_cloudwatch: - self.setup_cloudwatch_handler(level_num) - def setup_console_handler(self, level_num: int): - '''Set up the console handler.''' - try: - from apv.plugins.console import setup_console_handler - setup_console_handler(level_num, self.date_format, self.show_details) - except ImportError: - logging.error('Failed to import console handler') + ''' + Set up the console handler + + :param level_num: The logging level number + ''' + + # Create the console handler + console_handler = logging.StreamHandler() + console_handler.setLevel(level_num) + + # Create the formatter + formatter = JsonFormatter(datefmt=self.date_format) if self.json_log else ConsoleFormatter(datefmt=self.date_format, details=self.show_details) + console_handler.setFormatter(formatter) + + # Add the handler to the root logger + logging.getLogger().addHandler(console_handler) def setup_file_handler(self, level_num: int): - '''Set up the file handler.''' - try: - from apv.plugins.file import setup_file_handler - setup_file_handler( - level_num=level_num, - log_to_disk=self.log_to_disk, - max_log_size=self.max_log_size, - max_backups=self.max_backups, - log_file_name=self.log_file_name, - json_log=self.json_log, - ecs_log=self.ecs_log, - date_format=self.date_format, - compress_backups=self.compress_backups - ) - except ImportError: - logging.error('Failed to import file handler') - - - def setup_graylog_handler(self, level_num: int): ''' - Set up the Graylog handler. + Set up the file handler - :param level_num: The logging level number. + :param level_num: The logging level number ''' - try: - from apv.plugins.graylog import setup_graylog_handler - setup_graylog_handler(level_num, self.graylog_host, self.graylog_port) - except ImportError: - logging.error('Failed to import Graylog handler') + # Create logs directory if it doesn't exist + logs_dir = os.path.join(sys.path[0], 'logs') + os.makedirs(logs_dir, exist_ok=True) + # Set up log file path + file_extension = '.json' if self.json_log else '.log' + log_file_path = os.path.join(logs_dir, f'{self.log_file_name}{file_extension}') - def setup_cloudwatch_handler(self, level_num: int): - ''' - Set up the CloudWatch handler. - - :param level_num: The logging level number. - ''' + # Create the rotating file handler + handler_class = GZipRotatingFileHandler if self.compress_backups else logging.handlers.RotatingFileHandler + file_handler = handler_class(log_file_path, maxBytes=self.max_log_size, backupCount=self.max_backups) + file_handler.setLevel(level_num) - try: - from apv.plugins.cloudwatch import setup_cloudwatch_handler - setup_cloudwatch_handler( - level_num, - self.cloudwatch_group_name, - self.cloudwatch_stream_name, - self.date_format - ) - except ImportError: - logging.error('Failed to import CloudWatch handler') + # Set up the appropriate formatter + formatter = JsonFormatter(datefmt=self.date_format) if self.json_log else logging.Formatter(fmt='%(asctime)s ┃ %(levelname)-8s ┃ %(module)s ┃ %(funcName)s ┃ %(lineno)d ┃ %(message)s', datefmt=self.date_format) + file_handler.setFormatter(formatter) + logging.getLogger().addHandler(file_handler) def setup_logging(**kwargs): - '''Set up logging with various handlers and options.''' + '''Set up logging with various handlers and options''' + # Create a LoggerSetup instance with the provided keyword arguments logger_setup = LoggerSetup(**kwargs) - logger_setup.setup() \ No newline at end of file + + # Set up the logging system + logger_setup.setup() \ No newline at end of file diff --git a/apv/plugins/__init__.py b/apv/plugins/__init__.py deleted file mode 100644 index 9237af8..0000000 --- a/apv/plugins/__init__.py +++ /dev/null @@ -1 +0,0 @@ -# Empty file to make plugins a package \ No newline at end of file diff --git a/apv/plugins/cloudwatch.py b/apv/plugins/cloudwatch.py deleted file mode 100644 index bc5ec03..0000000 --- a/apv/plugins/cloudwatch.py +++ /dev/null @@ -1,100 +0,0 @@ -import logging -import json -import boto3 -from botocore.exceptions import ClientError - -class CloudWatchHandler(logging.Handler): - def __init__(self, group_name, stream_name): - super().__init__() - self.group_name = group_name - self.stream_name = stream_name - self.client = boto3.client('logs') - self._initialize_log_group_and_stream() - - def _initialize_log_group_and_stream(self): - # Create log group if it doesn't exist - try: - self.client.create_log_group(logGroupName=self.group_name) - except ClientError as e: - if e.response['Error']['Code'] != 'ResourceAlreadyExistsException': - raise e - - # Create log stream if it doesn't exist - try: - self.client.create_log_stream( - logGroupName=self.group_name, - logStreamName=self.stream_name - ) - except ClientError as e: - if e.response['Error']['Code'] != 'ResourceAlreadyExistsException': - raise e - - def _get_sequence_token(self): - try: - response = self.client.describe_log_streams( - logGroupName=self.group_name, - logStreamNamePrefix=self.stream_name, - limit=1 - ) - log_streams = response.get('logStreams', []) - return log_streams[0].get('uploadSequenceToken') if log_streams else None - except Exception: - return None - - def emit(self, record): - try: - log_entry = self.format(record) - timestamp = int(record.created * 1000) - - event = { - 'timestamp': timestamp, - 'message': log_entry - } - - kwargs = { - 'logGroupName': self.group_name, - 'logStreamName': self.stream_name, - 'logEvents': [event] - } - - sequence_token = self._get_sequence_token() - if sequence_token: - kwargs['sequenceToken'] = sequence_token - - self.client.put_log_events(**kwargs) - except Exception: - self.handleError(record) - -def setup_cloudwatch_handler(level_num: int, group_name: str, stream_name: str, date_format: str): - '''Set up the CloudWatch handler.''' - try: - import boto3 - except ImportError: - raise ImportError('boto3 is required for CloudWatch logging. (pip install boto3)') - - if not group_name or not stream_name: - logging.error('CloudWatch log group and log stream must be specified for CloudWatch handler.') - return - - cloudwatch_handler = CloudWatchHandler(group_name, stream_name) - cloudwatch_handler.setLevel(level_num) - - class JsonFormatter(logging.Formatter): - def format(self, record): - log_record = { - 'time' : self.formatTime(record, date_format), - 'level' : record.levelname, - 'module' : record.module, - 'function' : record.funcName, - 'line' : record.lineno, - 'message' : record.getMessage(), - 'name' : record.name, - 'filename' : record.filename, - 'threadName' : record.threadName, - 'processName' : record.processName, - } - return json.dumps(log_record) - - cloudwatch_formatter = JsonFormatter(datefmt=date_format) - cloudwatch_handler.setFormatter(cloudwatch_formatter) - logging.getLogger().addHandler(cloudwatch_handler) \ No newline at end of file diff --git a/apv/plugins/console.py b/apv/plugins/console.py deleted file mode 100644 index 5cfbd7d..0000000 --- a/apv/plugins/console.py +++ /dev/null @@ -1,70 +0,0 @@ -import logging - -class LogColors: - '''ANSI color codes for log messages.''' - RESET = '\033[0m' - DATE = '\033[90m' # Dark Grey - DEBUG = '\033[96m' # Cyan - INFO = '\033[92m' # Green - WARNING = '\033[93m' # Yellow - ERROR = '\033[91m' # Red - CRITICAL = '\033[97m\033[41m' # White on Red - FATAL = '\033[97m\033[41m' # Same as CRITICAL - NOTSET = '\033[97m' # White text - SEPARATOR = '\033[90m' # Dark Grey - MODULE = '\033[95m' # Pink - FUNCTION = '\033[94m' # Blue - LINE = '\033[33m' # Orange - -class ColoredFormatter(logging.Formatter): - def __init__(self, datefmt=None, show_details=False): - super().__init__(datefmt=datefmt) - self.show_details = show_details - self.LEVEL_COLORS = { - 'NOTSET' : LogColors.NOTSET, - 'DEBUG' : LogColors.DEBUG, - 'INFO' : LogColors.INFO, - 'WARNING' : LogColors.WARNING, - 'ERROR' : LogColors.ERROR, - 'CRITICAL' : LogColors.CRITICAL, - 'FATAL' : LogColors.FATAL - } - - def format(self, record): - log_level = record.levelname - message = record.getMessage() - asctime = self.formatTime(record, self.datefmt) - color = self.LEVEL_COLORS.get(log_level, LogColors.RESET) - separator = f'{LogColors.SEPARATOR} ┃ {LogColors.RESET}' - - if self.show_details: - formatted = ( - f'{LogColors.DATE}{asctime}{LogColors.RESET}' - f'{separator}' - f'{color}{log_level:<8}{LogColors.RESET}' - f'{separator}' - f'{LogColors.MODULE}{record.module}{LogColors.RESET}' - f'{separator}' - f'{LogColors.FUNCTION}{record.funcName}{LogColors.RESET}' - f'{separator}' - f'{LogColors.LINE}{record.lineno}{LogColors.RESET}' - f'{separator}' - f'{message}' - ) - else: - formatted = ( - f'{LogColors.DATE}{asctime}{LogColors.RESET}' - f'{separator}' - f'{color}{log_level:<8}{LogColors.RESET}' - f'{separator}' - f'{message}' - ) - return formatted - -def setup_console_handler(level_num: int, date_format: str, show_details: bool): - '''Set up the console handler with colored output.''' - console_handler = logging.StreamHandler() - console_handler.setLevel(level_num) - console_formatter = ColoredFormatter(datefmt=date_format, show_details=show_details) - console_handler.setFormatter(console_formatter) - logging.getLogger().addHandler(console_handler) \ No newline at end of file diff --git a/apv/plugins/file.py b/apv/plugins/file.py deleted file mode 100644 index b42d1cf..0000000 --- a/apv/plugins/file.py +++ /dev/null @@ -1,77 +0,0 @@ -import logging -import logging.handlers -import json -import os -import gzip - -class GZipRotatingFileHandler(logging.handlers.RotatingFileHandler): - '''RotatingFileHandler that compresses old log files using gzip.''' - - def doRollover(self): - '''Compress old log files using gzip.''' - super().doRollover() - if self.backupCount > 0: - for i in range(self.backupCount, 0, -1): - sfn = f'{self.baseFilename}.{i}' - if os.path.exists(sfn): - with open(sfn, 'rb') as f_in: - with gzip.open(f'{sfn}.gz', 'wb') as f_out: - f_out.writelines(f_in) - os.remove(sfn) - -class JsonFormatter(logging.Formatter): - def __init__(self, date_format): - super().__init__() - self.date_format = date_format - - def format(self, record): - log_record = { - 'time' : self.formatTime(record, self.date_format), - 'level' : record.levelname, - 'module' : record.module, - 'function' : record.funcName, - 'line' : record.lineno, - 'message' : record.getMessage(), - 'name' : record.name, - 'filename' : record.filename, - 'threadName' : record.threadName, - 'processName' : record.processName, - } - return json.dumps(log_record) - -def setup_file_handler(level_num: int, log_to_disk: bool, max_log_size: int, - max_backups: int, log_file_name: str, json_log: bool, - ecs_log: bool, date_format: str, compress_backups: bool): - '''Set up the file handler for logging to disk.''' - if not log_to_disk: - return - - # Create 'logs' directory if it doesn't exist - logs_dir = os.path.join(os.getcwd(), 'logs') - os.makedirs(logs_dir, exist_ok=True) - - # Use the specified log file name and set extension based on json_log - file_extension = '.json' if json_log else '.log' - log_file_path = os.path.join(logs_dir, f'{log_file_name}{file_extension}') - - # Create the rotating file handler - handler_class = GZipRotatingFileHandler if compress_backups else logging.handlers.RotatingFileHandler - file_handler = handler_class(log_file_path, maxBytes=max_log_size, backupCount=max_backups) - file_handler.setLevel(level_num) - - if ecs_log: - try: - import ecs_logging - except ImportError: - raise ImportError("The 'ecs-logging' library is required for ECS logging. Install it with 'pip install ecs-logging'.") - file_formatter = ecs_logging.StdlibFormatter() - elif json_log: - file_formatter = JsonFormatter(date_format) - else: - file_formatter = logging.Formatter( - fmt='%(asctime)s ┃ %(levelname)-8s ┃ %(module)s ┃ %(funcName)s ┃ %(lineno)d ┃ %(message)s', - datefmt=date_format - ) - - file_handler.setFormatter(file_formatter) - logging.getLogger().addHandler(file_handler) \ No newline at end of file diff --git a/apv/plugins/graylog.py b/apv/plugins/graylog.py deleted file mode 100644 index c5505f2..0000000 --- a/apv/plugins/graylog.py +++ /dev/null @@ -1,58 +0,0 @@ -import logging -import json -import socket -import zlib - -class GraylogHandler(logging.Handler): - def __init__(self, host, port): - super().__init__() - self.host = host - self.port = port - self.sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) - - # Mapping from Python logging levels to Graylog (syslog) levels - self.level_mapping = { - logging.CRITICAL : 2, # Critical - logging.ERROR : 3, # Error - logging.WARNING : 4, # Warning - logging.INFO : 6, # Informational - logging.DEBUG : 7, # Debug - logging.NOTSET : 7 # Default to Debug - } - - def emit(self, record): - try: - log_entry = self.format(record) - graylog_level = self.level_mapping.get(record.levelno, 7) - - gelf_message = { - 'version' : '1.1', - 'host' : socket.gethostname(), - 'short_message' : record.getMessage(), - 'full_message' : log_entry, - 'timestamp' : record.created, - 'level' : graylog_level, - '_logger_name' : record.name, - '_file' : record.pathname, - '_line' : record.lineno, - '_function' : record.funcName, - '_module' : record.module, - } - - message = json.dumps(gelf_message).encode('utf-8') - compressed = zlib.compress(message) - self.sock.sendto(compressed, (self.host, self.port)) - except Exception: - self.handleError(record) - -def setup_graylog_handler(level_num: int, graylog_host: str, graylog_port: int): - '''Set up the Graylog handler.''' - if graylog_host is None or graylog_port is None: - logging.error('Graylog host and port must be specified for Graylog handler.') - return - - graylog_handler = GraylogHandler(graylog_host, graylog_port) - graylog_handler.setLevel(level_num) - graylog_formatter = logging.Formatter(fmt='%(message)s') - graylog_handler.setFormatter(graylog_formatter) - logging.getLogger().addHandler(graylog_handler) \ No newline at end of file diff --git a/setup.py b/setup.py index 2bef877..d95fc48 100644 --- a/setup.py +++ b/setup.py @@ -4,12 +4,13 @@ from setuptools import setup, find_packages + with open('README.md', 'r', encoding='utf-8') as fh: long_description = fh.read() setup( name='apv', - version='1.0.4', + version='4.0.0', description='Advanced Python Logging', author='acidvegas', author_email='acid.vegas@acid.vegas', @@ -25,10 +26,6 @@ setup( install_requires=[ # No required dependencies for basic functionality ], - extras_require={ - 'cloudwatch': ['boto3'], - 'ecs' : ['ecs-logging'], - }, classifiers=[ 'Programming Language :: Python :: 3', 'License :: OSI Approved :: ISC License (ISCL)', diff --git a/unit_test.py b/unit_test.py index 222c5c4..c0b3312 100644 --- a/unit_test.py +++ b/unit_test.py @@ -1,95 +1,95 @@ -#! /usr/bin/env python3 +#!/usr/bin/env python3 # Advanced Python Logging - Developed by acidvegas in Python (https://git.acid.vegas/apv) -# unittest.py +# unit_test.py import logging +import os import random +import sys import time -# prevent bytecode files (.pyc) from being written -from sys import dont_write_bytecode -dont_write_bytecode = True +sys.dont_write_bytecode = True # FUCKOFF __pycache__ -import apv +import apv.apv as apv -# Test console logging with custom date format -apv.setup_logging(level='DEBUG', date_format='%H:%M:%S') -logging.debug('Testing debug message in console.') -logging.info('Testing info message in console.') -logging.warning('Testing warning message in console.') -logging.error('Testing error message in console.') -logging.critical('Testing critical message in console.') -print() +def test_console_logging(): + '''Test basic console logging functionality''' -# Test console logging with details -time.sleep(2) -apv.setup_logging(level='DEBUG', date_format='%Y-%m-%d %H:%M:%S', show_details=True) -logging.debug('Testing debug message in console with details.') -logging.info('Testing info message in console with details.') -logging.warning('Testing warning message in console with details.') -logging.error('Testing error message in console with details.') -logging.critical('Testing critical message in console with details.') + print('\nTesting Console Logging...') + apv.setup_logging(level='DEBUG', date_format='%H:%M:%S') + for level in ['debug', 'info', 'warning', 'error', 'critical']: + getattr(logging, level)(f'Testing {level} message in console.') + time.sleep(1) -print() -# Test disk logging with JSON and regular rotation -logging.debug('Starting test: Disk logging with JSON and regular rotation...') -time.sleep(2) -apv.setup_logging(level='DEBUG', log_to_disk=True, max_log_size=1024, max_backups=3, log_file_name='json_log', json_log=True, show_details=True) -for i in range(100): - log_level = random.choice([logging.DEBUG, logging.INFO, logging.WARNING, logging.ERROR, logging.CRITICAL]) - logging.log(log_level, f'Log entry {i+1} for JSON & regular rotation test.') - time.sleep(0.1) +def test_json_console_logging(): + '''Test JSON console logging''' -print() + print('\nTesting JSON Console Logging...') + apv.setup_logging(level='DEBUG', date_format='%H:%M:%S', json_log=True, log_to_disk=False) + logging.info('Test JSON console message with custom field', extra={'_custom_field': 'test value'}) + logging.warning('Test JSON console warning with error', exc_info=Exception('Test error')) + time.sleep(1) -# Test disk logging with rotation & compression -logging.debug('Starting test: Disk logging with rotation & compression...') -time.sleep(2) -apv.setup_logging(level='DEBUG', log_to_disk=True, max_log_size=1024, max_backups=3, log_file_name='plain_log', show_details=True, compress_backups=True) -for i in range(100): - log_level = random.choice([logging.DEBUG, logging.INFO, logging.WARNING, logging.ERROR, logging.CRITICAL]) - logging.log(log_level, f'Log entry {i+1} for disk rotation & compression test.') - time.sleep(0.1) -logging.info('Test completed. Check the logs directory for disk logging & JSON logging tests.') +def test_detailed_logging(): + '''Test console logging with details''' -print() + print('\nTesting Detailed Logging...') + apv.setup_logging(level='DEBUG', show_details=True) + for level in ['debug', 'info', 'warning', 'error', 'critical']: + getattr(logging, level)(f'Testing {level} message with details.') + time.sleep(1) -try: - import ecs_logging -except ImportError: - pass -else: - # Test ECS logging - logging.debug('Starting test: ECS logging...') - time.sleep(2) - apv.setup_logging(level='DEBUG', ecs_log=True) - logging.debug('This is a test log message to ECS.') - logging.info('This is a test log message to ECS.') - logging.warning('This is a test log message to ECS.') - logging.error('This is a test log message to ECS.') - logging.critical('This is a test log message to ECS.') -print() +def test_file_logging(): + '''Test file logging with rotation''' -# Test Graylog handler (Uncomment & configure to test) -# logging.debug('Starting test: Graylog handler...') -# time.sleep(2) -# apv.setup_logging(level='DEBUG', enable_graylog=True, graylog_host='your_graylog_host', graylog_port=12201) -# logging.debug('This is a test log message to Graylog.') -# logging.info('This is a test log message to Graylog.') -# logging.warning('This is a test log message to Graylog.') -# logging.error('This is a test log message to Graylog.') -# logging.critical('This is a test log message to Graylog.') + print('\nTesting File Logging...') + log_file = 'logs/test_log.log' + apv.setup_logging(level='DEBUG', log_to_disk=True, max_log_size=1024, max_backups=3, log_file_name='test_log') + for i in range(50): + level = random.choice(['debug', 'info', 'warning', 'error', 'critical']) + getattr(logging, level)(f'File logging test message {i}') + + assert os.path.exists(log_file), "Log file was not created" + time.sleep(1) -# Test CloudWatch handler (Uncomment & configure to test) -# logging.debug('Starting test: CloudWatch handler...') -# time.sleep(2) -# apv.setup_logging(level='DEBUG', enable_cloudwatch=True, cloudwatch_group_name='your_log_group', cloudwatch_stream_name='your_log_stream') -# logging.debug('This is a test log message to CloudWatch.') -# logging.info('This is a test log message to CloudWatch.') -# logging.warning('This is a test log message to CloudWatch.') -# logging.error('This is a test log message to CloudWatch.') -# logging.critical('This is a test log message to CloudWatch.') + +def test_json_logging(): + '''Test JSON format logging''' + + print('\nTesting JSON Logging...') + apv.setup_logging(level='DEBUG', log_to_disk=True, log_file_name='json_test', json_log=True) + logging.info('Test JSON formatted log message') + assert os.path.exists('logs/json_test.json'), "JSON log file was not created" + time.sleep(1) + + +def test_compressed_logging(): + '''Test compressed log files''' + + print('\nTesting Compressed Logging...') + apv.setup_logging(level='DEBUG', log_to_disk=True, max_log_size=512, max_backups=2, log_file_name='compressed_test', compress_backups=True) + for i in range(100): + logging.info(f'Testing compression message {i}') + time.sleep(1) + # Check for .gz files + gz_files = [f for f in os.listdir('logs') if f.startswith('compressed_test') and f.endswith('.gz')] + assert len(gz_files) > 0, 'No compressed log files were created' + + +if __name__ == '__main__': + # Create logs directory if it doesn't exist + os.makedirs('logs', exist_ok=True) + + # Run all tests + test_console_logging() + test_json_console_logging() + test_detailed_logging() + test_file_logging() + test_json_logging() + test_compressed_logging() + + print('\nAll tests completed successfully!') \ No newline at end of file