acidvegas/apv

Fork 0

acidvegas 1f928c4a2f

Initial commit

2024-10-21 06:34:40 -04:00

9.4 KiB

Raw Permalink Blame History

Advanced Python Logging (APV)

Flexible & powerful logging solution for Python applications

Introduction
Requirements
Features
Installation
Configuration Options
Usage
Testing

Introduction

APV emerged from a simple observation: despite the abundance of logging solutions, there's a glaring lack of standardization in application logging. As a developer deeply entrenched in Elasticsearch, AWS, and Graylog ecosystems, I found myself repeatedly grappling with inconsistent log formats and cumbersome integrations. APV is my response to this challenge – a logging library that doesn't aim to revolutionize the field, but rather to streamline it.

This project is rooted in pragmatism. It offers a clean, efficient approach to generating logs that are both human-readable and machine-parseable. APV isn't about reinventing logging; it's about refining it. It provides a unified interface that plays well with various logging backends, from local files to cloud services, without sacrificing flexibility or performance.

While there's no shortage of logging libraries out there, APV represents a distillation of best practices I've encountered and challenges I've overcome. It's designed for developers who appreciate clean, consistent logs and seamless integration with modern logging infrastructures. If you're tired of wrestling with logging inconsistencies in production environments, APV might just be the solution you didn't know you were looking for.

Requirements

Python 3.10+
- boto3 (Optional: for CloudWatch logging)
- ecs-logging (Optional: for ECS logging)

Features

Console Logging with Colors: Enhanced readability with colored log messages in the console.
File Logging: Write logs to files with support for log rotation based on size and number of backups.
Log Compression: Automatically compress old log files using gzip to save disk space.
JSON Logging: Output logs in JSON format for better structure and integration with log management systems.
ECS Logging: Output logs in ECS format for better integration with Elasticsearch
Detailed Log Messages: Option to include module name, function name, and line number in log messages.
Graylog Integration: Send logs to a Graylog server using GELF over UDP.
AWS CloudWatch Integration: Send logs to AWS CloudWatch Logs.
Customizable Logging Levels: Set the logging level to control verbosity.

Installation

To install APV, you can use the provided setup.py script:

Clone the repository or download the source code.
```
git clone https://github.com/acidvegas/apv.git
```
Navigate to the project directory.
```
cd apv
```
Install the package using setup.py.
```
python setup.py install
```
- With CloudWatch support:
  
  If you plan to use AWS CloudWatch logging, install with the cloudwatch extra:
```
pip install .[cloudwatch]
```
- Alternatively, install boto3 separately:
```
pip install boto3
```
- With ECS support:
  
  If you plan to use ECS logging, install with the ecs extra:
```
pip install .[ecs]
```
  - Alternatively, install ecs-logging separately:
```
pip install ecs-logging
```

Configuration Options

The setup_logging function accepts the following keyword arguments to customize logging behavior:

Name	Default	Description
`level`	`INFO`	The logging level. (`DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`)
`date_format`	`'%Y-%m-%d %H:%M:%S'`	The date format for log messages.
`log_to_disk`	`False`	Whether to log to disk.
`max_log_size`	`1010241024` (10 MB)	The maximum size of log files before rotation (in bytes).
`max_backups`	`7`	The maximum number of backup log files to keep.
`log_file_name`	`'app'`	The base name of the log file.
`json_log`	`False`	Whether to log in JSON format.
`ecs_log`	`False`	Whether to log in ECS format.
`show_details`	`False`	Whether to include module name, function name, & line number in log messages.
`compress_backups`	`False`	Whether to compress old log files using gzip.
`enable_graylog`	`False`	Whether to enable logging to a Graylog server.
`graylog_host`	`None`	The Graylog server host. (Required if `enable_graylog` is `True`)
`graylog_port`	`None`	The Graylog server port. (Required if `enable_graylog` is `True`)
`enable_cloudwatch`	`False`	Whether to enable logging to AWS CloudWatch Logs.
`cloudwatch_group_name`	`None`	The name of the CloudWatch log group. (Required if `enable_cloudwatch` is `True`)
`cloudwatch_stream_name`	`None`	The name of the CloudWatch log stream. (Required if `enable_cloudwatch` is `True`)

Usage

Basic Console Logging

import logging
import apv

# Set up basic console logging
apv.setup_logging(level='INFO')

logging.info('This is an info message.')
logging.error('This is an error message.')

Console Logging with Details

import logging
import apv

# Set up console logging with detailed information
apv.setup_logging(level='DEBUG', show_details=True)

logging.debug('Debug message with details.')

File Logging with Rotation

import logging
import apv

# Set up file logging with log rotation
apv.setup_logging(
    level='INFO',
    log_to_disk=True,
    max_log_size=10*1024*1024,  # 10 MB
    max_backups=5,
    log_file_name='application_log'
)

logging.info('This message will be logged to a file.')

File Logging with Compression and JSON Format

import logging
import apv

# Set up file logging with compression and JSON format
apv.setup_logging(
    level='DEBUG',
    log_to_disk=True,
    max_log_size=5*1024*1024,  # 5 MB
    max_backups=7,
    log_file_name='json_log',
    json_log=True,
    compress_backups=True
)

logging.debug('This is a debug message in JSON format.')

Graylog Integration

import logging
import apv

# Set up logging to Graylog server
apv.setup_logging(
    level='INFO',
    enable_graylog=True,
    graylog_host='graylog.example.com',
    graylog_port=12201
)

logging.info('This message will be sent to Graylog.')

AWS CloudWatch Integration

import logging
import apv

# Set up logging to AWS CloudWatch Logs
apv.setup_logging(
    level='INFO',
    enable_cloudwatch=True,
    cloudwatch_group_name='my_log_group',
    cloudwatch_stream_name='my_log_stream'
)

logging.info('This message will be sent to AWS CloudWatch.')

ECS Logging

import logging
import apv

# Set up logging to AWS CloudWatch Logs
apv.setup_logging(
    level='INFO',
    ecs_log=True
)

Mixing it all together

import logging
import apv

# Set up logging to all handlers
apv.setup_logging(
    level='DEBUG',
    log_to_disk=True,
    max_log_size=10*1024*1024,
    max_backups=7,
    log_file_name='app',
    json_log=True,
    compress_backups=True,
    enable_graylog=True,
    graylog_host='graylog.example.com',
    graylog_port=12201,
    enable_cloudwatch=True,
    cloudwatch_group_name='my_log_group',
    cloudwatch_stream_name='my_log_stream',
    show_details=True
)

Testing

To run the test suite, use the following command:

python unittest.py

The test suite will run all the tests and provide output for each test.

9.4 KiB Raw Permalink Blame History Unescape Escape