Library requirements

Log Bull is a system for logs collection and viewing. To collect logs, there are libraries for different languages.

Usually, Log Bull library for particular language has:

Log Bull class as a standalone logger
Log Bull as a handler for language standard logger (logger for Python, slog for Go, etc.)
Log Bull as a handler for different libraries popular within language (Loguru, structlog for Python, zap for Go, etc.)

The main priority of the library - is to be simple for use (with minimalistic interface) and support the most popular libraries within language.

Configuration

All library implementations must support these configuration parameters:

Required parameters

project_id (string, UUID format) - Target project identifier
host (string, URL) - LogBull server endpoint (e.g., http://localhost:4005)

Optional parameters

api_key (string) - API key for authentication (if server requires it)
log_level (string) - Minimum log level to process (default: INFO)

Never include in configuration

batch_size (integer) - Maximum logs per batch always 1000 by design
batch_interval (duration) - Time between batch sends always 1 second by design

Log Entry Format

Standard log structure


{
  "level": "INFO",
  "message": "User logged in successfully",
  "timestamp": "2025-01-15T10:30:45.123456789Z",
  "fields": {
    "user_id": "12345",
    "session_id": "sess_abc",
    "ip": "192.168.1.100"
  }
}

Field requirements

level: Uppercase string (DEBUG, INFO, WARNING, ERROR, CRITICAL)
message: Non-empty string, max 10,000 characters
timestamp: RFC3339Nano format with nanosecond precision in UTC timezone (e.g., 2025-01-15T10:30:45.123456789Z), must be unique and increasing to ensure correct order of logs
fields: Object/map with custom key-value pairs (max 100 fields)

Field constraints

Field keys: Max 100 characters, must be non-empty strings
Field values: Must be JSON-serializable (convert non-serializable values to strings)
Total fields: Maximum 100 per log entry

HTTP API integration

Endpoint


POST {host}/api/v1/logs/receiving/{project_id}

Request headers


Content-Type: application/json
User-Agent: LogBull-{Language}-Client/{Version}
X-API-Key: {api_key}  # Optional, if configured

Request body


{
  "logs": [
    {
      "level": "INFO",
      "message": "Log message",
      "timestamp": "2025-01-15T10:30:45.123456789Z",
      "fields": {}
    }
  ]
}

Response format


{
  "accepted": 99,
  "rejected": 1,
  "errors": [
    {
      "index": 5,
      "message": "Invalid log level"
    }
  ]
}

Response fields:

accepted (integer): Number of logs successfully accepted
rejected (integer): Number of logs rejected
errors (array, optional): Present only if rejected > 0. Contains details about rejected logs:
- index (integer): Index of the rejected log in the batch (0-based)
- message (string): Reason for rejection

Error handling

HTTP errors: Log error message, log sending entry to stderr/console, mark batch as failed
Server errors (5xx) or network errors: Consider retry with exponential backoff (max 3 tries), print error message to stderr/console
Client errors (4xx): Log error, do not retry (likely configuration issue), print error message to stderr/console
Rejected logs: If server response indicates rejected logs (rejected > 0), see section “8. Rejected logs output” for detailed requirements

Core features

1. Asynchronous log sending

Use background thread/goroutine for batch processing
Log operations should only add logs to internal queue (immediate return from log methods like .debug(), .info(), etc.)
Internal queue for pending logs
Automatic batch sending based on size or time interval

2. Batch processing

Collect logs into batches (up to 1000 logs)
Send batches at regular intervals (default: 1 second) even if batch is not full
Send immediately when batch reaches size limit
Concurrent HTTP requests using thread/connection pool - do not wait until previous request is completed

3. Context management

Context allows attaching persistent fields to all subsequent logs:


# Python example
base_logger = LogBullLogger(host="...", project_id="...")
request_logger = base_logger.with_context({"request_id": "req_123"})
request_logger.info("Processing started")  # Includes request_id

Implementation requirements:

Create new logger instance with merged context
Share same sender instance (don’t create duplicate senders)
Context fields should merge/override: child context > parent context

4. Timestamp generation

Use nanosecond precision timestamps
Ensure monotonic timestamps (each timestamp must be unique and increasing)
Format: RFC3339Nano (e.g., 2025-01-15T10:30:45.123456789Z)
All timestamps must be in UTC timezone (indicated by Z suffix)
Use thread-safe timestamp generation

5. Resource management

Auto-register senders for cleanup on application exit
Implement flush() method to send all pending logs immediately
Implement shutdown() method to stop processing and cleanup
Wait for in-flight requests during shutdown (with timeout)

6. Thread/Concurrency safety

All public methods must be thread-safe
Use locks/mutexes for shared state
Queue operations must be concurrent-safe
Executor/pool management must be thread-safe

7. Console output

Standalone logger should print logs to console in addition to sending them:

Format:


[2025-01-15T10:30:45.123456789Z] [INFO] User logged in (user_id=12345, ip=192.168.1.100)

Requirements:

Print before queuing for sending
Include timestamp, level, message, and fields
Use stderr for ERROR/CRITICAL levels (language-specific best practice)

8. Rejected logs output

When server response indicates rejected logs (rejected > 0), print detailed information about each rejected log to stderr/console:

Example output:


LogBull: Rejected 1 log entries
LogBull: Rejected log details:
  - Log #5 rejected (Invalid log level):
    Level: INVALID
    Message: Test message
    Timestamp: 2025-01-15T10:30:45.123456789Z
    Fields: {"user_id": "123"}

Requirements:

Print number of rejected logs
For each rejected log, include:
- Index in the batch (0-based)
- Rejection reason from server
- Full log content (level, message, timestamp, fields)
Only print when server successfully processes the request but rejects specific logs
Do not print for network errors or HTTP errors (those should be handled separately)

Library

Library should not include dependencies on third-party libraries (only for dev and testing, but not for production).

Integration patterns (Python example, similar for other languages)

Standalone logger pattern


# Initialization
logger = LogBullLogger(
    host="http://localhost:4005",
    project_id="uuid-here",
    api_key="optional-key",
    log_level="INFO"
)
 
# Basic logging
logger.info("Message", fields={"key": "value"})
logger.error("Error occurred", fields={"error_code": 500})
 
# Context management
request_logger = logger.with_context({"request_id": "req_123"})
request_logger.info("Processing")
 
# Cleanup
logger.shutdown()

Standard library handler pattern


import logging
from logbull import LogBullHandler
 
logger = logging.getLogger(__name__)
handler = LogBullHandler(
    host="http://localhost:4005",
    project_id="uuid-here"
)
logger.addHandler(handler)
 
# Use standard logging
logger.info("Message", extra={"field": "value"})

Third-party library pattern


from loguru import logger
from logbull import LoguruSink
 
logger.add(
    LoguruSink(host="http://localhost:4005", project_id="uuid-here"),
    level="INFO"
)
 
logger.info("Message", user_id=123)

Error handling philosophy

Never break user’s application: Catch all exceptions in log methods
Fail silently: Log errors to stderr/console, but don’t throw exceptions
Informative errors: Provide clear error messages for configuration issues
Validation on init: Validate configuration at initialization, not at runtime
Graceful degradation: If LogBull server is unavailable, continue logging locally

Testing requirements

Every library implementation must include:

Unit tests for all core components
Integration tests with mock HTTP server
Concurrency tests to verify thread-safety
Handler/integration tests for each supported library
Configuration validation tests

Documentation requirements

Each library must include:

README.md with:
- Installation instructions
- Quick start guide
- Usage examples for all integrations
- Configuration reference
- API documentation
Code comments for:
- Complex logic and algorithms
- Non-obvious design decisions
- Thread-safety considerations
- Not for self-explanatory code
Type annotations/documentation (language-specific):
- Python: Type hints
- Go: GoDoc comments
- TypeScript: TypeScript types
- Java: Javadoc
- etc.

License and versioning

License: Apache 2.0
Versioning: Semantic versioning (MAJOR.MINOR.PATCH)
Commit format:
- FEATURE (area): description → MINOR bump
- FIX (area): description → PATCH bump
- REFACTOR (area): description → No bump

After PR

Update code example on main website
Update code example in website docs
Update code example in Log Bull “How to send logs?” component