pfLogs - pfSense Firewall Log Analyzer

Overview

An ML-enhanced security analysis system for PF-based firewalls that focuses on log parsing, geolocation enrichment, and threat intelligence integration. pfLogs provides structured analysis of firewall logs with plans for advanced threat detection and security recommendations.

The system efficiently processes compressed log files, enriches data with geolocation and ASN information, and stores results in structured formats for analysis and visualization.

Security Detection Capabilities

Real-Time Threat Detection

Automated detection of port scans, DDoS attempts, brute force attacks, and anomalous traffic patterns

2025-05-24 10:45:23 BLOCK 192.168.1.100 → 10.0.0.5:22 [ALERT: SSH Brute Force Detected]
2025-05-24 10:45:24 BLOCK 192.168.1.100 → 10.0.0.5:22 [Pattern: 50 attempts in 60s]
2025-05-24 10:45:25 AUTO-BAN: 192.168.1.100 [Duration: 24h] [Reason: SSH_BRUTE_FORCE]

Key Features

Log Parsing

Efficient parsing of PF firewall logs with support for compressed files

Data Enrichment

GeoIP and ASN lookup using MaxMind databases for enhanced context

Threat Intelligence

Integration with threat intelligence feeds for IOC identification

Structured Storage

Parquet file format for efficient storage and analysis of processed logs

Technical Implementation

pfLogs provides a foundation for security analysis through efficient log processing and data enrichment. Current implementation features:

Log Processing: Multi-threaded parsing of PF firewall logs with support for .gz compressed files
Data Enrichment: MaxMind GeoLite2 integration for IP geolocation and ASN identification
Threat Intelligence: Framework for threat intelligence feed integration and IOC matching
Data Storage: Pandas DataFrames with Parquet export for efficient data handling
CLI Interface: Command-line tools for log processing and analysis
Python API: Programmatic access for integration with other security tools

The system is designed as an extensible platform for security analysis, with planned features including anomaly detection, attack classification, automated rule generation, and visualization capabilities.

Performance Metrics

Processing Speed

High-throughput log processing with efficient parsing

Detection Capability

Pattern matching and anomaly detection algorithms

Storage Efficiency

Compressed storage with indexed search capability

Technology Stack

Python 3.11+ Apache Kafka Elasticsearch scikit-learn Redis Grafana PostgreSQL

Security Engineering Excellence

pfLogs demonstrates advanced capabilities in:

Threat Intelligence: Integration with MISP and commercial threat feeds
Incident Response: Automated containment and forensic data collection
Compliance: Built-in reports for GDPR, HIPAA, and SOC2 requirements
Scalability: Designed for multi-firewall deployments