Apache Tomcat

Monitor Apache Tomcat application server logs including access logs, catalina logs, and application-specific logs using the File Stream plugin

Apache Tomcat Integration

Monitor and analyze Apache Tomcat application server logs in real-time using LogFlux Agent’s File Stream plugin. This configuration-based approach provides comprehensive log parsing, application performance monitoring, and Java web application analytics.

Overview

The Apache Tomcat integration leverages LogFlux Agent’s File Stream plugin to:

Real-time monitoring of access logs, catalina logs, and application logs
Java application analytics with request processing and error tracking
Performance monitoring with response times and thread pool analysis
Security monitoring with authentication and authorization tracking
Multi-application support for deployed web applications
JVM monitoring through garbage collection and memory logs

Installation

The File Stream plugin is included with LogFlux Agent. Enable it for Tomcat log monitoring:

1
2
3
4
5


# Enable File Stream plugin
sudo systemctl enable --now logflux-filestream

# Verify plugin status
sudo systemctl status logflux-filestream

Tomcat Configuration

Configure Tomcat logging in server.xml and logging.properties:

Access Log Configuration (server.xml)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


<Host name="localhost" appBase="webapps" unpackWARs="true" autoDeploy="true">
    <!-- Access log with detailed pattern -->
    <Valve className="org.apache.catalina.valves.AccessLogValve" 
           directory="logs"
           prefix="localhost_access_log" 
           suffix=".txt"
           pattern="%h %l %u %t &quot;%r&quot; %s %b %D %{Referer}i %{User-Agent}i %{jsessionid}s"
           resolveHosts="false"/>
    
    <!-- JSON Access Log for structured logging -->
    <Valve className="org.apache.catalina.valves.JsonAccessLogValve"
           directory="logs"
           prefix="localhost_access_json"
           suffix=".log"/>
</Host>

Application Logging (logging.properties)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


# Root logger configuration
.handlers = java.util.logging.FileHandler, java.util.logging.ConsoleHandler

# Catalina logs
org.apache.catalina.level = INFO
org.apache.catalina.handlers = java.util.logging.FileHandler

# File handler for catalina logs
java.util.logging.FileHandler.pattern = ${catalina.base}/logs/catalina.%g.log
java.util.logging.FileHandler.limit = 50000000
java.util.logging.FileHandler.count = 10
java.util.logging.FileHandler.formatter = java.util.logging.SimpleFormatter

# Application-specific logging
com.example.myapp.level = DEBUG
com.example.myapp.handlers = java.util.logging.FileHandler

Basic Configuration

Configure the File Stream plugin to monitor Tomcat logs by creating /etc/logflux-agent/plugins/filestream-tomcat.toml:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26


[filestream.tomcat_access]
paths = ["/opt/tomcat/logs/localhost_access_log*.txt"]
format = "tomcat_access"
tags = ["tomcat", "access", "java"]
fields = {
  service = "tomcat",
  log_type = "access"
}

[filestream.tomcat_catalina]
paths = ["/opt/tomcat/logs/catalina*.log"]
format = "java_log"
tags = ["tomcat", "catalina", "java"]
fields = {
  service = "tomcat",
  log_type = "catalina"
}

[filestream.tomcat_application]
paths = ["/opt/tomcat/logs/*application*.log"]
format = "java_log"
tags = ["tomcat", "application", "java"]
fields = {
  service = "tomcat",
  log_type = "application"
}

Tomcat Log Formats

Access Log Format (Common Pattern)

Standard Tomcat access log format:

1
2
3
4
5
6
7


[filestream.tomcat_access_standard]
paths = ["/opt/tomcat/logs/localhost_access_log*.txt"]
format = "regex"
regex = '^(?P<client_ip>\S+) (?P<remote_logname>\S+) (?P<remote_user>\S+) \[(?P<timestamp>[^\]]+)\] "(?P<request_method>\S+) (?P<request_uri>\S+) (?P<request_protocol>[^"]*)" (?P<status>\d+) (?P<bytes_sent>\S+) (?P<response_time>\d+) "(?P<referer>[^"]*)" "(?P<user_agent>[^"]*)" (?P<session_id>\S+)$'
parse_timestamp = true
timestamp_field = "timestamp"
timestamp_format = "02/Jan/2006:15:04:05 -0700"

JSON Access Log Format

For structured access logs:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


[filestream.tomcat_json_access]
paths = ["/opt/tomcat/logs/localhost_access_json*.log"]
format = "json"
parse_timestamp = true
timestamp_field = "timestamp"
timestamp_format = "2006-01-02T15:04:05.000Z"
tags = ["tomcat", "json", "access"]
fields = {
  service = "tomcat",
  log_type = "json_access"
}

Catalina Log Format

Java application server logs:

1
2
3
4
5
6
7


[filestream.tomcat_catalina]
paths = ["/opt/tomcat/logs/catalina*.log"]
format = "regex"
regex = '^(?P<timestamp>\w{3} \d{1,2}, \d{4} \d{1,2}:\d{2}:\d{2} [AP]M) (?P<level>\w+) \[(?P<thread>[^\]]+)\] (?P<class>[^ ]+) (?P<message>.*)$'
parse_timestamp = true
timestamp_field = "timestamp"
timestamp_format = "Jan 2, 2006 3:04:05 PM"

Application-Specific Logs

Monitor deployed application logs:

1
2
3
4
5
6
7
8


[filestream.tomcat_app_logs]
paths = ["/opt/tomcat/logs/myapp*.log"]
format = "regex"
regex = '^(?P<timestamp>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3}) \[(?P<thread>[^\]]+)\] (?P<level>\w+) (?P<class>[^ ]+) - (?P<message>.*)$'
parse_timestamp = true
timestamp_field = "timestamp"
timestamp_format = "2006-01-02 15:04:05,000"
tags = ["tomcat", "myapp", "application"]

Advanced Configuration

Multi-Application Monitoring

Monitor multiple deployed applications:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


[filestream.tomcat_webapp1]
paths = ["/opt/tomcat/logs/webapp1*.log"]
format = "java_log"
tags = ["tomcat", "webapp1"]
fields = {
  service = "tomcat",
  application = "webapp1",
  log_type = "application"
}

[filestream.tomcat_webapp2]
paths = ["/opt/tomcat/logs/webapp2*.log"]
format = "java_log"
tags = ["tomcat", "webapp2"]
fields = {
  service = "tomcat",
  application = "webapp2",
  log_type = "application"
}

Performance Monitoring

Track performance metrics with calculated fields:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21


[filestream.tomcat_performance]
paths = ["/opt/tomcat/logs/localhost_access_log*.txt"]
format = "tomcat_access"
tags = ["tomcat", "performance"]
fields = {
  service = "tomcat",
  log_type = "performance"
}

# Add calculated performance metrics
[filestream.tomcat_performance.processors.add_fields]
fields = {
  response_time_seconds = "{{ div .response_time 1000 }}",
  requests_per_minute = "{{ counter .client_ip 60 }}",
  error_rate = "{{ if ge .status 400 }}1{{ else }}0{{ end }}"
}

# Filter slow requests
[filestream.tomcat_performance.processors.threshold]
field = "response_time"
threshold = 5000  # 5 seconds

Security Monitoring

Track authentication and authorization events:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21


[filestream.tomcat_security]
paths = ["/opt/tomcat/logs/catalina*.log", "/opt/tomcat/logs/*security*.log"]
format = "java_log"
tags = ["tomcat", "security"]
fields = {
  service = "tomcat",
  log_type = "security"
}

# Filter for security-related events
[filestream.tomcat_security.processors.grep]
patterns = [
  "authentication",
  "authorization",
  "login",
  "logout",
  "session",
  "security",
  "WARN",
  "ERROR"
]

Garbage Collection Monitoring

Monitor JVM garbage collection logs:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


[filestream.tomcat_gc]
paths = ["/opt/tomcat/logs/gc*.log"]
format = "regex"
regex = '^(?P<timestamp>\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3}[+-]\d{4}): (?P<gc_time>\d+\.\d+): \[(?P<gc_type>[^\]]+)\] (?P<memory_before>\d+)K->(?P<memory_after>\d+)K\((?P<heap_size>\d+)K\), (?P<gc_duration>\d+\.\d+) secs\]$'
parse_timestamp = true
timestamp_field = "timestamp"
timestamp_format = "2006-01-02T15:04:05.000-0700"
tags = ["tomcat", "gc", "jvm"]
fields = {
  service = "tomcat",
  log_type = "garbage_collection"
}

Usage Examples

Monitor Tomcat Applications

1
2
3
4
5
6
7
8


# Stream all Tomcat logs
logflux-cli stream --filter 'service:tomcat'

# Monitor specific application
logflux-cli stream --filter 'service:tomcat AND application:webapp1'

# Track access logs
logflux-cli stream --filter 'service:tomcat AND log_type:access'

Performance Analysis

1
2
3
4
5
6
7
8


# Monitor slow requests (>5 seconds)
logflux-cli stream --filter 'service:tomcat AND response_time:>5000'

# Track 5xx errors
logflux-cli stream --filter 'service:tomcat AND status:>=500'

# Monitor memory usage
logflux-cli stream --filter 'service:tomcat AND log_type:garbage_collection'

Security Monitoring

1
2
3
4
5
6
7
8


# Track failed logins
logflux-cli stream --filter 'service:tomcat AND message:authentication AND level:ERROR'

# Monitor security events
logflux-cli stream --filter 'service:tomcat AND tags:security'

# Track session management
logflux-cli stream --filter 'service:tomcat AND message:session'

Application-Specific Configuration

Spring Boot Applications

1
2
3
4
5
6
7
8


[filestream.tomcat_springboot]
paths = ["/opt/tomcat/logs/spring-boot-app*.log"]
format = "regex"
regex = '^(?P<timestamp>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}\.\d{3}) (?P<level>\w+) (?P<pid>\d+) --- \[(?P<thread>[^\]]+)\] (?P<class>[^ ]+) : (?P<message>.*)$'
parse_timestamp = true
timestamp_field = "timestamp"
timestamp_format = "2006-01-02 15:04:05.000"
tags = ["tomcat", "springboot"]

Servlet Container Logs

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18


[filestream.tomcat_servlet]
paths = ["/opt/tomcat/logs/servlet*.log"]
format = "java_log"
tags = ["tomcat", "servlet"]
fields = {
  service = "tomcat",
  log_type = "servlet"
}

# Track servlet lifecycle events
[filestream.tomcat_servlet.processors.grep]
patterns = [
  "init",
  "destroy",
  "service",
  "filter",
  "listener"
]

Database Connection Pool Monitoring

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18


[filestream.tomcat_dbcp]
paths = ["/opt/tomcat/logs/*dbcp*.log"]
format = "java_log"
tags = ["tomcat", "database", "connection-pool"]
fields = {
  service = "tomcat",
  log_type = "database_pool"
}

# Monitor connection pool events
[filestream.tomcat_dbcp.processors.grep]
patterns = [
  "connection",
  "pool",
  "datasource",
  "SQLException",
  "timeout"
]

Monitoring and Alerting

Key Metrics to Monitor

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27


# High response time alert
[alerts.tomcat_slow_response]
query = "service:tomcat AND response_time:>10000"
threshold = 10
window = "2m"
message = "Tomcat slow response times detected"

# Application error alert
[alerts.tomcat_application_error]
query = "service:tomcat AND level:ERROR"
threshold = 5
window = "1m"
message = "High error rate in Tomcat applications"

# Memory pressure alert
[alerts.tomcat_memory_pressure]
query = "service:tomcat AND log_type:garbage_collection AND gc_duration:>1.0"
threshold = 3
window = "5m"
message = "Tomcat memory pressure detected"

# Session timeout alert
[alerts.tomcat_session_timeout]
query = "service:tomcat AND message:session AND message:timeout"
threshold = 10
window = "1m"
message = "High session timeout rate in Tomcat"

Dashboard Metrics

Monitor these key Tomcat metrics:

Request rate (requests per second)
Response times (average, 95th percentile)
Error rates (4xx, 5xx responses)
Application performance (by deployed application)
JVM metrics (garbage collection, memory usage)
Session management (active sessions, timeouts)
Thread pool (active threads, queue depth)
Database connections (pool utilization, failures)

Troubleshooting

Common Issues

Tomcat logs not appearing:

1
2
3
4
5
6
7
8


# Check Tomcat is running
systemctl status tomcat

# Verify log file permissions
sudo ls -la /opt/tomcat/logs/

# Check Tomcat configuration
grep -r "AccessLogValve" /opt/tomcat/conf/

Log format parsing errors:

1
2
3
4
5
6
7
8


# Test regex pattern
echo "sample tomcat log line" | grep -P "your_regex_pattern"

# Check LogFlux Agent logs
sudo journalctl -u logflux-filestream -f

# Validate Tomcat log format
tail -n 10 /opt/tomcat/logs/localhost_access_log.txt

Missing application logs:

1
2
3
4
5
6
7
8


# Check application logging configuration
find /opt/tomcat -name "logging.properties" -exec cat {} \;

# Verify log4j configuration
find /opt/tomcat -name "log4j*.xml" -exec cat {} \;

# Check application deployment
ls -la /opt/tomcat/webapps/

JVM monitoring issues:

1
2
3
4
5


# Enable GC logging in Tomcat startup
export CATALINA_OPTS="-XX:+UseG1GC -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -Xloggc:/opt/tomcat/logs/gc.log"

# Check JVM parameters
ps aux | grep tomcat

Best Practices

Performance

Tune JVM parameters for optimal garbage collection
Monitor thread pool usage and adjust connector settings
Use connection pooling for database connections
Implement caching strategies for better performance

Security

Secure management interfaces (manager, host-manager apps)
Implement proper authentication and authorization
Monitor security logs for suspicious activity
Use HTTPS for sensitive applications

High Availability

Deploy in cluster configuration with load balancing
Monitor health checks and application availability
Implement graceful shutdown procedures
Use external session storage for session persistence

Log Management

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


# Optimize log rotation for Tomcat
/opt/tomcat/logs/*.log /opt/tomcat/logs/*.txt {
    daily
    rotate 30
    missingok
    notifempty
    compress
    delaycompress
    copytruncate
    postrotate
        systemctl reload logflux-filestream.service > /dev/null 2>&1 || true
    endpostrotate
}

Application Development

1
2
3
4
5
6


<!-- Use structured logging in applications -->
<dependency>
    <groupId>net.logstash.logback</groupId>
    <artifactId>logstash-logback-encoder</artifactId>
    <version>7.0.1</version>
</dependency>

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


<!-- logback-spring.xml -->
<configuration>
    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
            <providers>
                <timestamp/>
                <logLevel/>
                <loggerName/>
                <message/>
                <mdc/>
                <stackTrace/>
            </providers>
        </encoder>
    </appender>
    
    <root level="INFO">
        <appender-ref ref="STDOUT"/>
    </root>
</configuration>

Integration Examples

Docker Deployment

1
2
3
4
5
6


FROM tomcat:9.0-jdk11
COPY logging.properties $CATALINA_HOME/conf/
COPY server.xml $CATALINA_HOME/conf/
COPY myapp.war $CATALINA_HOME/webapps/
VOLUME ["/usr/local/tomcat/logs"]
EXPOSE 8080

Kubernetes Deployment

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25


apiVersion: apps/v1
kind: Deployment
metadata:
  name: tomcat-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: tomcat-app
  template:
    metadata:
      labels:
        app: tomcat-app
    spec:
      containers:
      - name: tomcat
        image: tomcat:9.0
        ports:
        - containerPort: 8080
        volumeMounts:
        - name: tomcat-logs
          mountPath: /usr/local/tomcat/logs
      volumes:
      - name: tomcat-logs
        emptyDir: {}

This comprehensive Apache Tomcat integration provides real-time application server monitoring, Java web application analytics, and JVM performance tracking using LogFlux Agent’s File Stream plugin. The configuration-based approach offers detailed insights into servlet container operations, application performance, and Java runtime behavior.

Disclaimer

Apache Tomcat and the Apache Tomcat logo are trademarks of The Apache Software Foundation. LogFlux is not affiliated with, endorsed by, or sponsored by The Apache Software Foundation or the Apache Tomcat project. The Apache Tomcat logo is used solely for identification purposes to indicate compatibility with Apache Tomcat logs.

Apache Tomcat Integration

Overview

Installation

Tomcat Configuration

Access Log Configuration (server.xml)

Application Logging (logging.properties)

Basic Configuration

Tomcat Log Formats

Access Log Format (Common Pattern)

JSON Access Log Format

Catalina Log Format

Application-Specific Logs

Advanced Configuration

Multi-Application Monitoring

Performance Monitoring

Security Monitoring

Garbage Collection Monitoring

Usage Examples

Monitor Tomcat Applications

Performance Analysis

Security Monitoring

Application-Specific Configuration

Spring Boot Applications

Servlet Container Logs

Database Connection Pool Monitoring

Monitoring and Alerting

Key Metrics to Monitor

Dashboard Metrics

Troubleshooting

Common Issues

Best Practices

Performance

Security

High Availability

Log Management

Application Development

Integration Examples

Docker Deployment

Kubernetes Deployment

Disclaimer

Cookie Preferences

Essential Cookies

Analytics Cookies