Dashboard GitHub

Azure Monitor Integration

Query and stream logs from Azure Monitor and Application Insights with LogFlux Agent

Microsoft Azure Monitor

The LogFlux Azure Monitor integration queries and streams logs from Azure Monitor Log Analytics workspaces and Application Insights, enabling centralized log analysis from your Microsoft Azure infrastructure. This plugin provides seamless integration with Azure Monitor services using KQL (Kusto Query Language) for powerful log querying capabilities.

Overview

The Azure Monitor plugin provides:

  • Azure Monitor Integration: Direct connection to Log Analytics workspaces
  • Application Insights Support: Query telemetry data from Application Insights
  • KQL Query Support: Use powerful Kusto Query Language for log analysis
  • Multiple Authentication Methods: Managed Identity, Service Principal, Azure CLI, Device Code
  • Multi-Workspace Support: Query across multiple Log Analytics workspaces
  • Follow Mode: Continuously poll for new log entries in real-time
  • Batch Processing: Efficient batching for high-volume log retrieval
  • Rich Metadata: Extract Azure resource information and table data
  • Flexible Time Ranges: Query historical logs or stream real-time data

Installation

The Azure Monitor plugin is included with the LogFlux Agent but disabled by default.

Prerequisites

  • LogFlux Agent installed (see Installation Guide)
  • Azure credentials configured (Managed Identity, Service Principal, or Azure CLI)
  • Access to Azure Monitor Log Analytics workspaces
  • Network connectivity to Azure Monitor endpoints

Required Azure Permissions

For Log Analytics workspaces:

1
2
3
4
5
6

{
  "actions": [
    "Microsoft.OperationalInsights/workspaces/query/read",
    "Microsoft.OperationalInsights/workspaces/read"
  ]
}

For Application Insights:

1
2
3
4
5
6

{
  "actions": [
    "Microsoft.Insights/components/query/read",
    "Microsoft.Insights/components/read"
  ]
}

Enable the Plugin

1
2
3
4
5

# Enable and start the Azure Monitor plugin
sudo systemctl enable --now logflux-azuremonitor

# Check status
sudo systemctl status logflux-azuremonitor

Configuration

Basic Configuration

Create or edit the Azure Monitor plugin configuration:

1

sudo nano /etc/logflux-agent/plugins/azuremonitor.yaml

Basic configuration:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56

# Azure Monitor Plugin Configuration
name: azuremonitor
version: 1.0.0
source: azuremonitor-plugin

# Agent connection
agent:
  socket_path: /tmp/logflux-agent.sock

# Azure Configuration
azure:
  # Authentication method
  auth_method: "default"  # default, service-principal, managed-identity, azure-cli, device-code
  
  # Azure tenant ID
  tenant_id: "your-tenant-id"
  
  # Service Principal (optional)
  client_id: ""
  client_secret: ""

# Log Analytics workspaces
workspaces:
  - workspace_id: "your-workspace-id"

# Application Insights (optional)
app_insights:
  - app_id: "your-app-insights-id"

# Query settings
query:
  # Default KQL query
  query: "Heartbeat | limit 100"
  
  # Time range
  timespan: "PT1H"  # Last hour (ISO8601 duration)
  
  # Follow mode for real-time streaming
  follow: true
  poll_interval: 30s
  
  # Maximum results per request
  max_results: 10000

# Metadata and labeling
metadata:
  verbose: false
  labels:
    plugin: azuremonitor
    source: azure

# Batching for efficiency
batch:
  enabled: true
  size: 100
  flush_interval: 5s

Advanced Configuration

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99

# Advanced Azure Monitor Configuration
name: azuremonitor
version: 1.0.0
source: azuremonitor-plugin

# Enhanced agent settings
agent:
  socket_path: /tmp/logflux-agent.sock
  connect_timeout: 30s
  max_retries: 5
  retry_delay: 10s

# Azure Configuration
azure:
  # Authentication
  auth_method: "service-principal"
  tenant_id: "12345678-1234-1234-1234-123456789012"
  client_id: "87654321-4321-4321-4321-210987654321"
  client_secret: "your-client-secret"
  
  # Request timeout
  server_timeout: 60s

# Multiple Log Analytics workspaces
workspaces:
  - workspace_id: "prod-workspace-id"
    name: "production"
  - workspace_id: "staging-workspace-id"
    name: "staging"
  - workspace_id: "dev-workspace-id"
    name: "development"

# Multiple Application Insights
app_insights:
  - app_id: "prod-app-insights-id"
    name: "prod-web-app"
  - app_id: "staging-app-insights-id"
    name: "staging-web-app"

# Advanced query settings
query:
  # Complex KQL query
  query: |
    union
      (SecurityEvent | where TimeGenerated > ago(1h) | where EventID in (4624, 4625)),
      (Event | where TimeGenerated > ago(1h) | where EventLevel <= 3),
      (Syslog | where TimeGenerated > ago(1h) | where SeverityLevel in ("error", "critical"))
    | order by TimeGenerated desc
  
  # Query from file
  query_file: "/etc/logflux-agent/queries/security.kql"
  
  # Time range options
  start_time: "-1h"     # Relative time
  end_time: ""          # Current time
  # OR use absolute timespan
  timespan: "2024-01-20T10:00:00Z/2024-01-20T11:00:00Z"
  
  # Real-time following
  follow: true
  poll_interval: 15s
  
  # Request limits
  max_results: 50000
  
  # Table filtering
  include_tables: "SecurityEvent,Event,Syslog"
  exclude_tables: "Heartbeat,Usage"

# Enhanced metadata
metadata:
  verbose: true
  labels:
    plugin: azuremonitor
    source: azure
    environment: production
    tenant: corporate
  
  # Custom field mapping
  field_mapping:
    workspace_id: "azure_workspace"
    table_name: "azure_table"
    time_generated: "azure_timestamp"
    app_id: "azure_app_insights"

# Advanced batching
batch:
  enabled: true
  size: 500
  flush_interval: 10s
  
  # Memory management
  max_memory: 100MB

# Health monitoring
health:
  check_interval: 60s
  max_api_errors: 10
  alert_on_quota_exceeded: true

Usage Examples

Azure Virtual Machines

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

# Monitor VM security events
sudo logflux-azuremonitor \
  -workspace-id "your-workspace-id" \
  -query "SecurityEvent | where TimeGenerated > ago(1h) | where Computer startswith 'VM'" \
  -follow

# Monitor VM performance
sudo logflux-azuremonitor \
  -workspace-id "your-workspace-id" \
  -query "Perf | where TimeGenerated > ago(1h) | where CounterName == '% Processor Time'" \
  -follow

Azure Kubernetes Service (AKS)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

# AKS cluster monitoring
azure:
  tenant_id: "your-tenant-id"
  auth_method: "managed-identity"

workspaces:
  - workspace_id: "aks-workspace-id"

query:
  query: |
    union
      (ContainerLog | where TimeGenerated > ago(1h)),
      (KubeEvents | where TimeGenerated > ago(1h)),
      (KubePodInventory | where TimeGenerated > ago(1h))
    | order by TimeGenerated desc
  
  follow: true
  poll_interval: 20s

metadata:
  labels:
    service: aks
    cluster: production

Application Insights Telemetry

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

# Web application monitoring
azure:
  tenant_id: "your-tenant-id"
  auth_method: "service-principal"
  client_id: "your-client-id"
  client_secret: "your-client-secret"

app_insights:
  - app_id: "web-app-insights-id"

query:
  query: |
    union
      (requests | where timestamp > ago(1h) | where success == false),
      (exceptions | where timestamp > ago(1h)),
      (dependencies | where timestamp > ago(1h) | where success == false)
    | order by timestamp desc
  
  follow: true
  poll_interval: 30s

metadata:
  labels:
    service: web_application
    monitoring_type: apm

Security Monitoring

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

# Security event monitoring
workspaces:
  - workspace_id: "security-workspace-id"

query:
  query: |
    union
      (SecurityEvent | where TimeGenerated > ago(1h) | where EventID in (4624, 4625, 4648)),
      (SigninLogs | where TimeGenerated > ago(1h) | where ResultType != 0),
      (AuditLogs | where TimeGenerated > ago(1h) | where Result != "success")
    | order by TimeGenerated desc
  
  follow: true
  poll_interval: 60s

metadata:
  labels:
    log_type: security
    compliance: required

Command Line Usage

Basic Commands

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

# Query specific workspace
logflux-azuremonitor -workspace-id "workspace-id" -query "Heartbeat | limit 10"

# Follow mode for real-time logs
logflux-azuremonitor -workspace-id "workspace-id" -query "Event | where EventLevel <= 2" -follow

# Query Application Insights
logflux-azuremonitor -app-insights-id "app-id" -query "requests | where timestamp > ago(1h)"

# Multiple workspaces
logflux-azuremonitor -workspace-ids "ws1,ws2,ws3" -query "Heartbeat | limit 10"

# Historical query with time range
logflux-azuremonitor -workspace-id "workspace-id" -query "Event" -timespan "PT2H"

Advanced Options

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

# Service Principal authentication
logflux-azuremonitor -workspace-id "workspace-id" \
  -auth-method "service-principal" \
  -tenant-id "tenant-id" \
  -client-id "client-id" \
  -client-secret "secret" \
  -query "SecurityEvent | limit 100"

# Query from file
logflux-azuremonitor -workspace-id "workspace-id" \
  -query-file "/path/to/query.kql"

# Custom time range
logflux-azuremonitor -workspace-id "workspace-id" \
  -query "Event" \
  -start-time "2024-01-20T10:00:00Z" \
  -end-time "2024-01-20T11:00:00Z"

# Custom batch settings
logflux-azuremonitor -workspace-id "workspace-id" \
  -query "Heartbeat" \
  -batch-size 200 \
  -flush-interval 10s \
  -max-results 5000

# Verbose output
logflux-azuremonitor -workspace-id "workspace-id" \
  -query "Event | limit 5" \
  -verbose

# Configuration file
logflux-azuremonitor -config /etc/logflux-agent/plugins/azuremonitor.yaml

Authentication Methods

1
2
3
4

# System-assigned managed identity
azure:
  auth_method: "managed-identity"
  tenant_id: "your-tenant-id"

1
2
3
4
5

# Enable system-assigned managed identity for VM
az vm identity assign --name MyVM --resource-group MyRG

# Enable for AKS
az aks update --name MyAKS --resource-group MyRG --enable-managed-identity

Service Principal

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

# Create service principal
az ad sp create-for-rbac --name "LogFluxMonitor" \
  --role "Log Analytics Reader" \
  --scopes "/subscriptions/SUBSCRIPTION_ID"

# Use in configuration
azure:
  auth_method: "service-principal"
  tenant_id: "your-tenant-id"
  client_id: "service-principal-id"
  client_secret: "service-principal-secret"

Azure CLI Credentials

1
2
3
4
5
6
7

# Login with Azure CLI
az login

# Use in configuration
azure:
  auth_method: "azure-cli"
  tenant_id: "your-tenant-id"

Device Code Authentication

1
2
3
4

# Interactive device code flow
azure:
  auth_method: "device-code"
  tenant_id: "your-tenant-id"

Environment Variables

1
2
3
4
5
6
7

# Azure Service Principal
export AZURE_TENANT_ID="your-tenant-id"
export AZURE_CLIENT_ID="your-client-id"
export AZURE_CLIENT_SECRET="your-client-secret"

# Or use Azure CLI credentials
az login

KQL Query Examples

Basic Queries

-- All events from last hour
Event | where TimeGenerated > ago(1h)

-- Security events with specific IDs
SecurityEvent | where EventID in (4624, 4625, 4648) | where TimeGenerated > ago(2h)

-- Application logs with errors
AppServiceConsoleLogs | where TimeGenerated > ago(1h) | where Level == "Error"

-- Performance counters
Perf | where TimeGenerated > ago(1h) | where CounterName == "% Processor Time"

Advanced Queries

-- Failed login attempts by user
SecurityEvent 
| where TimeGenerated > ago(24h) 
| where EventID == 4625 
| summarize FailedAttempts = count() by Account, Computer 
| where FailedAttempts > 5 
| order by FailedAttempts desc

-- Application Insights error analysis
exceptions 
| where timestamp > ago(1h) 
| summarize ErrorCount = count() by operation_Name, type 
| order by ErrorCount desc

-- Container resource usage
KubePodInventory 
| where TimeGenerated > ago(1h) 
| join (Perf | where TimeGenerated > ago(1h)) on Computer 
| summarize AvgCPU = avg(CounterValue) by PodName, Namespace

-- Network security group flow analysis
AzureNetworkAnalytics_CL 
| where TimeGenerated > ago(1h) 
| where FlowStatus_s == "D" 
| summarize BlockedConnections = count() by SrcIP_s, DstPort_d 
| order by BlockedConnections desc

Application Insights Queries

-- Web app performance
requests 
| where timestamp > ago(1h) 
| summarize 
    RequestCount = count(),
    AvgDuration = avg(duration),
    P95Duration = percentile(duration, 95)
by bin(timestamp, 5m)

-- Dependency failures
dependencies 
| where timestamp > ago(1h) 
| where success == false 
| summarize FailureCount = count() by name, type 
| order by FailureCount desc

-- User session analysis
pageViews 
| where timestamp > ago(24h) 
| summarize SessionCount = dcount(session_Id) by bin(timestamp, 1h)

Metadata and Output Format

Metadata Fields

The plugin adds Azure-specific metadata:

Field Description Example
source_type Always “plugin” plugin
source_name Always “azuremonitor” azuremonitor
azure_workspace Log Analytics workspace ID 12345678-1234-1234-1234-123456789012
azure_table Azure Monitor table name SecurityEvent
azure_timestamp Original Azure timestamp 2024-01-20T14:30:50.123Z
azure_app_insights Application Insights app ID abcd1234-5678-9012-3456-789012345678
azure_tenant Azure tenant ID 87654321-4321-4321-4321-210987654321

LogFlux Output Format

Input Azure Monitor Log:

1
2
3
4
5
6
7
8
9

{
  "TenantId": "87654321-4321-4321-4321-210987654321",
  "TimeGenerated": "2024-01-20T14:30:50.123Z",
  "Computer": "VM-WEB-01",
  "EventID": 4624,
  "EventData": "Successful logon",
  "Account": "user@domain.com",
  "LogonType": 2
}

Output LogFlux Log:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

{
  "timestamp": "2024-01-20T14:30:50.123Z",
  "level": "info",
  "message": "Successful logon",
  "node": "azure",
  "metadata": {
    "source_type": "plugin",
    "source_name": "azuremonitor",
    "azure_workspace": "12345678-1234-1234-1234-123456789012",
    "azure_table": "SecurityEvent",
    "azure_timestamp": "2024-01-20T14:30:50.123Z",
    "azure_tenant": "87654321-4321-4321-4321-210987654321",
    "computer": "VM-WEB-01",
    "event_id": 4624,
    "account": "user@domain.com",
    "logon_type": 2,
    "plugin": "azuremonitor",
    "environment": "production"
  }
}

Performance Optimization

High-Volume Configuration

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

# High-throughput settings
query:
  max_results: 50000
  poll_interval: 10s
  
batch:
  size: 1000
  flush_interval: 30s
  max_memory: 500MB

# Use efficient KQL queries
query: |
  SecurityEvent 
  | where TimeGenerated > ago(15m) 
  | where EventID in (4624, 4625) 
  | project TimeGenerated, Computer, Account, EventID

Cost Optimization

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

# Reduce Azure Monitor costs
query:
  # Use time filters to limit data
  query: "Event | where TimeGenerated > ago(30m)"
  
  # Project only needed columns
  query: "SecurityEvent | project TimeGenerated, Computer, EventID, Account"
  
  poll_interval: 60s  # Less frequent polling
  max_results: 1000   # Smaller result sets

Query Performance

-- Optimize queries with filters early
SecurityEvent 
| where TimeGenerated > ago(1h)  -- Filter by time first
| where EventID == 4625          -- Then by specific criteria
| project TimeGenerated, Computer, Account  -- Project only needed columns

-- Use summarize for aggregations
SecurityEvent 
| where TimeGenerated > ago(1h) 
| summarize count() by Computer, bin(TimeGenerated, 5m)  -- Aggregate data

Monitoring and Alerting

Plugin Health Monitoring

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

#!/bin/bash
# check-azure-plugin.sh

if ! systemctl is-active --quiet logflux-azuremonitor; then
    echo "CRITICAL: LogFlux Azure Monitor plugin is not running"
    exit 2
fi

# Check Azure connectivity
if ! az account show &>/dev/null; then
    echo "CRITICAL: Cannot authenticate with Azure"
    exit 2
fi

# Check recent log processing
if ! journalctl -u logflux-azuremonitor --since="10 minutes ago" | grep -q "results processed"; then
    echo "WARNING: No results processed in last 10 minutes"
    exit 1
fi

echo "OK: LogFlux Azure Monitor plugin is healthy"
exit 0

Azure Monitor Metrics

1
2
3
4
5
6
7
8
9

# Check workspace usage
az monitor log-analytics workspace show \
  --workspace-name "MyWorkspace" \
  --resource-group "MyRG"

# Monitor query performance
az monitor log-analytics query \
  --workspace "workspace-id" \
  --analytics-query "Usage | where TimeGenerated > ago(1d) | summarize TotalDataMB = sum(Quantity) by bin(TimeGenerated, 1h)"

Common Use Cases

Infrastructure Monitoring

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

# VM and server monitoring
workspaces:
  - workspace_id: "infra-workspace-id"

query:
  query: |
    union
      (Event | where TimeGenerated > ago(1h) | where EventLevel <= 2),
      (SecurityEvent | where TimeGenerated > ago(1h) | where EventID in (4624, 4625)),
      (Perf | where TimeGenerated > ago(1h) | where CounterName in ("% Processor Time", "Available MBytes"))
    | order by TimeGenerated desc
  
  follow: true
  poll_interval: 30s

metadata:
  labels:
    service: infrastructure
    type: monitoring

Application Performance Monitoring

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

# APM with Application Insights
app_insights:
  - app_id: "web-app-insights-id"
  - app_id: "api-app-insights-id"

query:
  query: |
    union
      (requests | where timestamp > ago(1h) | where duration > 5000),
      (dependencies | where timestamp > ago(1h) | where success == false),
      (exceptions | where timestamp > ago(1h))
    | order by timestamp desc
  
  follow: true
  poll_interval: 15s

metadata:
  labels:
    monitoring_type: apm
    focus: performance

Security Monitoring

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

# Security event monitoring
workspaces:
  - workspace_id: "security-workspace-id"

query:
  query: |
    union
      (SecurityEvent | where TimeGenerated > ago(1h) | where EventID in (4625, 4648, 4672)),
      (SigninLogs | where TimeGenerated > ago(1h) | where ResultType != 0),
      (AuditLogs | where TimeGenerated > ago(1h) | where Category == "UserManagement")
    | order by TimeGenerated desc
  
  follow: true
  poll_interval: 60s

metadata:
  labels:
    log_type: security
    compliance: sox_pci

Container Monitoring (AKS)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

# Kubernetes monitoring
workspaces:
  - workspace_id: "aks-workspace-id"

query:
  query: |
    union
      (ContainerLog | where TimeGenerated > ago(1h) | where LogEntry contains "error"),
      (KubeEvents | where TimeGenerated > ago(1h) | where Reason != "Scheduled"),
      (KubePodInventory | where TimeGenerated > ago(1h) | where PodStatus != "Running")
    | order by TimeGenerated desc
  
  follow: true
  poll_interval: 20s

metadata:
  labels:
    platform: aks
    service: containers

Security Considerations

RBAC Best Practices

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

{
  "properties": {
    "roleName": "LogFlux Monitor Reader",
    "description": "Read access to Log Analytics for LogFlux",
    "assignableScopes": ["/subscriptions/SUBSCRIPTION_ID"],
    "permissions": [
      {
        "actions": [
          "Microsoft.OperationalInsights/workspaces/query/read",
          "Microsoft.Insights/components/query/read"
        ],
        "notActions": [],
        "dataActions": [],
        "notDataActions": []
      }
    ]
  }
}

Network Security

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

# Configure private endpoints for Log Analytics
az monitor log-analytics workspace create \
  --workspace-name "secure-workspace" \
  --resource-group "security-rg" \
  --public-network-access-for-ingestion Disabled \
  --public-network-access-for-query Disabled

# Use Azure Private Link
az network private-endpoint create \
  --name "loganalytics-pe" \
  --resource-group "security-rg" \
  --subnet "private-subnet" \
  --private-connection-resource-id "/subscriptions/SUB/resourceGroups/RG/providers/Microsoft.OperationalInsights/workspaces/WORKSPACE"

Credential Management

1
2
3
4
5
6
7

# Use Key Vault for secrets
azure:
  auth_method: "service-principal"
  tenant_id: "tenant-id"
  client_id: "client-id"
  # Store client_secret in Azure Key Vault
  client_secret: "@Microsoft.KeyVault(SecretUri=https://vault.vault.azure.net/secrets/logflux-secret/version)"

Troubleshooting

Common Issues

Authentication Failures:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

# Check Azure CLI authentication
az account show

# Test workspace access
az monitor log-analytics query \
  --workspace "workspace-id" \
  --analytics-query "Heartbeat | limit 1"

# Check service principal permissions
az role assignment list --assignee "service-principal-id"

No Data Retrieved:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

# Verify data exists in workspace
az monitor log-analytics query \
  --workspace "workspace-id" \
  --analytics-query "search * | take 5"

# Check query syntax
az monitor log-analytics query \
  --workspace "workspace-id" \
  --analytics-query "Event | where TimeGenerated > ago(1h) | limit 5"

# Test Application Insights
az monitor app-insights query \
  --app "app-id" \
  --analytics-query "requests | limit 5"

Query Performance Issues:

-- Check query performance
Event 
| where TimeGenerated > ago(1h)  -- Always filter by time first
| limit 1000                     -- Limit results for testing

-- Use query performance statistics
set query_results_cache_max_age = time(0m);
Event | where TimeGenerated > ago(1h) | count

High Costs:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

# Monitor and optimize costs
query:
  # Use specific time ranges
  query: "Event | where TimeGenerated > ago(30m)"
  
  # Project only needed columns
  query: "SecurityEvent | project TimeGenerated, Computer, EventID"
  
  # Increase poll interval
  poll_interval: 300s  # 5 minutes

Debugging

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

# Enable verbose logging
sudo systemctl edit logflux-azuremonitor
# Add:
[Service]
Environment="LOGFLUX_LOG_LEVEL=debug"

# Monitor API calls
az monitor log-analytics query \
  --workspace "workspace-id" \
  --analytics-query "Usage | where TimeGenerated > ago(1h)" \
  --debug

# Check plugin logs
sudo journalctl -u logflux-azuremonitor -f

# Test connectivity
az rest --method get \
  --url "https://api.loganalytics.io/v1/workspaces/WORKSPACE_ID/metadata"

Best Practices

Configuration Management

  1. Use Managed Identity when possible instead of service principals
  2. Optimize KQL queries with time filters and column projections
  3. Set appropriate poll intervals based on data freshness requirements
  4. Monitor Azure costs associated with query volume

Performance

  1. Filter by time early in KQL queries to reduce data processing
  2. Use summarize for aggregated data rather than raw logs
  3. Project only needed columns to reduce data transfer
  4. Batch similar queries to improve efficiency

Security

  1. Follow least privilege principle for Azure permissions
  2. Use Azure Key Vault for credential management
  3. Enable private endpoints for sensitive workspaces
  4. Monitor query access through Azure Activity Log

Cost Management

  1. Use efficient KQL queries to minimize data processing
  2. Set up cost alerts for Log Analytics workspaces
  3. Archive old data to lower-cost storage tiers
  4. Monitor data ingestion and retention policies

Disclaimer

Microsoft Azure, Azure Monitor, Application Insights, and the Microsoft Azure logo are trademarks of Microsoft Corporation. LogFlux is not affiliated with, endorsed by, or sponsored by Microsoft Corporation. The Microsoft Azure services and logos are referenced solely for identification purposes to indicate compatibility with Azure Monitor and Application Insights.

Next Steps