Skip to content
Home » Linux Dual Network Interface Routing Configuration: Complete Guide to Multi-Interface Setup

Linux Dual Network Interface Routing Configuration: Complete Guide to Multi-Interface Setup

  • by

Introduction to Dual Network Interface Routing

Configuring dual network interfaces on Linux servers is a common requirement in enterprise environments, but it often leads to routing challenges that can prevent services from functioning correctly. This comprehensive guide addresses the specific routing issues that occur when multiple network interfaces share the same subnet and provides practical solutions for Ubuntu and other Linux distributions.

When running services like Zabbix, web servers, or database applications on servers with multiple network interfaces, proper routing configuration becomes critical for ensuring reliable connectivity from all client networks.

Understanding the Network Setup Problem

In our example scenario, we have:

  • Server: Ubuntu 20.04 LTS running Zabbix monitoring service
  • Interface 1 (ens5): 192.168.1.2/24, gateway 192.168.1.254
  • Interface 2 (ens6): 192.168.1.1/24, gateway 192.168.1.254
  • Service Port: 10051 (Zabbix Server listening on 0.0.0.0:10051)

The typical problem manifests as:

  • ✅ Connections to 192.168.1.2:10051 work correctly
  • ❌ Connections to 192.168.1.1:10051 fail despite the service listening on all interfaces

Root Cause Analysis

The issue stems from Linux’s default routing behavior. When multiple interfaces exist in the same subnet, the kernel typically uses the first configured interface for outbound traffic, regardless of which interface received the incoming connection. This creates an asymmetric routing situation where:

  1. Incoming packets arrive on ens6 (192.168.1.1)
  2. The application receives the request and generates a response
  3. Outgoing response packets are routed through ens5 (192.168.1.2) due to default routing rules
  4. The client rejects the response because it came from a different IP address

Linux Routing Fundamentals

Understanding Routing Tables

Linux uses multiple routing tables managed by the iproute2 package. You can view the available tables:

# View routing table configuration
cat /etc/iproute2/rt_tables

## Reserved values
#255	local
254	main
253	default
0	unspec

## Custom tables
100	primary
200	secondary

Examining Current Routes

Check your current routing configuration:

# Display main routing table
ip route show

# Example output showing the problem:
default via 192.168.1.254 dev ens5 proto dhcp src 192.168.1.2 metric 100
192.168.1.0/24 dev ens6 proto kernel scope link src 192.168.1.1
192.168.1.0/24 dev ens5 proto kernel scope link src 192.168.1.2 metric 100

Policy-Based Routing Rules

View current routing policies:

# Display routing policies
ip rule list

# Typical output:
0:	from all lookup local
32766:	from all lookup main
32767:	from all lookup default

Netplan Configuration for Dual Interfaces

Ubuntu uses Netplan for network configuration management. Proper Netplan configuration is essential for resolving dual interface routing issues.

Primary Interface Configuration

Edit /etc/netplan/50-cloud-init.yaml:

network:
    ethernets:
        ens5:
            dhcp4: true
            dhcp6: false
            match:
                macaddress: 00:11:22:33:44:55
            set-name: ens5
            routing-policy:
              - from: 192.168.1.2
                table: 100
              - to: 192.168.1.2
                table: 100
            routes:
              - to: default
                via: 192.168.1.254
                table: 100
                metric: 200  # Higher value = lower priority
    version: 2

Secondary Interface Configuration

Create /etc/netplan/99-ens6.yaml:

network:
  ethernets:
    ens6:
      addresses:
        - 192.168.1.1/24
      routing-policy:
        - from: 192.168.1.1
          table: 200
        - to: 192.168.1.1
          table: 200
      routes:
        - to: default
          via: 192.168.1.254
          table: 200
        - to: default
          via: 192.168.1.254
          metric: 50  # Lower value = higher priority

Applying Netplan Configuration

# Test the configuration before applying
sudo netplan try

# Apply the configuration permanently
sudo netplan apply

# Verify the configuration
ip route show
ip rule list

Advanced Diagnostic Procedures

Network Connectivity Testing

Use these commands to diagnose routing issues:

# Test basic connectivity
ping -I ens5 192.168.1.254
ping -I ens6 192.168.1.254

# Test service connectivity
telnet 192.168.1.1 10051
telnet 192.168.1.2 10051

# Check if service is listening on all interfaces
netstat -tlnp | grep :10051
ss -tlnp | grep :10051

Packet Flow Analysis

Capture and analyze network traffic:

# Monitor traffic on specific interface
sudo tcpdump -i ens6 port 10051 -nn -v

# Monitor traffic on all interfaces
sudo tcpdump -i any port 10051 -nn

# Advanced packet analysis with detailed output
sudo tcpdump -i any port 10051 -nn -X -s 1500

Kernel Parameter Verification

Check and adjust critical kernel parameters:

# Check reverse path filtering settings
sysctl -a | grep rp_filter

# Check IP forwarding status
sysctl -a | grep ip_forward

# View all network-related kernel parameters
sysctl -a | grep net.ipv4

System-Level Configuration Adjustments

Kernel Parameter Optimization

Apply these kernel parameter changes for multi-interface environments:

# Enable IP forwarding (if needed)
sudo sysctl -w net.ipv4.ip_forward=1

# Set reverse path filtering to loose mode
sudo sysctl -w net.ipv4.conf.all.rp_filter=2
sudo sysctl -w net.ipv4.conf.ens5.rp_filter=2
sudo sysctl -w net.ipv4.conf.ens6.rp_filter=2

# Disable source route verification
sudo sysctl -w net.ipv4.conf.all.accept_source_route=0

Make these changes permanent by adding them to /etc/sysctl.conf:

# Add to /etc/sysctl.conf
net.ipv4.ip_forward=1
net.ipv4.conf.all.rp_filter=2
net.ipv4.conf.ens5.rp_filter=2
net.ipv4.conf.ens6.rp_filter=2
net.ipv4.conf.all.accept_source_route=0

# Apply the changes
sudo sysctl -p

Manual Route Priority Adjustment

When Netplan configuration isn’t sufficient, manually adjust route priorities:

# Remove existing default route
sudo ip route del default via 192.168.1.254 dev ens5

# Add new default route with correct priority
sudo ip route add default via 192.168.1.254 dev ens6 metric 50

# Add backup route through secondary interface
sudo ip route add default via 192.168.1.254 dev ens5 metric 100

# Verify changes
ip route show

Policy-Based Routing Implementation

Creating Custom Routing Tables

For complex scenarios, implement policy-based routing:

# Add custom routing table entries
echo "100 rt_ens5" >> /etc/iproute2/rt_tables
echo "200 rt_ens6" >> /etc/iproute2/rt_tables

# Create rules for each interface
sudo ip rule add from 192.168.1.2 table rt_ens5
sudo ip rule add from 192.168.1.1 table rt_ens6

# Add routes to custom tables
sudo ip route add default via 192.168.1.254 dev ens5 table rt_ens5
sudo ip route add default via 192.168.1.254 dev ens6 table rt_ens6

# Add local network routes
sudo ip route add 192.168.1.0/24 dev ens5 table rt_ens5
sudo ip route add 192.168.1.0/24 dev ens6 table rt_ens6

Dynamic Route Management

Create a script for dynamic route management:

#!/bin/bash
# dual-interface-routing.sh

# Function to setup routing for dual interfaces
setup_dual_routing() {
    local primary_if="ens5"
    local secondary_if="ens6"
    local primary_ip="192.168.1.2"
    local secondary_ip="192.168.1.1"
    local gateway="192.168.1.254"
    
    # Flush existing rules for these IPs
    ip rule del from $primary_ip 2>/dev/null
    ip rule del from $secondary_ip 2>/dev/null
    
    # Add policy routing rules
    ip rule add from $primary_ip table 100 priority 100
    ip rule add from $secondary_ip table 200 priority 200
    
    # Setup routes in custom tables
    ip route add default via $gateway dev $primary_if table 100
    ip route add default via $gateway dev $secondary_if table 200
    
    echo "Dual interface routing configured successfully"
}

# Execute the function
setup_dual_routing

Cloud Platform Specific Considerations

AWS Configuration Requirements

When running on AWS EC2, additional configuration may be required:

  • Security Groups: Ensure both network interfaces have appropriate security group rules
  • Source/Destination Check: Disable source/destination checking on secondary interfaces
  • Route Tables: Configure VPC route tables to properly route traffic to both interfaces
  • Elastic IPs: Consider using Elastic IP addresses for consistent connectivity
# Disable source/destination check (via AWS CLI)
aws ec2 modify-network-interface-attribute \
    --network-interface-id eni-12345678 \
    --no-source-dest-check

Azure and GCP Considerations

Similar considerations apply for other cloud platforms:

  • Azure: Configure Network Security Groups and User Defined Routes
  • GCP: Set up proper firewall rules and custom routes in VPC

Troubleshooting Common Issues

Service Binding Problems

Ensure services are properly bound to all interfaces:

# Check service binding
sudo netstat -tlnp | grep :10051

# For Zabbix server, verify configuration
grep "ListenIP" /etc/zabbix/zabbix_server.conf

# Service should listen on 0.0.0.0 or both specific IPs
# ListenIP=0.0.0.0

Firewall Configuration

Verify firewall rules don’t block traffic:

# Check UFW status
sudo ufw status verbose

# Check iptables rules
sudo iptables -L -n -v

# Allow traffic on service port
sudo ufw allow 10051/tcp

DNS and Hostname Resolution

Ensure proper hostname resolution:

# Check hostname resolution
hostname -I
cat /etc/hosts

# Add entries if needed
echo "192.168.1.1 server.example.com server" >> /etc/hosts
echo "192.168.1.2 server.example.com server" >> /etc/hosts

Performance Optimization and Monitoring

Network Performance Tuning

Optimize network performance for dual interfaces:

# Increase network buffer sizes
echo 'net.core.rmem_max = 16777216' >> /etc/sysctl.conf
echo 'net.core.wmem_max = 16777216' >> /etc/sysctl.conf
echo 'net.ipv4.tcp_rmem = 4096 87380 16777216' >> /etc/sysctl.conf
echo 'net.ipv4.tcp_wmem = 4096 65536 16777216' >> /etc/sysctl.conf

# Apply changes
sudo sysctl -p

Monitoring Network Traffic

Set up monitoring for dual interface performance:

# Monitor interface statistics
watch -n 1 'cat /proc/net/dev'

# Monitor routing table changes
watch -n 5 'ip route show'

# Check connection states
watch -n 2 'ss -tuln | grep :10051'

Best Practices and Recommendations

Network Design Principles

  1. Subnet Separation: When possible, configure network interfaces on different subnets to avoid routing complexity
  2. Clear Routing Policies: Always define explicit routing policies and tables for each IP address
  3. Reverse Path Filtering: Use loose mode (rp_filter=2) in multi-interface environments
  4. Metric-Based Prioritization: Set different metric values to control route priority
  5. Persistent Configuration: Use Netplan or systemd-networkd for configuration persistence

Security Considerations

  • Implement proper firewall rules for each interface
  • Use specific interface binding for security-sensitive services
  • Monitor traffic patterns for unusual routing behavior
  • Regularly audit routing table configuration

Maintenance and Documentation

  • Document all custom routing configurations
  • Create backup scripts for routing table restoration
  • Test failover scenarios regularly
  • Monitor system logs for routing-related errors

Conclusion

Configuring dual network interfaces in Linux requires careful attention to routing policies, kernel parameters, and service configuration. By following the comprehensive approach outlined in this guide, you can successfully resolve routing issues and ensure reliable connectivity through multiple network interfaces.

The key success factors include:

  • Proper understanding of Linux routing fundamentals
  • Correct Netplan configuration with policy-based routing
  • Appropriate kernel parameter settings
  • Thorough testing and monitoring procedures
  • Platform-specific considerations for cloud deployments

Remember that network configuration changes can impact system connectivity, so always test thoroughly in a controlled environment before implementing in production systems. Regular monitoring and maintenance of routing configurations ensure continued reliable operation of your multi-interface Linux servers.

Leave a Reply

Your email address will not be published. Required fields are marked *