General load balancer configuration

Here is the general configuration and the explaination for BalanceNG. At the end of this section, some examples are given.
More examples, please refer to BalaneNG website: http://www.inlab.de/balanceng/examples.html


Before setup:

If server machine shut down, the setting on the server has to be done again.

At web1:

service httpd start
ifconfig lo:0 <the virtual IP> netmask 255.255.255.255 -arp up

Error if not doing this:  In the DSR mood, this target actual server cannot connect to the virtual server (IP: H) .

# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
# echo 1 > /proc/sys/net/ipv4/conf/eth0/arp_ignore
# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
# echo 2 > /proc/sys/net/ipv4/conf/eth0/arp_announce

Error if not doing this: In the DSR mood, the bng log file will show: IP address conflict <MAC of web1> claims to be local VIP <the virtual IP>

At web2:

service httpd start
ifconfig lo:0
<the virtual IP> netmask 255.255.255.255 -arp up

# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
# echo 1 > /proc/sys/net/ipv4/conf/eth0/arp_ignore
# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
# echo 2 > /proc/sys/net/ipv4/conf/eth0/arp_announce

Some useful commands to check bng cheatsheet


--------------------------------------------------------------------------

Configuration and files in a glance

The network connected to the Router the "Access Network", and the network to which the Targets are connected the "Target Network".
A BalanceNG configuration consists of the following sections in exactly that order:

   1. hostname, remark, license
   2. interfaces section
   3. vrrpsection
   4. network definitions
   5. network register/enable section
   6. gateway section
   7. server definitions
   8. server register/enable section
   9. target section
  10.target register/enable section
 

Configuration files

BalanceNG makes use of the following configuration files:
/etc/bng.global Global configuration file for all instances
/etc/bng.conf Standard configuration file for BalanceNG default instance 0
/etc/bngN.conf Configuration file for BalanceNG instance N (N: 1 ... 127)
/etc/bng.private Node specific private data for BalanceNG default instance 0
/etc/bngN.private Node specific private data for BalanceNG instance N (N: 1 ... 127)
BalanceNG instance 0 is called the default instance
(Note: sessions in green color are necessary while those in blue color are additional. )

hostname, remark, license

Example 1
hostname [bng-master]
license TEST001 7e68aa2c7d022712a76ee13befd6c607
Example 2
hostname [bng-backup]
license TEST002 2ebe1bf6643061dcdc573ca0db06a1a6

Example 3
hostname master
remark "DSR configuration with dual node HA and session synchronisation"
license MASTER-TEST d5b787e61745533a8c5858a3a5f2e855

parameter settings ("set"section)

BalanceNG uses a set of internal parameters. Using set these parameters may be changed.
All parameters are numeric with a minimum, a maximum and a default. If the parameter is currently set to the default value then no "set" line appears in the configuration file (see
"show parameters").

set {
arplookup 40
arprefresh 180
ipforwarding 1
}

interfaces section

interface nge0
interface eth0

vrrpsection

vrrp      {
vrid 9
priority 200
network 1
}

network definitions

Example 1
network 1 {
name "local network"
addr 172.16.1.0
mask 255.255.255.0
real 172.16.1.252            //For ARP-requests and health checks
interface eth0
}

Example 2
network
  1 {
name "Balancer Network"
addr 172.17.2.0
mask 255.255.255.0
real 172.17.2.70
virt 172.17.2.71            //Being addressable as a routing endpoint for external devices
                        
interface eth0
}
network 2 {
name "Internal Network"
addr 10.10.10.0
mask 255.255.255.0
real 10.10.10.70
virt 10.10.10.71
interface eth1
}
network 3 {
name "Private VRRP Network"
addr 10.0.0.0
mask 255.255.255.0
real 10.0.0.1
interfaces eth0,eth1                   //One network definition and the address are associated to one or many interfaces. Dual legged.
}

network register/enable section

register  network 1,2,3
enable network 1,2,3

gateway section

Example 1
gateway {
ipaddr 172.16.1.254
}

tnet {
10.1.1.4 10.55.55.233 udp 53
10.1.1.4 10.55.55.233 udp 53
10.1.1.4 10.55.55.233 udp 53
10.1.1.4 10.55.55.233 udp 53
}

Example 2
gateway {
ipaddr 10.10.10.254
ping 2,7
trackval 12
}

server definitions

Example 1
server 1 {  
ipaddr 172.16.1.10
targets 1,2
port 80,443
protocol tcp
method agent
}
Example 2
server 1 {
name "BalanceNG LLB Router"
ipaddr any
method session
targets 1,2,3,4
}

server register/enable section

register  server 1
enable server 1

target section

Example 1
target 1 {
ipaddr 172.16.1.100
agent 439,2,5
script "/usr/lib/mon/mon.d/http.monitor -p $port$ -u /index.html $ipaddr$",2,7
dsr enable
}
target 2 {
ipaddr 172.16.1.101
agent 439,2,5
script "/usr/lib/mon/mon.d/http.monitor -p $port$ -u /index.html $ipaddr$",2,7
dsr enable
}

Example 2
target 1 {
name "Link 1"
ipaddr 172.17.2.254
ping 2,5
trackval 3
}

target register/enable section

register  targets 1,2
enable targets 1,2





--------------------------------------------------------------------------


Setup 1: Direct server return mode 

/etc/bng.conf

This setup has the router, balancer, targets under the same network. One balancer with two targets. 


Router(LAN):      192.168.10.1      (B)
Network:          192.168.10.0      (D)
Network mask:     255.255.255.0     (F)
Balancer(archie): 192.168.10.119    (G)         eth0
Target1 (web1):   192.168.10.220    (T1)        eth0
Target2 (web2):   192.168.10.213    (T2)        eth0

Virtual server IP is set by us:
                  192.168.10.20     (H)

 

//  Configuration taken on 05 June 2008 by Weiwei
interface eth0 (archie’s?)
network 1 {
          name "Weiwei's test network"
          addr D
          mask F
          real 192.168.10.252
          interface eth0 (archie’s?)

real cannot be G: log will show error:
    Unable to enable network 1: real address locally occupied by OS
    Before set up this real address, 192.168.10.252 is not reachable
    The "real" address in the network section is being used as source address for ARP resolving and health checks

}
register  network 1
enable    network 1
server    1 {
          ipaddr H
          port 80
          protocol tcp
          targets 1,2
}
register  server 1
enable    server 1
target    1 {
          ipaddr T1
          port 80
          protocol tcp
          tcpopen 80,3,10            //tcpopen <p>,<i>,<t> perform tcpopen healthcheck (port,interval,timeout)
          dsr enable

}
target    2 {
          ipaddr T2
          port 80
          protocol tcp
          tcpopen 80,3,10
          dsr enable
}
register  targets 1,2
enable    targets 1,2
//        end of configuration



--------------------------------------------------------------------------


Setup 2: "All service load balancing": Not specifying ports at server and target definitions


// Configuration taken on 10 June 2008 by Weiwei
// Remark "test configuration, DSR to two target"
// Ref: http://www.inlab.de/balanceng/configs.html conf001.txt
interface eth1
network 1 {
          name "No specifiying ports"
          addr
192.168.10.0
          mask 255.255.255.0
          real 192.168.10.252
          interface eth1
(archie’s eth now change to eth1 as there seems to have some problem with eth1.)
}
register network 1
enable network 1
server 1 {
ipaddr 192.168.10.20
target 1,2
}
register server 1
enable server 1
target 1 {
ipaddr 192.168.10.220
alert "/usr/bin/echo target $target$ down $ipaddr$ >> /tmp/a1.log" specify external alert script Need license
         upalert "/usr/bin/echo target $target$ up $ipaddr$ >> /tmp/a1.log" specify external upalert script Need license
      dsr enable
}

target 2 {
ipaddr 192.168.10.213
dsr enable
}

register target 1,2
enable target 1,2
// end of configuration



--------------------------------------------------------------------------


Setup 3: DSR load balancing FTP on port 20 and 21 to two targets


Note: Here we specify the port is 20 and 21 where 21 is the command port and 20 is the data port. This is an active FTP mood.  Therefore in the FTP configuration file we must have passive mood disable. The sample FTP config file:
/etc/vsftpd/vsftpd.conf
// Configuration taken on 11 June 2008 by Weiwei
// Remark "test configuration, DSR to two target, FTP on port 21 the command port"
// Ref: http://www.inlab.de/balanceng/configs.html conf005.txt One node DSR load balancing SMTP on port 25 to two targets
interface eth1
network 1 {
name "Test FTP port20 21"
addr 192.168.10.0
        
mask 255.255.255.0
real 192.168.10.252
         interface eth1
}
register network 1
enable network 1

server 1 {
ipaddr 192.168.10.20
protocol tcp
portrel on
port 20,21
targets 1,2
}
register server 1
enable server 1
target 1 {
ipaddr 192.168.10.220
tcpopen 21,2,5
dsr enable
}
target 2 {
ipaddr 192.168.10.213
tcpopen 21,2,5
dsr enable
}
register targets 1,2
enable targets 1,2
// end of configuration



--------------------------------------------------------------------------

Setup 4: DSR balance of LDAP



The working copy of config file: bng38.conf

// Configuration taken on 2 July 2008 by Weiwei
// Remark    "DSR to two target, LDAP on port 389"
interface eth1
network   1 {
          name "access LDAP"
          addr 192.168.10.0
          mask 255.255.255.0  
          real 192.168.10.253   //for health check. cannot be the same as the other bng.conf
          interface eth1
}

network 2 {
    name "target LDAP"
    addr 10.0.11.0
    mask 255.255.255.0
    real 10.0.11.253
    interface eth1
}
register  network 1,2
enable    network 1,2

server    1 {
          ipaddr 192.168.10.200
          protocol tcp
          port 389
          portrel on
          targets 1,2
}
register  server 1
enable    server 1
target    1 {
          ipaddr 10.0.11.5
          ping 2,10          //ip health check
          tcpopen 389,5,20   //port health check
          dsr enable
}
target    2 {
          ipaddr 10.0.11.6
          ping 2,10
          tcpopen 389,5,20
          dsr enable
}
register  targets 1,2
enable    targets 1,2
//        end of configuration


______________________________________________________________________________________

Setup 0: virtual machines - Balancer & Targets in one machine, 2 network eth.

Just for experiment.


This setup has the router, balancer, targets under the same network. One balancer with two targets.
Balancer has two network cards: eth0 and eth1.


Router(LAN):      192.168.84.1      (B)
Access Network:   192.168.84.0      (
C1)
Target Network:   192.168.199.0     (C2)
Network mask:     255.255.255.0     (F)
Balancer:         192.168.84.220    (G0)        eth0
Balancer:         192.168.84.221    (G1)        eth1
Target1:          192.168.199.150   (T1)        eth0
Target2:          192.168.199.151   (T2)        eth0

Virtual server IP is set by us:
                  192.168.84.222     (H)


// configuration taken Fri Sep 28 21:38:37 2007
// BalanceNG 1.898 (created 2007/09/28)
hostname bng1
interface eth0
interface eth1
network 1 {
name access
addr 192.168.84.0
mask 255.255.255.0
real 192.168.84.220
virt 192.168.84.221
interface eth1
}
network 2 {
name target
addr 192.168.199.0
mask 255.255.255.0
real 192.168.199.100
virt 192.168.199.252
interface eth0
}
register networks 1,2
enable networks 1,2
gateway {
ipaddr 192.168.84.1
}
tnat {
192.168.199.150 192.168.84.233 udp 53
192.168.199.151 192.168.84.231 udp 53
}
server 1 {
ipaddr 192.168.84.222
portrel on
targets 1,2
}
register server 1
enable server 1
target 1 {
ipaddr 192.168.199.150
ping 2,10
tcpopen 80,5,20
}
target 2 {
ipaddr 192.168.199.151
ping 2,10
tcpopen 80,5,20
}
register targets 1,2
enable targets 1,2
// end of configuration

Comments