如下VPP命令創建bond接口，模式選擇LACP協議，負載均衡模式選擇網絡三四層。新創建的接口為BondEthernet0，添加兩個子接口GigabitEtherneta/0/0和GigabitEthernetb/0/0。

vpp# create bond mode lacp load-balance l34
BondEthernet0
vpp#
vpp#
vpp# bond add BondEthernet0 GigabitEtherneta/0/0
vpp#
vpp# bond add BondEthernet0 GigabitEthernetb/0/0

查看創建的bond接口，顯示成員數量為兩個。隨后將兩個成員接口狀態設置為UP。

vpp# show bond
interface name   sw_if_index  mode          load balance  active members members
BondEthernet0    24           lacp          l34           0              2
vpp#
vpp# set interface state GigabitEtherneta/0/0 up
vpp# set interface state GigabitEthernetb/0/0 up

LCP插件創建以上bond接口的對等接口，取名為be0。

vpp# lcp create BondEthernet0 host-if be0
vpp#
vpp# show interface
              Name               Idx    State  MTU (L3/IP4/IP6/MPLS)     Counter          Count
BondEthernet0                     24    down         9000/0/0/0
GigabitEtherneta/0/0              13     up          9000/0/0/0
GigabitEthernetb/0/0              14     up          9000/0/0/0
GigabitEthernetc/0/0              15     up          9000/0/0/0
GigabitEthernetd/0/0              16    down         9000/0/0/0
tap19                             25     up          9000/0/0/0
vpp# show lcp
lcp default netns '<unset>'
lcp lcp-auto-subint on
lcp lcp-sync on
itf-pair: [4] BondEthernet0 tap19 be0 27 type tap

在linux中查看，創建了bond接口be0。

# ip -d link
27: be0: <BROADCAST,MULTICAST> mtu 9000 qdisc mq state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:60:e0:6f:5a:f3 brd ff:ff:ff:ff:ff:ff promiscuity 0 minmtu 68 maxmtu 65521
    tun type tap pi off vnet_hdr on multi_queue numqueues 2 numdisabled 0 persist off addrgenmode eui64 numtxqueues 256 numrxqueues 256 gso_max_size 65536 gso_max_segs 65535

接口be0的MAC地址與VPP中對等接口BondEthernet0的MAC地址相同。

MAC相同。

vpp# show hardware-interfaces BondEthernet0
              Name                Idx   Link  Hardware
BondEthernet0                     19     up   BondEthernet0
  Ethernet address 00:60:e0:6f:5a:f3
vpp#
vpp#
vpp# show hardware-interfaces GigabitEtherneta/0/0
              Name                Idx   Link  Hardware
GigabitEtherneta/0/0              13    up    GigabitEtherneta/0/0
  Ethernet address 00:60:e0:6f:5a:f3
vpp#
vpp#
vpp# show hardware-interfaces GigabitEthernetb/0/0
              Name                Idx   Link  Hardware
GigabitEthernetb/0/0              14    up    GigabitEthernetb/0/0
  Ethernet address 00:60:e0:6f:5a:f3
vpp#
vpp# set interface state BondEthernet0 up

Ubuntu配置

在另外一臺Ubuntu主機上創建接口bond0，模式為802.3ad，負載均衡使用網絡3/4層hash算法。

# ip link add bond0 type bond mode 802.3ad xmit_hash_policy layer3+4
# 
# 
# ip -d link show bond0
7: bond0: <BROADCAST,MULTICAST,MASTER> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 26:dc:49:57:fa:b3 brd ff:ff:ff:ff:ff:ff promiscuity 0 minmtu 68 maxmtu 65535 
    bond mode 802.3ad miimon 100 updelay 0 downdelay 0 peer_notify_delay 0 use_carrier 1 arp_interval 0 arp_validate none arp_all_targets any primary_reselect always fail_over_mac none xmit_hash_policy layer3+4 resend_igmp 1 num_grat_arp 1 all_slaves_active 0 min_links 0 lp_interval 1 packets_per_slave 1 lacp_rate slow ad_select stable tlb_dynamic_lb 1 addrgenmode eui64 numtxqueues 16 numrxqueues 16 gso_max_size 65536 gso_max_segs 65535 

將接口ens35和ens36添加為bond0的子接口。

/ # ip link set ens35 down
/ # ip link set ens36 down
/ #
/ # ip link set ens35 master bond0
[82160.305319] 8021q: adding VLAN 0 to HW filter on device ens35
[82160.311192] bond0: Enslaving ens35 as a backup interface with an up link
/ #
/ # ip link set ens36 master bond0
[82172.266467] 8021q: adding VLAN 0 to HW filter on device ens36
[82172.272367] bond0: Enslaving ens36 as a backup interface with an up link
/ #
/ # ip link set ens35 up
/ # ip link set ens36 up
/ #
/ # ip link set bond0 up
[82194.363870] IPv6: ADDRCONF(NETDEV_UP): bond0: link is not ready
[82194.369815] 8021q: adding VLAN 0 to HW filter on device bond0

子接口和bond接口使用相同的mac地址。

/ # ip link show master bond0
9: ens35: <NO-CARRIER,BROADCAST,MULTICAST,PROMISC,SLAVE,UP> mtu 1500 qdisc mq master bond0 state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:60:e0:68:9f:ee brd ff:ff:ff:ff:ff:ff
10: ens36: <NO-CARRIER,BROADCAST,MULTICAST,PROMISC,SLAVE,UP> mtu 1500 qdisc mq master bond0 state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:60:e0:68:9f:ee brd ff:ff:ff:ff:ff:ff
/ #
/ # ip -d link show master bond0
9: ens35: <NO-CARRIER,BROADCAST,MULTICAST,PROMISC,SLAVE,UP> mtu 1500 qdisc mq master bond0 state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:60:e0:68:9f:ee brd ff:ff:ff:ff:ff:ff promiscuity 1
    bond_slave state BACKUP mii_status UP link_failure_count 0 perm_hwaddr 00:60:e0:68:9f:ee queue_id 0 ad_aggregator_id 1 ad_actor_oper_port_state 69 ad_partner_oper_port_state 1 addrgenmode eui64 numtxqueues 8 numrxqueues 8
10: ens36: <NO-CARRIER,BROADCAST,MULTICAST,PROMISC,SLAVE,UP> mtu 1500 qdisc mq master bond0 state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:60:e0:68:9f:ee brd ff:ff:ff:ff:ff:ff promiscuity 1
    bond_slave state BACKUP mii_status UP link_failure_count 0 perm_hwaddr 00:60:e0:68:9f:ef queue_id 0 ad_aggregator_id 2 ad_actor_oper_port_state 69 ad_partner_oper_port_state 1 addrgenmode eui64 numtxqueues 8 numrxqueues 8

測試

設置VPP接口BondEthernet0地址90.0.1.1/24，并且，設置其對等接口be0為相同的地址。

vpp# set interface ip address BondEthernet0 90.0.1.1/24

設置Ubuntu主機的bond0接口地址為90.0.1.2/24。在此主機上執行ping地址90.0.1.1的操作。

/ # ip address add dev bond0 90.0.1.2/24

如下為VPP的trace顯示。可見報文在dpdk-input節點之后進入了bond-input節點。

Packet 1

03:23:12:144922: dpdk-input
  GigabitEthernetb/0/0 rx queue 0
  buffer 0x4f662: current data 0, length 98, buffer-pool 0, ref-count 1, trace handle 0x2000000
                  ext-hdr-valid
  PKT MBUF: port 13, nb_segs 1, pkt_len 98
    buf_len 2176, data_len 98, ol_flags 0x180, data_off 128, phys_addr 0x8fdd9900
    packet_type 0x11 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
    rss 0x0 fdir.hi 0x0 fdir.lo 0x0
    Packet Offload Flags
      PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid
      PKT_RX_IP_CKSUM_NONE (0x0090) no IP cksum of RX pkt.
      PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid
      PKT_RX_L4_CKSUM_NONE (0x0108) no L4 cksum of RX pkt.
    Packet Types
      RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet
      RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension headers
  IP4: 00:60:e0:68:9f:ee -> 00:60:e0:6f:5a:f3
  ICMP: 90.0.1.2 -> 90.0.1.1
    tos 0x00, ttl 64, length 84, checksum 0xff73 dscp CS0 ecn NON_ECN
    fragment id 0xc532
  ICMP echo_request checksum 0x1bfc id 768
03:23:12:144925: bond-input
  src 00:60:e0:68:9f:ee, dst 00:60:e0:6f:5a:f3, GigabitEthernetb/0/0 -> BondEthernet0
03:23:12:144926: ethernet-input
  IP4: 00:60:e0:68:9f:ee -> 00:60:e0:6f:5a:f3
03:23:12:144926: ip4-input
  ICMP: 90.0.1.2 -> 90.0.1.1
    tos 0x00, ttl 64, length 84, checksum 0xff73 dscp CS0 ecn NON_ECN
    fragment id 0xc532
  ICMP echo_request checksum 0x1bfc id 768
03:23:12:144927: ip4-lookup
  fib 0 dpo-idx 11 flow hash: 0x00000000
  ICMP: 90.0.1.2 -> 90.0.1.1
    tos 0x00, ttl 64, length 84, checksum 0xff73 dscp CS0 ecn NON_ECN
    fragment id 0xc532
  ICMP echo_request checksum 0x1bfc id 768
03:23:12:144928: ip4-receive
    ICMP: 90.0.1.2 -> 90.0.1.1
      tos 0x00, ttl 64, length 84, checksum 0xff73 dscp CS0 ecn NON_ECN
      fragment id 0xc532
    ICMP echo_request checksum 0x1bfc id 768
03:23:12:144928: ip4-icmp-input
  ICMP: 90.0.1.2 -> 90.0.1.1
    tos 0x00, ttl 64, length 84, checksum 0xff73 dscp CS0 ecn NON_ECN
    fragment id 0xc532
  ICMP echo_request checksum 0x1bfc id 768
03:23:12:144929: ip4-icmp-echo-request
  ICMP: 90.0.1.2 -> 90.0.1.1
    tos 0x00, ttl 64, length 84, checksum 0xff73 dscp CS0 ecn NON_ECN
    fragment id 0xc532
  ICMP echo_request checksum 0x1bfc id 768
03:23:12:144929: ip4-load-balance
  fib 0 dpo-idx 24 flow hash: 0x00000000
  ICMP: 90.0.1.1 -> 90.0.1.2
    tos 0x00, ttl 64, length 84, checksum 0x85ec dscp CS0 ecn NON_ECN
    fragment id 0x3eba
  ICMP echo_reply checksum 0x23fc id 768
03:23:12:144930: ip4-rewrite
  tx_sw_if_index 24 dpo-idx 24 : ipv4 via 90.0.1.2 BondEthernet0: mtu:9000 next:9 flags:[] 0060e0689fee0060e06f5af30800 flow hash: 0x00000000
  00000000: 0060e0689fee0060e06f5af30800450000543eba0000400185ec5a0001015a00
  00000020: 0102000023fc03001800be766a490000000090430800000000000000
03:23:12:144931: BondEthernet0-output
  BondEthernet0
  IP4: 00:60:e0:6f:5a:f3 -> 00:60:e0:68:9f:ee
  ICMP: 90.0.1.1 -> 90.0.1.2
    tos 0x00, ttl 64, length 84, checksum 0x85ec dscp CS0 ecn NON_ECN
    fragment id 0x3eba
  ICMP echo_reply checksum 0x23fc id 768

在Ubuntu主機上，可通過PROC文件bond0查看其狀態信息。

/ # cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer3+4 (1)
MII Status: up
MII Polling Interval (ms): 0
Up Delay (ms): 0
Down Delay (ms): 0

802.3ad info
LACP rate: slow
Min links: 0
Aggregator selection policy (ad_select): stable
System priority: 65535
System MAC address: 00:60:e0:68:9f:ee
Active Aggregator Info:
        Aggregator ID: 1
        Number of ports: 1
        Actor Key: 9
        Partner Key: 24
        Partner Mac Address: 00:60:e0:6f:5a:f3

Slave Interface: ens35
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:60:e0:68:9f:ee
Slave queue ID: 0
Aggregator ID: 1
Actor Churn State: none
Partner Churn State: none
Actor Churned Count: 1
Partner Churned Count: 1
details actor lacp pdu:
    system priority: 65535
    system mac address: 00:60:e0:68:9f:ee
    port key: 9
    port priority: 255
    port number: 1
    port state: 61
details partner lacp pdu:
    system priority: 65535
    system mac address: 00:60:e0:6f:5a:f3
    oper key: 24
    port priority: 255
    port number: 1
    port state: 63

Slave Interface: ens36
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: 00:60:e0:68:9f:ef
Slave queue ID: 0
Aggregator ID: 2
Actor Churn State: churned
Partner Churn State: churned
Actor Churned Count: 2
Partner Churned Count: 2
details actor lacp pdu:
    system priority: 65535
    system mac address: 00:60:e0:68:9f:ee
    port key: 0
    port priority: 255
    port number: 2
    port state: 69
details partner lacp pdu:
    system priority: 65535
    system mac address: 00:00:00:00:00:00
    oper key: 1
    port priority: 255
    port number: 1
    port state: 1