Hello all!
Thank you for reading my topic.
Our network consists of 2x MX and 2x EX9200 directly connected to eachother in full mesh. No P (MPLS only) routers are bieng used. Popping happends directly on the same device.
Our top PE routers have internet connectivity, the bottom EX9200 devices recieve 2 default routes from the PE routers. I want both the PE's to handle the internet traffic.
Our configuration did not have multipath loadbalancing enabled yet. This caused all traffic to be routed to one PE (TC5-PE01) because it has the lower router ID.
Last night i have enabled multipathing;
set routing-instances OUTSIDE-VRF routing-options multipath
I also tried to add the vpn-unequal-cost command but this did not change anything.
After this command, some traffic that follow the default route is bieng loadbalanced (unkown which and why) but most of the traffic still only has one active default route path. The next-hop router ID seems to ruin the party.
fyi: The other xxxMbits of traffic on the "empty connections" is shortest path EVPN traffic for a known /32 destination
I now have default routes with an @ (Routing Use Only), an > (Active Route) and an # (Forwarding use only)
username@NOS722-CORE01> show route table OUTSIDE-VRF.inet.0 0.0.0.0/0
@ = Routing Use Only, # = Forwarding Use Only
+ = Active Route, - = Last Active, * = Both
0.0.0.0/0 @[BGP/170] 21w0d 23:57:56, localpref 100, from x.x.254.2
AS path: I, validation-state: unverified
> to x.x.254.137 via ae1.0, Push 16
[BGP/170] 21w0d 23:56:01, localpref 100, from x.x.254.3
AS path: I, validation-state: unverified> to x.x.254.129 via ae5.0, Push 16
#[Multipath/255] 13:01:47, metric2 10> to x.x.254.137 via ae1.0, Push 16
to x.x.254.129 via ae5.0, Push 16I understand the function of the routing tables (routing& forwarding table) but i don't understand it's use/reason in the current scenario.
Why is the routing path only active with 1 path@BGP/170 and why is there now a "ForwardingOnly" multipath path with a cost of 255?
What is the difference between these two and why?
How can we make the multipath active for all traffic?
Thank you in advance for thinking with me!
Extra output:
username@MGW111-CORE01> show route table OUTSIDE-VRF.inet.0 0.0.0.0/0 extensive
0.0.0.0/0 (3 entries, 2 announced)
State: <CalcForwarding>
TSI:
KRT in-kernel 0.0.0.0/0 -> {indirect(1048703), indirect(1048695)}
Page 0 idx 1, (group internal-vssmgw-v4 type Internal) Type 1 val 0xb77f18c (adv_entry)
Advertised metrics:
Flags: Nexthop Change
Nexthop: Self
Localpref: 100
AS path: [1234] I
Communities: target:100:1009
Page 0 idx 2, (group HST-FW-v4 type Internal) Type 1 val 0xc15b2a4 (adv_entry)
Advertised metrics:
Flags: Nexthop Change
Nexthop: Self
Localpref: 100
AS path: [1234] I
Communities: target:100:1009
Path 0.0.0.0 from x.x.254.2 Vector len 4. Val: 1 2
@BGP Preference: 170/-101
Route Distinguisher: x.x.254.2:9
Next hop type: Indirect
Address: 0xa762f40
Next-hop reference count: 85
Source: x.x8.254.2
Next hop type: Router, Next hop index: 1672
Next hop: x.x.254.133 via ae5.0, selected
Label operation: Push 16
Label TTL action: prop-ttl
Load balance label: Label 16: None;
Session Id: 0x144
Protocol next hop: x.x8.254.2
Label operation: Push 16
Label TTL action: prop-ttl
Load balance label: Label 16: None;
Indirect next hop: 0xa7a4dd0 1048703 INH Session ID: 0x167
State: <Secondary Active Int Ext ProtectionCand>
Local AS: 1234 Peer AS: 1234
Age: 38w2d 14:02:15 Metric2: 10
Validation State: unverified
Task: BGP_1234.x.x.254.2+179
Announcement bits (3): 1-RT 2-BGP_RT_Background 5-Resolve tree 8
AS path: I
AS path: Recorded
Communities: target:100:1009
Import Accepted
VPN Label: 16
Localpref: 100
Router ID: x.x.254.2
Primary Routing Table bgp.l3vpn.0
Indirect next hops: 1
Protocol next hop: x.x.254.2 Metric: 10
Label operation: Push 16
Label TTL action: prop-ttl
Load balance label: Label 16: None;
Indirect next hop: 0xa7a4dd0 1048703 INH Session ID: 0x167
Indirect path forwarding next hops: 1
Next hop type: Router
Next hop: x.x.254.133 via ae5.0
Session Id: 0x144
x.x.254.2/32 Originating RIB: inet.0
Metric: 10 Node path count: 1
Forwarding nexthops: 1
Nexthop: x.x8.254.133 via ae5.0
BGP Preference: 170/-101
Route Distinguisher: x.x.254.3:10
Next hop type: Indirect
Address: 0x9daf530
Next-hop reference count: 31
Source: x.x.254.3
Next hop type: Router, Next hop index: 1963
Next hop: x.x.254.125 via ae1.0, selected
Label operation: Push 16
Label TTL action: prop-ttl
Load balance label: Label 16: None;
Session Id: 0x1c1
Protocol next hop: x.x.254.3
Label operation: Push 16
Label TTL action: prop-ttl
Load balance label: Label 16: None;
Indirect next hop: 0xa7a4660 1048695 INH Session ID: 0x166
State: <Secondary NotBest Int Ext ProtectionCand>
Inactive reason: Not Best in its group - Router ID
Local AS: 1234 Peer AS: 1234
Age: 25w6d 0:52:17 Metric2: 10
Validation State: unverified
Task: BGP_1234.x.x.254.3+179
AS path: I
AS path: Recorded
Communities: target:100:1009
Import Accepted
VPN Label: 16
Localpref: 100
Router ID: x.x.254.3
Primary Routing Table bgp.l3vpn.0
Indirect next hops: 1
Protocol next hop: x.x.254.3 Metric: 10
Label operation: Push 16
Label TTL action: prop-ttl
Load balance label: Label 16: None;
Indirect next hop: 0xa7a4660 1048695 INH Session ID: 0x166
Indirect path forwarding next hops: 1
Next hop type: Router
Next hop: x.x.254.125 via ae1.0
Session Id: 0x1c1
x.x.254.3/32 Originating RIB: inet.0
Metric: 10 Node path count: 1
Forwarding nexthops: 1
Nexthop: x.x.254.125 via ae1.0
#Multipath Preference: 255
Next hop type: Indirect
Address: 0x97b4254
Next-hop reference count: 2
Next hop type: Router, Next hop index: 1672
Next hop: x.x.254.133 via ae5.0, selected
Label operation: Push 16
Label TTL action: prop-ttl
Load balance label: Label 16: None;
Session Id: 0x144
Next hop type: Router, Next hop index: 1963
Next hop: x.x.254.125 via ae1.0
Label operation: Push 16
Label TTL action: prop-ttl
Load balance label: Label 16: None;
Session Id: 0x1c1
Protocol next hop: x.x.254.2
Label operation: Push 16
Label TTL action: prop-ttl
Load balance label: Label 16: None;
Indirect next hop: 0xa7a4dd0 1048703 INH Session ID: 0x167
Protocol next hop: x.x.254.3
Label operation: Push 16
Label TTL action: prop-ttl
Load balance label: Label 16: None;
Indirect next hop: 0xa7a4660 1048695 INH Session ID: 0x166
State: <ForwardingOnly Int Ext>
Inactive reason: Forwarding use only
Age: 13:06:16 Metric2: 10
Validation State: unverified
Task: RT
Announcement bits (1): 0-KRT
AS path: I
AS path: Recorded
Communities: target:100:1009Forwarding table:
username@MGW111-CORE01> show route forwarding-table table OUTSIDE-VRF extensive
Routing table: OUTSIDE-VRF.inet [Index 16]
Internet:
Destination: default
Route type: user
Route reference: 0 Route interface-index: 0
Multicast RPF nh index: 0
Flags: sent to PFE
Next-hop type: unilist Index: 1048902 Reference: 1
Next-hop type: indirect Index: 1048703 Reference: 21
Weight: 0x0
Nexthop: 5c:45:27:cb:b0:ca
Next-hop type: Push 16 Index: 1672 Reference: 2
Load Balance Label: None
Next-hop interface: ae5.0 Weight: 0x0
Next-hop type: indirect Index: 1048695 Reference: 10
Weight: 0x0
Nexthop: cc:e1:7f:ad:47:c1
Next-hop type: Push 16 Index: 1963 Reference: 2
Load Balance Label: None
Next-hop interface: ae1.0 Weight: 0x0
Destination: default
Route type: permanent
Route reference: 0 Route interface-index: 0
Multicast RPF nh index: 0
Flags: none
Next-hop type: reject Index: 669 Reference: 3