BGP in Data Center Fabrics
Articles » Technology Resources » BGP in Data Center Fabrics
You must use BGP as the endpoint reachability routing protocol in your data center fabric if you decide to run VXLAN with EVPN control plane... but should you also run it as the transport (underlay) routing protocol instead of OSPF or IS-IS? The resources collected on this page might help you make that decision.
We covered this topic in these webinars, articles, and podcasts:
- Leaf-and-Spine Fabric Architectures
- EVPN Technical Deep Dive
- Using BGP in a Data Center Leaf-and-Spine Fabric (free article)
- BGP Navel Gazing (Software Gone Wild episode 112)
- Can We Trust Routing Protocols (The Hedge episode 43)
- Cool or Hot? Lapukhov + Nkposong’s BGP SDN (Packet Pushers episode 164)
Petr Lapukhov (the author of BGP-as-better-IGP idea) initially proposed to use BGP as a data center SDN mechanism. I covered that use case in these blog posts and webinars:
- BGP-Based SDN Solutions
- The Four Paths to SDN (2014)
- Exception Routing with BGP: SDN Done Right (2013)
We also tried to answer the question "Do we need a new routing protocol for data center fabrics?" in these Software Gone Wild episodes and related blog posts (TL&DL: No):
- Data Center Routing with RIFT with Dr. Tony Przygienda (2018)
- OpenFabric with Russ White (2018)
- Is BGP Good Enough with Dinesh Dutt (2018)
- Is EBGP Really Better than OSPF in Leaf-and-Spine Fabrics? (2018)
- Is OSPF or IS-IS Good Enough for My Data Center? (2018)
- BGP as a Better IGP? When and Where? (2017)
- BGP or OSPF? Does Topology Visibility Matter? (2016)
- Questions about BGP in the Data Center (with a Whiff of SRv6) (2021)
I covered the design aspects of using BGP in data centers (in particular in combination with EVPN) in these blog posts:
- Using BGP for Firewall High Availability: Design and Software Upgrades (2019)
- Tuning BGP Convergence in High-Availability Firewall Cluster Design (2019)
- Typical EVPN BGP Routing Designs (2018)
- BGP in EVPN-Based Data Center Fabrics - follow-up to the Data Center BGP article (2018)
- BGP AS Numbers on MLAG Members (2020)
- Improving BGP Convergence without Tweaking BGP Timers (2017)
- To BFD or not to BFD? (2017)
- Avoid Summarization in Leaf-and-Spine Fabrics (2018)
- Leaf-and-Spine Fabric Myths (Part 2) (2018)
- When EVPN EBGP Session between Loopbacks Makes Sense (2020)
- EVPN: The Great Unifying Theory of VPN Control Planes? (2020)
- EVPN Control Plane in Infrastructure Cloud Networking (2020)
It's also popular to run BGP on redundantly connected servers, or on edge appliances connecting overlay virtual networks with physical world. More details in these blog posts and in the Routing on Servers part of Leaf-and-Spine Fabric Architectures
- Running BGP on Servers (2016)
- Running BGP between Virtual Machine and ToR Switch (2016)
- Why Would I Use BGP and not OSPF between Servers and the Network? (2016)
- Don’t Run OSPF with Your Customers (2016)
- Sysadmins Shouldn’t Be Involved with Routing (2016)
- Host-to-Network Multihoming Kludges (2016)
The designers of Cumulus Linux preferred the EBGP-only data center design, and added numerous features to their BGP routing daemon (now FRRouting). We covered those features in these blog posts and webinars:
- FRRouting Architecture and Features
- Cumulus Linux deep dive
- BGP Configuration Made Simple with Cumulus Linux (2015)
- Auto-MLAG and Auto-BGP in Cumulus Linux (2019)
Some vendors love making your life overly complex, inventing ridiculous architectures like running IBGP (EVPN) over EBGP (IP routing). Here are a few choice blog posts on that topic:
- Is BGP Really that Complex? (2016)
- Dissecting IBGP+EBGP Junos Configuration (2018)
- Update: Automatic EVPN Route Targets in EBGP Environments (2018)
- The EVPN/BGP Saga Continues (2020)
- Networking, Engineering and Safety (2020)
Two other interesting topics often pop up in BGP-related discussions: anycast and multipathing:
- Local TCP Anycast Is Really Hard
- Using Unequal-Cost Multipath to Cope with Leaf-and-Spine Fabric Failures
- Unequal-Cost Multipath with BGP DMZ Link Bandwidth
You might also find these blog posts somewhat relevant to your data center BGP designs:
Finally a few more abstract blog posts to tickle your gray cells:
- Valley-Free Routing (2018)
- Valley-Free Routing in Data Center Fabrics (2018)
- Implications of Valley-Free Routing in Data Center Fabrics (2018)
- Can We Trust BGP Next Hops (Part 1)? (2020)
- Response: Next-Hop and VTEP Reachability in EVPN Networks (2020)
- Can We Trust BGP Next Hops (Part 2)? (2020)
- BGP: the Tragedy of the Commons (2017)
- Routing Protocols: a Perfect Example of RFC 1925 Rule 5 (2017)