Uncloaking DMVPN

Many of you may have heard about the wonders of Dynamic Multi point Virtual Private Network (DMVPN), or seen it work somewhere on a Marine Corps networks.

Well today, I’m going to try to demystify the cloud of mystery surrounding DMVPN.

Obviously, we need to start out with that we would like to accomplish

  1. Simplify Hub configuration, by providing a single tunnel interface which multiple spokes are terminated on.
  2. Provide Direct Spoke-to-Spoke communication, with minimal interaction to the hub (we’ll talk about this…)

Let’s get on to how our lab is going to be setup, for this lab I am using GNS3 and using IOSv version 15.6.2T:

  1. 1 Hub Router
  2. 1 Router, emulating TDMA Cloud (imagine this as Linkway modems running RIP, not trying to get too fancy here, just trying to visualize this for you guys)
  3. 3 Spoke routers
  4. All routers source interface will have dedicated VRF-Lite

Here’s a rough diagram:

 

During this discussion, we’re going to talk about the following protocols and technologies:

  1. Next-Hop Resolution Protocol (NHRP)
  2. Virtual Route/Forwarding Instance (VRF, specifically VRF-Lite/FVRF)
  3. Generic Routing Encapsulation (GRE)
  4. Enhanced Interior Gateway Routing Protocol (EIGRP)

GRE

GRE, or generic routing encapsulation is the simplest form of tunneling we can utilize. We should all already be relatively familiar with it, however, if you’re not, we’ll talk about it real quick.

GRE is a form of encapsulation, it will take an original IPv4 packet, and encapsulate it within a new packet. This new packet will have 2 additional header fields, while still maintaining the original IP Header, as well as the payload.

The new fields we will see are the IP Header (outside) and the GRE Header. The IP Header outside includes the source and destination IP address, not to be confused though. This field is specific to the underlay, or publicly routable address, whereas your inner IP header is going to consist of those source and destination private addresses. You might often hear this specific outside IP header referred to as the “Delivery Header”. After this, you have the GRE header, which will include information about the specific protocols being used by the original packet.

I won’t go too much further into depth with that, quite frankly because it’s not needed at all for this discussion, just know that GRE will be adding an additional 24 Bytes to every packet transmitted. If you’re interested in getting into the nitty gritty of GRE, here’s a good link from the Cisco Learning Network:

https://learningnetwork.cisco.com/blogs/vip-perspectives/2017/03/14/anatomy-of-gre-tunnels

NHRP

NHRP, or next-hop resolution protocol is the technology used by DMVPN. NHRPs sole purpose in life is to associate public (referred to as NBMA, or non-broadcast multi-access) addresses, to the associated private address of the tunnel.

NHRP will consist of a Next-Hop Server (NHS), which maintains the copy of the NHRP cache database, and Next Hop Clients (NHC), which will dynamically register with the NHS.

Phases of DMVPN

Phase 1:

During phase 1 of DMVPN, the only member of the DMVPN cloud with a multi-point tunnel will be the hub. The downside to this is that there is no ability for direct spoke-to-spoke communication, however, the simplicity of adding new spokes to the cloud is a plus. Realistically, adding a new spoke to phase requires two additional commands, we would just need to specify our NHRP network ID and NHS. The rest would look like a normal GRE tunnel (with source and destination).

Phase 2:

During phase 2 of DMVPN, all members are multi-point tunnels. This allows for direct spoke-to-spoke communication, there are some catches with this:

  1. We cannot summarize, from the hub, to spokes.
  2. We cannot propagate a default route.

For those two reasons, I’m not going to touch on Phase 2, because it’s just not a feasible (or Cisco recommended) solution. If you were to summarize, or propagate a default route where the specific subnets for each spokes next-hop are not in each spokes routing table, then this would generate Phase 1 behavior.

Phase 3:

Phase 3 DMVPN is most recent, and Cisco recommended implementation. During phase 3, there is spoke-to-spoke connectivity, and you are able to summarize and propagate default routes. The key to this is the NHRP redirect message.

As traffic comes from one spoke, destined for another, the hub router will notice that the destination is in it’s NHRP cache (or if backend network, will notice that the next-hop is in NHRP cache) and generate an NHRP redirect message. This NHRP redirect message will contain all the information the spoke needs to generate a dynamic (and temporary) tunnel to the other spoke. Once this happens, with NHRP shortcuts enabled, the spoke router will re-write the CEF for all routes where the spoke is the next-hop, allowing traffic for backend network to flow directly to the spoke, without the hub being involved whatsoever.

Phase 3 is the implementation of DMVPN that we will be focusing on.

Here is a diagram of what phase 3 DMVPN would look like. The solid lines define the tunnels which would always persist, and the dashed lines define the tunnels which would be created, dynamically, as needed.

And lastly, for those of you (like me) who like to get into the weeds of things, checkout RFC 2332, which defines NHRP:

https://tools.ietf.org/html/rfc2332

VRF

VRFs, or Virtual Route/Forward Instance (sometimes referred to as VPN Route/Forward Instance), are similar to VLANs. As we know, VLANs can segregate a physical switch into multiple different broadcast domains. VRFs may not seperate things into their own broadcast domain, but they do seperate them into their own routing domain.

Using a VRF is essentially having a separate routing table. (key concept)

On top of having it’s own routing table, each VRF will have it’s own RIB (routing information base), and when CEF is utilized, it’s own CEF table.

Because VRFs are essentially a router within a router, you can have multiple interfaces with the same IP addresses and routing processes which would otherwise interfere. Inter-VRF communication can be configured, but is not a default whatsoever.

For our purpose, we will use a simple Front-Door VRF which will provide a default static route pointing towards what would (in real life anyway) be our Linkway modem. In real life we would need to ensure that we create full-mesh PVCs on our MRT, as well as enable RIP on all the sites.

For those interested in learning more, here is a link to Cisco’s VRF Design guide.

https://www.cisco.com/c/en/us/td/docs/voice_ip_comm/cucme/vrf/design/guide/vrfDesignGuide.html

 

EIGRP

We all know this one, and we all know exactly why I’m covering it. For that reason, I won’t go to into it. Just know that for DMVPN, we must disable split-horizon for EIGRP to work properly.

Step 1 – Interface Addressing, and VRF Routing

First and foremost we must define our VRF. We will do this before we assign our source interface  an IP address. This is because once you remove an interface from the global routing table,  and assign it to a VRF, the router will remove the IP address from the interface. To create a VRF, follow the steps below
Router(config)#ip vrf [name]
Router(config-vrf)#description [string]

Now that you have created the VRF, you can assign your source interface to the VRF, to do this, enter interface configuration mode and type the following
Router(config-if)#ip vrf forwarding [vrf-name]
After doing this, you can now procede to configure your IP address.
Once you have your interface configured, and assigned to the VRF you created, you can now create a static default route using the following command.
ip route vrf [vrf-name] 0.0.0.0 0.0.0.0 [next-hop-ip] name [name-this-route]
Note: if you were on a multi-access network (not point-to-point) it would be essential that you specify the next-hop IP address, instead of the exit-interface. I follow this standard altogether, and when I’m feeling frisky I’ll even fully specify it!

Pro-tip: Name your routes so when the other guy, who doesn’t know what you did, does a show run, he might be able to figure out what that route is for, and not delete it!

Now that you’ve created the route, you can verify it by viewing that VRF’s routing table
Router#show ip route name [vrf-name]

Note: anytime you need to ping, or traceroute something which utilizes this VRF, you must use the VRF keyword in your command.

Router#ping vrf [vrf-name] [destination]
Router#traceroute vrf [vrf-name] [destination]

Here’s what my Hub config looks like at this point in time (anything that is default has been redacted) this is what your config should look like for every router in the topology so far:
!
ip vrf TDMA
description FVRF for TDMA Cloud (All TDMA Modems)
!
interface GigabitEthernet0/0
ip vrf forwarding TDMA
ip address 10.10.10.2 255.255.255.252
duplex auto
speed auto
media-type rj45
!
ip route vrf TDMA 0.0.0.0 0.0.0.0 10.10.10.1 name TDMA-Modem-Default-Route
!

Here’s the output of both my global and VRF routing table, so far:

Global

VRF

Step 2: Creating the Tunnel (Hub)

Before we continue, we should probably discuss our network ID, for all intensive purposes, we’re going to use 1775 as our network ID… To be completely transparent, the NHRP network-ID is locally significant, and you could definitely have every router in the topology with a different network ID, it would still process the NHRP associations.

Something that is significant to all, and must match on all is the tunnel key, this is optional and used for multiplexing multiple DMVPN clouds over the same source interface (there are some instances where you might want to separate DMVPN clouds).

For our tunnel key, you guessed it, 1775.

Our tunnel network is going to be the 172.16.1.0/24 network, our NHS (the hub) will be 172.16.1.1

Something to note is that since we are utilizing the VRF, we will have to do something just a little different when we create our tunnel

The first thing you should do is create the tunnel (quite obvious here), but once you do this, before you do anything else, be sure to inform the tunnel that it will be using a VRF to route all destination traffic (not to route actual payload traffic).
Router(config-if)#tunnel vrf [vrf-name]
Next up, let’s define our tunnel mode as “gre multipoint”
Router(config-if)#tunnel mode gre multipoint
Now that you have done that, define your source, and assign your IP address, the interface at this point will show Up/Up (or atleast should…)
Next, we need to specify our NHRP commands, there’s only a few we need to enter:
ip nhrp network-id [id]

ip nhrp map multicast dynamic


ip nhrp redirect


The first command defines our NHRP domain, locally significant, similar to a process ID for OSPF.
Next, we tell the tunnel that it will listen for registration messages and dynamically map that “ip nhrp map multicast dynamic”
Lastly, we input the redirects command. This will tell the router that when it receives traffic from one spoke, destined for another spoke, that it needs to generate that NHRP redirect message and inform the spoke that it can create a spoke-to-spoke tunnel and leave the hub out of the communication.

Something you could do, and is extremely easy to implement, is NHRP authentication. Note that NHRP authentication is sent in clear-text and really just utilized to prevent unauthorized devices from entering your DMVPN cloud
ip nhrp authentication [string]

At this point, you should be done configuring the hub tunnel, there are obviously some more important things we would need to get into, such as timers, but since we’re just labbing it out, I’ll let you do the research for all that, as everyone will need different registration timers based on situation.

Now I’m going to enable routing, my EIGRP ASN is going to be 1775, and I’m going to propagate a summary default route to all of my spokes. This is generally what the start of my EIGRP configuration looks like.
Router(config)#router eigrp [asn]
Router(config-rtr)#network [network] [wildcard]
Router(config-rtr)#no auto-summary
Router(config-rtr)#passive-interface default
Router(config-rtr)#no passive-interface [interface-name]

And now, to disable split-horizon, and add that summary default route.
Router(config-if)#no ip split-horizon eigrp [asn]

Router(config-if)#ip summary-address eigrp [asn] 0.0.0.0 0.0.0.0

Here’s what my hub configuration looks like at the moment:
!
router eigrp 1775
network 172.16.1.0 0.0.0.255
passive-interface default
no passive-interface Tunnel1
!
interface Tunnel1
ip address 172.16.1.1 255.255.255.0
no ip redirects
no ip split-horizon eigrp 1775
ip nhrp map multicast dynamic
ip nhrp network-id 1775
ip nhrp redirect
ip mtu 1400
ip mtu 1368
ip summary-address eigrp 1775 0.0.0.0 0.0.0.0
tunnel source GigabitEthernet0/0
tunnel mode gre multipoint
tunnel vrf TDMA
!

Something I didn’t touch here is MTU and MSS, but since we know we’re adding 20 Bytes to the packet, we know that we need to adjust, I put it at 1400 on mine, because I do intend to show you guys (at the very end) how to implement IPSEC.

Step 3: The Spokes!

Configuring the spokes is relatively easy, knowing what we know, and given that we already created the VRF and assigned the source interface to it, we should be all set to start seeing some associations pop-up.

First, I want to start by stating that there are two additional commands we will need to enter. This is something that I’ve had to do only when the spokes are utilizing an FVRF, if you don’t do it, the hub will never form an adjacency with the spoke (the spoke will form an adjacency, because it received the hellos, but the hub has not received any hellos, although they are being sent, bizarre, I’ll have to post some PCAPs sometime…Update: interestingly enough, running “debug nhrp”, the router is having an issue with resolving the NBMA address, because it cant find it in the global RIB, sort of what I figured to begin with).
Router(config-if)#ip nhrp map mutlicast [hub-nbma-address]
Router(config-if)#ip nhrp map multicast dynamic
Router(config-if)#ip nhrp map [hub-nbma-ip] [hub-tunnel-ip]
Router(config-if)#ip nhrp nhs [nhs-ip]
Router(config-if)#ip nhrp network-id [network-id]
Router(config-if)#ip nhrp shortcut

The first command will tell the router where to send multicast packets to (important for EIGRP).
The second command will inform the router that it can dynamically associate itself with other routers.
The third will create a static NHRP associate with the hub, something that when using an FVRF, I found necessary, although it could be a bug in the IOS I’m using.
The fouth command will tell the router who the NHS is, the NHS is the next-hop server and is the router which maintains the NHRP cache database (the hub).
The fifth command, we already know.
The size command informs the router that it can expect NHRP redirect messages, this is what truly makes your configuration phase 3. This will allow the router to accept those NHRP redirect messages and generate a spoke-to-spoke tunnel, as needed.

After you configure all that, make sure that we configured it similar (obviously not identical) to the hub. Here is the output of my Spoke 1 Router configuration:

!

interface Tunnel1


ip address 172.16.1.2 255.255.255.0


no ip redirects


ip mtu 1400


no ip split-horizon eigrp 1775


ip nhrp map multicast dynamic


ip nhrp map 172.16.1.1 10.10.10.2


ip nhrp map multicast 10.10.10.2


ip nhrp network-id 1775


ip nhrp nhs 172.16.1.1


ip nhrp shortcut


ip tcp adjust-mss 1360


tunnel source GigabitEthernet0/1


tunnel mode gre multipoint


tunnel vrf TDMA


!


After completing the first spoke, go to the hub and just verify real quick that the spoke is registered, you can do this with the following commands:
Router#show ip nhrp
Router#show dmvpn

Here is the output of the hub:

An important note is that we should not rely on the spoke output for the “show ip nhrp” command, as we have statically created that association to the hub, and even when it’s not working correctly, the association will still show. You should also verify your EIGRP neighbor table, and topology table (important to make sure routes are actually being exchanged, if not you might have the situation I talked about earlier).

Step 4: Let’s Generate those Spoke-Spoke tunnels

This part is extremely simple; all I’m going to do is show you how that tunnel gets created. For this I’m just going to ping spoke 2’s tunnel interface from spoke 1, capture the traffic in Wireshark, and do some traceroutes…
As you can see from the traceroute, traffic originally traveled through the hub, but after a few packets (after the hub sent the NHRP redirect message) traffic was sent directly to the spoke. You can also see that the router has the other spoke in it’s NHRP cache. Please take note that this tunnel states “Dynamic”, this means that the information was retrieved from the NHS.

How bout that PCAP, here’s what that NHRP redirect looks like when running wireshark! Although Wireshark does not refer to it as an “NHRP Redirect”, it does refer to it as an “NHRP Traffic Indication”.

Verification and troubleshooting commands

Router#show ip nhrp
Router#show dmvpn
Router#show ip cef
Router#show adjacency
Router#show ip route [vrf vrf-name]
Router#show ip eigrp neighbors
Router#show ip eigrp topology
Router#show ip protocols
Router#debug ip eigrp
Router#debug nhrp

 

Something I failed to mention earlier in the post is the actual underlying difference between Phase 2 and Phase 3 DMVPN. During phase 2, when a spoke would communicate to a spoke, it would generate an NHRP Resolution Request, directly to the spoke, the spokes would initiate their tunnels. During phase 3, however, the spoke is not yet aware that there is another spoke, he only knows of the hub. When traffic is sent destined for another spoke, the hub recognizes this and sends and NHRP redirect message, which contains the NBMA and private addresses required for those spokes to communicate. Phase 3, the hub would initiate the spoke-to-spoke tunnels.

 

 

Posted in RoutingTagged , , , , , , ,

Leave a Reply

Your email address will not be published. Required fields are marked *