Configuring source address based routing on my Unifi USG

Updated 10/24/2018 since routing didn’t work anymore. You have to disable source-validation, thanks to Roelf for the comment with the correct command.

For some time now I wanted to be able to test some network stuff. I want to be able to connect certain devices over a VPN to the Netherlands but without the need to configure every client with VPN connections.

With this scenario it is possible to test different geo stuff accessing my network from different places in the world, it also helps me test the different latencies when going across the ocean and back. It also could be used to access certain video services in another country or access a different Netflix catalog, but I would never use it for something like that obviously Smile

After reading up on the different forums and asking some questions I was able to configure my USG in a way which gives me the most flexibility possible for my scenario. This is the step by step guide how to configure your USG and network so all your network on that special network will be routed over the VPN connection to the Netherlands.

The first step is to configure my ‘hoekstraonline NL’ network as described in this blogpost. Connecting through my ‘hoekstraonline NL’ wireless network and specific ports on my router (tagged with the same VLAN 100) will be the basis of my configuration. I want all that network going over the VPN connection to NL. All me regular traffic will go over my Comcast connection as usual but machines connected to that wireless network and specific ports on my routers will be routed over the VPN connection.

So lets create the VPN Client network first. Nowadays this can be done through the UI (I am running Unify version 5.6.20 stable candidate when I am writing this)

VPNClient

After you create this network you can check on your USG how the routing table looks like. It should have added the VPN NL network. Enter the following command on your USG (via SSH):

ubnt@USG:~$ netstat -r

My routing table looks like this:

netstat2USG

the pptpc0 interface is the VPN connection I just defined, you can see from the flags the connection is up (U). The eth1.100 is the virtual network which was added in the previous blogpost.

The next step is to change the routing depending on the source address. Unfortunately this can’t be done through the GUI from Unifi. They add more and more functionality every month, but this has to be done through the command line. so fire up your bash shell or putty and connect to your firewall (USG in my case).

In the shell type; configure

ubnt@USG:~$ configure

[edit]

We have to define a new routing table we call table 1 which will route traffic to my VPN connection on the 10.0.0.0/24 network.

ubnt@USG# set protocols static table 1 route 0.0.0.0/0 next-hop 10.0.0.1

[edit]

Now we have to define the modify policy. A modify policy allows us to modify various items when the rule matches. So if the source address came from 192.168.2.0, then we want to use routing table 1:

ubnt@RTR# set firewall modify SOURCE_ROUTE rule 10 description ‘traffic from eth1.100 to VPN NL’

ubnt@RTR# set firewall modify SOURCE_ROUTE rule 10 source address 192.168.2.0/24

ubnt@RTR# set firewall modify SOURCE_ROUTE rule 10 modify table 1

Now we need to apply this policy to the interface. When it comes to applying a policy to an interface, it needs to be done on the input interface before the routing lookup takes place.

ubnt@USG# set interfaces ethernet eth1 vif 100 firewall in modify SOURCE_ROUTE

A last step which you need to add (this changed so this step was added 10/24/2018) is to disable source validation (thanks to Roelf for the comment and help)

ubnt@USG# set firewall source-validation disable

[edit]

After this you can give the commit and save command and you can test your network routing. From a client in the 192.168.1.x range nothing should be different. But when you test it from a 192.168.2.x client you see the traceroute change to the 10.0.0.1 hop and than off to the Netherlands.

The first tracert is from a machine in the 192.168.1.x range. You see the first hop is my USG gateway and than it goes out to the internet.

The second tracert is from the machine when it’s in the 192.168.2.x range. You see the second hop goes through the 10.0.0.0 VPN gateway and you also see the response times go up since it’s traveling the ocean now.

Mission accomplished!.

The last step is to add these settings to the provisioning script stored on my cloudkey, so when I reset the USG the settings won’t be lost.

One of the sources I used to write this article.

Comments

Comment by Phil on 2017-10-29 23:22:54 -0800

great guide, thank you

but one question: “The last step is to add these settings to the provisioning script stored on my cloudkey, so when I reset the USG the settings won’t be lost.”

could you please explain that too?

Comment by Matthijs Hoekstra on 2017-10-30 08:30:22 -0800

I will in the next post, I still have to figure out what exactly to copy to the gateway json file on my cloud key to make sure it works 100%. Hopefully somewhere this week/

Comment by Phil on 2017-10-31 02:59:29 -0800

Matth, you’re my fracking hero!

my goal was slightly different, as I wanted to route traffic through WAN2 based on the source address 192.168.1.189/24 (in m case a VLAN ID 189). but now it is working…damn, that is great! 🙂

one tiny little problem now occurred: I cant reach the client in the new subnet/vlan 189 from my untagged subnet/vlan 192.168.1.1/24.

I guess there is something missing….

Comment by Phil on 2017-10-31 08:48:26 -0800

ah…nevermind.

I took one step forward, and two back:

in my “test-setup” it worked fine, but the target-router had an static IP.

actually I wanna force specific clients to use my WAN2, which is a PPPoE connection. and there seems to be the problem, since I can’t address it with a static rule (or I haven#t figured it out yet).

Comment by Matthijs Hoekstra on 2017-11-11 15:59:47 -0800

No clue either 🙂 the folks on the unifi forum where very helpful and most of the info I posted I gathered from different posts over there.

Comment by Adrian on 2017-12-13 23:35:13 -0800

Hi Matthijs,

What if I don’t want to route trough VPN, but trough an another IP in the local subnet?

Example: 192.168.1.50 traffic –> to –> 192.168.1.60

Comment by Matthijs Hoekstra on 2017-12-23 16:59:40 -0800

you have to route to another subnet and just add the route to the routing table (like I did the next hop to 10.0.0.0 that happened to be the VPN network, but doesn’t have to be. In your example it depends on your subnet mask. If that’s 255.255.255.0 the 2nd ip is not in another subnet.

Comment by Mike on 2017-12-30 08:49:34 -0800

Hi, I followed your tips on 5.6.26 and unfortunately the router didn’t accept the changes.

It failed with the following:

Router configuration commit error. Error message: { “COMMIT” : { “error” : “\ufffe[ firewall modify SOURCE_ROUTE ]\nError: [sudo /sbin/iptables-restore -n -v 2> /tmp/iptables.out] = 256\nIptables restore OK\n\n\uffff0\nCommit failed\n” , “failure” : “1” , “success” : “1”} , “DELETE” : { “failure” : “0” , “success” : “1”} , “SESSION_ID” : “5eb3818f81edab21ff85dd0c08” , “SET” : { “failure” : “0” , “success” : “1”}}

Do you have any ideas?

Comment by Matthijs Hoekstra on 2018-01-02 07:37:23 -0800

I would try to configure line by line. So first try to set the table and commit. If that works, set the modify rule in your firewall (3 lines) and commit etc.

Are the ip ranges the same for you? If not, did you change accordingly?

Comment by Gerardo Martinez on 2018-01-11 11:55:13 -0800

Below is the config and the error message. Any ideas why I get the error message?

set protocols static table 2 route 0.0.0.0/0 next-hop 192.168.4.254

set firewall modify SOURCE_ROUTE rule 30 description ‘traffic from eth1 vif 10 to WLAN_2’

set firewall modify SOURCE_ROUTE rule 30 source address 192.168.5.0/24

set firewall modify SOURCE_ROUTE rule 30 modify table 2

set interface ethernet eth1 vif 10 firewall in modify SOURCE_ROUTE

commit

==================================================

[ firewall modify SOURCE_ROUTE ]

Error: [sudo /sbin/iptables-restore -n -v 2> /tmp/iptables.out] = 512

Iptables restore OK

[ interfaces ethernet eth1 vif 10 firewall in modify SOURCE ROUTE ]

Firewall config error: Rule set SOURCE_ROUTE is not configured

Commit failed

Comment by Matthijs Hoekstra on 2018-01-11 21:26:12 -0800

what happens if you do a commit after every line? you could also try vlan 100 instead of using 10?

Comment by Mike on 2018-01-15 12:26:25 -0800

Got it working. No idea why it failed first time.

Keep posting, you’re doing good job sharing the know-how.

Comment by Matthijs Hoekstra on 2018-01-18 07:55:27 -0800

that’s great to hear! Thank you.

Comment by Rafal Sydor on 2018-07-31 13:28:29 -0800

hi, could this be configure with site to site VPN? I have tried and so far I am not getting any luck. I have tried two static route configurations (target GW and vti64 which is site to site VPN tunnel) but my traffic from 192.168.11.0/24 does not get routed to VPN tunnel. Any tips would be much appreciated.

set protocols static table 1 route 0.0.0.0/0 next-hop 172.16.0.1

set protocols static table 1 interface-route 0.0.0.0/0 next-hop-interface vti64 distance 5

set firewall modify SOURCE_ROUTE rule 100 description “Traffic from Ancaster VLAN 4 to Frisco DMZ”

set firewall modify SOURCE_ROUTE rule 100 source address 192.168.11.0/24

set firewall modify SOURCE_ROUTE rule 100 modify table 1

set interfaces ethernet eth1 vif 4 firewall in modify SOURCE_ROUTE

Comment by Karl on 2018-08-15 04:36:44 -0800

Rafal, Did you ever get this working?

Comment by Tomasz on 2018-08-24 19:14:41 -0800

I know it’s kind of an “old” post but I have a question. I followed the instruction until checking the routing table via SSH. When I do that I can see the newly created network 192.168.2.0 but I don’t see the VPN connection (pptpc0). I checked the VPN network configuration a few times. I even deleted it and recreated a few times. Still the same. What am I missing/doing wrong?

Thanks

Comment by Roelf on 2018-10-23 09:32:03 -0800

In some cases (especially the later firmware versions), you need to disable the source-validation in the firewall. Else you will have a connection, you can ping to any endpoint over the VPN, but full traffic will not be allowed. If that happens ,execute : set firewall source-validation disabled

or in JSON:

“firewall”: {

“modify”: {

….},

“source-validation”: “disable”

},

Comment by Roelf on 2018-10-23 09:33:40 -0800

try to execute:

set firewall source-validation disable

Comment by Roelf on 2018-10-23 09:42:09 -0800

when you are in the device, run cat /var/log/messages and check the logfiles for any indication why the VPN would not come up.. even when connected it doesnt show by default anymore in the netstat -r. That is why putting in an interface-route is better than a static route to another gw. .

Comment by Matthijs Hoekstra on 2018-10-24 20:20:23 -0800

Awesome tip, this solved my issue which suddenly happened (traffic didn’t route anymore)

Comment by Lucas Janin on 2018-11-03 14:57:28 -0800

Thanks for this tutorial!

A beginner question: It’s possible just configure to use VPN only for spetific IP?

Thanks

Comment by Steve on 2019-01-21 15:32:13 -0800

I have this same issue using site-to-site VPN. Would like to route all of one VLAN through the site-to-site VPN. I’ve followed all the guides and haven’t gotten it to work. Any ideas would be great.

Comment by Matt on 2019-08-11 18:30:24 -0800

Matthijs, thanks for writing this great blog. I was wondering, if it’s also possible to do port-based routing rules (instead of IP-based). This way I could for example set all port 80 traffic through the VPN, but my game ports (i.e. 27000-27015) not through VPN.

Comment by Matthijs Hoekstra on 2019-08-11 21:43:10 -0800

I am not sure. Not an expert, but port forwarding based on ip address is probably what you are looking for. No idea how to do that for this.

Comment by Matt on 2019-08-11 21:56:06 -0800

Ok well thanks for the reply. I appreciate that.

Comment by Andreas on 2019-09-08 12:11:02 -0800

Hello Matthijs and thank you for this post! A have been looking for a way of doing this.

But I so have a site to site VPN. This is my Routing table:

Destination Gateway Genmask Flags MSS Window irtt Iface

default xxx.250.145.254 0.0.0.0 UG 0 0 0 eth0.832

10.10.10.0 * 255.255.255.0 U 0 0 0 eth1.10

10.255.254.1 * 255.255.255.255 UH 0 0 0 vti0

xxx.250.144.0 * 255.255.254.0 U 0 0 0 eth0.832

loopback * 255.0.0.0 U 0 0 0 lo

192.168.22.0 * 255.255.255.0 U 0 0 0 vti0

192.168.48.0 * 255.255.255.0 U 0 0 0 eth1

When connected to the Wifi I can ping the remote GW on 192.168.22.1

Then I set the rules:

set protocols static table 1 interface-route 0.0.0.0/0 next-hop-interface vti0 distance 5

set firewall modify SOURCE_ROUTE rule 10 description ‘traffic from eth1.10 over site-to-site-VPN’

set firewall modify SOURCE_ROUTE rule 10 source address 10.10.10.0/24

set firewall modify SOURCE_ROUTE rule 10 modify table 1

set interfaces ethernet eth1 vif 10 firewall in modify SOURCE_ROUTE

set firewall source-validation disable

commit ; save ; exit

After I have done that, I cannot ping remote GW on 192.168.22.1, and all traffic is still not routed over VPN.

What am I doing wrong?

Comment by Bryan on 2019-11-25 20:57:05 -0800

Do you happen to have the syntax for the controller config.gateway.json file? When i commit these changes, my USG seems to reprovision and the changes are lost

Comment by neil on 2020-01-10 09:30:55 -0800

Bryan, this took a while for me to get right as well, ended up using the filemerge app to spot the differences between the before and after dumped files.

My config looks like the one below – 10.1.50.0 is my alternate wifi and 10.0.25.1 is my remote network.

{

“firewall”: {

“modify”: {

“SOURCE_ROUTE”: {

“rule”: {

“10”: {

“action”: “modify”,

“description”: “traffic from eth1.100 to vpn”,

“modify”: {

“table”: “1”

},

“source”: {

“address”: “10.1.50.0/24”

}

}

}

}

},

“source-validation”: “disable”

},

“interfaces”: {

“ethernet”: {

“eth1”: {

“vif”: {

“100”: {

“firewall”: {

“in”: {

“modify”: “SOURCE_ROUTE”

}

}

}

}

}

}

},

“protocols”: {

“static”: {

“table”: {

“1”: {

“route”: {

“0.0.0.0/0”: {

“next-hop”: {

“10.0.25.1”: “””

}

}

}

}

}

}

}

}

Comment by Adrian on 2020-04-19 09:30:39 -0800

Hello, I found your link looking for a solution for what I am trying to do with USG routers.

I’ve posted on the Unifi forum but can’t get a response. The truth is that I would need help because I am not an expert in this.

My question is if I could forward all the incoming traffic from my company where I have 3 USG routers installed and each one of them has a public IP in their WAN and the LAN1 of each one of them are within the same LAN 198.96.90.0/24 , manage to forward all the incoming traffic to another public IP address that is in another geographical place, where I have a USG Pro 4 router.

The problem is generated because the public IPs do not transfer to the new site and I receive traffic for these 3 public IPs.

The LAN2 of each USG is free, but I think it could create a VPN from these 3 USGs to the USG Pro 4 and somehow forward all incoming traffic to the VPN.

Would this be possible according to your knowledge?

Link of my query in Spanish in the Ubuquiti forum:

https://forum-es.ui.com/discussion/1505707/usg-reenvio-de-trafico-traffic-forwarding

Very grateful for your help.

Comment by Matthijs Hoekstra on 2020-04-19 12:09:15 -0800

I wish I could help, I have no clue. I am not a network specialist, I just describe what I got to work and hope it helps 🙂