WAN Bonding with ROOter and OpenMPTCProuter

How To Tutorials related to Routers and Firmware
User avatar
bnhf2
Posts: 37
Joined: Sun Aug 05, 2018 10:03 am
Has thanked: 0
Been thanked: 21 times

Re: WAN Bonding with ROOter and OpenMPTCProuter

Post by bnhf2 » Wed Jul 29, 2020 9:02 am

ciarlill wrote:
Wed Jul 29, 2020 7:56 am
In the meantime I noticed something strange... I wanted to disable WiFi on the ROOter device to cut down on extraneous networks floating around. Although I can access the configuration page at 192.168.3.1 I cannot actually save any configurations, it just times out and appears to be redirecting me back to 192.168.1.1. It works fine if I plug directly in and access on 192.168.1.1. Wondering if you have run into this and have any ideas on anything I might have missed in setup. It seems like one of the additional benefits of this approach is actually being able reach all the devices on their own VLANs for additional configuration as necessary.
Problems with not being able to save changes is usually related to a corrupt SD Card filesystem (which can happen easily on an RPi). Do you see "Unsaved Changes" in the upper right corner? If so, more recent RPi versions of ROOter include a filesystem check and repair menu item. Run that and reboot.

Whether that turns out to be the issue or not, be sure to always do a proper shutdown from the ROOter menu system. Pulling the power cord on an RPi actively running any version of OpenWRT, very often results in the need to do a filesystem repair.

If that's not the culprit, I'd suggest confirming your new VLAN is part of the LAN firewall group. This can be easily verified by checking the header colors under Network - Interfaces. Those that are part of the same firewall group will have the same color header. Mine is green for the LAN group and pink for WAN.

ciarlill
Posts: 27
Joined: Wed Jul 15, 2020 11:45 am
Has thanked: 4 times
Been thanked: 0

Re: WAN Bonding with ROOter and OpenMPTCProuter

Post by ciarlill » Tue Aug 04, 2020 8:19 am

bnhf2 wrote:
Wed Jul 29, 2020 9:02 am
Problems with not being able to save changes is usually related to a corrupt SD Card filesystem (which can happen easily on an RPi). Do you see "Unsaved Changes" in the upper right corner? If so, more recent RPi versions of ROOter include a filesystem check and repair menu item. Run that and reboot.

Whether that turns out to be the issue or not, be sure to always do a proper shutdown from the ROOter menu system. Pulling the power cord on an RPi actively running any version of OpenWRT, very often results in the need to do a filesystem repair.

If that's not the culprit, I'd suggest confirming your new VLAN is part of the LAN firewall group. This can be easily verified by checking the header colors under Network - Interfaces. Those that are part of the same firewall group will have the same color header. Mine is green for the LAN group and pink for WAN.
Thanks. That was not the issue (although I did run into this issue on my initial ROOter setup, so good thought!). I'm not quite sure what resolved this, but after hooking up a working modem my changes started saving. I don't think that was the cause, I think it is more likely I had something cached in my browser which was wanting to target or reload from the non-VLAN IP address (192.168.1.1 vs 192.168.3.1).

Anyway, onto my next hurdle! The run I want to do is from house -> detached garage -> barn (where antennas and modems will be). I want to put a switch in the garage to add another wireless AP for more range. So I got the following 2 devices: TRENDnet TPE-TG50g (5port gigabit PoE+ switch) and TRENDnet TPE-P521ES (5 port PoE+ powered w/ passthrough, EdgeSmart). So the 5 port PoE+ switch would be put in the garage and powered from there, and then I could do a single run to the barn, plug into the pass through switch to power it, and 2 modems - sounds great. But as soon as I introduced the PoE passthrough "smart" switch I was unable to access the device on VLAN3. Somehow the managed device, even totally unconfigured (so it should be acting just like an unmanaged device?), is screwing up the VLAN traffic. I verified this same exact setup works with 2 unmanaged switches, I can pass VLAN traffic across both of them no problem. Same exact cofiguration, just replacing the smart switch with a tp-link dumb switch I had laying around. I know this is getting pretty far off topic from the original post, was just wondering if maybe you had some ideas I could try.

Thanks again for your help!

EDIT: Think I got it working. I made the mistake of only adding the port the rpi was connected to to VLAN3. I had to add both that port, and the port connected to the unmanaged switch so that VLAN3 traffic could exit the switch.

ciarlill
Posts: 27
Joined: Wed Jul 15, 2020 11:45 am
Has thanked: 4 times
Been thanked: 0

Re: WAN Bonding with ROOter and OpenMPTCProuter

Post by ciarlill » Thu Aug 06, 2020 5:06 pm

The quest continues...

Wondering if you ever ran into any latency issues with the VLAN on a ROOter device? For some running in this configuration is giving me latency spikes. Even just pinging from OpenMPTCP router directly to the ROOter device. More info here if you are curious: https://github.com/Ysurac/openmptcprouter/issues/1123

For a test I just disabled the VLAN on ROOTer, and plugged the modem directly into eth1 on the OpenMPTCP router device and configured a static interface on 192.168.1.1 as WAN1. This works and I have internet, and the ping issue is gone. But the worrying part is I cannot seem to actually modify the default LAN interface address in ROOTer, it just doesn't apply, so I'm not sure this will be a valid solution.

User avatar
bnhf2
Posts: 37
Joined: Sun Aug 05, 2018 10:03 am
Has thanked: 0
Been thanked: 21 times

Re: WAN Bonding with ROOter and OpenMPTCProuter

Post by bnhf2 » Thu Aug 06, 2020 7:21 pm

ciarlill wrote:
Thu Aug 06, 2020 5:06 pm
Wondering if you ever ran into any latency issues with the VLAN on a ROOter device? For some running in this configuration is giving me latency spikes. Even just pinging from OpenMPTCP router directly to the ROOter device. More info here if you are curious: https://github.com/Ysurac/openmptcprouter/issues/1123

For a test I just disabled the VLAN on ROOTer, and plugged the modem directly into eth1 on the OpenMPTCP router device and configured a static interface on 192.168.1.1 as WAN1. This works and I have internet, and the ping issue is gone. But the worrying part is I cannot seem to actually modify the default LAN interface address in ROOTer, it just doesn't apply, so I'm not sure this will be a valid solution.
I have had problems with VLANs on the Raspberry Pi 3B+ -- so I'd be sure your issue doesn't lie there. Check the log for errors related to the Ethernet port. I'd also suggest verifying your Ethernet run to the garage and barn aren't part of the problem. Cable testers are cheap these days, so if you don't have one -- get one. I typically verify my cables using a tester, and also by using a pair of unmanged switches that have different colored LEDs based on the speed of the link.

ciarlill
Posts: 27
Joined: Wed Jul 15, 2020 11:45 am
Has thanked: 4 times
Been thanked: 0

Re: WAN Bonding with ROOter and OpenMPTCProuter

Post by ciarlill » Thu Aug 06, 2020 10:48 pm

bnhf2 wrote:
Thu Aug 06, 2020 7:21 pm
I have had problems with VLANs on the Raspberry Pi 3B+ -- so I'd be sure your issue doesn't lie there. Check the log for errors related to the Ethernet port. I'd also suggest verifying your Ethernet run to the garage and barn aren't part of the problem. Cable testers are cheap these days, so if you don't have one -- get one. I typically verify my cables using a tester, and also by using a pair of unmanged switches that have different colored LEDs based on the speed of the link.
Okay good to know. I did some googling but didn't come across anything about pi 3b+ vlan issues but that is the device in question. I'll take a peek in the logs after I reconfigure it again.

No cables have been run yet, everything is still on my workbench until I'm 100% sure it is and will work how I want, and I have swapped out all cables involved to eliminate that possibility as well.

At this point I'm also considering trying to run the modems directly of the USB ports from the qotom appliance and just sticking the whole kit out in the barn (there is power out there to some lights, would just need to wire in an outlet) and reversing the whole run back to the house lol. Just a little worried about the qotom dealing with 90+ degree days out there.

I am having doubts about the ability to power the pi+modem off PoE, or at least with the switch I got, so I might just need to bite the bullet and do some wiring regardless.

User avatar
bnhf2
Posts: 37
Joined: Sun Aug 05, 2018 10:03 am
Has thanked: 0
Been thanked: 21 times

Re: WAN Bonding with ROOter and OpenMPTCProuter

Post by bnhf2 » Fri Aug 07, 2020 8:13 am

ciarlill wrote:
Thu Aug 06, 2020 10:48 pm
I am having doubts about the ability to power the pi+modem off PoE, or at least with the switch I got, so I might just need to bite the bullet and do some wiring regardless.
I think you're on the right track with the network architecture you have in mind, with the cellular routers in the barn, and the OMR router in the house. I'm fairly confident the 3B+ is the problem. I used the Raspberry Pi myself originally with ROOter, and had great success with the 3B. The 3B+ though was never as solid, particularly with VLANs in use. That, and the lack of a stable OpenWRT for the 4B, resulted in me moving to the MikroTik RBM33G.

I'd highly recommend the RBM33G, as it has a speedy processor (for a router product), 2x mPCIe slots, a USB3 port, real gigabit Ethernet, an m.2 SSD slot, a wide range of power voltages including passive PoE, and a price point similar to the RPi. For me the 24V PoE was a huge plus, as I already use Ubiquiti for all of my WiFi.

If you're in an urban area, having your cellular routers in close proximity to the OMR router is usually fine. I'm guessing that since you have a barn, you're rural, and as such want to be able to carefully locate your cellular routers and pair them with high performance antennas. Stick with the network of gateways approach in this scenario, and it will serve you well for years to come.

ciarlill
Posts: 27
Joined: Wed Jul 15, 2020 11:45 am
Has thanked: 4 times
Been thanked: 0

Re: WAN Bonding with ROOter and OpenMPTCProuter

Post by ciarlill » Fri Aug 07, 2020 8:28 am

bnhf2 wrote:
Fri Aug 07, 2020 8:13 am
I think you're on the right track with the network architecture you have in mind, with the cellular routers in the barn, and the OMR router in the house. I'm fairly confident the 3B+ is the problem. I used the Raspberry Pi myself originally with ROOter, and had great success with the 3B. The 3B+ though was never as solid, particularly with VLANs in use. That, and the lack of a stable OpenWRT for the 4B, resulted in me moving to the MikroTik RBM33G.

I'd highly recommend the RBM33G, as it has a speedy processor (for a router product), 2x mPCIe slots, a USB3 port, real gigabit Ethernet, an m.2 SSD slot, a wide range of power voltages including passive PoE, and a price point similar to the RPi. For me the 24V PoE was a huge plus, as I already use Ubiquiti for all of my WiFi.

If you're in an urban area, having your cellular routers in close proximity to the OMR router is usually fine. I'm guessing that since you have a barn, you're rural, and as such want to be able to carefully locate your cellular routers and pair them with high performance antennas. Stick with the network of gateways approach in this scenario, and it will serve you well for years to come.
Oh wow that microtik device looks awesome. Do you plug the modems into it directly instead of over USB then? Sucks that I wasted so much time and money on raspberry pis, usb enclosures, sd cards, adapters, etc but if these are as good as the look, definitely worth another $100 after everything I've already invested.

User avatar
bnhf2
Posts: 37
Joined: Sun Aug 05, 2018 10:03 am
Has thanked: 0
Been thanked: 21 times

Re: WAN Bonding with ROOter and OpenMPTCProuter

Post by bnhf2 » Fri Aug 07, 2020 8:52 am

ciarlill wrote:
Fri Aug 07, 2020 8:28 am
Oh wow that microtik device looks awesome. Do you plug the modems into it directly instead of over USB then? Sucks that I wasted so much time and money on raspberry pis, usb enclosures, sd cards, adapters, etc but if these are as good as the look, definitely worth another $100 after everything I've already invested.
Yes, you can use an mPCIe modem directly (both Sierra and Quectel modems need to be configured to use USB2 "lines" only -- which is easily done by AT command), or if you have an m.2 modem in an m.2 to mPCIe adapter it's plug-and-play. I just ordered some RBM33G units from Streakwave Wireless, and they shipped the next day.

Baltic Networks sells an enclosure called the "Maxxwave UBTik 433 MikroTik Access Point (Enclosure Only)" that's pretty slick, though I frequently build my own outdoor boxes using Bud Industries boxes. There's also a pretty good 3D printer indoor enclosure design floating around.

Flashing the RBM33G with ROOter is easy once you get the hang of it, using tftp and the reset button. It's a really sweet board for ROOter, and addresses all of the downfalls of the RPi as far as I'm concerned. Other than setting mPCIe modems for USB2 lines only, the other thing to know is that the current "stable release" of ROOter doesn't have the board's Ethernet ports organized as it should, so you might want to use the 2019-03-10 build, which I'll link for you here:

https://www.dropbox.com/s/edc036nkcvsp0 ... -03-10.zip

User avatar
bnhf2
Posts: 37
Joined: Sun Aug 05, 2018 10:03 am
Has thanked: 0
Been thanked: 21 times

Re: WAN Bonding with ROOter and OpenMPTCProuter

Post by bnhf2 » Fri Aug 07, 2020 8:59 am

Oh, and the RBM11G is a nice (and cheaper) board too -- it's the 33G with a single Ethernet port and single mPCIe slot.

The RPi 3B+ makes an awesome Kodi (OSMC) box, Pi-hole DNS server or low-power Linux (Raspbian) file server by the way -- I've got at least half-a-dozen in daily use on my LAN!

ciarlill
Posts: 27
Joined: Wed Jul 15, 2020 11:45 am
Has thanked: 4 times
Been thanked: 0

Re: WAN Bonding with ROOter and OpenMPTCProuter

Post by ciarlill » Fri Aug 07, 2020 9:17 am

bnhf2 wrote:
Fri Aug 07, 2020 8:52 am
Yes, you can use an mPCIe modem directly (both Sierra and Quectel modems need to be configured to use USB2 "lines" only -- which is easily done by AT command), or if you have an m.2 modem in an m.2 to mPCIe adapter it's plug-and-play. I just ordered some RBM33G units from Streakwave Wireless, and they shipped the next day.

Baltic Networks sells an enclosure called the "Maxxwave UBTik 433 MikroTik Access Point (Enclosure Only)" that's pretty slick, though I frequently build my own outdoor boxes using Bud Industries boxes. There's also a pretty good 3D printer indoor enclosure design floating around.

Flashing the RBM33G with ROOter is easy once you get the hang of it, using tftp and the reset button. It's a really sweet board for ROOter, and addresses all of the downfalls of the RPi as far as I'm concerned. Other than setting mPCIe modems for USB2 lines only, the other thing to know is that the current "stable release" of ROOter doesn't have the board's Ethernet ports organized as it should, so you might want to use the 2019-03-10 build, which I'll link for you here:

https://www.dropbox.com/s/edc036nkcvsp0 ... -03-10.zip
So you cannot use an m.2 modem directly in the available m2 slot?

The other nice thing is it looks like I can ditch the PoE splitters I was using too.

I already have 2 nice NEMA encloures I can put them in, only concern is securing the antenna pigtails in some way since I won't have any bulkhead to secure the SMA side to inside the enclosure, but I'm sure I can rig up a little plate.

It's been a while since I flashed anything using tftp - i think way back flashing XBMC on an original Xbox or trying to unbrick one - but I can dust off those skills. Last question: are you still using your same 4 port PoE injector? Any idea how many watts your devices are drawing? This PoE pass-through switch is only rated for a total of 18W, so I think I will need to go another direction. Either the Ubiquiti passthrough one or just run 2 PoE lines from the garage and do the injection there. I've seen a lot of conflicting things about total power requirements on here.

Post Reply