Lately my router seems to be struggling under all my devices and after some googling, it seems my Archer AX1800 isnt really meant for more than 20-40 devices and I’m currently sitting at 50 with heavy torrent traffic and a media server with 15-20 friends and family connected (typically not all at once of course).
I was looking at the Omada networking gear but then thought of trying OPNSense since I have pretty much everything I need on hand versus spending $400+ for the Omada stuff.
I’d like to run it on the media server since I have tons of compute/RAM available but my question is about the ethernet ports. My mobo has a dedicated 2.5Gb ethernet port built in and I have a second 2.5Gb card that I could toss in there as well, but my concern is that I may saturate the ports with all the additional traffic. My LAN is just 1Gb outside of these two PCs.
My other option is an unused Optiplex 3050 micro that I’ve installed a second ethernet port on but these would both be 1Gb ports.
Am I likely to run into saturation issues on the media server or should the 2.5Gb ports be more than enough to handle the load? Apart from the torrents and video streaming, the rest of the network is mostly typical devices like phones/tablets/consoles, 6 4K POE cameras connected to an NVR and frigate, and a bunch of IoT devices connected to Home Assistant.
I don’t necessarily mind using the Optiplex but I’d definitely prefer to keep everything in one place if I can. I realize that this means my network goes down if I restart the host but it has been a pretty rock solid setup and Proxmox means I can just restart individual containers/VMs rather than the host most of the time.
Edit: forgot to mention that I’m open to buying another quad port NIC for the server if recommended, but would definitely prefer to not spend money if possible. I was planning on using the current router in AP mode along with its ports for a switch and I have another 4 port 2.5Gb switch in the mix as well
Just ask yourself one question: is your household OK with networking going offline when you need to reboot proxmox?
If the answer is yes, then by all means save yourself some hardware and virtualize it.
If the answer is no, then baremetal OPNsense on a miniPC is the way.
This is why I have two piholes. So if I’m updating and rebooting one, the network doesn’t cut out. My router has both programmed in.
2.5Gb won’t be your bottleneck on the WAN side — 3-4TB a month is only about 10Mbps averge, and even torrent bursts won’t saturate that unless your internet is over a gig. The place I’d watch is traffic hairpinning through the router, like cameras and media going via OPNsense instead of staying on the switch — keep the NVR/Frigate traffic local where you can.
If you do virtualise it, the security bit that matters most is isolation: keep the WAN side separate from your media containers, and keep the Proxmox managment interface on the LAN only, not reachable from WAN. FYI, that’s also how a misconfigured bridge ends up exposing the host.
And keep your old Archer handy as a fallback router with the OPNsense config backed up — if Proxmox needs a reboot, you can swap one cable and be back online. YMMV, but for a house with 50 devices, I’d rather have that fallback than squeeze everything onto one box.
I don’t know about your speeds, but for me it has worked fine for somewhere around 7 years. I went form OPNsense on it own hardware and moved the HDD into a PVE server, and have migrated that to a new PVE box and virtual SSD without instance.
So:
- OPNsense is on it’s own HDD can work well under PVE
- Running on a virtual SSD works just as well (as long as you have the internal bandwidth if you are doing lots of logging etc)
- Have a router always available to swap in it it goes sideways, I can swap 1 cable and change 1 setting (on the wifi) and I be back online
I’m a bare metal guy when it comes to routers.
With that said you are fretting over something that probably won’t matter and that is the ethernet port speed. Your current router only has gigabit ports and the router can only pass data from the LAN to the WAN at speeds fast enough to saturate your internet connection speeds. The only advantage of 2.5gbps on the router would be IF you are running multiple vLAN’s. Since you are already using the router as a point for your switch to connect to the internet you are already limited to 1Gbps speeds.
Install OpnSense on the device you have available, set it up for the basics (DHCP reservations, IP addresses for the LAN, etc.), change the ip address in the original router to something that will work with your new setup and turn off the DHCP server. Then do some testing with your new router and see what you think. The old router will work as an ok access point and switch for now, grab a cheap switch to expand if you need to since the WAN port on the archer will no longer be usable.
Later on when you have some extra cash you can grow and expand where you feel the need. I’m using 10Gbps for my backbone with 1Gbps Dell 5548 PowerConnect switches for the majority of devices. You can get the PowerConnect switches on ebay for around 50 to 100 bucks easily. Most clients don’t need more than 1Gbps (even 4k cameras) and a 10Gbps card dropped in your server will give you great speeds for cheap. An Intel X520 10G NIC can be picked up for 30 bucks add a couple sfp+ modules for about 10 each plus an OM3 fiber cable for a few bucks and you are set for a while. If you get a Dell PowerConnect 5548P (the P makes it a POE switch) and you can power POE cameras and POE access points without anything extra and still use the rest of they’re ports for regular computer connections.
Depends. How much traffic are you actually pushing at any given time? Don’t you want to be able to get full speed when you’re doing a transfer?
Personally I don’t recommend running a virtual router. If something on the hypervisor breaks, your Internet access goes down too.
I ran OPNSense on a Proxmox VM for a few months and felt nervous about it the whole time. If you’re running a cluster, unless you can do routing without OPNSense, then you can’t get a quorum with the other nodes so restarting that node can require manually force starting the OPNSense VM. Same with a cold startup after a power outage or such.
Not sure on traffic numbers but I can easily do 3-4TB of data per month externally plus whatever chatter the IoT devices are sending
back to Chinaacross the network. The cameras record to the NVR and only the substreams are running to Frigate for object detection.As far as “full speed” goes, my torrents and media files are in a shared ZFS pool and the HDDs that comprise the pool seem to be the limiting factor in transfer speeds. Currently the 2.5Gb port on the server is connected to a 1Gb port on the router anyway as these two cards and the 4-port switch are the only 2.5Gb devices on the network. My ISP speed is only “up to” 1.2Gbps down as well.
I get that a dedicated router is going to be more robust but the market isn’t the best for people in our situation as there’s “cheap consumer grade” and “super expensive prosumer grade” with nothing much in between. As I said this option seems worth exploring since I already have (most?) everything I need but I dont want to go through all the effort if it isn’t going to work in the end or solve one issue only to introduce two new issues. Going with Omada or Ubiquiti means not only a new router but a switch, controller, and AP at the very least.
Opnsense VM proxmox is how I’ve been set up for years now. Whatever you choose now will be a piece of cake to migrate later, either as VM backup/restore or opnsense’s, so don’t fret too much.
I’d use the tiny PC if the 1 Gbps cards are good enough. “Up to 1.2Gbps” that’s pretty close. Reason for tiny PC is to keep internet in separate hardware than the rest. For me, limiting when I could do anything hardware related on the homeserver to when internet outages are OK was too much. I use a mini PC n100 for opnsense and pihole (so the core network stuff, routing firewall dhcp dns).
Whichever hardware you’re on, I would have the wan nic on it’s own bridge in proxmox, cable straight to ISP. Opnsense VM gets a vnic on that bridge for wan. Then lan one is just like your other VMs on the lan/inside bridge. Same setup whichever box you end up on. And backup/restore the VM if you want to migrate or test and compare speeds on both. or explore HA and have both involved.
To parent’s point about skipping the hypervisor: that works great too. Opnsense install on a new drive and restoring cfg backup is super quick, which would be one of the big benefits of using a VM. You can do a lot of add-on stuff in opnsense like adblocking, so possibly eliminate need for running those as side VMs on a “network box” like mine.
If you’ve got the time and desire, test them all out. Once you get migrated to opnsense itll be easy to swap around
