Any ASIC switch released in the last decade from Cisco/Juniper/Arista supports EVPN/VXLAN in hardware. EVPN is built on BGP. This has become the industry standard for new enterprise and cloud deployments.
The lack of support for hardware EVPN is one of the many reasons that Mikrotik is not considered for professional deployments.
If I can source an enterprise Cisco/Juniper/Arista ASIC switch that is 1) rock-solid 2) full featured 3) cheaper - which I can - there is unfortunately no rationale where Mikrotik would be applicable in any professional project of mine.
With that said, I love Mikrotik for what it is: it is very approachable and it fills a niche. I believe it has added a lot of value to the industry and I'm excited to see their products mature.
You're referencing backbone, not edge. It has only been a few years that MikroTik had offered a 100G solution, let alone became competitive in it. You won't find it in the backbone yet. However, many European ISP's have largely upgraded their distro and aggregation switches to MikroTik over the last five years. There's a sovereignty push, too. I would guess edge is similar, but there's too many cheap options there so probably not that much.
If your impression is based on data circa ~2020, you should re-evaluate your priors with the recent packages in mind. See https://mikrotik.com/product/crs812_ddq
CE (Customer Edge) is what you are referring to. ISPs would be the PE (Provider Edge). I am aware it can be popular for SMB CE devices, however that is simply not the case for PE devices.
Service Provider ISPs cannot use Mikrotik - It is impossible. RouterOS supports none of the features required for a service provider. VRFs are even still unsupported in HW [1]. I am confused why this is even a discussion as anyone with experience working at an ISP/SP would come to the same conclusion.
There are many ISP's that successfully run their networks on BGP, without VRF unless their customers specifically require it. It simply means that VRF-heavy architectures (like dense MPLS L3VPN etc.) would require additional hardware. Nobody says you have to use MikroTik for everything, and nobody says it's the ultimate solution to all ISP problems. I don't get it where this maximalist view comes from—all or nothing. The typical MPLS VPN scenario has to do with overlapping address spaces, and for customer separation most aggregation layer deployments use pure L3 routing with VLAN segmentation in the first place.
There's a famous use-case from 10 years ago (sic!) of using MikroTik for serving over 400 customers, see https://mum.mikrotik.com/presentations/ID16/presentation_340... proving you could do it on small scale many years ago. Needless to say, A LOT has improved since. MikroTik has become a serious, and affordable means to power a small-to-midsize ISP in the recent years. Of course there are "enterprise" features for some people to get knickers in a twist over, but they are well beyond necessity. It's often that people were taught certain techniques, a certain way to do things (which more often than not includes all this domain over-extension madness and all that it carries with it up to L7!) so they struggle to adapt to alternative architectures.
To say that it's "impossible" to provide ISP services with MikroTik is reaching.
I have a mix of equipment from heavyweight juniper mxs at peeing points to arista dcs/ccs in large sites to £50 mikrotiks in the smallest branch offices.
Right tool for the right job, mikrotik is often but not always the right tool.
You're delusional on price. I wouldn't touch severely overpriced and backdoored American switches with a 10-foot pole! Meanwhile, MikroTik just released a 400G switch in under two grand. To buy Cisco/Juniper/Arista with your own money in 2025 you have to be super rich and super stupid. And I say this as a guy that buys 100G stuff from Xilinx.
I have not seen a case where I could not source a Juniper switch (for example) for lower $/port than Mikrotik, even at 400GE. It is unheard of to pay MSRP. YMMV.
Agreed. They've also been extremely finnicky from my experience - had cases where large EVPN deployments just blackholed some arbitrary destination MAC until GARPs were sent out of them.
Also IME EVPN is mostly deployed/pushed when clueless app developers expect to have arbitrary L2 reachability across any two points in a (cross DC!) fabric [1], or when they want IP addresses that can follow them around the DC or other dumb shit that they just assumed they can do.
[1] "What do you mean I can't just use UDP broadcast as a pub sub in my application? It works in the office, fix your network!" and the like.
The good clouds don't support L2, they use a centralized control plane instead of brittle EVPN, and they virtualize in the hypervisor instead of in the switches. People are being sold EVPN as "we have cloud at home" and it's not really true.
AWS/GCE/Azure's network implementations pre-date EVPN and are proprietary to their cloud. EVPN is for on-premise. You don't exactly have the opportunity to use their implementation unless you are on their cloud, so I am not sure comparing the merits of either is productive.
> Also IME EVPN is mostly deployed/pushed when clueless app developers expect to have arbitrary L2 reachability across any two points in a (cross DC!) fabric [1], or when they want IP addresses that can follow them around the DC or other dumb shit that they just assumed they can do.
Sorry, but that's really reductive and backwards. It's usually pushed by requirements from the lower regions of the stack, operators don't want to let VMs have downtime so they live migrate to other places in the DC. It's not a weird requirement to let those VM's keep the same IP once migrated. I never had a developer ask me for L2 reachability.
VXLAN over WireGuard is acceptable if you require a shared L2 boundary.
IPSec over VXLAN is what I recommend if you are doing 10G or above. There is a much higher performance ceiling than WireGuard with IPSec via hardware firewalls. WireGuard is comparatively quite slow performance-wise. Noting Tailscale, since it has been mentioned, has comparatively extremely slow performance.
edit: I'm noticing that a lot of the other replies in this thread are not from network engineers. Among network engineers WireGuard is not very popular due to performance & absence of vendor support. Among software engineers, it is very popular due to ease of use.
At this time, there is no commercial offering for hardware/ASIC WireGuard implementations. The standard WireGuard implementation cannot reach 10G.
The fastest I am aware of is VPP (open-source) & Intel QAT [1], which while it is achieves impressive numbers for large packets (70Gbps @ 512 / 200Gbps @ 1420 on a $20k+ MSRP server), is still not comparable with commercial IPsec offerings [2][3][4] that can achieve 800Gbps+ on a single gateway (and come with the added benefit of relying on a commercial product with support).
There are also solutions like Arista TunnelSec [1] that can achieve IPsec and VXLANsec at line-rate performance (21.6Tbps per chassis)! This is fairly new and fancy though.
This lack of ASIC is interesting to me. If it existed, that would very much change the game. And, given the simplicity of WG encryption it would be a comparatively small design (lower cost?)
While that's true, I'm not sure it's because of something inherent in IPsec vs WireGuard. It's more likely due to the fact that hardware accelerators have been designed to offload encryption routines that IPsec uses.
One wonders what WG perf would look like if it could leverage the same hardware offload.
Exactly this. I would love to see a commercial product with a hardware implementation for WireGuard, but it does not yet exist. IPsec, however, is well supported.
Thanks for your answers. I wonder though, from the perspective of a small user that doesn’t have requirements for such bandwidth, how does ipsec compare with wg on other metrics/features? Is it worth looking into?
I'd use WireGuard in that case. The main reason WireGuard is popular at all is because it is approachable. IPsec is much more complicated and is designed for network engineers, not users.
Well yeah, so except being more complex and having hardware support, is there anything useful in ipsec? I meant a user in the general sense, not necessarily meaning a clueless non technical home user.
> Noting Tailscale, since it has been mentioned, has comparatively extremely slow performance.
Isn't this mainly because Tailscale relies on userspace WG (wireguard-go)? I'd imagine the perf ceiling is much higher for kernel WG, which I believe is what Netbird uses.
wireguard-go is indeed very slow. For example, the official WireGuard Mac client uses it, and performance on my M1 Max is CPU capped at 200Mbps. The kernel WireGuard implementation available for Linux is certainly faster, but I would not consider it fast.
Tailscale however, although it derives from WireGuard libraries and the protocol, is really not WireGuard at all- so comparing it is a bit apples to oranges. With that said, it is still entirely userspace and its performance is less than stellar.
Well, according to this[1] bench, you can get ~10 Gbps with kernel WG.
I'm interested in this because I'm working on a small hobby project to learn eBPF. The idea is to implement a "Tailscale-lite" that eliminates context switches by keeping both Wireguard and L3 and L4 policy handling in kernel space. To me, the bulk of Tailscale's overhead comes from the fact that the dataplane is running between user and kernel space.
That's a large packet benchmark, not mixed packet size, and it just barely hits it. If you need consistent 10Gbps for a business use case, I would not consider that sufficient.
> "To me, the bulk of Tailscale's overhead comes from the fact that the dataplane is running between user and kernel space."
Yes and no, it's more complicated. DPDK is the industry standard library for fast packet processing, and it is in entirely user space. The Linux kernel netstack is just not very fast.
Sure, but who is going to ship a DPDK application to end users? And how exactly would that work for existing user applications that are not DPDK aware?
I think kernel networking is the only option for Tailscale (or any similar mesh VPN solution). Given this key constraint, the best you can do is do more work in kernel space and reduce context switches.
> I'd much prefer they enforce the laws evenly and then fix them where they're broken rather than disadvantaging everyone going through the legal process while those that cheat get to jump ahead.
That is incredibly optimistic to believe that any legislation reforming immigration will be passed in the next decade.
There is a reason ICE was neutralized until now. Life is short. We don't have time for congress to play politics while Americans and their spouses suffer. Let people live their lives.
Maybe I'm too stiff, but even if they don't get around to updating the laws, I'd still prefer they enforce the ones that exist so it's clear, fair, and safe. And so upstanding citizens aren't spending years separated from their spouses while they keep getting skipped by people willing to cheat the system.
It's not even a law that results in the years-long wait; it's just because the system is clogged up with other junk and understaffed. As other's have mentioned; there's no formal waiting for citizen spouses—it's supposed to be immediate—it's just that they don't even get to look at your application for years.
What would be your definition of “upstanding citizens”.
I’ve found that people tend to respond as you have until the laws impact themselves or their friends. Then it’s very much a case of - I didn’t think this applied to us…
People following the actual immigration law, like I did. This very much impacts me and has determined my family's country of residence for many years. The process is horrible, but we still aren't cheating and I don't appreciate being skipped by people that are cheating.
I've spent hours on immigrations forums trying to understand the law and have filed many US Federal forms (and have mad mistakes) taken years to get action done while following the law.
It really feels like a slap in the face to the people who do the "right thing" to allow others to be allowed in freely not following the law.
I assume you’re insinuating some kind of insult? I’m honestly not sure which one. For not risking my family to cheat US immigration law?
Edit: And now browsing the latest on this thread, it seems all the commenters here who have actually filed petitions agree—the law should be enforced evenly.
Negotiating is difficult if you show your hand. It is arguably beneficial to both the state and Elon that the e-mails stay redacted. I agree it is unfortunate though.
Sensitive negotiations like those should be handled by low-level bureaucrats with supervision, not by the damned Governor. Again, negotiating directly with a state governor is obviously corrupt no matter what the contents. This would have been instantly clear to any American back when we were an advanced nation.
Could you elaborate why an executive having sensitive conversations with a governor is corrupt? I'm having difficulty understanding this outlook.
My industry has historically been extremely corrupt. This is why rules were defined for reportable expenses. Could you boil down your viewpoint into rules that can be applied in an organization, rather than a vibe test? I think I could better understand you, then.
If the result of the negotiations are in good faith and beneficial for his constituents, then it is not corruption. Of course, that's open to interpretation.
Corruption is usually when there is personal benefit to the politician themselves.
Use-after-free bugs (such as the vulnerability in question, https://issuetracker.google.com/issues/440183164) usually can be exploited to result in remote code execution, but not always. It wouldn't be prudent to bet that this case is one of the exceptions, of course.
That behaviour is indeed totally unacceptable. At your job. Where they're paying you, and especially if they're paying you at FAANG type pay scales.
If you're an unpaid volunteer? Yeah - nah. They can tell you "Sorry, I'm playing with my cat for the next 3 months, maybe I'll get to it after that?", or just "Fuck off, I don't care."
(I'm now playing out a revenge fantasy in my head where the ffmpeg team does nothing, and Facebook or Palantir or someone similar get _deeply_ hacked via the exploit Google published and theat starts the planets biggest ever pointless lawyers-chasing-the-deepest-pockets fight.)
Or perhaps you’re a FAANG security researcher and your time will be better spent serving the OSS community as a whole by submitting as many useful bug reports as possible, instead of slightly fewer reports with patches included.
In this particular case it’s hardly obvious which patch you should submit. You could fix this particular bug (and leave in place the horrible clunky codec that nobody ever uses) OR you could just submit a patch that puts it behind a compile flag. This is really a decision for the maintainers, and submitting the latter (much better!) patch would not save the maintainers any meaningful amount of time anyway.
I don’t understand how it helps the community to publicly release instructions for attacking people, unless you’re trying to incentivize a company to fix their crap. In this case, there is no company to incentivize, so just report it privately.
You can say publicly that “there is an ABC class vulnerability in XYZ component” so that users are aware of the risk.
It’s OSS so somebody who cares will fix it, and if nobody cares then it doesn’t really matter.
This also informs users that it’s not safe to use ffmpeg or software derived from it to open untrusted files, and perhaps most importantly releasing this tells the distro package maintainers to disable the particular codec when packaging.
This bug might lead to vulnerability and that's enough. It makes no sense to waste lot of time and research whether it is possible or not - it is faster to remove the buggy codec nobody needs or make a fix.
The lack of support for hardware EVPN is one of the many reasons that Mikrotik is not considered for professional deployments.
reply