Hi list,
During the pub session after Zanog2024/iWeek, Sander mentioned his friend that did a Linux router using VPP, so I asked him more information about it, and he responded below. Anybody else that is/have tested similar and could share advice on locally (ie. South African) sourceable parts ?
Hendrik
On 21 Mar 2024, at 09:20, Sander Steffann sander@nogalliance.org wrote:
Hi!
The hardware list is in this blog post: https://ipng.ch/s/articles/2021/09/21/vpp-7.html
But do look at his full collection of posts: https://ipng.ch/s/articles/
There is really good stuff in there!
Cheers, Sander --- for every complex problem, there’s a solution that is simple, neat, and wrong
Op 21 mrt 2024 om 05:19 heeft Hendrik Visage hvisage@lnx-solutions.com het volgende geschreven:
Good day Sander,
You mentioned this 1U switch/router that somebody did, and I’m interested in the bill of materials, ie. which 1U server/chassis, which SPF+ adapters etc. as I need to cost/price it (especially w.r.t. Watts power consumed per port).
Any pointers where to source those would also be appreciated!
Hendrik Visage
hi,
During the pub session after Zanog2024/iWeek, Sander mentioned his friend that did a Linux router using VPP, so I asked him more information about it, and he responded below. Anybody else that is/have tested similar and could share advice on locally (ie. South African) sourceable parts ?
In the mean time I also talked to Pim van Pelt (the friend mentioned above). He’d be willing to come to a future ZANOG if travel expenses can be covered (ZANOG covered my travel expenses this time, so it has been done before :) and the timing works with his full-time job at Google. It’s amazing that he manages to do all this work next to having a full-time job in the first place! :D
Anyway, I can really recommend ZANOG inviting Pim in the future!
Cheers, Sander
This is probably best pointed at the PC for 2025…
FWIW Pim’s neighbour is Max Stucchi who came down this year.
f
On 24 Mar 2024 at 12:04:32, Sander Steffann via zanog-discuss < zanog-discuss@lists.nog.net.za> wrote:
hi,
During the pub session after Zanog2024/iWeek, Sander mentioned his friend that did a Linux router using VPP, so I asked him more information about it, and he responded below. Anybody else that is/have tested similar and could share advice on locally (ie. South African) sourceable parts ?
In the mean time I also talked to Pim van Pelt (the friend mentioned above). He’d be willing to come to a future ZANOG if travel expenses can be covered (ZANOG covered my travel expenses this time, so it has been done before :) and the timing works with his full-time job at Google. It’s amazing that he manages to do all this work next to having a full-time job in the first place! :D
Anyway, I can really recommend ZANOG inviting Pim in the future!
Cheers, Sander
zanog-discuss mailing list -- zanog-discuss@lists.nog.net.za To unsubscribe send an email to zanog-discuss-leave@lists.nog.net.za
On 24 Mar 2024, at 17:19, Fearghas Mckay fearghas@gmail.com wrote:
This is probably best pointed at the PC for 2025…
Glad you noticed ;)
FWIW Pim’s neighbour is Max Stucchi who came down this year.
Yep, the internet community is close in many ways :D Sander
On 24 Mar 2024 at 13:52:43, Sander Steffann sander@steffann.nl wrote:
Glad you noticed ;)
Remind him to submit when the CFP comes out for 2025 and it can be a reminder for us 🙂
f
PS yes I know Pim is on the cc line…
Hoi folks,
(apologies if zanog-discuss@ is not open for posting from non-members; I can certainly subscribe if that's the case).
On 24.03.2024 17:04, Sander Steffann wrote:
During the pub session after Zanog2024/iWeek, Sander mentioned his friend that did a Linux router using VPP, so I asked him more information about it, and he responded below. Anybody else that is/have tested similar and could share advice on locally (ie. South African) sourceable parts ?
In the mean time I also talked to Pim van Pelt (the friend mentioned above). He’d be willing to come to a future ZANOG if travel expenses can be covered (ZANOG covered my travel expenses this time, so it has been done before 🙂 and the timing works with his full-time job at Google. It’s amazing that he manages to do all this work next to having a full-time job in the first place! :D
By means of Introduction, I'm PBVP1-RIPE and live in Switzerland. In my spare time I operate AS8298 which is a small ISP with presence in some european cities. We're reasonably well connected [1] but it also fully runs open source. I write about [2], and contribute to [3], Vector Packet Processing that Sander mentioned at the zanog social. I'm an active member of NLNOG (my home country) and SWINOG (my home since 2006). I'll use Sander to help me navigate the program committee for an upcoming meeting, and see if I can offer some interesting content about open source dataplanes and contribute some stories about my little European ISP.
groet, Pim
[1] as8298.peeringdb.com and ipng.ch/s/articles/2021/02/27/network.html [2] ipng.ch/s/articles/ [3] ipng.ch/s/articles/2021/08/12/vpp-1.html or ipng.ch/s/articles/2023/05/07/vpp-mpls-1.html
Hi Pim
Good to hear from you herein...
Would you know how far things are going? In terms of the active development of vvp addon to the next LTS version of VyOs?
Besides vvp, does VyOs form part of your stack for your oss-based ISP?
Cheers, *.**/noah*
On Mon, Mar 25, 2024 at 5:06 PM Pim van Pelt via zanog-discuss < zanog-discuss@lists.nog.net.za> wrote:
Hoi folks,
(apologies if zanog-discuss@ is not open for posting from non-members; I can certainly subscribe if that's the case).
On 24.03.2024 17:04, Sander Steffann wrote:
During the pub session after Zanog2024/iWeek, Sander mentioned his friend that did a Linux router using VPP, so I asked him more information about it, and he responded below. Anybody else that is/have tested similar and could share advice on locally (ie. South African) sourceable parts ?
In the mean time I also talked to Pim van Pelt (the friend mentioned above). He’d be willing to come to a future ZANOG if travel expenses can be covered (ZANOG covered my travel expenses this time, so it has been done before 🙂 and the timing works with his full-time job at Google. It’s amazing that he manages to do all this work next to having a full-time job in the first place! :D
By means of Introduction, I'm PBVP1-RIPE and live in Switzerland. In my spare time I operate AS8298 which is a small ISP with presence in some european cities. We're reasonably well connected [1] but it also fully runs open source. I write about [2], and contribute to [3], Vector Packet Processing that Sander mentioned at the zanog social. I'm an active member of NLNOG (my home country) and SWINOG (my home since 2006). I'll use Sander to help me navigate the program committee for an upcoming meeting, and see if I can offer some interesting content about open source dataplanes and contribute some stories about my little European ISP.
groet, Pim
[1] as8298.peeringdb.com and ipng.ch/s/articles/2021/02/27/network.html [2] ipng.ch/s/articles/ [3] ipng.ch/s/articles/2021/08/12/vpp-1.html or ipng.ch/s/articles/2023/05/07/vpp-mpls-1.html
-- Pim van Pelt pim@ipng.ch pim@ipng.ch PBVP1-RIPE https://ipng.ch/
zanog-discuss mailing list -- zanog-discuss@lists.nog.net.za To unsubscribe send an email to zanog-discuss-leave@lists.nog.net.za
Hoi Noah,
On 26.03.2024 08:44, Noah wrote:
Would you know how far things are going? In terms of the active development of vvp addon to the next LTS version of VyOs?
IPng Networks deployed VPP based routers (based on that vpp-7 article that Sander showed) in 2021. They are vanilla Debian machines with Intel i350 (4x1G) and Intel i710 (quad 10G and dual 25G) network cards. It's been running quite well, with one dataplane bug (that yielded VPP crashes) that was tracked down together with the folks at Netgate in early 2022. Otherwise I've had no surprises with ~14 routers and ~2100 BGP adjacencies or so, for nearing three years, so I think I could claim that VPP is as stable as merchant silicon. Famous last words!
Besides vvp, does VyOs form part of your stack for your oss-based ISP?
I speak to the folks from VyOS from time to time. Their journey integrating VPP into VyOS has just started and there will be some ways to go - but you can download nightly builds with the feature turned on and experiment with it. Be aware that some features work well (forwarding, OSPF, BGP), while others don't yet (for example, ACLs or NAT or ISIS). I've promised them to take VyOS out for a spin and write about it, but I haven't started that yet :)
I do not use VyOS at IPng. It's "just" Debian Bookworm with the VPP debian packages, and Bird2 as a controlplane.
groet, Pim
I also spoke to Sander about that and said her should invite the guy to ZA to present. These parts are difficult to source in ZA and it would be a good idea for someone to stock Intel whiteboxes. I import the stuff myself. I approached Boston who does DC hosting equipment but I've been ignored. Only place I have picked up anything is CME. Regards
-----Original Message----- From: hvjunk via zanog-discuss zanog-discuss@lists.nog.net.za Sent: Thursday, March 21, 2024 10:43 AM To: zanog-discuss@lists.nog.net.za Cc: hvjunk hvjunk@gmail.com Subject: [zanog-discuss] Linux/VPP router
Hi list,
During the pub session after Zanog2024/iWeek, Sander mentioned his friend that did a Linux router using VPP, so I asked him more information about it, and he responded below. Anybody else that is/have tested similar and could share advice on locally (ie. South African) sourceable parts ?
Hendrik
On 21 Mar 2024, at 09:20, Sander Steffann sander@nogalliance.org wrote:
Hi!
The hardware list is in this blog post: https://ipng.ch/s/articles/2021/09/21/vpp-7.html
But do look at his full collection of posts: https://ipng.ch/s/articles/
There is really good stuff in there!
Cheers, Sander --- for every complex problem, there’s a solution that is simple, neat, and wrong
Op 21 mrt 2024 om 05:19 heeft Hendrik Visage hvisage@lnx-solutions.com het volgende geschreven:
Good day Sander,
You mentioned this 1U switch/router that somebody did, and I’m interested in the bill of materials, ie. which 1U server/chassis, which SPF+ adapters etc. as I need to cost/price it (especially w.r.t. Watts power consumed per port).
Any pointers where to source those would also be appreciated!
Hendrik Visage _______________________________________________ zanog-discuss mailing list -- zanog-discuss@lists.nog.net.za To unsubscribe send an email to zanog-discuss-leave@lists.nog.net.za
On 25 Mar 2024, at 09:13, Ron B ronald@amastelek.com wrote:
I approached Boston who does DC hosting equipment but I've been ignored.
We are resellers to Boston and that is why they do respond to us. I’ve already reached out and awaiting some quotes to discuss.
Only place I have picked up anything is CME.
Does seem to have the type I’m looking for
Excellent - tell me how it goes with Boston as its always good to have an alternative source. Attached is my kit via Oz.
-----Original Message----- From: hvjunk hvjunk@gmail.com Sent: Monday, March 25, 2024 5:45 PM To: Ron B ronald@amastelek.com Cc: zanog-discuss@lists.nog.net.za Subject: Re: [zanog-discuss] Linux/VPP router
On 25 Mar 2024, at 09:13, Ron B ronald@amastelek.com wrote:
I approached Boston who does DC hosting equipment but I've been ignored.
We are resellers to Boston and that is why they do respond to us. I’ve already reached out and awaiting some quotes to discuss.
Only place I have picked up anything is CME.
Does seem to have the type I’m looking for
Who did you speak to at Boston?
I'll get this sorted out for you.
Regards
Jade Benson Absolute Hosting absolutehosting.co.za +27 12 004 0112
-----Original Message----- From: Ron B via zanog-discuss zanog-discuss@lists.nog.net.za Sent: Tuesday, March 26, 2024 8:03 AM To: hvjunk hvjunk@gmail.com Cc: zanog-discuss@lists.nog.net.za; Ron B ronald@amastelek.com Subject: [zanog-discuss] Re: Linux/VPP router
Excellent - tell me how it goes with Boston as its always good to have an alternative source. Attached is my kit via Oz.
-----Original Message----- From: hvjunk hvjunk@gmail.com Sent: Monday, March 25, 2024 5:45 PM To: Ron B ronald@amastelek.com Cc: zanog-discuss@lists.nog.net.za Subject: Re: [zanog-discuss] Linux/VPP router
On 25 Mar 2024, at 09:13, Ron B ronald@amastelek.com wrote:
I approached Boston who does DC hosting equipment but I've been ignored.
We are resellers to Boston and that is why they do respond to us. I’ve already reached out and awaiting some quotes to discuss.
Only place I have picked up anything is CME.
Does seem to have the type I’m looking for
Don't want to drop names but it was the sales manager based on an integration at Africomm. I'm certain many on here would be interested in the same hardware as used by PIM that can which can be used for bird and frr.
I haven't been too worried as my imports have been fine and there has been no supply crisis.
-----Original Message----- From: Jade Benson jade@absolutehosting.co.za Sent: Tuesday, March 26, 2024 8:16 AM To: zanog-discuss@lists.nog.net.za; hvjunk hvjunk@gmail.com Cc: Ron B ronald@amastelek.com Subject: RE: [zanog-discuss] Re: Linux/VPP router
Who did you speak to at Boston?
I'll get this sorted out for you.
Regards
Jade Benson Absolute Hosting absolutehosting.co.za +27 12 004 0112
-----Original Message----- From: Ron B via zanog-discuss zanog-discuss@lists.nog.net.za Sent: Tuesday, March 26, 2024 8:03 AM To: hvjunk hvjunk@gmail.com Cc: zanog-discuss@lists.nog.net.za; Ron B ronald@amastelek.com Subject: [zanog-discuss] Re: Linux/VPP router
Excellent - tell me how it goes with Boston as its always good to have an alternative source. Attached is my kit via Oz.
-----Original Message----- From: hvjunk hvjunk@gmail.com Sent: Monday, March 25, 2024 5:45 PM To: Ron B ronald@amastelek.com Cc: zanog-discuss@lists.nog.net.za Subject: Re: [zanog-discuss] Linux/VPP router
On 25 Mar 2024, at 09:13, Ron B ronald@amastelek.com wrote:
I approached Boston who does DC hosting equipment but I've been ignored.
We are resellers to Boston and that is why they do respond to us. I’ve already reached out and awaiting some quotes to discuss.
Only place I have picked up anything is CME.
Does seem to have the type I’m looking for
Hoi folks,
On 26.03.2024 12:30, Ron B via zanog-discuss wrote:
I'm certain many on here would be interested in the same hardware as used by PIM that can which can be used for bird and frr.
I haven't been too worried as my imports have been fine and there has been no supply crisis.
Granted i don't know much about the reseller and distributor market in South Africa, but in case it helps: - IPng uses these Supermicro machines, as I really do like the IPMI and power-to-performance (38W all up for a router that can do 60Gbps and 35Mpps). - IPng also uses regular Dell R620/R630 from the refurb market and ~any reasonable Xeon E5 would do, as would Zen3+ CPUs (like Epyc, Romes and Milans).
For vendors, pretty much anything that runs Debian will do: HPE, Dell, Supermicro, whitelabel boxes, and so on. As long as the network card is DPDK enabled (most Intel 1G/2.5G/10G+, Mellanox Cx4+), you should be totally fine.
A modern E5 Xeon will yield between 8Mpps-15Mpps per CPU thread, and it's relatively straight forward to get a machine with 40 cores or even 80 cores. So realistically, the main bottleneck will be PCI bus and network card throughput. "Any modern PC will do" and "Vendor does not matter". I'd love to learn more about the local market, what you can and cannot buy.
groet, Pim
Hi Pim,
On 26 Mar 2024, at 16:31, Pim van Pelt via zanog-discuss zanog-discuss@lists.nog.net.za wrote:
As long as the network card is DPDK enabled (most Intel 1G/2.5G/10G+, Mellanox Cx4+), you should be totally fine.
Where/how do I get a list of supported devices?
Especially asking for when I can get some refurbished NICs
Hendrik
Hoi Hendrik,
On 26.03.24 21:12, hvjunk via zanog-discuss wrote:
Where/how do I get a list of supported devices?
Especially asking for when I can get some refurbished NICs
Take a look at https://core.dpdk.org/supported/ There's one caveat. For VPP, devices are found using PCI bus enumeration. Some drivers (for example DPAA2) do not expose the NIC on the PCI bus but some other means. For those more esoteric network interface cards, DPDK works, but VPP may not.
Roughly speaking: Intel and Mellanox Cx4 or higher will do just fine. https://core.dpdk.org/supported/nics/intel/ https://core.dpdk.org/supported/nics/mellanox/
Note: the support page here says that Cx3 is also supported, but precisely the problem above is true: Mellanox Cx3 cards with 2 ports will expose them under only 1 PCI bus:slot:addr, which implies code changes will be necessary to make VPP pick up both ports (I happen to have a patch for Cx3, but again: you're safe with Cx4+ and Intel NICs).
Hope that helps!
groet, Pim
Excellent. Dou you have a precise model or hardware reference for those Supermicro machines so that we can have them sourced locally.
I’ll buy you beers when I see you.
Regards
*From:* Pim van Pelt via zanog-discuss zanog-discuss@lists.nog.net.za *Sent:* Tuesday, March 26, 2024 4:31 PM *To:* zanog-discuss@lists.nog.net.za *Cc:* Pim van Pelt pim@ipng.ch *Subject:* [zanog-discuss] Re: Linux/VPP router
Hoi folks,
On 26.03.2024 12:30, Ron B via zanog-discuss wrote:
I'm certain many on here would be interested in the same hardware as used by
PIM that can which can be used for bird and frr.
I haven't been too worried as my imports have been fine and there has been
no supply crisis.
Granted i don't know much about the reseller and distributor market in South Africa, but in case it helps: - IPng uses these Supermicro machines, as I really do like the IPMI and power-to-performance (38W all up for a router that can do 60Gbps and 35Mpps). - IPng also uses regular Dell R620/R630 from the refurb market and ~any reasonable Xeon E5 would do, as would Zen3+ CPUs (like Epyc, Romes and Milans).
For vendors, pretty much anything that runs Debian will do: HPE, Dell, Supermicro, whitelabel boxes, and so on. As long as the network card is DPDK enabled (most Intel 1G/2.5G/10G+, Mellanox Cx4+), you should be totally fine.
A modern E5 Xeon will yield between 8Mpps-15Mpps per CPU thread, and it's relatively straight forward to get a machine with 40 cores or even 80 cores. So realistically, the main bottleneck will be PCI bus and network card throughput. "Any modern PC will do" and "Vendor does not matter". I'd love to learn more about the local market, what you can and cannot buy.
groet, Pim
Hoi Ron, colleagues,
On 27.03.24 08:03, Ron B wrote:
Excellent. Dou you have a precise model or hardware reference for those Supermicro machines so that we can have them sourced locally.
I’ll buy you beers when I see you.
I described it in detail on https://ipng.ch/s/articles/2021/09/21/vpp-7.html both the hardware and the software bits. I'm still running this exact configuration at AS8298. For the list, the chassis I chose is a *Supermicro SYS-5018D-FN8T* [link https://www.supermicro.com/en/products/system/1u/5018/sys-5018d-fn8t.cfm], which includes:
* Full IPMI support (power, serial-over-lan and kvm-over-ip with HTML5), on a dedicated network port. * A 4-core, 8-thread Xeon D1518 CPU which runs at 35W * Two independent Intel i210 NICs (Gigabit) * A Quad Intel i350 NIC (Gigabit) * Two Intel X552 (TenGig) * (optional) One Intel X710 Quad-TenGig NIC in the expansion bus * m.SATA 120G boot SSD * 2x16GB of ECC RAM
Regarding that Intel 4x10G, it's the rather popular *X710-DA4* [link https://ark.intel.com/content/www/us/en/ark/products/83965/intel-ethernet-converged-network-adapter-x710-da4.html]. As an alternative, the same card with 2x25G is available as *XXV710-DA2* [link https://ark.intel.com/content/www/us/en/ark/products/95260/intel-ethernet-network-adapter-xxv710-da2.html] and I operate that card at some places.
But just to re-iterate: any PC with an Intel Xeon D or E, or AMD Zen3+ will do absolutely fine. Some folks may want more PCI slots or dual PSU, so a fine alternative that I also run in production is the *Dell R630* [link https://i.dell.com/sites/doccontent/shared-content/data-sheets/en/documents/dell-poweredge-r630-spec-sheet.pdf] because it allows for up to 16x10G at line rate, using ~120W. In case you want to binge watch some VPP materials - take a look at my [NOG recordings] https://video.ipng.ch/c/ipngnetworks/videos?s=1.
Good luck!
groet, Pim
On 27 Mar 2024, at 11:16, Pim van Pelt via zanog-discuss wrote:
But just to re-iterate: any PC with an Intel Xeon D or E, or AMD Zen3+ will do absolutely fine. Some folks may want more PCI slots or dual PSU, so a fine alternative that I also run in production is the *Dell R630* [link https://i.dell.com/sites/doccontent/shared-content/data-sheets/en/documents/dell-poweredge-r630-spec-sheet.pdf ] because it allows for up to 16x10G at line rate, using ~120W. In case you want to binge watch some VPP materials - take a look at my [NOG recordings] https://video.ipng.ch/c/ipngnetworks/videos?s=1 .
for those following this, thread, thanks pim for posting your talk from last week’s GRNOG : https://video.ipng.ch/w/nnYWXW3Gw9dA7kWGswf1g1
-n.
Hi
120W seems pretty low power.
Last time I checked 8 core Xeon D uses around 85 Watt when doing basic Linux routing at low-ish speeds.
Always figured VPP would be much more power hungry? keeping all the cores busy all the time.
I just had a look at a dell r630 with 2 x Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz
At a moderate workload, (VMs) With 2 x spinning disks for boot disks
1 x dual port 10G card.
Power draw : currently 125W, peak of last 24 hours, 135W
So it sounds ok
I guess if you are going to run 16 x 10G ports as suggested it may draw a bit more :-)
Kind regards
Edd
On 1 May 2024, at 18:01, Joe Botha via zanog-discuss wrote:
Hi
120W seems pretty low power.
Last time I checked 8 core Xeon D uses around 85 Watt when doing basic Linux routing at low-ish speeds.
Always figured VPP would be much more power hungry? keeping all the cores busy all the time.
-- Swimmingly, Joe
swimgeek.com/blog +27 82 562 6167 instagram.com/joe.swimgeek "...all progress depends on the unreasonable man.”
On 25 Apr 2024, at 11:10, Nishal Goburdhan via zanog-discuss zanog-discuss@lists.nog.net.za wrote:
On 27 Mar 2024, at 11:16, Pim van Pelt via zanog-discuss wrote:
But just to re-iterate: any PC with an Intel Xeon D or E, or AMD Zen3+ will do absolutely fine. Some folks may want more PCI slots or dual PSU, so a fine alternative that I also run in production is the *Dell R630* [link https://i.dell.com/sites/doccontent/shared-content/data-sheets/en/documents/dell-poweredge-r630-spec-sheet.pdf ] because it allows for up to 16x10G at line rate, using ~120W. In case you want to binge watch some VPP materials - take a look at my [NOG recordings] https://video.ipng.ch/c/ipngnetworks/videos?s=1 .
for those following this, thread, thanks pim for posting your talk from last week’s GRNOG : https://video.ipng.ch/w/nnYWXW3Gw9dA7kWGswf1g1
-n. _______________________________________________ zanog-discuss mailing list -- zanog-discuss@lists.nog.net.za To unsubscribe send an email to zanog-discuss-leave@lists.nog.net.za
zanog-discuss mailing list -- zanog-discuss@lists.nog.net.za To unsubscribe send an email to zanog-discuss-leave@lists.nog.net.za
Hi
That sounds like normal Linux workload (most cores fairly idle).
I think the poll mode drivers keep all the CPU cores 100% busy when doing VPP, resulting in more power use?
So, having gone through some pricing/discusions with SuperMicro distributor for South Africa.
All prices below is ROE dependent, ex-VAT, ex-Shipping (~R15k needs to confirm etc.)
Supermicro SYS-5018D-FN8T
This is basically replaced by the 5019D 8core/16thread The single PSU, internal 3.5/2.5”drives: SYS-5019D-FN8TP: R35k chassis Redundant PSU, hotswap 2.5” drives: SYS-5019D-RN8TP R45k chassis -> “standard” PCIe full height x8 dual 25G SFP28 CX-4) - R8k quad 25G SFP28 - R16k (Intel) dual 100G QSFP28 - R15k (Mellanox CX-4)
The upgraded option: SYS-110D-8C-FRDN8TP with 2x25G SFP28 (Haven’t checked bigger core counts)
The more interesting option IMHO: SYS-1019D-FRN5TP just like the 5018D/5019D standard 4x 1GigE, 2x 10GigE RJ45, 2x 10GigE SPF+ Chassis prices: 8/12/14/16c Chassis ~R45k/R55k/R66k/R80k
What is the nice part is 4x PCIe x8 AOM slots AOM 4x10G SPF+ Intel X710 ~R7.5k
On 27 Mar 2024, at 11:16, Pim van Pelt via zanog-discuss zanog-discuss@lists.nog.net.za wrote:
Hoi Ron, colleagues,
On 27.03.24 08:03, Ron B wrote:
Excellent. Dou you have a precise model or hardware reference for those Supermicro machines so that we can have them sourced locally. I’ll buy you beers when I see you.
I described it in detail on https://ipng.ch/s/articles/2021/09/21/vpp-7.html both the hardware and the software bits. I'm still running this exact configuration at AS8298. For the list, the chassis I chose is a Supermicro SYS-5018D-FN8T [link], which includes: • Full IPMI support (power, serial-over-lan and kvm-over-ip with HTML5), on a dedicated network port. • A 4-core, 8-thread Xeon D1518 CPU which runs at 35W • Two independent Intel i210 NICs (Gigabit) • A Quad Intel i350 NIC (Gigabit) • Two Intel X552 (TenGig) • (optional) One Intel X710 Quad-TenGig NIC in the expansion bus • m.SATA 120G boot SSD • 2x16GB of ECC RAM Regarding that Intel 4x10G, it's the rather popular X710-DA4 [link]. As an alternative, the same card with 2x25G is available as XXV710-DA2 [link] and I operate that card at some places. But just to re-iterate: any PC with an Intel Xeon D or E, or AMD Zen3+ will do absolutely fine. Some folks may want more PCI slots or dual PSU, so a fine alternative that I also run in production is the Dell R630 [link] because it allows for up to 16x10G at line rate, using ~120W. In case you want to binge watch some VPP materials - take a look at my [NOG recordings].
Good luck!
groet, Pim -- Pim van Pelt pim@ipng.ch PBVP1-RIPE - https://ipng.ch/ _______________________________________________ zanog-discuss mailing list -- zanog-discuss@lists.nog.net.za To unsubscribe send an email to zanog-discuss-leave@lists.nog.net.za
On 27 Mar 2024, at 11:16, Pim van Pelt via zanog-discuss zanog-discuss@lists.nog.net.za wrote:
For the list, the chassis I chose is a Supermicro SYS-5018D-FN8T [link https://www.supermicro.com/en/products/system/1u/5018/sys-5018d-fn8t.cfm],
I’ve tried to sent summary information about the prices I got for the 5019D & 1019D etc. but seemed that email go somestuff in it that was deemed needing moderation approval, so if anybody else is interested in the pricings, let me know and I can direct email you
Hi Pim
Digging up a bit of an older thread,
But out of interest, do you still use Kees for your implementations as the configuration tool? Or is there something better to use now?
Kind regards
Edd
On 27 Mar 2024, at 11:16, Pim van Pelt via zanog-discuss wrote:
Hoi Ron, colleagues,
On 27.03.24 08:03, Ron B wrote:
Excellent. Dou you have a precise model or hardware reference for those Supermicro machines so that we can have them sourced locally.
I’ll buy you beers when I see you.
I described it in detail on https://ipng.ch/s/articles/2021/09/21/vpp-7.html both the hardware and the software bits. I'm still running this exact configuration at AS8298. For the list, the chassis I chose is a *Supermicro SYS-5018D-FN8T* [link https://www.supermicro.com/en/products/system/1u/5018/sys-5018d-fn8t.cfm ], which includes:
- Full IPMI support (power, serial-over-lan and kvm-over-ip with HTML5), on a dedicated network port.
- A 4-core, 8-thread Xeon D1518 CPU which runs at 35W
- Two independent Intel i210 NICs (Gigabit)
- A Quad Intel i350 NIC (Gigabit)
- Two Intel X552 (TenGig)
- (optional) One Intel X710 Quad-TenGig NIC in the expansion bus
- m.SATA 120G boot SSD
- 2x16GB of ECC RAM
Regarding that Intel 4x10G, it's the rather popular *X710-DA4* [link https://ark.intel.com/content/www/us/en/ark/products/83965/intel-ethernet-converged-network-adapter-x710-da4.html ]. As an alternative, the same card with 2x25G is available as *XXV710-DA2* [link https://ark.intel.com/content/www/us/en/ark/products/95260/intel-ethernet-network-adapter-xxv710-da2.html ] and I operate that card at some places.
But just to re-iterate: any PC with an Intel Xeon D or E, or AMD Zen3+ will do absolutely fine. Some folks may want more PCI slots or dual PSU, so a fine alternative that I also run in production is the *Dell R630* [link https://i.dell.com/sites/doccontent/shared-content/data-sheets/en/documents/dell-poweredge-r630-spec-sheet.pdf ] because it allows for up to 16x10G at line rate, using ~120W. In case you want to binge watch some VPP materials - take a look at my [NOG recordings] https://video.ipng.ch/c/ipngnetworks/videos?s=1 .
Good luck!
groet, Pim
-- Pim van Peltpim@ipng.ch PBVP1-RIPE -https://ipng.ch/
zanog-discuss mailing list -- zanog-discuss@lists.nog.net.za To unsubscribe send an email to zanog-discuss-leave@lists.nog.net.za
*> From:* Pim van Pelt via zanog-discuss zanog-discuss@lists.nog.net.za
I'd love to learn more about the local market, what you can and cannot
buy.
There is no local whitebox network appliance market in ZA and if you want it you need to import it yourself. In ZA the preferred kit is plastic floaters that are cheaper than a plate of slaptjips! 😊😊😊(At least some of them you can flash to OpenWRT).
zanog-discuss@lists.nog.net.za
-
Edrich de Lange -
Fearghas Mckay -
hvjunk -
Jade Benson -
Joe Botha -
Nishal Goburdhan -
Noah -
Pim van Pelt -
Ron B -
Sander Steffann