09 March 2021 Reading time: 12 minutes

How to choose core level network equipment

Viktor Tkachenko

Viktor Tkachenko

Viktor Tkachenko, Head of Open Network Solutions Department at NAG

ISPSystem

Contents

How to choose network equipment

First, let us define what we mean by the network core. In classical terminology, this is the element that ensures connectivity of all network components and redundancy of connections. Typically, these are high-performance L2/L3 switches with a minimal set of features sufficient to switch/route all traffic within the network.

Sometimes other, more functional network elements are used at the core level. For example, border routers and service gateways (NAT, BNG, DPI, NGFW). Any additional functionality beyond full connectivity within a private network will not be considered here.

The following recommendations are not unique know-how and are mainly intended for young professionals.

Determine the purpose of your purchase

If you are planning to build a new network from scratch, you must first develop a general design. This is a topic for a separate conversation, so we will not go into it.

If the task is to upgrade the network, describe the existing architecture, as well as the one that we wish to obtain (considering that the design is already described). The purpose of the upgrade can be not only to improve the overall performance of the network, but also to introduce new functionality. Often this is not easy to implement without downtime. Prepare a solid plan for transition from the old network to the new network. It is important to consider the requirements for each stage of the upgrade, as well as the characteristics of the old and new architecture.

As a result, the main requirements for all elements of the network, as well as each stage of the upgrade should be identified. Based on them, you should choose the equipment.

Note the basic characteristics of the devices

Key functionality. Often the network infrastructure already uses technology that you do not want to give up. It is important to check whether the equipment offered by the vendor supports all the required functionality.

Device performance. Not only in bits per second, but also in packets per second, which is important in a number of scenarios. If you are planning a significant growth in business in the near future, then choose equipment with a performance surplus.

The need for surplus was particularly acute for telecom operators in 2020, when the coronavirus epidemic caused the total traffic of many operators to increase by 20-30%.

Port capacity. The number of ports on the switch/router should also be selected with allowance for expansion and contingency. Note that standard switches with a small packet buffer provide efficient utilization of up to 75% on average. This is important if you want to ensure minimal delays and prevent losses. That is, the number of ports should be calculated taking into account their incomplete load.

Internet traffic is inherently uneven. This is due to the fact that traffic generators deliver content in large portions within short intervals (see micro-burst ). НFor example, such a phenomenon could be observed in iperf3.

Overlapping bursts of traffic that arrive at different ports on the switch but have the same output port. They can cause short-term packet buffer overflow and loss of some packets.

Sometimes there is a need for maximum utilization of specific devices (e.g. DPI). In doing so, it is necessary to avoid losses on the switch to which this device is connected. Then you can use switches with a deep packet buffer. These switches allow you to survive prolonged interface congestion. But this raises another problem — a significant increase in delay. This can be critical for a number of services. You can get around the problem by configuring QoS correctly.

Switching/routing tables. As a rule, the MAC/ARP/route table size of the switches of the required performance are sufficient for the scenarios in question. And if it isn't, you'll probably have to rethink the design of the network.

Suppose you do decide to choose not a core switch, but an edge router that needs to handle multiple BGP FullView (full routing table on the Internet). In this case, you need to make sure that the routing table is large enough and not just accommodates all the routes today, but also has a reserve (BGP FullView is steadily growing).

We have seen many times that devices with a routing table barely larger than a BGP FullView were purchased as an edge router to save money. As a result, after a year the resources were already insufficient and we had to either give up receiving BGP FullView, or buy new, more expensive equipment.

Of course, such situations are more likely to occur when using outdated equipment, but you should remain vigilant.

Find out the lifetime of the product

Check the product's lifetime, scheduled date of discontinuation and support. This is especially important when using proprietary solutions and in particular chassis switches. Equipment that has been discontinued will be difficult to retrofit with line cards.

If you plan to use a proprietary technology stack, however, it is important to justify its necessity and make sure that the manufacturer will support it on other switch models as well. Otherwise, in the future you run the risk of having an infrastructure, the expansion of which will require a complete replacement of equipment.

In general, we recommend using pizza-box switches without expansion modules and with a standard technology stack that is supported by most manufacturers. In this case, further expansion of the network is possible by adding devices of other vendors.

Study the documentation

Studying the documentation is one of the most important points when choosing equipment. Especially when it comes to new models of equipment.

At the stage of choosing from several models by different manufacturers that fit the functionality, the customer will start comparing prices. Let us say one product is more expensive, but it has a number of advantages (bigger performance margin, additional functionality), and the other is simpler and cheaper, but also meets basic requirements. Even if laboratory testing has already been performed and compliance with the basic requirements has been confirmed in practice, you cannot be 100% sure that there will be no problems during operation. As a rule, lab testing does not cover all scenarios that arise in the final installation. Therefore, it is important to have documentation and support from the vendor.

It is not uncommon to encounter situations where the vendor or developer of the equipment does not provide sufficient technical support. In this case, it is especially important to have quality documentation with options to diagnose and solve problems or recommendations on how to build a network.

It is worth checking open sources for documentation:

  • the developer's website (as a rule, there is quite detailed documentation there);
  • forums (if the equipment has been actively used in the CIS for a long time);
  • other open sources (you can find many user manuals directly from the developer).

The corporate segment often resorts to the services of systems integrators. This allows you to rely on the competence of the solution provider and not to delve into the documentation. However, if a business is looking for options to reduce the cost of equipment, you should be especially careful about the quality of documentation.

I would especially like to mention the segment of telecom operators, where in recent years, saving on purchases has already become a survival tool. When buying new inexpensive equipment, be sure to check the availability and quality of documentation, as well as the supplier's willingness to participate in solving problems that arise on the network without making previously unannounced investments.

Trends in the network equipment market

Performance growth. Perhaps the most obvious trend, because the volume of information is growing and this is directly reflected in the requirements for network performance.

The 800GBASE-R standard was published in 2020. We will probably soon see it implemented in the largest data centers. In the meantime, 32x400GE switches are becoming more affordable, and the 25/100GE devices are becoming comparable in price with 10/40GE, gradually replacing them. At the same time, 10GE on campus networks is getting cheaper and gigabit uplinks are less common at the access level.

Campus network access layer switches are also growing in performance, with more 2.5GE switches hitting the market. This allows to increase the density of wireless access connections, as well as solve some of the problems with traffic buffering.

Disaggregation. What once happened in computing is also happening with networking equipment — a growing range of bare metal or white box switches. Developers offer new network operating systems that can be installed on different platforms.

Disaggregated solutions generate business interest. The attraction here is the ability to replace the software on the switches without buying new hardware, to change the hardware platform vendor at any time, and to maintain compatibility with specific software. Of course, most people still prefer proven closed solutions. Nevertheless, as the competence of the integrator engineers grows so will the confidence in the new approach.

Free software distribution. The rapid growth of the industry owes much to enthusiasts who freely distribute their own designs. The technology is further developed with the support of users. Industry leaders do not simply observe but open the source code of their developments. This allows to form communities for joint development and later sell not the software itself, but professional services for its maintenance.

Operating systems for switches are among the open-source developments. In 2017, we saw the release of SONiC by Microsoft, which is primarily targeted for use in data centers. In 2019, AT&T introduced a free version of DANOS for telecom operators, and at the end of 2020 DentOS 1.0 was released, which is compatible with less expensive 1/10GE switches and provides basic functionality for campus networks. These initiatives are supported by both the consumer and the hardware vendors on whose platforms the open source software being developed is deployed. Large expert communities gather around such initiatives aimed at the free distribution of software and hardware platform designs.

For example, huge communities gather under the leadership of projects such as Open Compute Project, Open Networking Foundation and Telecom Infra Project. They consolidate various problems of the industry, discuss, find solutions and plan their development and promotion.

Such developments do not yet provide the full range of features that we are used to seeing on switches. But they are already capable of solving specific problems, and the development does not stop there. In this decade, such developments will find their use even in small enterprises.

DCImanager for managing physical networks

Try DCImanager platform for managing physical hardware: servers, switches, networks, PDUs.

  • Supports multivendor infrastructure;
  • Manages network equipment, VLANs, IPv4 and IPv6 addresses;
  • Automates routine tasks: OS installation, server diagnostics;
  • Keeps record of equipment assets based on ITAM system.