SAN JOSE, Calif. – PLX Technology is developing a variant of its PCI Express switches to act as a fabric for a variety of servers and network appliances. The effort is one of several proposals to expand the reach and role of PCIe, now the subject of a lively debate.
Server designers at Facebook want to use a PCIe fabric as the basis for future processor cards that ease the task of upgrading CPUs in the thousands of racks in its large data centers. Currently the company uses server cards that include relatively expensive 10 Gbit/second Ethernet links.
“Our goal is to use only PCIe in the rack so we can refresh every component on its own natural life cycle,” said Amir Michael, a senior hardware designer at Facebook which leads the Open Compute Project, trying to set standards in server design.
[ARM TechCon 2012, the largest ARM design ecosystem under one roof, is Oct. 30 - Nov. 1 in Santa Clara. Click here to check out agenda.]“I believe [a PCIe fabric] is one of the next big frontiers for servers,” Michael said. “It has a lot of benefits, but also some technology challenges because PCIe was not designed for that and there are no long distance, cost-effective PCIe components now,” he added.
PLX (San Jose) hopes to fill the gap with a chip and software it currently has in development with unnamed partners. It includes “standards-compliant extensions to its switches” and is aimed at data-center and high-performance computing servers as well as network appliances such as dedicated storage systems, said a PLX representative.
The PLX approach will offer twice the throughput of 10G Ethernet with slightly lower CPU utilization. PLX is working with standards groups to get its approach adopted.
Sources at two server makers said the PLX effort is one of several similar proposals in the industry. However, none are actively under consideration at the PCI Special Interest Group, said one senior I/O engineer who asked not to be named.
“Perhaps for some purpose-built applications, [PLX] will find an interesting market, but for the volume space of cloud and enterprise data centers, the world is dominated by Ethernet,” the engineer said. “With the rise of processors with integrated Ethernet NICs across multiple design points, the technical challenges may be the least to overcome,” he said.
“Convincing customers to implement two fabrics, two management domains, and etc., will be a challenge,” the engineer said.
He noted Infiniband has faced similar challenges and thus “still remains a niche for purpose-built environments.” In addition, Intel scrapped several years ago its Advanced Switching initiative that tried to create a version of PCIe acting as a broad clustering and network fabric, he added.
Intel recently said it is developing a clustering interconnect it will put on its future server processors, borrowing techniques from many technologies including Ethernet, it said. Separately, archrival Advanced Micro Devices is promoting the Freedom Fabric acquired with startup SeaMicro.
Related stories:Intel preps interconnect for server CPUsAMD fabric takes first step to broader market
Intel pulls plug on Advanced Switching