Brocade VDX Switches provide automated network foundation for Yahoo Japan
Bangalore, India, September 17, 2014: Brocade today announced that Yahoo Japan Corporation, the country’s largest Internet company and a global digital market leader, has completed deployment of a Brocade Ethernet fabric solution as the network foundation for an enterprise-wide big data project. The new Brocade Ethernet fabric is now delivering high-performance data throughput for Apache Hadoop, while cutting operational costs in half compared to the prior infrastructure.
Big data is a key strategic initiative at Yahoo Japan. The project was implemented to help the company maintain its innovation leadership in providing Internet services to users. Hadoop is used to analyze vast amounts of information obtained from its portal site and leverages such data for improving the quality of Yahoo Japan’s services, as well as creating new services. Hadoop is also used to support e-commerce and to enable millions of daily personalized and relevant content recommendations for the company’s user base.
At Yahoo Japan, Brocade VDX Ethernet Fabric modular and fixed-port network switches replaced a legacy Spanning Tree Protocol-based network. The new automated network infrastructure connects hundreds of servers supporting thousands of nodes dedicated to supporting Hadoop analysts across the entire enterprise.
As the demand and perceived value for big data analysis significantly grew within the organization, Yahoo Japan quickly recognized it needed to build a new, larger Hadoop infrastructure that could be shared across the entire organization. It also became immediately apparent that the legacy network wasn’t suitable for the new Hadoop design, because of limitations in data throughput, the inability to easily add capacity and increasing operational complexities that combined to put application performance and availability at risk.
Providing a Solid Foundation for Big Data
Network requirements to support the new Hadoop environment at Yahoo Japan included the ability to scale-out the environment for additional nodes quickly and cost-effectively, and the need to support bursty, high-speed communications between nodes. Following an analysis of alternatives, Yahoo Japan determined that an Ethernet fabric architecture was best suited for their new Hadoop environment to enable simple and flexible expansion, combined with high performance data throughput.
After determining the network requirements to best support Hadoop, Yahoo Japan meticulously compared and evaluated several Ethernet fabric solutions for performance, functionality, cost and operational management. Upon completion, the company decided on Brocade VCS Fabric technology and Brocade VDX switches because of the performance, low latency, deep buffers as well as automated deployment and operational simplicity, which saves time by eliminating the need for manual configuration.
Today, the newly built, enterprise-wide shared Hadoop network infrastructure consists of multiple clusters with thousands of nodes. High-density Brocade VDX 8770 modular chassis switches are used as aggregation switches that integrate a host of racks to create fabrics between the switches. The VDX chassis are connected to Brocade MLXe core routers using the switches’ high bandwidth multi-chassis trunking (MCT) feature. The company has also deployed fixed-port Brocade VDX switches to build fabrics for smaller Hadoop infrastructures used for other projects at Yahoo Japan group companies.
Analyst Perspective: Ethernet Fabrics for Big Data
According to Gartner, “Big data servers generate intense networking requirements, heavy volume and bursty server-to-server (i.e., east/west) traffic. To address these characteristics network designers must ensure their network switching infrastructure meets key requirements.”
In the same report, Gartner states that switches within a Big Data deployment should address the “High volume of traffic between servers” with “high-bandwidth access and uplink interfaces,” as well as “lower latency.” The switching infrastructure must also address the “bursty traffic patterns and varying data collection patterns” characteristic of Big Data with “line-rate capability with low oversubscription ratios, deep packet buffering capability and lower latency. Organizations that have already implemented a modern ‘Ethernet Fabric’ (i.e., a one- or two-tier physical mesh or partial mesh topology) will likely meet the above networking requirements.”
“In addition to delivering the high-performance needed to support enormous traffic processing for our new Hadoop environment, the Brocade VCS Fabric has allowed us to reduce the time spent on network deployment and operation by more than half, which has significantly lowered operational costs,” said Kenya Murakoshi, Manager of Data Center Network, Infrastructure Engineering Department, Site Operation Division, Yahoo Japan. “Compared to other fabric technologies, the simplicity of Brocade VCS fabrics is incredible, allowing us to quickly scale our network with tremendous flexibility.”
“Yahoo Japan is representative of the increased adoption we’ve seen for our Ethernet fabric switches supporting enterprise big data initiatives,” said Jason Nolet, vice president, Data Center and Campus Networking, at Brocade. “We are extremely pleased to be a key component of the Hadoop solution at Yahoo Japan and are proud to deliver best-in-class network automation and performance to help Yahoo Japan dramatically lower their operational costs while ensuring their mission-critical applications perform at optimal levels.”
