Performance Optimization of Hadoop Using InfiniBand RDMA

DK Panda

“The Hadoop framework has become the most popular open-source solution for Big Data processing. Traditionally, Hadoop communication calls are implemented over sockets and do not deliver best performance on modern clusters with high-performance interconnects. This talk will examine opportunities and challenges in optimizing performance of Hadoop with Remote DMA (RDMA) support, as available with InfiniBand, RoCE (RDMA over Converged Enhanced Ethernet) and other modern interconnects.”

Mellanox InfiniBand Powers Yahoo! Japan

mellanox-logo

Today Mellanox Technologies announced that Yahoo! Japan has deployed Mellanox’s end-to-end FDR 56Gb/s InfiniBand solutions for database acceleration at several data center sites in Japan.

Interview: Mellanox Covers Exascale Computing, the Cloud and Big Data

eyal

“With a novel data analytics approach, Mellanox is providing double performance to data Extract, Transform, Load (ETL) solutions based on Hadoop Map Reduce. The code to enable such performance gains is part of the Hadoop community code. With acceleration of the networking stack, Mellanox is quadrupling the number of clients served on a single Memcached server, a dominating key-value caching service for large scale web applications.”

Mellanox Collaborates with IBM to Speed In-Memory Data Grids

mellanox-logo

Today Mellanox announced a collaboration with IBM to deliver a high-performance infrastructure for NoSQL Databases and In-Memory Data Grids.

IBM Goes Deep for Big Data Market with POWER8 Open Server Innovation

Slide7

“This is the first truly disruptive advancement in high-end server technology in decades, with radical technology changes and the full support of an open server ecosystem that will seamlessly lead our clients into this world of massive data volumes and complexity,” said Tom Rosamilia, Senior Vice President, IBM Systems and Technology Group. “There no longer is a one-size-fits-all approach to scale out a data center. With our membership in the OpenPOWER Foundation, IBM’s POWER8 processor will become a catalyst for emerging applications and an open innovation platform.”

Video: RDMA Accelerated Big Data Applications

In this video from the HPC Advisory Council European Conference 2013, Eyal Gutkind from Mellanox presents: RDMA Accelerated Big Data Applications. Download the slides (PDF) and check out more videos at the HPC Advisory Council European Conference Video Gallery.

Video: Accelerating Big Data over RDMA

In this video from the 2013 Open Fabrics Developer Workshop, Sreev Doddabalapur from Mellanox presents: Accelerating Big Data over RDMA. You can check out more OFA videos at our Open Fabrics Workshop Video Gallery.

Video: Hadoop Acceleration with RDMA

In this video, Eyal Gutkind from Mellanox presents: Hadoop Acceleration with RDMA. The presentation was recorded at the HPC Advisory Council Stanford Conference 2013. Download the slides (PDF).

Mellanox Accelerates Teradata Unified Big Analytics Appliance

Today Mellanox announced that Teradata has chosen its InfiniBand interconnect solution to accelerate the Teradata Aster Big Analytics Appliance. Designed for demanding analytics which require high computational power and the fastest data movement, the Teradata Aster Big Analytics Appliance offers up to 19 times better data throughput and performs analytics up to 35 times faster […]

Colfax and Mellanox Introduce Hadoop Appliance

This week Colfax unveiled the joint demonstration of a high performance Hadoop appliance over Mellanox end-to-end FDR 56Gb/s InfiniBand solution during SC12 at the Mellanox booth. Colfax’s Hadoop appliance provides users with the best in class performance and ease of use in one consolidated solution. Featuring IntelXeon E5 processor based servers and Mellanox FDR 56Gb/s InfiniBand, it […]