Performance Optimization of Hadoop Using InfiniBand RDMA

DK Panda

“The Hadoop framework has become the most popular open-source solution for Big Data processing. Traditionally, Hadoop communication calls are implemented over sockets and do not deliver best performance on modern clusters with high-performance interconnects. This talk will examine opportunities and challenges in optimizing performance of Hadoop with Remote DMA (RDMA) support, as available with InfiniBand, RoCE (RDMA over Converged Enhanced Ethernet) and other modern interconnects.”

Spark Panel Discussion with Cloudera, MapR & Pivotal

Spark_logo_feature

The panel discussion video below comes from the Los Angeles Spark Users Group. The talk fosters a lively discussion on Spark’s initial goals, where it came from and what the future holds for Spark. Many leading Big Data vendors are responding by introducing Spark’s capabilities into their architectures. The panel discussion is between the top Hadoop distribution vendors – Cloudera, MapR, and Pivotal.

Interview: Replacing HDFS with Lustre for Maximum Performance

Gabriele Paciucci

“When organizations operate both Lustre and Apache Hadoop within a shared HPC infrastructure, there is a compelling use case for using Lustre as the file system for Hadoop analytics, as well as HPC storage. Intel Enterprise Edition for Lustre includes an Intel-developed adapter which allows users to run MapReduce applications directly on Lustre. This optimizes the performance of MapReduce operations while delivering faster, more scalable, and easier to manage storage.”

Datameer Introduces Smart Execution Compute Framework

Datameer, an end-to-end big data analytics application for the Hadoop ecosystem, today introduced Datameer 5.0 with Smart Execution, a new patent-pending technology that intelligently and dynamically selects the best-of-breed compute framework at each step in the big data analytics process.

Intel Steps up with Enterprise Edition for Lustre Software

Brent Gorda

In this video from the 2014 Lustre Administrators and Developers Conference, Brent Gorda from Intel describes how the company is adding enterprise features to the Lustre File System.

Dirk Slama Keynote on The Internet of Things

Dirk Slama, Director of Business Development, Bosch Software Innovations

“The vision for the Internet of Things is very powerful – a world in which assets, devices, machines, and cloud-based applications seamlessly interoperate, enabling new business models and services; with big data analytics as a foundation to support intelligent decision making in this connected world. As with every vision, the question is how to make it happen. This presentation provides key success factors for IoT, as well as a detailed overview of concrete IoT uses cases in the areas of automotive and transport, manufacturing and supply chain, as well as energy. Finally, a framework for IoT implementation is presented, which helps making your IoT projects a success.”

Continuent Announces Tungsten 3.0

continuent_logo

Continuent, Inc., a leading provider of open source database clustering and replication solutions, today announced Continuent Tungsten 3.0, a powerful solution that combines advanced clustering and replication technologies to meet the transaction processing and analytic needs of the entire business.

Earn Your Masters in Predictive Analytics from Northwestern

northwestern-university-logo

As businesses seek to maximize the value of vast new stores of available data, Northwestern University’s Master of Science in Predictive Analytics program prepares students to meet the growing demand in virtually every industry for data-driven leadership and problem solving.

Performance Comparison of Intel Enterprise Edition Lustre and HDFS for MapReduce

tata

In this video from the LAD’14 Lustre Administrators and Developers Conference in Reims, Rekha Singhal from Tata Consultancy Services presents: Performance Comparison of Intel Enterprise Edition Lustre and HDFS for MapReduce Applications.

Slidecast: Fortissimo Foundation – A Clustered, Pervasive, Global Direct-remote I/O Access System

a3cube

“Fortissimo Foundation is a clustered, pervasive, global direct-remote I/O access system that linearly scales I/O bandwidth, memory, Flash and hard disk storage capacity and server performance to provide an “in-memory” scale-out solution that intelligently aggregates all resources of a data center cluster into a massive global name space, bridging all remote compute and storage resources to look and act as if they were local.”