Discovering Gold with Big Data Analytics and Data-Intensive Computing

Entries filed under “Lustre”

Video: Warp Mechanics ZFS Array

In this video from the Lustre User Group 2013 conference, Josh Judd from Warp Mechanics presents: Warp Mechanics ZFS Array.

The WARP Mechanics 39830 is a turnkey network-attached non-volatile RAM + SSD system with industry-leading price, performance, and scalability. This system maximizes the IOPs performance for the most demanding application profiles. It is an ultra-dense space and power saving solution. This is optimal for large-scale IO intensive workloads with large live data sets. The 50x high capacity 2TB SSD modules per 4U enclosure are configured into five 10-disk RAID 6 sets to maximize protection and performance. Each RAID set has a two NV-RAM modules serving as write cache. These RAID sets are added to the overall ZFS storage pool and can be allocated to a nearly limitless number of any sized volumes presented to hosts. This yields a flexible 100TB of usable RAID protected SSD storage.

Check out more Lustre presentations at our LUG 2013 Video Gallery.


Also posted in Events, Hardware, HPC, Software, Storage, Video, ZFS | Leave a comment

Video: Lustre on Amazon Web Services

In this video from the Lustre User Group 2013 conference, Robert Read from Intel presents: Lustre on Amazon Web Services.

You can check out more Lustre presentations at our LUG 2013 Video Gallery.


Also posted in Cloud, Events, HPC, I/O, Software, Video | Leave a comment

Video: Hadoop MapReduce over Lustre

In this video from the Lustre User Group 2013 conference, Omkar Kulkarni from Intel presents: Hadoop MapReduce over Lustre.

You can check out more Lustre presentations at our LUG 2013 Video Gallery.


Also posted in Events, Hadoop, Software, Video | Leave a comment

You Can’t Keep a Good god Down

In Greek mythology the Titans, known as the elder gods, ruled the earth until the Olympians tossed them out of power.

At Oak Ridge National Laboratory, one of the Titans is not only still with us and doing well, but this modern day diety just got a boost that will help keep it king of the hill for some time to come.

The world’s fastest supercomputer, Titan is capable of delivering a peak capability of over 27 petaflops, ten times more powerful than previous generations of ORNL computers such as Jaguar. This week the Lab announced that it has selected DataDirect Networks to build the world’s fastest file storage system to ensure Titan’s ascendancy in the world of HPC.

Built around DDN’s SFA 12K-40, the system is being designed with 40 petabytes of raw capacity capable of  “ingesting, storing, processing and distributing research data at unprecedented speed,” according to a DDN press release.  The DDN system will work with the Lab’s Lustre parallel file system with a Lustre performance of over one terabyte per second to handle the demands of Titan’s 299,008 CPU cores.

The world’s toughest questions demand the toughest storage and the fastest technology to drive new levels of scientific insight. DDN has spent the better part of a decade engineering a platform that is built precisely and efficiently for today’s Big Data challenges,” comments Jean-Luc Chatelain, chief technology officer at DDN. “As applications everywhere – from energy exploration to climate modeling to energy efficient car manufacturing – continue to drive extreme levels of computational simulation and data analytics, we’re proud to provide the data storage technology that makes such innovation and economic competitiveness possible. We’re honored to continue our long-standing partnership with ORNL today and to be part of the future of Big Data and exascale computing tomorrow.”

ORNL points out that Titan is architecturally unique in a variety of ways, and is a showcase for tomorrow’s computational requirements as Big Data continues to make inroads into the enterprise.

When building the world’s fastest system for data intensive computing, we carefully considered all aspects of high-throughput I/O infrastructure and how efficient storage platforms can complement our supercomputer’s efficiency,” says Buddy Bland, project director for the Oak Ridge Leadership Computing Facility. “The ORNL and DDN teams have worked together to architect a file system designed to enhance the performance of our Titan supercomputer and enable our users to achieve unprecedented simulations and big data insights through massively scalable computing.”

The Olympians may have thought they dethroned the Titans, but as Yogi Berra once said, “It ain’t over till it’s over.”  Titan at ORNL, with some help from DDN, rules supreme…at least for now.

Read the Full Story.


Also posted in HPC, Software, Storage | Leave a comment

Video: Architecting High Availability Lustre Storage Solution – ClusterStor 6000

In this video from the HPC Advisory Council Switzerland Conference, Torben Kling Petersen from Xyratex presents: Architecting High Availability Lustre Storage Solution – ClusterStor 6000.

Part of the ClusterStor family, ClusterStor 6000 is designed to support installations with linear performance scalability in less space, scaling from up to 6 gigabytes per second to installations providing 1 terabyte per second file system throughput, as well as linear data storage capacity from terabytes up to tens of petabytes.

Download the Slides (PDF).


Also posted in Business of Big Data, Events, HPC, Software, Storage, Video | Leave a comment

With an Eye on Big Data, HPAC Workshop Returns to Lugano March 13-15

The HPC Advisory Council Switzerland Conference returns to Lugano March 13-15, 2013.

The conference will focus on the following topics: Progress of Exascale in the European Union, high-performance interconnects, Accelerators and Parallel I/O, communication libraries (MPI, SHMEM, PGAS), GPU computing (CUDA, OpenCL) Big Data, advanced topics / technologies / development including server and storage systems, and hands-on clustering, network, troubleshooting, tuning, optimizations. The conference is open to the public and will bring together system managers, researchers, developers, computational scientists and industry affiliates.

Having been to this event several times, I can tell you that Lugano is one of the most beautiful towns in the world. It’s a solid three-day workshop, and this year they’ll be treating attendees to a boat trip on lake Lugano with an on-board apero and dinner.

Check out the Preliminary Agenda and Register now.


Also posted in Events, File Systems, HPC, Research, Software, Storage | Leave a comment

Xyratex Acquires Lustre Assets from Oracle, and there is Much Rejoicing

Today Xyratex announced that the company has acquired the original Lustre trademark, logo, website and associated intellectual property from Oracle, and will assume responsibility for providing support to Lustre customers going forward. The company say it plans to advance the global Lustre portfolio by supporting the community-oriented development of Lustre as an open source file system and continuing to work in conjunction with the broader community to help chart the best path forward for this key technology.

Lustre is a powerful open source file system, and Xyratex strongly believes that all members of the Lustre community need to continue to play a part in the evolution of the code and the benefits it delivers over the long term,” said Steve Barber, CEO of Xyratex. “We want to ensure that current Lustre customers get the best possible feature roadmap and support, and we intend to engage the entire community to advance the Lustre technology. We also appreciate Oracle’s support of Lustre, and their efforts to ensure the long-term success of the technology.”

From this reporter’s perspective, this is really good news for the Lustre community. Oracle owned the Lustre assets since it acquired Sun Microsystems in 2010, and the company didn’t take kindly to organizations using the Lustre trademark in the past. Now with these assets in the hands of an active Lustre community member (Xyratex is a board-level sponsor of OpenSFS), one could say that Jack has effectively rescued the Open Source Goose from the mean old Giant.

OpenSFS actively supports the HPC open source file system community of which Xyratex is an active Member,” said Norm Morse, CEO at OpenSFS. “This acquisition gives Xyratex a great opportunity in concert with other members of the Lustre community to continue the stability needed to ensure Lustre remains a vital part of HPC going forward.  We look forward to working with Xyratex in the future.”

With their ClusterStor products now being distributed by the likes of Cray, HP, and Dell, Xyratex is now well-positioned to move forward with their increasing focus on HPC and Big Data markets. Powered by Lustre, ClusterStor technology not only holds the title for world’s fastest file system at over 1 Terabyte/sec, they managed to accomplish that number with about half the storage devices of the next runner-up.

Read the Full Story.


Also posted in Business of Big Data, HPC, Software, Video | Leave a comment

Video: Brent Gorda on The Future of Network-Based Storage

In this video, Brent Gorda from Intel’s High Performance Data Division presents: The Future of Network-Based Storage. Gorda joined Intel in July 2012 as part of Intel’s acquisition of Whamcloud. Since then, Gorda’s team has continued to work on Lustre as well as conduct R&D on Darpa’s Fast Forward Storage & IO program.

The presentation was recorded at the HPC Advisory Council Stanford Conference 2013. Download the slides (PDF).


Also posted in Events, HPC, Software, Storage, Video | Leave a comment

Video: Architecting High Availability Lustre Storage with ClusterStor 6000

In this video, John Fragalla from Xyratex presents: Architecting High Availability Lustre Storage with ClusterStor 6000.

ClusterStor 6000 is designed to support installations with linear performance scalability in less space, scaling from up to 6 gigabytes per second to installations providing 1 terabyte per second file system throughput, as well as linear data storage capacity from terabytes up to tens of petabytes.

The presentation was recorded at the HPC Advisory Council Stanford Conference 2013. Download the slides (PDF).


Also posted in Hardware, HPC, I/O, Software, Storage, Video | Leave a comment

Video: Intel’s Brent Gorda on the Company’s Move to the Board Level of OpenSFS

In this video, Brent Gorda from Intel’s High Performance Data Division provides an update on the Lustre File System development. Gorda joined Intel in July 2012 as part of Intel’s acquisition of Whamcloud. Since then, Gorda’s team has continued to work on Lustre as well as conduct R&D on Darpa’s Fast Forward Storage & IO program.

Gorda goes on to share that Intel has now upgraded their participation in the OpenSFS community to the Board (Promoter) level, joining Cray, DDN, LLNL, ORNL, and Xyratex.


Also posted in HPC, Software, Video | Leave a comment

Advertisement


View All Videos

inside-bigdata.com is a production of insideHPC, LLC. © 2011-2013 Sitemap