Discovering Gold with Big Data Analytics and Data-Intensive Computing

Advertisement

Entries filed under “Lustre”

Video: Architecting High Availability Lustre Storage with ClusterStor 6000

By Rich | Published February 11, 2013

In this video, John Fragalla from Xyratex presents: Architecting High Availability Lustre Storage with ClusterStor 6000.

ClusterStor 6000 is designed to support installations with linear performance scalability in less space, scaling from up to 6 gigabytes per second to installations providing 1 terabyte per second file system throughput, as well as linear data storage capacity from terabytes up to tens of petabytes.

The presentation was recorded at the HPC Advisory Council Stanford Conference 2013. Download the slides (PDF).


Also posted in Hardware, HPC, I/O, Software, Storage, Video | Leave a comment

Video: Intel’s Brent Gorda on the Company’s Move to the Board Level of OpenSFS

By Rich | Published February 6, 2013

In this video, Brent Gorda from Intel’s High Performance Data Division provides an update on the Lustre File System development. Gorda joined Intel in July 2012 as part of Intel’s acquisition of Whamcloud. Since then, Gorda’s team has continued to work on Lustre as well as conduct R&D on Darpa’s Fast Forward Storage & IO program.

Gorda goes on to share that Intel has now upgraded their participation in the OpenSFS community to the Board (Promoter) level, joining Cray, DDN, LLNL, ORNL, and Xyratex.


Also posted in HPC, Software, Video | Leave a comment

Slidecast: Wrangling Unstructured Data with SGI Infinite Storage & Scality Ring

By Rich | Published February 5, 2013

In this slidecast, Floyd Christofferson from SGI describes how the combination of the company’s Infinite Storage platform and Scality Ring technology provide a new, unified scale-out storage system. The solution is designed to provide both extreme scale and high performance, allowing customers to manage storage of massive stores of unstructured data.

Scale-out object-based solutions are designed to address this particular set of problems by minimizing manual intervention for storage expansions, migrations, and recoveries from storage system failure,” said Ashish Nadkarni, research director, Storage Systems at IDC. “Such a dispersed, fault-tolerant architecture enables IT organizations to more efficiently absorb data growth in a manner that is predicable for the long term.”

Read the Full Story * Download the MP3 * Download the Slides (PDF)* Subscribe on iTunes * If Dropbox is blocked, download audio from Google Drive.


Also posted in Business of Big Data, HPC, Podcasts, Software | Leave a comment

Slidecast: Eric Barton Updates Progress on Fast Forward Storage & IO Program

By Rich | Published January 29, 2013

In this slidecast, Eric Barton, Lead Architect for Intel’s High Performance Data Division presents a progress update on the Fast Forward I/O & Storage program.

Back in July 2012, Whamcloud was awarded the Storage and I/O Research & Development subcontract for the Department of Energy’s FastForward program. Shortly afterward, the company was acquired by Intel. The two-year contract scope includes key R&D necessary for a new object storage paradigm for HPC exascale computing, and the developed technology will also address next-generation storage mechanisms required by the Big Data market.

The subcontract incorporates application I/O expertise from the HDF Group, system I/O and I/O aggregation expertise from EMC Corporation, object storage expertise from DDN, and scale testing facilities from Cray, teamed with file system, architecture, and project management skills from Whamcloud. All components developed in the project will be open sourced and benefit the entire Lustre community.

This is a fascinating presentation for those interested in how an Exascale system might handle data, and the prototype that comes out of it may well represent the roadmap to the future of supercomputing.

Download the MP3 * Download the Slides (PDF) * Subscribe on iTunes * If Dropbox is blocked, download audio from Google Drive.


Also posted in HPC, Podcasts, Software, Video | Leave a comment

Henry Newman on File System Interface Futures

By Rich | Published January 15, 2013

Over at Enterprise Storage Forum, Henry Newman looks at the future of file systems and examines whether REST will overtake POSIX as an interface of choice for all applications.

We do not have a lot of POSIX file systems that scale today to 10s of PB and billions of files. There are three file systems in production with a parallel namespace (Gluster, PAN-FS, Lustre, and GPFS) and a new entry called Ceph. Ceph, GPFS Lustre and Pan-FS support parallel I/O, which is I/O from multiple threads (these threads could be running on multiple nodes) to a single file, but Gluster does not. On the other side there are dozens of vendors developing REST- and SOAP-based object management interfaces. Vendors are trying to create systems that support billions of objects in a single namespace. Given that the vendors are not constrained by the POSIX atomicity requirements and support for parallel I/O, this is far easier than developing this support inside a POSIX file system.

Read the Full Story.


Also posted in Ceph, I/O, Software, Storage | Leave a comment

Aeon Computing Showcases Award-Winning Data Oasis at SC12

By Rich | Published December 2, 2012

In this video from SC12, Doug Johnson from Aeon Computing describes the company’s innovative Data Oasis technology powered by the Lustre file system.

What advantages does Lustre offer as a foundation for a storage system? Bandwidth. Its performance scales out linearly as the file system scales in build out. The more object servers you have, the more network paths you have, the faster your potential. It is the opposite of a large scale monolithic NFS appliance with one spigot.”

For more information, check out our exclusive interview with Aeon’s co-founder, Jeff Johnson.


Also posted in Events, Hardware, Software, Storage, Video | Leave a comment

Video: OpenSFS and EOFS Foster the Lustre Community at SC12

By Rich | Published November 22, 2012

In this video from SC12, Norm Morse from OpenSFS and Hugo Falter from the European EOFS get together over a couple of beers to discuss the latest developments in the Lustre community on the road to Exascale.

The group’s initial focus is the Lustre parallel file system, which supports many of the requirements of leadership class HPC simulation environments, has a diverse development community and is open-source software.”

In related news, LUG 2013 will be held in San Diego April 15-17, 2013.


Also posted in HPC, Software, Video | Leave a comment

Interview: Aeon’s Data Oasis Storage Platform Powered by Lustre

By Rich | Published November 19, 2012

We are big fans of Lustre here at insideHPC. And when I learned that Aeon Computing was building their Data Oasis storage platform on the open source file system, I sought out the company’s co-founder Jeff Johnson to learn more

insideHPC: Why build a Lustre appliance? What was the customer problem you set out to solve?

Jeff Johnson: This is all about performance. We originally set out to plan a Lustre deployment and brought in storage solutions from various well known manufacturers for Lustre benchmarking. We were surprised to discover that many storage devices that looked to be high performance on paper performed quite poorly in a Lustre environment. As a result we decided to develop one ourselves. In this particular case the customer needed Lustre performance without all of the extra, and costly, active failover high-availability features found in the very high end Lustre solutions available. That is what we built.

insideHPC: What sets the Oasis Appliance from other available storage solutions available out there?

Jeff Johnson: There are many storage solutions available in the market but not all of them do Lustre well. We set out to design a Lustre platform that was good at Lustre data and I/O profiles. Part of that design, in addition to performance, is that it follows Aeon Computing’s business philosophy in that there is no unnecessary, extraneous bull___t that gets in the way.

insideHPC: Is Oasis just for the HPC market, or does it have appeal to the Big Data and Cloud-based applications?

Jeff Johnson: The Data Oasis filesystem is specifically an HPC application. The EclipseSL appliance is well suited for any application where reliable, dense high-performance storage is needed. We have mainly focused on Lustre, the EclipseSL would perform well in Gluster and other distributed filesystem applications.

insideHPC: How does the Aeon design change the way you architect a Lustre system?

Jeff Johnson: We designed the EclipseSL with no bottlenecks. The data flow from the network interfaces through bus infrastructures and memory to block storage and back out encounter no bottlenecks or oversubscription. I don’t know that what we did necessarily changes the industry in any way. We simply started with a blank sheet of paper.

insideHPC: What advantages does Lustre offer as a foundation for a storage system?

Jeff Johnson: Bandwidth. Its performance scales out linearly as the file system scales in build out. The more object servers you have, the more network paths you have, the faster your potential. It is the opposite of a large scale monolithic NFS appliance with one spigot.

insideHPC: What was your first reaction when you learned you won the Best of Show Award?

Jeff Johnson: We checked the calendar to make sure it wasn’t April Fools Day. There may have been some expletives of disbelief uttered in the office. Once it sunk in, humbled, flattered and still a bit surprised.

Check out Aeon this week at SC12 booth #2119.


Also posted in Hardware, HPC, Software, Storage | Leave a comment

Speed Big Data Results with the New NetApp HPS Rack Solution

By Rich | Published November 9, 2012

Today, NetApp announced the new High-Performance Storage (HPS) Rack. Prepackaged and preconfigured, the HPS Rack includes servers, management and monitoring software, and the open-source Lustre parallel file system, enabling organizations to shorten deployment time from months to days, lower risk, reduce total cost of ownership, and speed time to results.

For fast insight to action, the best software and hardware needs to be employed to achieve the highest levels of performance for our researchers,” said Scott Kornblum, managing director, Thesys Technologies, the infrastructure affiliate of Tradeworx. “Once we started running our NetApp E5400 and E2600 with Terascala’s LustreStack, it became dramatically easier for us to manage resources, helping us reduce costs. We can now monitor performance in real time and provide 5GB per second of throughput to our research group, giving them the tools they need to test and model new algorithms concurrently.”

The NetApp HPS Rack is built on the NetApp E-Series platform. Additionally, the NetApp HPS Rack can integrate Terascala LustreStack software to provide proven management and extensive analytics. With LustreStack, customers get a Lustre-based solution that deploys in minutes, is easy to manage, and is optimally tuned for applications and workflows. Administrators can get all the performance and scalability benefits of Lustre without the need for any in-house expertise, bringing rapid deployment, simplicity, and faster time to results to big data and HPC environments.

The NetApp HPS Rack will be demonstrated at next week’s SC12 Conference in Salt Lake City, Utah, from November 10–16. Come and visit NetApp at booth #4626.


Also posted in Business of Big Data, Hardware, Software, Startups | Leave a comment

NEC Goes with Dot Hill for LXFS Lustre-Powered Storage

By Rich | Published September 20, 2012

Today SAN storage provider Dot Hill Systems announced that it has been selected by NEC’s HPC group in Europe to supplement the LXFS parallel file system based on Lustre technology. The joint solution integrates Dot Hill Assured SAN 3000 and 3003 storage arrays with NEC’s file system products.

Technical and scientific computing creates huge amounts of data at a high rate, therefore scalable file systems became critical components for building HPC systems. Dot Hill storage complements our LXFS parallel file system by exceeding our customers’ expectations in terms of scalability, reliability, performance and cost-effectiveness,” said Dr. Erich Focht, Senior Manager R&D and Storage Products for NEC High Performance Computing Europe. “We take advantage of the 6GB SAS host interface option offered by the AssuredSAN platform connected in a DAS configuration which provides a robust and scalable infrastructure for our performance-intensive applications.”

According to NEC, all Linux HPC Clusters with InfiniBand infrastructure can now be equipped with LXFS, a high performance parallel file system based on Lustre technology. Jointly developed with a customer in the automotive industry, LXFS complements the performance of Lustre by adding outstanding reliability.

The Dot Hill storage portfolio offers up to 288 terabytes of capacity in a single system. Read the Full Story.


Also posted in Business of Big Data, HPC, Software, Storage | Leave a comment

Advertisement


View All Videos

inside-bigdata.com is a production of insideHPC, LLC. © 2011-2013 Sitemap