In this podcast, the Radio Free HPC team discusses Lustre and LUG 2013 with Brent Gorda. Now part of Intel in their High Performance Data Division, Gorda was CEO of Whamcloud when the company was acquired last summer.
Gorda recently wrote a post about the rapid growth of the Lustre community, so we started our discussion there and learned a good deal more about the popular file system.
Download the MP3 * Subscribe on iTunes * Subscribe to RSS
In this video from the Lustre User Group 2013, John Fragalla from Xyratex presents: High Availability in Lustre.
Download the slides (PDF) or check out our LUG 2013 Video Gallery.
In this video from the Lustre User Group 2013, Jeff Denworth from DDN presents: DataDirect Networks Update.
DDN has developed a Hadoop solution that is all about time to value: It simplifies rollout so that enterprises can get up and running more quickly, provides typical DDN performance to accelerate data processing, and reduces the amount of time needed to maintain a Hadoop solution.” said Dave Vellante, Chief Research Officer, Wikibon.org. “For enterprises with a deluge of data but a limited IT budget, the DDN hScaler appliance should be on the short list of potential solutions.”
Download the slides (PDF) or check out our LUG 2013 Video Gallery.
In this video from LUG 2013, Jeff Johnson from Aeon Computing presents an overview of the company’s innovative Lustre storage solutions.
Our focus is on the design and deployment of highly-integrated HPC, clustered computing, and storage solutions for all areas of computer-aided research and production. With over 55 years of staff experience in high-performance computing, enterprise computing architectures, and data storage, our focus is on architecting a perfectly-suited solution for your needs. We do not adhere to the large manufacturer approach of “one size fits most”. Every application or research methodology is different. We prefer to learn about our customer’s research, their needs and challenges. Our strength comes from being able to draw from many different types of technologies and configuration approaches. Our goal is to design and deploy a cluster or system for our customers that avoids needless bottlenecks, limitations or design and configuration flaws that are commonly found in solutions designed by companies that are more focused on profits or sales incentives from manufacturers.”
When it comes to security, Big Data is a two-edged sword.
On one hand it can be used to analyze mountains of data in order to foil intruders, head off attacks and neutralize a wide variety of other threats. But the network architecture required to support Big Data analytics is itself vulnerable to attack.
Writing in CSO Magazine, John P. Mello, Jr. notes that Hadoop is frequently used in order to manage the computer clusters that are at the heart of Big Data deployments. This, he says, can create problems for security people, especially if they are relying on traditional security tools.
He quotes a white paper from Zettaset, a Big Data security company, which asserts, “Incumbent data security vendors believe that Hadoop and distributed cluster security can be addressed with traditional perimeter security solutions such as firewalls and intrusion detection/prevention technologies. But no matter how advanced, traditional approaches that rely on perimeter security are unable to adequately secure Hadoop clusters and distributed file systems.”
Traditional security products are designed to protect a single database. But when these products are called upon to protect a distributed cluster of computers that may number in the thousands, they fall short.
Mello interviewed Zettaset CTO Brian Christian.
When you put them (traditional security products) on a large scale distributed computing environment, they become either a choke point or a single point of failure for the entire cluster,” Christian said. “They could potentially be extremely dangerous running them on a cluster, because if they do fail, there is the potential to deny everybody on the cluster access to petabytes of data or a corruption of data in some of the encryption security technologies.”
Other problems arise when security is “bolted on” to an existing Big Data infrastructure, a costly and often ineffective procedure.
And, the story notes, when it comes to business versus security, business requirement takes precedence over implementing an ideal security solution. Says Chris Petersen, CTO of LogRhythm, “While security catches up, there is going to vulnerability. My guess is that there is a lot of vulnerability right now in organizations adopting Hadoop.”
Read the Full Story.
In this video from the Strata 2013 Conference, David Smith from Revolution Analytics describes the five stages of real-time analytics deployment, and the technologies supporting each stage, including Hadoop, R, and database warehousing systems. He also shares some best practices for setting up a the technology stack and processes for model deployment, based on some real-life case studies.
In this video from the Lustre User Group 2013 conference, Josh Judd from Warp Mechanics presents: Lustre & ZFS go to Hollywood.
WARP Mechanics Ltd. is a leading provider of high performance computing (HPC) solutions. The company mission is to bring these super computing technologies into broader IT markets. Each WARP product is factory-optimized for vertical markets such as public-sector “Big Science”, commercial Bio/Life, Cloud, or Media/Entertainment, and can be rolled out in a turn-key fashion.
In this video from the 2013 Open Fabrics Developer Workshop, D.K. Panda from Ohio State University presents: High Performance RDMA-based Design for Big Data and Web 2.0 memcached.
You can check out more OFA videos at our Open Fabrics Workshop Video Gallery.
In this video from the Lustre User Group 2013 conference, Sage Weil from Inktank presents: An Intro to Ceph for HPC.
Ceph is a free software unified storage platform designed to present object, block, and file storage from a single distributed cluster. Ceph’s main goals are to be completely distributed without a single point of failure, scalable to the exabyte level, and freely-available. The data is seamlessly replicated, making it fault tolerant. Ceph is a software-based solution and runs on commodity hardware. The system is designed to be both self-healing and self-managing and strives to reduce both administrator and budget overhead.
Check out more presentations at our LUG 2013 Video Gallery.
In this video from the Lustre User Group 2013 conference, Josh Judd from Warp Mechanics presents: Warp Mechanics ZFS Array.
The WARP Mechanics 39830 is a turnkey network-attached non-volatile RAM + SSD system with industry-leading price, performance, and scalability. This system maximizes the IOPs performance for the most demanding application profiles. It is an ultra-dense space and power saving solution. This is optimal for large-scale IO intensive workloads with large live data sets. The 50x high capacity 2TB SSD modules per 4U enclosure are configured into five 10-disk RAID 6 sets to maximize protection and performance. Each RAID set has a two NV-RAM modules serving as write cache. These RAID sets are added to the overall ZFS storage pool and can be allocated to a nearly limitless number of any sized volumes presented to hosts. This yields a flexible 100TB of usable RAID protected SSD storage.
Check out more Lustre presentations at our LUG 2013 Video Gallery.
Advertisement
inside-bigdata.com is a production of insideHPC, LLC. © 2011-2013 Sitemap
