In this video, D.K. Panda from Ohio State University presents: Accelerating Big Data with Hadoop and Memcached. The presentation was recorded at the HPC Advisory Council Stanford Conference 2013. Download the slides (PDF).
This week Xyratex announced a partnership with analytics leader Pentaho Corporation to develop the industry’s first fully integrated Big Data analytics and scalable storage solutions. The combined offerings, which will be released later this year, will help organizations decrease the amount of hardware and software required to complete major data analysis projects – and unlock the limitless potential of Big Data while delivering lower total cost of ownership (TCO) to end users.
This tight integration of the analytics engine and the data storage into the same solution will remove performance bottlenecks, reduce deployment complexity, simplify management and ease the scaling of an organization’s big data infrastructure, enabling our customers to garner valuable insights into their business sooner,” said Ken Claffey, senior vice president of the ClusterStor business at Xyratex. “Today, in collaboration with our partners, we’re helping end users achieve best-in-class performance, reliability and scalability – including implementing the fastest data storage system in the world. We’re confident that the combined power of our ClusterStor data storage with Pentaho’s leading analytics will re-define what’s possible with Big Data.”
Read the Full Story.
In this video, Brent Gorda from Intel’s High Performance Data Division provides an update on the Lustre File System development. Gorda joined Intel in July 2012 as part of Intel’s acquisition of Whamcloud. Since then, Gorda’s team has continued to work on Lustre as well as conduct R&D on Darpa’s Fast Forward Storage & IO program.
Gorda goes on to share that Intel has now upgraded their participation in the OpenSFS community to the Board (Promoter) level, joining Cray, DDN, LLNL, ORNL, and Xyratex.
In this slidecast, Floyd Christofferson from SGI describes how the combination of the company’s Infinite Storage platform and Scality Ring technology provide a new, unified scale-out storage system. The solution is designed to provide both extreme scale and high performance, allowing customers to manage storage of massive stores of unstructured data.
Scale-out object-based solutions are designed to address this particular set of problems by minimizing manual intervention for storage expansions, migrations, and recoveries from storage system failure,” said Ashish Nadkarni, research director, Storage Systems at IDC. “Such a dispersed, fault-tolerant architecture enables IT organizations to more efficiently absorb data growth in a manner that is predicable for the long term.”
Ed. Note: The launch event starts at the 28:22 minute mark.
The Pittsburg Supercomputer Center is streaming live video from their launch event for their new Sherlock supercomputer. A uRiKA graph-analytics appliance from YarcData, Sherlock is designed to discover unknown relationships or patterns hidden in extremely large and complex bodies of information.
Sherlock gives PSC the first system available to researchers that is optimized for a particularly difficult family of questions regarding, for example, security, medicine, public health, and social dynamics,” says Nick Nystrom, Director of Strategic Applications, PSC. “These problems cost individuals and society in time, money, and human suffering. Sherlock also helps keep Pittsburgh — and Pennsylvania — at the forefront of high performance computing.”
In this slidecast, Eric Barton, Lead Architect for Intel’s High Performance Data Division presents a progress update on the Fast Forward I/O & Storage program.
Back in July 2012, Whamcloud was awarded the Storage and I/O Research & Development subcontract for the Department of Energy’s FastForward program. Shortly afterward, the company was acquired by Intel. The two-year contract scope includes key R&D necessary for a new object storage paradigm for HPC exascale computing, and the developed technology will also address next-generation storage mechanisms required by the Big Data market.
The subcontract incorporates application I/O expertise from the HDF Group, system I/O and I/O aggregation expertise from EMC Corporation, object storage expertise from DDN, and scale testing facilities from Cray, teamed with file system, architecture, and project management skills from Whamcloud. All components developed in the project will be open sourced and benefit the entire Lustre community.
This is a fascinating presentation for those interested in how an Exascale system might handle data, and the prototype that comes out of it may well represent the roadmap to the future of supercomputing.
This award underscores the rapid adoption of our ClusterStor family of storage solutions, and the tremendous value it brings to data-intensive computing environments,” said Ken Claffey, senior vice president of the ClusterStor business at Xyratex. “The introduction of the ClusterStor 6000 was an important milestone for us, and in collaboration with our partners we’re helping end users achieve best-in-class performance, reliability and scalability – including implementing the fastest data storage system in the world.”
Read the Full Story.
The breakthrough technology – silicon nanophotonics – allows the integration of different optical components side-by-side with electrical circuits on a single silicon chip – using, for the first time, sub-100nm semiconductor technology.
Silicon nanophotonics takes advantage of pulses of light for communication and provides a super highway for large volumes of data to move at rapid speeds between computer chips in servers, large data centers, and supercomputers, thus alleviating the limitations of congested data traffic and high-cost traditional interconnects.
This technology breakthrough is a result of more than a decade of pioneering research at IBM,” said John Kelly, senior vice president and director of IBM Research. “This allows us to move silicon nanophotonics technology into a real-world manufacturing environment that will have impact across a range of applications.”
The amount of data being created and transmitted over enterprise networks continues to grow due to an explosion of new applications and services. Silicon nanophotonics, now primed for commercial development, can enable the industry to keep pace with increasing demands in chip performance and computing power.
A team at the Georgia Institute of Technology has received a $2.7 million award from the Defense Advanced Research Projects Agency (DARPA) to develop technology to help address the challenges of Big Data – data sets that are both massive and complex.
The contract is part of DARPA’s XDATA program, a four-year research effort to develop computational techniques and open-source software tools for processing and analysing data, motivated by defence needs. Georgia Tech was selected to perform research in the area of scalable analytics and data-processing technology.
The team will focus on producing new machine-learning approaches capable of analyzing very large-scale data. Team members will also pursue development of distributed computing methods that can process data-analytics algorithms very rapidly with a variety of systems, including supercomputers, parallel-processing environments and networked, distributed computing systems.
‘This award allows us to build on the foundations we’ve already established in large-scale data analytics and visualisation,’ said Richard Fujimoto, leader of the Georgia Tech team. ‘The algorithms, tools and other technologies that we develop will all be open source, to allow them to be customised to address new problems arising in defence and other applications.’
The award is part of a $200 million multi-agency federal initiative for big-data research and development. It aims to improve the ability to extract knowledge and insights from the nation’s fast-growing volumes of digital data.