TWiki> DICE Web>WebLeftBar>ServiceCatalogue>ServiceCatalogueEntryHadoop (03 Feb 2011, ChrisCooke)EditAttach
ServiceCatalogueForm
Name Hadoop cluster
Description Hadoop consists of a distributed filesystem (HDFS) and a distributed Map/Reduce processing framework. See HadoopCluster for more information.
Unit Research & Teaching
End Users Research Staff, Research PG Students, Taught PG Students, UG Students
Software Hadoop
Hardware The cluster is administered by two dedicated machines borrowed from the Beowulf cluster. The rest of the Hadoop cluster runs on all the other machines in the Beowulf cluster - which do other jobs at the same time as running Hadoop - and a collection of spare desktops which are dedicated to the Hadoop cluster temporarily until allocated to desktop duties elsewhere.
Risk One Person
Review Status No University Equivalent
Review Details web search for 'site:ed.ac.uk Hadoop'
Dependencies

Redundancy Hadoop is designed to cope with node failure. Both the filesystem and the job processing can recover cleanly and automatically from the failure of a node. However there are two exceptions: the "namenode" is a single point of failure. The namenode's metadata is mirrored in real time to another node (the "secondary namenode") by Hadoop.
Connection Authentication Kerberos
Connection Authentication Details (if Other)

Connection Authorization LCFG, Roles/Capabilities
Connection Authorization Details (if Other)

Connection Configuration Management LCFG
Connection Configuration Management Details (if Other)

Connection Visibility Other
Connection Visibility Details (if Other) inf.ed.ac.uk
Connection Supported Clients any client in inf.ed.ac.uk that can ssh to the namenode
Connection Client Configuration

Backend Authentication Inherited from Connection
Backend Authentication Details (if Other)

Backend Authorization LCFG
Backend Authorization Details (if Other)

Backend Configuration Management LCFG
Backend Configuration Management Details (if Other)

User Feature Authentication Not Applicable
User Feature Authentication Details (if Other)

User Feature Authorization Not Applicable
User Feature Authorization Details (if Other)

User Feature Configuration Management LCFG
User Feature Configuration Management Details (if Other)

Link to End User Documentation HadoopCluster
Link to Technical Documentation HadoopCareAndFeeding
Miscellaneous Comments

Edit | Attach | Print version | History: r1 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r1 - 03 Feb 2011 - 14:11:02 - ChrisCooke
 
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback
This Wiki uses Cookies