Hadoop deployment made easy: HA and Kerberos secured cluster in 1 command
Ryba boostraps and manages a full secured Hadoop cluster with one command. This is an Open-source software (OSS) project released under the new BSD license developed for one of the World largest utility company and now operationnal.
Install Ryba locally or on a remote server and you are ready to go. It uses SSH to connect to each server of your cluster and will fully install all the components you wish. You don't need to prepare your cluster nodes as long as a minimal installation of RHEL or CentOS is installed with a root user or a user with sudo access.
- Use secured comminication with SSH
- No database used, full distribution across multiple servers relying on GIT
- No agent or pre-installation required on your cluster nodes
- Version control all your configuration and modifications (using GIT by default)
- Command-based to integrate with your Business Continuity Plan (BCP) and existing scripts
- For developer, as simple as learning Node.js and not a new framework
- Self-documented code written in Literate CoffeeScript
- Idempotent and executable on a running cluster without any negative impact
- Bootstrap the nodes from a fresh install
- Configure proxy environment if needed
- Optionnaly create a bind server (useful in Vagrant development environment)
- Install OpenLDAP and Kerberos and/or integrate with your existing infrastructure
- Deploy the latest Hortonworks Data Platform (HDP)
- Setup High Availabity for HDFS
- Integrate Kerberos with cross realm support
- Set IPTables rules and startup scripts
- Check the running components
- Provide convenient utilities such as global start/stop/status commands, distributed shell execution, ...
First download Node.js. You might need to adjust the name of the Node.js archive depending on the version you choose to install. Also, replace the path "/usr/local/node" to another location (eg "~/node") if you don't have the permission to write inside "/usr/local".
# Extract the Node.js packagetar xzf node-v0.10.28-linux-x64.tar.gz# Move Node.js into its final destinationsudo mv node-v0.10.28-linux-x64 /usr/local/node# Add path to Node.js binaryecho 'export PATH=/usr/local/node/bin:$PATH' >> ~/.bashrc# Source the update profile. ~/.bashrc# Check if node is installednode -v# Clean up uploaded archiverm -rf node-v0.10.28-linux-x64.tar.gz
If you are behind a proxy, configure the Node.js Pakage Manager (NPM) with the commands:
npm config set proxynpm config set https-proxy
npm install to download the project dependencies.
- David Worms: https://github.com/wdavidw