Job Title: Hadoop Database Administrator
Department: Technology Analytics (Enterprise Data Warehouse)
This position will be responsible for development and support of the open source Hadoop environment, including; infrastructure planning, scaling and administration. The administrator will work closely with infrastructure, network, database, business intelligence and application teams to insure business applications are highly available and performing within agreed on service levels.
ESSENTIAL DUTIES AND RESPONSIBILITIES include the following:
- Provides day to day production support of our Hadoop infrastructure including;
o Implement new Hadoop hardware infrastructure
o OS integration and application installation
o Cluster maintenance
o HDFS support and maintenance
o Adding and removing cluster nodes
o Backup and restores
o Troubleshooting MapReduce Jobs as required
o Cluster Monitoring and Troubleshooting
o Manage and review Hadoop log files
o Filesystem management and monitoring
- Hadoop Cluster capacity planning
- Design, implement and maintain security
- Manage and review data backups
- Execute system and disaster recovery processes as required
- Provide support and troubleshooting for BI issues associated with Hadoop
- Provide support for legacy ETL environment as required
- Work with the project and application development teams to implement new business initiatives as they relate to Hadoop
- Works with application teams to install operating system and Hadoop updates, patches, version upgrades as required.
- Implement vulnerability and operating system patches to insure compliance and system integrity.
- Works with business intelligence, ETL and application teams to troubleshoot complex problems that impact our customer's ability to perform required business functions.
- 3+ years supporting open source Linux operating systems (CENTOS, Debian, Fedora) and hardware in an enterprise environment.
- 3+ years experience installing, configuring Linux based systems.
- Strong scripting expertise including BASH, PHP, PERL, Java script and UNIX Shell.
- Knowledge of java virtual machines (JVM) and multithreaded processing.
- Expertise in typical system administration and programming skills such as storage capacity management, performance tuning, system dump analysis, and Tserver hardening (security).