Senior Hadoop Administrator
Klout - San Francisco, CA

This job posting is no longer available on Klout. Find similar jobs: Senior Hadoop Administrator jobs - Klout jobs

We're looking for a seasoned engineer to help us create cutting edge systems and tools to manage, maintain and monitor all aspects of the Klout production, staging, and development environments with a focus on our Big Data systems. Our environment is an intense combination of modern software and tried and true technologies including and leveraging Hadoop, HBase, Hive, ElasticSearch, Scala, Storm, and Node.js. We'll challenge your scalability chops with our high volume consumer site traffic coupled with the Big Data challenges of processing over 12 billion data signals a day for over 400 million people on multiple petabyte clusters.

Here are the values we look for in a team member:
You believe in automating everything. If you are repeating the same task more than once, you believe it should be automated
You believe configuration management isn't just a good idea but a must
You prove your ideas with testing and metrics, no guesswork
No pointing the finger or blame -- we're all in this together and everyone owns site up
We're here to serve, both our customers and our team members -- we provide tools to help everyone succeed

Here's what you need to have:
5+ years of experience in systems or database administration experience on Linux-based systems
2+ years of experience installing, configuring, monitoring, and administering large Hadoop clusters, preferably on Cloudera CDH3 and CDH4
Experience or a strong understanding of Hadoop architecture including HDFS, Hadoop MapReduce, Hive, Oozie, HBase, Hue, ZooKeeper, and ElasticSearch
Experience with current technologies like HAProxy, memcached, and Redis
Strong understanding of best practices and standards for Hadoop application design and implementation
Strong knowledge of Linux, OS level file system and network troubleshooting
Experience with Java, Python, and Bash (Ruby, Scala, and C++ a plus plus)
Expertise in at least one large-scale RDBMS
Experience running a monitoring system -- and participating in an on-call rotation
Klout score of 10 or higher

The things you'll do:
Perform HBase maintenance, splitting regions and performing compactions as necessary
Monitor and tune Hadoop clusters for performance
Monitor and report data usage across the clusters
Monitor Hadoop clusters through Nagios and Graphite
Support development, QA, and operations during new system deployments
Evaluate and propose new tools and technologies to meet the needs of the organization
Backup and recover NameNode and Hive metadata
Perform system health checks and monitor logs to ensure data integrity and fix block corruption
Develop automated tools and scripts to perform routine cluster maintenance
Deploy, configure, monitor, and adminster different systems like Hue, Hive, and ElasticSearch in an automated fashion
Patch, test, and deploy proprietary builds of different open source systems, including HBase and Hive
Document configs and upgrade procedures
Create runbooks for troubleshooting, cluster recovery, and routine cluster maintenance

Here’s what to expect:
Competitive pay, 100% paid health premium benefits and 50% dependent premiums
401k plan
Stock options
3 weeks vacation
Catered lunches, snacks, drinks
Passionate people building great products
A fun environment where people get the job done
Appreciation of exceptional science and engineering
Doing something no one has ever done before

Klout began with a very simple idea: Everyone has influence—the ability to drive action. Klout built on this idea to show anyone how he or she can influence the world and its future.
Klout was founded in 2008 to empower everyone to unlock their influence. We come to work every day inspired to help people understand the power of their voices and democratize influence.
Klout is a privately held company based in San Francisco, Calif. We’re a growing startup backed by Kleiner Perkins, among others.

Klout - 15 months ago - save job