Description :

Hadoop development engineer with good Knowledge on Hadoop echo systems HDFS (Cloudera preferred), Map Reduce, Pig, Sqoop and Hive for scalability, distributed computing and performance computing.

Responsibilities:

• Analyze Hadoop cluster using different bigdata analytic tools including MapReduce, Hive, Sqoop, Zookeeper, Spark, Spark-streaming, Kafka, Phoenix-HBase.
• Translate functional and technical requirements into a details programs unning on Hadoop and Spark.
• Develop Logic in Spark and Spark Streaming using Scala and HQL for fast processing of the data according to the business requirements.
• Design Multiple POC’s using Spark and Scala and deploy on cluster, compare the performance of Sparksql, with hive.
• Develop and create Hive tables to store high volumes of data receiving from different data sources into a single framework and, work on performance tuning.
• Implementation of spark jobs with parametrization and incremental data storage.
• Write Shell Scripts for Automation of job to run in all environments for various checks to monitor the cluster health.
• Work with team in analyzing large data sets to provide strategic direction on implementation.
• Maintain documentation of the code development and deliver detailed documentation for design and production support.
• Monitor Production Jobs, provide on call support for the project and resolve issues, tickets raised with in the business.

Requirements:

• Minimum of Bachelor’s degree in Computer Science, Information Systems, or related field or its Equivalent.
• Good Knowledge in HDFS (Cloudera preferred), Map Reduce, Pig, Sqoop, Hbase and Hive
• Candidate should have a good overall understanding of Data and Analytics.
• Should have scripting skills in Linux environment.
• Good communication skills.