Data Engineers
We are looking for proficient Data Engineers who can work well in a modern agile software engineering environment and knowledge of application architecture patterns. The role will be a technical solution development function working with technical architects.
Job Description:
Roles and Responsibilities:
- Create and maintain optimal data pipeline architecture,
- Assemble large, complex data sets that meet functional / non-functional business requirements.
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
Requirements:
- Expertise and hands-on experience in Python/Java – Must Have
- Expertise knowledge on SparkQL/Spark Dataframe/HiveQL – Must Have
- Good knowledge of SQL – Good to Have
- Good knowledge of Shell script/kafka/sqoop/mapreduce – Good to Have
- Good Knowledge of one of the Workflow engine like Oozie, Autosys – Good to Have