View all jobs

Sr. Data Engineer

San Mateo, CA

We are urgently looking for Data Engineer  for our Direct client requirement
TITLE:  Sr. Data Engineer
DURATION: 12+ Months
Rate: DOE


Position Summary
• Very Strong engineering skills. Should have an analytical approach and have good programming skills.
• Provide business insights, while leveraging internal tools and systems, databases and industry data
• Minimum of 5+ years’ experience. Experience in retail business will be a plus.
• Excellent written and verbal communication skills for varied audiences on engineering subject matter
• Ability to document requirements, data lineage, subject matter in both business and technical terminology.
• Guide and learn from other team members.
• Demonstrated ability to transform business requirements to code, specific analytical reports and tools
• This role will involve coding, analytical modeling, root cause analysis, investigation, debugging, testing and collaboration with the business partners, product managers other engineering team.

Must Haves
  • Excellent knowledge and experience with Hive and SQL
  • Experience with Spark SQL
  • Proficient with one programming language Java/Scala/Python
  • General understanding of how to build end-to-end data pipelines

Good to Have
  • Experience n architecting data pipelines – from Data model to the jobs and the sequence of jobs
  • Ability to build dashboards with Tableau or looker
  • Software Engineering knowledge – ability to build web applications using Java and AngularJS or ReactJS tech stacks

Technical Requirements
  •  Knowledge/experience on Teradata Physical Design and Implementation, Teradata SQL Performance Optimization
  • Advanced SQL (preferably Teradata)
  • Experience working with large data sets, experience working with distributed computing (MapReduce, Hadoop, Hive, Pig, Apache Spark, etc.).
  • Strong Hadoop scripting skills to process petabytes of data
  • Experience in Unix/Linux shell scripting or similar programming/scripting knowledge
  • Experience in ETL/ processes
  • Real time data ingestion (Kafka)

BS degree in specific technical fields like computer science, math, statistics preferred
Powered by