logo

View all jobs

Data Engineer Automation Controls

Irving, TX
Role: Data Engineer Automation Controls
Location : Irving, TX. 


Primary Responsibilities:
Candidates should possess strong knowledge and interest across big data technologies and have a
background in data engineering.
 Build data pipeline frameworks to automate high-volume and real-time data delivery for
our Spark and streaming data hub
 Transform complex analytical models in scalable, production-ready solutions
 Provide support and enhancements for an advanced anomaly detection machine learning
platform
 Continuously integrate and ship code into our cloud production environments
 Develop cloud based applications from the ground up using a modern technology stack
 Work directly with Product Owners and customers to deliver data products in a
collaborative and agile environment
Your Responsibilities Will Include
Developing sustainable data driven solutions with current new generation data technologies to
drive our business and technology strategies
 Building data APIs and data delivery services to support critical operational and
analytical applications
 Contributing to the design of robust systems with an eye on the long-term maintenance
and support of the application
 Leveraging reusable code modules to solve problems across the team and organization
 Handling multiple functions and roles for the projects and Agile teams
 Defining, executing and continuously improving our internal software architecture
processes
 Being a technology thought leader and strategist
Knowledge, Skills & Experience
Education
 BS degree in Computer Science, Data Engineering or similar
 Intermediate to senior level experience in an Apps Development role. Demonstrated
strong execution capabilities
Required
 5+ years of experience on designing and developing Data Pipelines for Data Ingestion or
Transformation using Java or Scala or Python
 At least4 years of experience in the following Big Data frameworks: File Format
(Parquet, AVRO, ORC), Resource Management, Distributed Processing and RDBMS
 5+ years of developing applications with Monitoring, Build Tools, Version Control, Unit
Test, TDD, Change Management to support DevOps
 At least 2 years of experience with SQL and Shell Scripting experience

 Experience of designing, building, and deploying production-level data pipelines using
tools from Hadoop stack (HDFS, Hive, Spark, HBase, Kafka, NiFi, Oozie, Apache Beam,
Apache Airflow etc).
 Experience with Spark programming (pyspark or scala or java).
 Experience troubleshooting JVM-related issues.
 Experience and strategies to deal with mutable data in Hadoop.
 Experience with Stream sets.
 Familiarity with machine learning implementation using PySpark.
 Experience in data visualization tools like Cognos, Arcadia, Tableau
Preferred
 Angular.JS 4 Development and React.JS Development expertise in a up to date Java
Development Environment with Cloud Technologies
 1+ years’ experience with Amazon Web Services (AWS), Google Compute or another
public cloud service
 2+ years of experience working with Streaming using Spark or Flink or Kafka or NoSQL
 2+ years of experience working with Dimensional Data Model and pipelines in relation
with the same
 Intermediate level experience/knowledge in at least one scripting language (Python, Perl,
JavaScript)
  Hands on design experience with data pipelines, joining data between structured and
unstructured data
 Familiarity of SAS programming will be a plus
 Experience implementing open source frameworks & exposure to various open source &
package software architectures (AngularJS, ReactJS, Node, Elastic Search, Spark, Scala,
Splunk, Apigee, and Jenkins etc.).
 Experience with various noSQL databases (Hive, MongoDB, Couchbase, Cassandra, and
Neo4j) will be a plus
 Experience in Ab Initio technologies including, but not limited to Ab Initio graph
development, EME, Co-Op, BRE, Continuous flow
Other
 Successfully complete assessment tests offered in Pluralsight, Udemy, etc. or complete
certifications to demonstrate technical expertise on more than one development platform.

Share This Job

Powered by