Search Jobs

Data Engineer 3

Plano, TX 75023

Posted: 01/10/2020 Industry: Other Job Number: 9721

Job Description


Description:

Responsible for completing our transition into fully automated operational reports across different functions within Care (including repair operations, contact center, digital support, product quality and finance) and for bringing our Care Big Data capabilities to the next level by designing and implementing a new analytics governance model, with emphasis on architecting consistent root cause analysis procedures resulting in enhanced operational and customer engagement results.

Summary:
The main function of the Data Engineer is to develop, evaluate, test and maintain architectures and data solutions within our organization. The typical Data Engineer executes plans, policies, and practices that control, protect, deliver, and enhance the value of the organizations data assets.

Job Responsibilities:
Design, construct, install, test and maintain highly scalable data management systems.
Ensure systems meet business requirements and industry practices.
Design, implement, automate and maintain large scale enterprise data ETL processes.
Build high-performance algorithms, prototypes, predictive models and proof of concepts.

Skills:
Ability to work as part of a team, as well as work independently or with minimal direction.
Excellent written, presentation, and verbal communication skills.
Collaborate with data architects, modelers and IT team members on project goals.
Strong PC skills including knowledge of Microsoft SharePoint.

Education/Experience:
Bachelor' s degree in a technical field such as computer science, computer engineering or related field required.
5-7 years of experience required.
Process certification, such as, Six Sigma, CBPP, BPM, ISO 20000, ITIL, CMMI.


- Analytical and problem solving skills, applied to Big Data domain
- Proven understanding and hands on experience with Hadoop, Hive, Pig, Impala, and Spark
- 5-8 years of Python or Java/J2EE development experience
- 3+ years of demonstrated technical proficiency with Hadoop and big data projects

Day to Day duties:
- Data collection - gather information and required data fields.
- Data manipulation - Join data from multiple data sources and build ETLs to be sent to Tableau for reporting purpose
- Measure & Improve - Implement success indicators to continuously measure and improve, while providing relevant insight and reporting to leadership and teams.



Interviews focus on conceptual questions around python, shell, Hadoop, hive. After that will be coding questions in SQL. After that, it' s coding in Python and Shell scripting.
Once we get to the next round, it is around performance tuning, ETL Design, business related questions to confirm they are a good fit for the project.

A lot of times candidates screened have been really good in SQL but not in python/shell/Hadoop or vice versa.
Mostly we are looking for someone with advanced skills in Hive/SQL and moderate skills in python/shell and Hadoop.

All backend is in Hadoop/AWS.
Mostly in house clusters, we do have the AWS Cloud platform though, so having that skill will really help.

About 75% of the work we do is using SQL in high query language. The other 25% is python, shell script, oozie, spark, etc.
1. Hadoop
2. SQL
3. Python/Shell Scripting (exchanging data between UNIX and other sources into Hadoop. All Hive tables we create will be pointed to the files in Hadoop)
4. AWS - not super required but some of our data comes from AWS S3


- Analytical and problem solving skills, applied to Big Data domain
- Proven understanding and hands on experience with Hadoop, Hive, Pig, Impala, and Spark
- 5-8 years of Python or Java/J2EE development experience
- 3+ years of demonstrated technical proficiency with Hadoop and big data projects
No items found.
 

 

Job Requirements

Scott

Meet Your Recruiter

William Smith

Apply Online
Apply with LinkedIn Apply with Facebook Apply with Twitter
Chat With A Recruiter

Send an email reminder to:

Share This Job:

Related Jobs:

Login to save this search and get notified of similar positions.