Senior Big Data Engineer
New Yesterday
Data Engineer Company: enGen
Job Summary: This job is responsible for designing and engineering data solutions for the enterprise and, working closely with business, analytic and IT stakeholders, assists with the development and maintenance of these solutions. This includes coding data ingestion pipelines, transformations and delivery programs/logic for people and systems to access data for operational and/or analytic needs. Duties include but are not limited to the coding, testing, and implementation of ingestion and extraction pipelines, transformation and cleansing processes, and processes that load and curate data in conformed, fit-for-purpose data structures. The incumbent is expected to partner with others throughout the organization (including other engineers, architects, analysts, data scientists, and non-technical audiences) in their daily work. The incumbent will work with cross-functional teams to deliver and maintain data products and capabilities that support and enable strategies at business unit and enterprise levels. The incumbent is expected to utilize technologies such as, but not limited to: Google Cloud Platform, Hadoop, Hive, NoSQL, Kafka, Spark, Python, Linux shell scripting, SAS, Teradata, Oracle, and Informatica.
Essential Responsibilities In partnership with other business, platform, technology, and analytic teams across the enterprise, design, build and maintain well-engineered data solutions in a variety of environments.
Work with internal and external platforms and systems to connect and align on data sourcing, flow, structure, and subject matter expertise.
Working across multiple, disparate systems and platforms, design, code, test, implement, and maintain scalable and extensible frameworks that support data engineering services.
Align with security, data governance and data quality programs by driving assigned components of metadata management, data quality management, and the application of business rules.
Develop, optimize and/or maintain machine learning and AI engineering processes (MLOps) that are deployed to cloud or big data environments.
Develop tasks across multiple projects with limited need for guidance. This includes providing guidance and education to intermediate and junior contributors within the team.
Establish standards and patterns for high performance data ingestion, transformation, and delivery of data analytic needs.
Education Required: Bachelor's Degree in Software Engineering, Information Systems, Computer Science, Data Science or related field
Substitutions: None
Preferred: Master's Degree in Software Engineering, Information Systems, Computer Science, Data Science or related field
Experience Required: 5 years in Data platform development, data engineering, software development, or data science + 3 years in Big data or cloud data platform
Preferred: 3 years in Healthcare Industry + 3 years of Data Warehousing + 3 years of Database Administration
Licenses and Certifications Required: None
Preferred: Cloud certification (GCP, Azure, AWS)
Skills SQL + Data Warehousing + Problem-Solving + Communication Skills + Analytical Skills + Spark or Python or related tool + Cloud Technologies
Language (Other than English): None
Travel Requirement: 0% - 25%
Physical, Mental Demands and Working Conditions
Position Type: Office-based Teaches / trains others regularly Rarely Travel regularly from the office to various work sites or from site-to-site Does Not Apply Works primarily out-of-the office selling products/services (sales employees) Never Physical work site required No Lifting: up to 10 pounds Constantly Lifting: 10 to 25 pounds Occasionally Lifting: 25 to 50 pounds Rarely
- Location:
- Boise, ID, United States
- Category:
- Computer And Mathematical Occupations