Data Engineer
New Yesterday
Job Title: Data Engineer
Location: Philadelphia, PA ( Remote for EST )
Duration: 4+ Months
Job Description:
We are seeking a Data Engineer with hands-on experience in Databricks, particularly in implementing Databricks Asset Bundles (DABs) and streamlining deployment workflows through CI/CD automation. This is a 4-month Project with a potential for an extention.
***Candidates located in the Tri-State area (NY, NJ, CT) will be given first preference, as the role requires onsite presence 1-2 days per week in Philadelphia.*** However, we are also open to remote candidates based in the Eastern Time Zone (EST).
Candidates should have experience with: Databricks Asset Bundling to support CI/CD automation
Conducting code reviews to ensure adherence to development standards
Contributing to architectural discussions and technical solution design
The ideal candidate will focus on optimizing the current Databricks environment by: Building and enhancing data transformations
Designing and implementing graph data integration patterns and relationship mapping strategies
Supporting the implementation of Medallion Architecture
Data Engineer Qualifications: 5+ years of experience in data engineering with at least 3 years focused on Databricks platform
Hands-on experience with Databricks Asset Bundles (DABs) for CI/CD automation
Proven experience implementing Medallion architecture patterns in production environments
Strong proficiency in Databricks platform capabilities and data processing frameworks
Experience with data lake management concepts and transactional data operations
Nice to Have Skills: Databricks certified professional credentials (Data Engineer Associate/Professional)
Experience with real-time streaming data processing using Databricks Structured Streaming
Knowledge of machine learning workflows and MLOps practices within Databricks
Experience with Infrastructure as Code tools and deployment automation
Familiarity with containerization technologies and modern DevOps practices
Previous consulting or managed services experience
Experience with data visualization tools and business intelligence platforms
Knowledge of graph algorithms and network analysis techniques
Data Engineer responsibilities: Design and implement robust data pipelines using Databricks platform capabilities
Develop and maintain ETL/ELT processes following best practices for performance and reliability
Build and optimize data transformations within Databricks environments
Design and implement graph data integration patterns and relationship mapping strategies
Implement and manage Databricks Asset Bundles (DABs) for streamlined deployment workflows
Configure and maintain CI/CD pipelines using Asset Bundles for development, staging, and production environments
Participate in architectural decisions and technical solution design
Stay current with Databricks platform updates and emerging data engineering technologies
- Location:
- Philadelphia, PA, United States
- Category:
- Computer And Mathematical Occupations