Data Engineer

New Yesterday

Job Title: Data Engineer Location: Philadelphia, PA ( Remote for EST ) Duration: 4+ Months
Job Description: We are seeking a Data Engineer with hands-on experience in Databricks, particularly in implementing Databricks Asset Bundles (DABs) and streamlining deployment workflows through CI/CD automation. This is a 4-month Project with a potential for an extention. ***Candidates located in the Tri-State area (NY, NJ, CT) will be given first preference, as the role requires onsite presence 1-2 days per week in Philadelphia.*** However, we are also open to remote candidates based in the Eastern Time Zone (EST). Candidates should have experience with: Databricks Asset Bundling to support CI/CD automation Conducting code reviews to ensure adherence to development standards Contributing to architectural discussions and technical solution design The ideal candidate will focus on optimizing the current Databricks environment by: Building and enhancing data transformations Designing and implementing graph data integration patterns and relationship mapping strategies Supporting the implementation of Medallion Architecture
Data Engineer Qualifications: 5+ years of experience in data engineering with at least 3 years focused on Databricks platform Hands-on experience with Databricks Asset Bundles (DABs) for CI/CD automation Proven experience implementing Medallion architecture patterns in production environments Strong proficiency in Databricks platform capabilities and data processing frameworks Experience with data lake management concepts and transactional data operations Nice to Have Skills: Databricks certified professional credentials (Data Engineer Associate/Professional) Experience with real-time streaming data processing using Databricks Structured Streaming Knowledge of machine learning workflows and MLOps practices within Databricks Experience with Infrastructure as Code tools and deployment automation Familiarity with containerization technologies and modern DevOps practices Previous consulting or managed services experience Experience with data visualization tools and business intelligence platforms Knowledge of graph algorithms and network analysis techniques
Data Engineer responsibilities: Design and implement robust data pipelines using Databricks platform capabilities Develop and maintain ETL/ELT processes following best practices for performance and reliability Build and optimize data transformations within Databricks environments Design and implement graph data integration patterns and relationship mapping strategies Implement and manage Databricks Asset Bundles (DABs) for streamlined deployment workflows Configure and maintain CI/CD pipelines using Asset Bundles for development, staging, and production environments Participate in architectural decisions and technical solution design Stay current with Databricks platform updates and emerging data engineering technologies
Location:
Philadelphia, PA, United States
Category:
Computer And Mathematical Occupations