Data Engineer

New Yesterday

Location Requirement:
This is a hybrid role based in New York City. Remote work is not available - candidates must reside in or be willing to relocate to the NY area.
Responsibilities:
· Build data applications using Python, SQL, and AWS
· Build ETL pipelines for batch data ingest from APIs, databases and other sources as needed
· Apply knowledge of basic principles, methods and practices to simple and moderately complex assignments
· Proactively identify and implement opportunities to automate tasks and develop reusable frameworks
· Adhere to standard methodologies for coding, testing and designing reusable code/component
· Work as a data engineer within the Information Security team that uses several Data, Search and AWS technologies
· Implement standardized, automated operational and quality control processes to deliver accurate and timely data and reporting to meet or exceed SLAs
· Collaborate with the other engineering team members to ensure all services are reliable, maintainable, and well-integrated into existing platforms
· Review functional and technical designs to identify areas of risk and/or missing requirements
· Provide technical write-ups and drawings to promote the proposed solutions
· Partner with engineers, program managers and management to understand data needs
· Design, build and deploy efficient data pipelines to third party and internally developed systems
· Craft and build reports / dashboards using visualization tools like Tableau, Python and R
Qualifications:
· 4-5 years experience AWS cloud and AWS services such as S3 Buckets, Lambda, API Gateway, SQS queues
· 4-5 years experience with batch job scheduling and identifying data/job dependencies
· 4-5 years experience with data engineering using AWS platform and Python
· Familiar with AWS Services like EC2, S3, Redshift/Spectrum, Glue, Athena, RDS, Lambda, and API gateway
· Familiar with software DevOps CI/CD tools, such Git, Linux, and Shell Script
· Ability to learn quickly, be organized and detail oriented
· Experience in Software Engineering and Development
· Understanding of database schema design
· Proficient in Python
· Experience analyzing data to discover opportunities and address gaps
· Experience with Python, Tableau, SQL and Excel
· Experience with working in cloud or on-prem big data technologies
· Experience with data quality validation
· Looking for individuals with security driven attitude
· 4-5+ years of Python & SQL
· 4-5+ years of data modeling
· 4-5+ years of ETL design implementation and maintenance
Apply
Location:
New York

We found some similar jobs based on your search