Data Engineer Analyst
Prefer a full account with a password? Sign Up to Apply
Job Description
What is this position about?
• Design, build, and maintain scalable data pipelines and ingestion frameworks supporting batch processing, streaming, and change data capture (CDC) patterns.
• Develop and optimize data processing workflows using Apache Spark and PySpark, ensuring reliable and performant transformation across large datasets.
• Implement and manage data lake architectures on AWS, leveraging S3 as the primary storage layer and Apache Iceberg for table format management, schema evolution, and time-travel capabilities.
• Work with AWS-native services including Glue for ETL orchestration, Athena for serverless querying, and Lambda for event-driven data processing.
• Manage data access controls and security configurations using AWS IAM, ensuring compliant and least-privilege access across the data platform.
• Collaborate with data analysts, data scientists, and platform engineers to understand data requirements and deliver production-ready solutions.
• Monitor pipeline health, troubleshoot data quality issues, and implement improvements to ensure data reliability and freshness.
• Contribute to the documentation of data flows, architecture decisions, and engineering standards within the team.
Requirements
Function: Marketing
Experience Level: Mid-Senior Level