Software Engineer - Big Data Engineer (Spark, Databricks, PySpark , Snowflake)
Prefer a full account with a password? Sign Up to Apply
Job Description
Our NielsenIQ Technology teams are working on revamping multiple platform, a unified, global, open data ecosystem powered by Microsoft Azure. Our clients around the world rely on NielsenIQ’s data and insights to innovate and grow.
As a Big Data Software Engineer, you’ll be part of a team of smart, highly skilled technologists who are passionate about learning and prototyping cutting-edge technologies. Ingestion Services is working on unifying and standardizing several varied ingestion methodologies and convergence initiatives with latest Big Data services on Spark, Databricks, PySpark, Snowflake and leverage GitHub to support things like CI/CD and integrations and continuing to adopt the best in class cloud-native, cloud-agnostic technologies. Expertise in Python, Spark, Databricks are mandatory. Skills in PySpark, Snowflake, GitHub, AI, Azure are welcome additions. AI readiness and DevSecOps exposure are key additional facets to possess and exhibit.
• Develop, test, troubleshoot, debug, and enhance data engineering applications using Spark, PySpark, Pandas, Databricks, Snowflake, Airflow, and SQL as core technologies.
• Design, build, and optimize scalable data pipelines and ETL/ELT workflows for large-scale data processing.
• Deploy and maintain application components through CI/CD pipelines, ensuring reliable and efficient releases.
• Build utilities, frameworks, and monitoring solutions to automate repetitive tasks and improve operational efficiency.
• Collaborate with Agile cross-functional teams, including Engineering, Operations, Infrastructure, Technical Operations, and external stakeholders, to deliver business solutions.
• Investigate production issues, perform root cause analysis, and implement timely bug fixes and performance improvements.
• Ensure data quality, reliability, and adherence to engineering best practices, coding standards, and security guidelines.
• Participate in code reviews, sprint planning, design discussions, and continuous improvement initiatives
Requirements
Function: General Business
Experience Level: Mid-Senior Level