Principal Member Technical Staff - Data engineer

Nielsen’s Digital Content AI team builds and operates data-processing pipelines that turn digital audience and interaction signals into reliable, analytics-ready datasets. As a principal-level individual contributor, you will bring a Solution first mindset and strong data understanding to the full lifecycle: understand new data sources, build reusable ingestion and transformation workflows, validate and enrich data, publish it to downstream data platforms, and keep production processing dependable. You will lead multiple cross-functional projects through measurable business deliverables, providing technical direction through expertise and influence rather than formal people management. You will work across mobile, Connected TV (CTV), desktop/laptop and web use cases, partnering with product, data collection, platform, quality and downstream analytics teams.

● Lead the technical design and delivery of batch and near-daily data pipelines, from raw inputs through curated outputs and publication to the Master Data Layer (MDL).

● Profile and interpret incoming telemetry from diverse data sources; identify data gaps and translate source formats into dependable processing logic.

● Apply an AI-first approach to engineering and operations: identify appropriate opportunities for AI-assisted data exploration, code and test generation, classification, anomaly detection and workflow automation; validate outputs rigorously and use AI responsibly with appropriate privacy and security controls.

● Design and operate real-time and streaming event workflows alongside batch pipelines, managing event validation, schema evolution, ordering and duplication, late or replayed events, processing latency, backpressure and failure recovery.

● Build and maintain parsers, configuration-driven workflows, event/session logic, business rules and schema mappings for application usage and in-app activity across mobile, CTV, desktop/laptop and web products.

● Develop data-stitching and transformation logic that correlates relevant sources, derives sessions and engagement measures, and produces consistent intermediate and curated datasets.

● Deliver data processing for streaming/video, e-commerce and advertising use cases, including event classification, purchase and product/category/search activity, app and web usage, and incremental dataset updates.

● Integrate reference and enrichment data—including content metadata and classification inputs—where required; collaborate with machine-learning and platform teams to operationalize model-based classification and enrichment.

● Build reusable frameworks for onboarding new applications, platforms, data sources and markets; support configuration, historical backfills, reprocessing and safe daily refreshes.

● Orchestrate and operate workflows using tools such as Airflow and AWS workflow services; monitor schedules, dependencies, retries, failures and data availability, and troubleshoot production issues through root-cause analysis.

● Implement and evolve automated data-quality controls, including schema and completeness checks, freshness and volume monitoring, null/range/duplicate checks, event consistency, stitching validation and reconciliation between processing layers and MDL.

● Create and use operational dashboards, alerts and job-level telemetry to make pipeline health, data quality, performance and processing costs visible; improve reliability and efficiency based on those signals.

● Optimize distributed processing and cloud data layouts for correctness, scalability, runtime and cost, working with technologies such as Spark, Parquet, Amazon S3, AWS Glue, Athena and EMR.

● Contribute to design reviews, code reviews, testing strategy, release readiness, runbooks and technical documentation; maintain clear data contracts, schemas and operational ownership boundaries.

● Provide technical direction and mentoring to engineers, lead moderately complex initiatives through influence, and communicate risks, dependencies, estimates and delivery status to cross-functional partners.

● Drive multiple projects or workstreams end to end, translating business needs into scope, milestones and technical plans; coordinate dependencies and stakeholders; and remain accountable for quality, adoption, timelines and measurable business outcomes—not people-management responsibilities.

● Handle sensitive data responsibly and follow Nielsen’s security, privacy, access-control and engineering practices.

Department: Technology
Function: Information Technology
Experience Level: Not Applicable