I will guide you through a complex data pipeline. I have worked on many pipelines, but this one is particularly intricate.

Consider a pipeline designed for polling, transforming, enhancing, and pushing data to Elasticsearch. This pipeline, which operates on AWS ECS, pulls data at regular intervals, sending 500 articles to Kinesis. From Kinesis, Lambda functions process and transform this data, as it must be converted into different formats due to the varied sources.

First, we segregate the useful data, validating and mapping fields while transforming data types as necessary. Next, we enhance the data. A second Lambda function, triggered by events, adds details such as publications, licenses, and ownership. We also retrieve additional information, like author details, from DynamoDB. Finally, we push the enhanced data to Elasticsearch.

Simultaneously, this data undergoes percolation, generating queries from the articles. Another complex pipeline involves Apache Beam and Kafka. Source data is pushed to a Kafka topic and consumed by parallel executors. The data is processed in batches, transformed, and enhanced within the pipeline. Successfully processed data is sent to a different Kafka topic, while failures are directed to a Dead Letter Queue (DLQ).

An application using Apache Flink and Beam pulls this processed data and indexes it in Elasticsearch. By utilizing batching, parallel processing, and robust error handling, these pipelines ensure scalability and maintainability.

Thus, our data pipelines are both complex and efficient.