Posted 1h ago
Loading this job…
Data Engineering Intern - ETL & Pipelines
Posted 1h ago0 views
Pipelines, not dashboards
If you would rather build the plumbing than the reports, this is the internship to apply for. MailerMen ingests job postings from eleven partner boards and the scrapers break quietly at least twice a week. Over six months, working fully remote from anywhere in India, you will make that ingestion boring and predictable.
The build list and how we work
Your first few weeks go into understanding the existing Airflow DAGs and adding alerting so failures reach a human instead of vanishing into a log file. After that you rewrite the two worst scrapers, add deduplication for postings that appear on several boards under different titles, and split the nightly full load into incremental batches. The retention policy that keeps our storage bill from doubling each quarter is yours to design as well. Everything here runs asynchronously: a written brief on Monday, a review on Thursday, and a lot of freedom in between. No industry experience is expected. You need enough Python to write a script you are not embarrassed by, and the discipline to log what your pipeline is actually doing.
Responsibilities
- Maintain and extend the Airflow DAGs that ingest postings from partner job boards
- Add alerting and retry logic so pipeline failures surface within minutes
- Rewrite two legacy scrapers with proper error handling and rate limiting
- Build deduplication logic for postings that appear across multiple sources
- Convert the nightly full load into incremental batch processing
- Implement a data retention and archival policy for raw ingest storage
- Write runbooks so anyone on call can recover a broken DAG
Requirements
- Final year student or recent graduate in computer science, IT or a related field
- Solid Python fundamentals and comfort working from the command line
- Basic SQL and an understanding of relational tables and indexes
- Familiarity with Git and reading through someone else's codebase
- Self directed enough to work in a fully asynchronous remote team
Skills
Benefits
- Monthly stipend with a review after your first three months
- Remote from anywhere in India with no fixed office days
- Written feedback every Thursday from the platform engineer
- Completion certificate and referral support for full time roles
- Reimbursement for cloud and tooling costs you incur