Data Extraction Specialist at ACLED

Specific tasks and responsibilities

The Data Extraction Specialist will be responsible for the following tasks: 

• Develop bespoke web scrapers and general web crawlers to extract data from online news sources.
• Deploy and monitor the performance of scrapers and crawlers to ensure reliable and continuous data acquisition.
• Implement logging and alerting systems for scraper performance and failures.
• Maintain and troubleshoot existing scrapers and crawlers, adapting them to changes in website structures and data formats.
• Deploy models and monitor ML/NLP models for filtering and classifying extracted news articles. 
• Develop and deploy scalable data engineering pipelines using cloud platforms (preferably AWS), Docker, or serverless technologies.
• Ensure all developed scrapers and crawlers adhere to best practices for web scraping, including ethical considerations and respect for website terms of service.
• Document all configurations, assumptions, and limitations to ensure consistency and transparency.
• Collaborate with other teams at ACLED to understand data needs and align scraper strategy with broader organizational goals.

Skills and competencies

Required

• Proven experience in developing and deploying web scrapers and crawlers for production use.
• Proven experience developing language models for practical applications.
• Strong proficiency in Python and JavaScript.
• Experience with major web scraping frameworks and libraries (Scrapy, Beautiful Soup, Selenium, Playwright, Puppeteer, etc.).
• Experience deploying machine learning models, including transformers-based NLP models, for processing source texts.
• Experience with data engineering tools and frameworks (e.g., Docker, Kubernetes, Airflow, Databricks).
• Understanding of ethical and legal considerations in web scraping, including compliance with robots.txt and terms of service.
• Experience working to deadlines with limited supervision.
• An extremely high level of attention to detail.
• Flexible team player, especially across a remote, global team.
• Fluency in English.
• Bachelor's degree required.