Riversand, a Syndigo company's engineering team is looking for a passionate Data Engineer / Scraping Extraordinaire to help develop web data mining robots and power our software. This is a team role to fulfill the Riversand's growing crawling needs. The role will require knowledge of writing sophisticated crawlers and scraping large volumes of unstructured data from public databases, websites, and social platforms using Python. The role will help ensure the accessibility, aesthetics, performance, and security of the application, and
will work with other members of the software engineering team to perform integration activities.
Essential Duties and Responsibilities:
Chrome, Linux, and AWS, and comfortable working in an agile, test-driven environment and is
experienced in continuous-delivery processes. Knowledge of digital advertising operations is
nice to have.
• Applicants must also be excellent problem solvers who know how to detect and evade antiscraping / bot technologies
• Connect to public databases to ingest data as well as executing one-off imports of data.
• Create new data ingestion and processing tooling to eliminate manual processes, inefficient or
repetitive work, and address quality issues.
• Make thoughtful judgements on data quality to clean data sources for import.
• Use third-party APIs and web scraping tools to source data at scale.
• Work with the team to scale and embed techniques and help with data ingestion projects.
• Demonstrate common sense in applying business logic to ontological/schema decisions.
• Extract data from a variety of relational databases, manipulate, explore data using quantitative,
statistical and visualization tools
• Develop and implement standards for clean code that maintain modularity, clarity, and
• Demonstrate passion for continued learning by staying abreast of new technology and trends.
• 3+ years of professional web/software development experience.
• 3+ years’ experience in Python.
• 2+ years database experience.
• 1+ year Selenium experience.
• Knowledge of Browser-Web Server interaction including DNS, HTTP, SSL, GET vs POST.
• SQL, ETL, Pentaho experience.
• Command of software engineering principles, frameworks and technologies.
• Experience prioritizing and performing multiple tasks in time-critical situations.
• Comfort working within a fast-paced, dynamic and distributed environment.
• Strong written and verbal communication skills.
Your application was submitted successfully.