• Write reliable software to ingest, transform and distribute data into our internal applications.
• Manage our data from ingestion through ETL to storage and batch processing.
• Design and develop resilient pipelines using a variety of different technologies.
• Drive the advancement of data infrastructure by developing and implementing underlying logic and structure for how data is set up, cleansed and stored.
• Scope points of failure and create alerts to ensure continual data availability, accuracy and cleanliness.
• Partner with development and SEO teams to deliver data solutions that meet internal requirements within agreed upon timelines.
• Monitor and remediate data quality issues.
• Process semi-structured data into a form suitable for analysis.
• Provide ad-hoc queries and analysis as needed.
• Maintain detailed requirements/design/technical documentation for new and existing components.
• Automate, test and harden all data workflows using tools such as AWS triggers and lambda handlers.
Other skills (Desired)
• Python PEP 8 code style
• ORM (SQLAlchemy, Django ORM, other)
• Python Packages: Scrapy, Pandas, Requests, Numpy
• Apache Spark / PySpark or AWS Glue
• Apache Airflow, Amazon Redshift, Amazon Athena
• 3+ years’ experience in processing data with Python, SQL, and other data tools.
• 2+ years’ experience working with back-end data systems to create, process, and clean data.
• 2+ years ‘experience in developing automated ETL solutions.
• Familiarity with version control systems (preferably Git).
• Familiarity with manipulating data frames and Pandas.
• Experience maintaining and updating a data warehouse.
• Experience accessing data via Web Services and external/non-SQL datasets.
• Proficient with Data Visualization Tools (e.g. - Matplotlib, Superset, or Other).
• Ability to refactor portions of an existing product currently implemented in Python, Perl and PostgreSQL.
• Amazon Web Services experience (AWS) - EC2, RDS, S3, Lambda, ECS, Elastic Beanstalk, Glue, Data Pipeline, Boot.
- Relocation Package
- Permanent - Full Time Job
- Work from Mon through Fri. WE LEAVE EARLY ON FRIDAYS!
- Upper MX Law Benefits including Medical Insurance with Dental Discounts
- 100% Payroll scheme
- Career Path
- Cool offices and a collaborative environment
- American Company culture
- Health & Wellness program for employees