We are looking for a hands-on Sr. Software Engineer with a focus on Data Engineering. This position requires extensive hands-on, data pipeline design and coding experience, developing modern data pipelines (AWS Step functions, Prefect, Airflow, Luigi, Python, Spark, SQL) and associated code in cloud/on-prem Linux/Windows environments.
This is a highly collaborative position that will be partnering and advising multiple teams by providing guidance throughout the creation and consumption of our data pipelines.
Responsibilities
- Participate in efforts to design, build, and develop Proof-of-Concept (POC) solutions and services.
- Review functional and technical designs to identify areas of risk and/or missing requirements.
- Provide technical write-ups and drawings to promote the proposed solutions.
- Build applications using Python, SQL, Snowflake, and AWS.
- Working understanding of Agile, Scrum, Design Thinking, and principles.
- Apply knowledge of basic principles, methods, and practices to simple and moderately complex data pipeline orchestration.
- Understand the principles of high availability, resiliency
- Proactively identify and implement opportunities to automate tasks and develop reusable frameworks.
- Adhere to standard methodologies for coding, testing, and designing reusable code/components.
- Participate in sprint planning meetings and provide estimations on technical implementation.
- Develop data pipelines on the AWS platform such as step function, lambda, etc.
- Collaborate with the other engineering team members to ensure all services are reliable, maintainable, and well-integrated into existing platforms.
- Contributed to the exploration and understanding of new tools and techniques and propose improvements to the data pipeline.
- Understand the overall SLA, analyze, and improve the processes to deliver accurate and timely data to meet or exceed the KPIs
Requirements
- BS or MSc in Computer Science, Information Systems, or an engineering field or relevant experience.
- 10+ years of related experience developing data solutions and data movement
- Hands-on experience designing and implementing data pipelines across large-scale systems.
- 3-5 years of data engineering experience in AWS cloud, platform, and services such as EC2, S3, Redshift/Spectrum, Glue, Athena, RDS, Lambda, API gateway, step function, and SQS queues.
- Proficient in Python, SQL, Java, Scala.
- Proficient in production delivery experience in Cloud-based PaaS Big Data related technologies Snowflake, Databricks, Synapse, SQL Server, EMR.
- Proficient with ETL, Orchestration Tools.
- Familiar with software DevOps CI/CD tools, such as Git, Jenkins, Linux, and Shell Script.
- Proficient with Power BI, Tableau.
- Understanding of Hadoop 2.0, Impala, Spark, Hive, Kafka, Kinesis, Spark Streaming, and Airflow.
- 3-5 years of experience with job scheduling and identifying data/job dependencies.
- Experience in Software Engineering and Development.
- Understanding of database schema design.
- Ability to learn quickly, be organized, and be detail-oriented.
- A collaborative and informative mentality.
Benefits
We offer amazing benefits for all our workers and try to make the process as easy as possible for any candidate interested in working in this amazing company.
- Remote work
- Telework costs support towards Electricity and Internet
- Permanent - Full Time Job
- Work from Mon through Fri. WE LEAVE EARLY ON FRIDAYS!
- Upper MX Law Benefits including Medical Insurance with Dental Discounts, Saving Funds, and Christmas Bonus.
- 100% Payroll scheme
- Career Path
- Collaborative environment
- Healthy Work Culture
- Health & Wellness program
- Life insurance in case of accidental death
- Internal Referral Program
External Referral Program
If this position is not the right fit for you, refer someone and if hired, receive a $100 USD AMAZON gift card per hire.
Back to All Job Postings