Senior Data Engineer – Mexico/India (Remote)
Full Time
India, Mexico
Posted 3 weeks ago
Position Type: Full-Time Employment
Location: 100% Remote – Mexico or India
Experience: 5-7 years of hands-on experience in delivering data engineering solutions
Qualifications/Must-haves:
- Strong hands-on experience in implementing data pipelines using Python, PySpark and SQL – advanced development, optimization and troubleshooting
- Experience with data engineering related services and technologies like Databricks or Cloudera (architecture, integration and optimization in at least one platform)
- Experience with Iceberg / Delta Lake (schema evolution, partitioning, CDC, optimization)
- Good understanding and experience implementing data quality management solutions, incl. data quality framework design and implementation
- Experience with DevOps processes, including experience implementing CI/CD pipelines for data engineering solution components, incl. automated deployment processes
- Solution design – technical evaluation and estimation
- Production support – root cause analysis, performance and reliability improvements
Nice to Haves:
- Strong hands-on experience in Azure / ADLS (cloud integration and security)
- Experience with SQL Server / SSIS (stored procedures, ETL modernization)
- Experience with Apache NiFi (framework design, reusable components, performance tuning) and Automic and/or ADF (complex workflows, dependencies, recovery)
- Experience with Docker / Kubernetes – configuration, deployment and troubleshooting
- Strong analytical and problem-solving skills
- Excellent communication and collaboration skills
Responsibilities:
- Lead technical design efforts for data engineering solution components, automation and accelerators
- Design and implement scalable, secure and resilient data engineering solutions on Azure, with a good understanding of data architecture patterns
- Evaluate and select appropriate technology stack for data engineering solution components – cloud native services, open-source tools, data storage, data processing frameworks etc
- Lead, mentor and manage a team of dynamic data engineers for development, testing and implementation of data pipelines ensuring quality, performance and on-time delivery
- Responsible for detailed technical design and data mapping specification
- Ensure adherence to industry best practices, coding standards and guidelines driving continuous improvement in code quality and delivery excellence
- Optimize data solutions for performance, cost-efficiency and scalability
- Ensure data quality and integrity for downstream systems and applications
- Plan, prioritize and manage data engineering projects, ensuring timely delivery with quality, incl. estimating effort
- Identify and mitigate risks, troubleshoot issues, handle customer escalations, incl. root cause analysis, performance and reliability improvements
- Ensure alignment with business requirements and priorities, working closely with stakeholders – both internal and customer
- Maintain comprehensive technical documentation, including architecture diagrams, high level / low level data integration design, data mapping specifications, database design etc
- Working closely with peers, BI engineers and data scientists to enable advanced analytics, machine learning models and reporting solutions
ID: P28
Job Features
| Job Category | Technical Leadership |