Data Engineer(Remote)

Summary Python + Airflow + Snowflake Data Engineer We are seeking a highly skilled Python / Airflow Data Engineer with 7 to 8 years of experience, specifically with a strong background in AWS technologies. The ideal candidate will have a deep understanding of Apache Airflow and its integration within AWS ecosystem, enabling efficient data pipeline orchestration and management. Responsibilities • Design, develop, and maintain complex data pipelines using Python for efficient data processing and orchestration. • Collaborate with cross-functional teams to understand data requirements and architect robust solutions within the AWS environment. • Implement data integration and transformation processes to ensure optimal performance and reliability of data pipelines. • Optimize and fine-tune existing data pipelines / Airflow to improve efficiency, scalability, and maintainability. • Troubleshoot and resolve issues related to data pipelines, ensuring smooth operation and minimal downtime. • Work closely with AWS services like S3, Glue, EMR, Redshift, and other related technologies to design and optimize data infrastructure. • Develop and maintain documentation for data pipelines, processes, and system architecture. • Stay updated with the latest industry trends and best practices related to data engineering and AWS services. Requirements • Bachelor’s degree in Computer Science, Engineering, or a related field. • Proficiency in Python, PySpark and SQL for data processing and manipulation. • Min 5 years of experience in data engineering, specifically working with Apache Airflow and AWS technologies. • Strong knowledge of AWS services, particularly S3, Glue, EMR, Redshift, and AWS Lambda. • Understanding of Snowflake Data Lake is preferred. • Experience with optimizing and scaling data pipelines for performance and efficiency. • Good understanding of data modeling, ETL processes, and data warehousing concepts. • Excellent problem-solving skills and ability to work in a fast-paced, collaborative environment. • Effective communication skills and the ability to articulate technical concepts to non-technical stakeholders. Preferred Qualifications: • AWS certification(s) related to data engineering or big data. • Experience working with big data technologies like Snowflake, Spark, Hadoop, or related frameworks. • Familiarity with other data orchestration tools in addition to Apache Airflow. • Knowledge of version control systems like Bitbucket, Git. Job Types: Full-time, Contractual / Temporary Pay: ₹500,911.64 - ₹1,841,614.32 per year Work Location: Remote Apply tot his job Apply To this Job