Data Engineer

Description: Mandatory skills Corporate Banking Knowledge. Strong hands-on experience in developing, optimizing, and managing ETL pipelines, particularly for large-scale, real-time data. Advanced proficiency in SQL. Expertise in Python programming languages. In-depth experience with Azure services (e.g., Azure Data Factory, Azure Databricks) Strong experience with Cloudera/Hadoop ecosystem, including HDFS, Hive, Spark, Knowledge of data governance, data quality, and data security best practices. Any Data engineer Professional Certification. Strong analytical and problem-solving skills Desired Skills Real-time Data Processing Basic experience with machine learning models and tools such as Azure ML Roles & Responsibilities Data Pipeline Management Design and optimize ETL pipelines to integrate corporate banking data into the data lake (Cloudera/Hadoop, Azure + Databricks) Data Integration & Transformation Integrate data from multiple sources, transforming it for analytics using Apache Spark and Databricks. Automation & Optimization Automate data processes and optimize pipeline performance. Data Quality & Validation Implement data quality checks to ensure accurate and reliable data.

Collaboration Work with business teams and IT to design and document data solutions. Overall years of experience 10+ years of experience in data engineering, with a focus on large-scale data pipelines.