Cloud Data Engineer (Azure Synapse Focus)

Cloud Data Engineer (Azure Synapse Focus)

Job Description

We are seeking a skilled Cloud Data Engineer with a focus on Azure Synapse Analytics to manage, troubleshoot, and optimize our cloud data environment. This role will involve collaborating with data engineers, cloud administrators, and developers to ensure that our data infrastructure operates efficiently, securely, and at scale.

Responsibilities

·       Manage, troubleshoot, and optimize Azure Synapse Analytics environments, including SQL pools, Spark pools, and pipelines.

·       Develop and maintain data integration workflows, using services such as Azure Data Factory (ADF) and Synapse pipelines.

·       Collaborate with data engineering and analytics teams to ensure data solutions align with business needs and performance goals.

·       Implement security best practices, such as managing private endpoints, access controls (ACLs), and virtual networks.

·       Monitor and optimize cloud resources for cost, performance, and scalability.

·       Automate deployments using infrastructure as code (Bicep, ARM, or Terraform).

·       Investigate and resolve operational issues in cloud services, including data pipelines and query performance bottlenecks.

·       Work cross-functionally with cloud architects, developers, and security teams to ensure seamless data processing across platforms.

Qualifications

·       3+ years of experience working in cloud data platforms, preferably Azure.

·       Strong experience with Azure Synapse Analytics or similar data platforms (e.g., Snowflake, Redshift, BigQuery).

·       Hands-on experience with Azure Data Factory, SQL, and Spark.

·       Familiarity with Azure services like Data Lake Storage, Blob Storage, and Virtual Networks.

·       Experience in infrastructure as code (e.g., Bicep, ARM, Terraform).

·       Understanding of data security best practices, including managing private endpoints, firewalls, and role-based access control (RBAC or ACL).

·       Problem-solving skills in large-scale data processing environments and ability to work on complex issues.

·       Experience with CI/CD pipelines in a DevOps environment (Azure DevOps or similar).

Preferred Qualifications

·       Certifications in Azure (e.g., Azure Solutions Architect, Azure Data Engineer, or Azure Administrator).

·       Experience working in big data ecosystems (Hadoop, Spark) and distributed computing.

·       Understanding of SQL performance tuning and optimization techniques.

Last updated on Sep 13, 2024