Job Description:
Unison Consulting Pte Ltd is seeking an experienced Big Data Developer to join our team. As a Big Data Developer, you will be responsible for designing, developing, and maintaining big data solutions to enable data-driven decision making and analytics capabilities for our clients. You will work with large volumes of structured and unstructured data using cutting-edge technologies and platforms.
Responsibilities:
- Design, develop, and maintain big data solutions using technologies such as Hadoop, HDFS, Spark, and Hive.
- Ingest, process, and analyze large volumes of structured and unstructured data from diverse sources.
- Develop and optimize ETL processes for efficient data extraction, transformation, and loading.
- Implement data models, schemas, and data governance frameworks to ensure data quality and integrity.
- Perform data exploration, profiling, and validation to identify patterns, trends, and anomalies.
- Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver actionable insights.
- Implement and optimize data storage and retrieval strategies, including data partitioning and indexing.
- Build and maintain scalable data pipelines and workflows for data processing.
- Perform performance tuning and optimization of big data applications and queries.
- Implement and maintain data security measures to protect sensitive information.
- Maintain technical documentation of big data solutions, processes, and procedures.
Requirements
Skills and Qualifications:
- Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
- Proven experience as a Big Data Developer with proficiency in technologies such as Hadoop, Spark, HDFS and Hive.
- Strong programming skills in languages such as Java, Scala, or Python.
- Experience with ETL tools and processes for data extraction, transformation, and loading.
- Good understanding of big data concepts, architectures, and technologies.
- Experience with SQL and NoSQL databases.
- Knowledge of data modeling and schema design.
- Excellent problem-solving and analytical skills.
- Strong communication and collaboration skills to work effectively with cross-functional teams.
- Understanding of data security and privacy principles.
- Ability to adapt to new technologies and learn quickly in a fast-paced environment.
•
Last updated on Apr 6, 2024