Tóm lược
Mô tả công việc
Tóm tắt công việc
Salary up to 4000USD with JOINING BONUS 30,000,000VND
As a Hadoop big data engineer, you will develop, operate and drive scalable and resilient data platform based on Hadoop ecosystem to address the business requirements:
• Ensure industry best practices around data pipelines, metadata management, data quality, data governance and data privacy
• Design and implement business-specific large-scale data processing pipelines
• Work with complex data structures, manipulate, cleanse data, and perform transformations to make insights from data.
• Responsible to Ingest data from files, streams, and databases. Process the data with PySpark, Kafka, Hive, Hive LLAP…
• Develop efficient software code for multiple use cases leveraging Spark and Big Data Technologies for various use cases built on the platform
• Provide high operational excellence guaranteeing high availability and platform stability.
Yêu cầu công việc
• Experience in Hadoop ecosystem including HDFS, MapReduce, YARN, HBase, Zookeeper, Pig, Hive…
• Experience with Hadoop distributions such as Cloudera, HortonWorks, GCP…
• Experience with Apache Spark using PySpark
• Experience with Apache Kafka
• Experience with Apache Beam
• Experience with SRE, Patching & Automation: Kubernetes or Docker & Containerization
• Good in programming language Python, Java
• Good to have cloud Skills and Experience, SQL/NoSQL database on Cloud, especially GCP
• Experience in building large-scale data processing (batch-processing, stream processing)
Ngôn ngữ
-
English
Nói: Intermediate - Đọc: Intermediate - Viết: Intermediate
Yêu cầu kỹ thuật
- Hadoop
- Apache Spark
- Big Data
- Java
- Python
- NoSQL
- MS SQL
- MapReduce
- Hbase
- Docker
- HDFS
- Pig script
- Apache Kafka
- Stream processing
- Kubernetes
- GCP
- Cloudera
- Apache Zookeeper
- Yarn
- BEAM
- Hortonworks
- PySpark
Thông tin doanh nghiệp
HCL Technologies is a next-generation global technology company.
We help enterprises reimagine their businesses for the digital age. With a worldwide network of R&D, innovation labs and delivery centers, and 150,000+ ‘Ideapreneurs’ working in 49 countries, HCL serves leading enterprises across key industries, including 250 of the Fortune 500 and 650 of the Global 2000. HCL generated consolidated
revenues of US$ 9.93 bn for 12 Months as of 30 th June, 2020.
We offer an integrated portfolio of products, solutions, services, and IP through our Mode 1-2-3 strategy built around Digital, IoT, Cloud, Automation, Cybersecurity, Analytics, Infrastructure Management and Engineering Services, amongst others, to help enterprises reimagine their businesses for the digital age.