Tóm lược
Mô tả công việc
Tóm tắt công việc
An accomplished Data Platform Engineer is desired to become a member of our dynamic team. The optimal candidate should have substantial knowledge and experience managing workloads on EC2 clusters using DataBricks (EMR or an equivalent level of experience will suffice), ensuring connectivity and data availability via Data Mesh, implementing least privileged access across AWS accounts and Databricks data platform, and operating distributed engines such as Spark. Furthermore, storage technologies, data orchestration, and CI/CD pipeline, and operational experience with Spark and other data warehousing tools are prerequisites for success in this position.
Your responsibilities
- Manage, capacity plan, and operate workloads utilizing EC2 clusters via DataBricks/EMR to ensure efficient and reliable data processing
- Collaborate with stakeholders to design and implement a Data Mesh architecture across multiple closely related but separate enterprise entities
- Utilize Infrastructure as Code (IaC) tools such as CloudFormation or Terraform to define and manage data platform user access to data and compute resources.
- Implement role-based access control (RBAC) mechanisms using IaC templates to enforce least privilege principles and ensure secure access to data and compute resources
- Collaborate with cross-functional teams to design, implement, and optimize data pipelines and workflows
- Utilize distributed engines such as Spark to process and analyze large volumes of data efficiently when required
- Develop and maintain operational best practices for Spark and other data warehousing tools to ensure system stability and performance
- Implement and manage storage technologies to efficiently store and retrieve data as per business requirements
- Troubleshoot and resolve platform-related issues in a timely manner to minimize downtime and disruptions
- Stay updated on emerging technologies and industry trends to continuously enhance the data platform infrastructure
- Document processes, configurations, and changes to ensure comprehensive system documentation.
Yêu cầu công việc
- Knowledge of one or more of the following AWS CloudFormation and Terraform for infrastructure provisioning
- Knowledge of the source control and its related concepts (Gitlab/Git flow, Trunk base, branches, etc.).
- Experience with CI/CD tools (such as Jenkins, TeamCity, AWS CodePipeline, CodeDeploy) or configuration management tools (such as Ansible, Chef, Puppet..)
- Familiarity with at least one programming language (Python, Bash, etc.).
- Familiarity a distributed compute engine such as Spark
- Familiarity with a data platform or data orchestration tool such as Databricks/Airflow
- Equipped with in-depth working knowledge and experience in using AWS IAM, VPC, EC2, RDS, DynamoDB and S3
- Knowledge of networking, IP, endpoints, firewalls, load balancers, high availability, etc..
- Basic skills around monitoring, observability, and building system dashboards are desired
- DevOps mindset with automation and operation excellence in mind
- Good skills in English and the ability to communicate effectively with business and technical teams
- Demonstrate good logical thinking and problem-solving skills
- Be curious and self-learning attitude
- AWS Certifications (Big Plus)
You are:
- Passionate about technology
- Independent but also a team player
- Comfortable with a high degree of ambiguity
- Focused on usability and speed
- Keen on presenting your ideas to your peers and management.
Ngôn ngữ
-
English
Nói: Intermediate - Đọc: Intermediate - Viết: Intermediate
Yêu cầu kỹ thuật
- AWS
- AWS CloudFormation
- Terraform
- Python
- Jenkins
- Puppet
- TeamCity
- Chef
- Apache Spark
- Ansible
- Amazon S3
- Amazon RDS
- DynamoDB
- Amazon EC2
- VPC
- Bash
- Gitlab
- Gitflow
- Apache Airflow
- AWS CodePipeline
- Databricks
- CI/CD
- IAM
NĂNG LỰC
- Communication Skills
- Logical Thinking
- Problem Solving Skills
Thông tin doanh nghiệp
Tyme is building multi-country digital banks with high-tech and high-touch solutions.
Tyme Vietnam is the international arm of Tyme Group with operations in Hong Kong, Vietnam, The Philippines, and South Africa. We own the technology platform and intellectual property globally and collaborate with in-country partners to launch new digital banks.
Our team is an ideal blend of entrepreneurs, engineers, and banking experts. We are serial bank builders who re-imagine the bank’s digital future through technology.
Our “high-tech, high-touch” philosophy has its roots deep in our culture. As a technology-powered organization, we leverage emerging technologies and collaboration tools to create a digital and mobile workplace that reinforces agility, creativity, and teamwork.
But we don’t forget about touch – the power that connects our people. We strive hard to find the right balance between the digital and physical worlds; the healthy mix that boosts productivity and employee well-being.
We are building a positive, fast-paced work environment that provides intellectual stimulation and inspiration to the best talents. Here at Tyme, you will work with our small, but great teams, and find exciting opportunities with great adventures.