The project is developing a large model trained with on-chain data, our team includes a former founding partner of Polychain, world-renowned algorithm and machine learning experts, and endorsed by world-leading industry investors and angels. We are currently looking for a Data Engineer/Data Engineering Lead to join our team for on-chain data processing and data infrastructure development.
Main Responsibilities: - Responsible for real-time data computation, data service development, and real-time data warehouse construction. Job Responsibilities: - Responsible for big data computation and data service development and design, as well as the development, optimization, and monitoring of Flink, Iceberg, Hadoop, and AWS-EMR clusters. - Responsible for upgrading and optimizing the data infrastructure and data processing system (both real-time and offline) to continuously improve system and service stability and efficiency. Job Requirements: - Proficient in Java or Python development, with knowledge of AWS EMR and related AWS ecosystem products. - Familiar with the principles of Zeppelin, Iceberg, Flink, Flink SQL, Kafka, HDFS, Yarn, Hive, and Airflow, with extensive project development experience. - Knowledge and skills in the data warehouse field are preferred, including but not limited to: data modeling, metadata management, data development testing tools and methods, data quality management. - Practical experience in real-time data warehouse is preferred, especially with experience in implementing stream-batch integration and lake-warehouse integration in the AWS ecosystem. - Ability to quickly identify problems and strong learning ability.
You will have the opportunity to work with arguably the best team in the industry to develop the best model, and you'll also be able to trade using the model, which you won't regret about joining. Working with us will refreshed your impression of how resourceful the team is. The team is globally distributed. The core team is based in the US, while some members are in Asia, including China.
Map For Jobs — Web3行业的求职地图