Spark Developer

• Contribute fixes and enhancements to the Apache Spark open-source project. • Upgrade and maintain compatibility with newer Spark releases. • Optimize Spark internals for performance, scalability, and reliability. • Design and develop distributed data processing solutions. • Debug complex Spark-related performance and stability issues. • Collaborate with product and platform teams on architecture and technical initiatives.

About the Role We are looking for a Senior Software Engineer with strong expertise in Apache Spark and distributed systems. The role involves working on Spark internals, contributing to the Apache Spark open-source community, and building scalable data processing solutions aligned with the platform requirements. Key Responsibilities • Contribute fixes and enhancements to the Apache Spark open-source project. • Upgrade and maintain compatibility with newer Spark releases. • Optimize Spark internals for performance, scalability, and reliability. • Design and develop distributed data processing solutions. • Debug complex Spark-related performance and stability issues. • Collaborate with product and platform teams on architecture and technical initiatives. Required Skills • Strong experience with Apache Spark internals and distributed systems. • Hands-on experience contributing to open-source projects. • Strong Java and/or Scala programming skills. • Expertise in query execution, distributed processing, and performance optimization. • Strong debugging and problem-solving abilities. • Ability to work independently and drive technical initiatives. Good to Have • Experience with Kubernetes and cloud-native environments. • Knowledge of JVM tuning and system optimization. • Contributions to Apache or other open-source communities. • Experience with large-scale analytics or data platforms. Originally posted on Himalayas Spark-Developer, Big-Data-Engineer, Software-Engineer, Distributed-Systems-Engineer, Apache-Spark-Developer, PySpark-Developer, Spark-Development, Apache-Spark-Development, PySpark-Development, Senior-Java-Spark-Developer, Databricks-Developer, Scala-Developer, Developer