We are looking for a savvy Data Engineer to join our growing team, who is responsible for building and maintaining the organization's entire data ecosystem. The one will be supporting our software developers, database architects, data analysts and data scientists on data initiatives.
• Design and develop the data platform to efficiently and cost effectively address various data needs across the business.
• Build software across our entire cutting-edge data platform, including event driven data processing, storage, and serving through scalable and highly available APIs, with awesome cutting-edge technologies.
• Ensure performance isn’t our weakness by implementing and refining robust data processing, REST services, RPC (in an out of HTTP), and caching technologies.
• Build process and tools to maintain Machine Learning pipelines in production.
• Develop and enforce data engineering, security, data quality standards through automation.
• Participate in supporting the data platforms 24X7.
• Bachelor’s degree in computer science or Similar discipline.
• 10+ years of experience in software /data engineering and in particular with Apache Spark, Databricks for at least 6+ year’s experience.
• Ability to work in fast paced, high pressure, agile environment and willingness to learn any new technologies and apply them at work in order to stay ahead of the curve.
• Expertise in programming languages Python and expertise in Java, Scala or similar will help.
• Expertise in building and managing large volume data processing (both streaming and batch) platform and performance optimization is a must.
• Expertise in stream processing systems such as Kafka, Kinesis, Pulsar or Similar.
• Expertise in building micro services and managing containerized deployments, preferably using Kubernetes.
• Expertise in distributed data processing frameworks such as Apache Spark, Databricks, Flink, Apache Airflow, Druid or Similar.
• Expertise in SQL, Spark SQL, Hive, MSSQL, Snowflake or Redshift, Postgres, MongoDB or similar is a plus.
• Strong team focus with outstanding organizational and interpersonal, management, communication and presentation skills.