Data Engineer

We are looking for a savvy Data Engineer to join our growing team, who is responsible for building and maintaining the organization's entire data ecosystem. The one will be supporting our software developers, database architects, data analysts and data scientists on data initiatives.

Responsibilities

• Design and develop the data platform to efficiently and cost effectively address various data needs across the business.

• Build software across our entire cutting-edge data platform, including event driven data processing, storage, and serving through scalable and highly available APIs, with awesome cutting-edge technologies.

• Ensure performance isn’t our weakness by implementing and refining robust data processing, REST services, RPC (in an out of HTTP), and caching technologies.

• Build process and tools to maintain Machine Learning pipelines in production.

• Develop and enforce data engineering, security, data quality standards through automation.

• Participate in supporting the data platforms 24X7.

Qualification

• Bachelor’s degree in computer science or Similar discipline.

• 10+ years of experience in software /data engineering and in particular with Apache Spark, Databricks for at least 6+ year’s experience.

• Ability to work in fast paced, high pressure, agile environment and willingness to learn any new technologies and apply them at work in order to stay ahead of the curve.

• Expertise in programming languages Python and expertise in Java, Scala or similar will help.

• Expertise in building and managing large volume data processing (both streaming and batch) platform and performance optimization is a must.

• Expertise in stream processing systems such as Kafka, Kinesis, Pulsar or Similar.

• Expertise in building micro services and managing containerized deployments, preferably using Kubernetes.

• Expertise in distributed data processing frameworks such as Apache Spark, Databricks, Flink, Apache Airflow, Druid or Similar.

• Expertise in SQL, Spark SQL, Hive, MSSQL, Snowflake or Redshift, Postgres, MongoDB or similar is a plus.

• Strong team focus with outstanding organizational and interpersonal, management, communication and presentation skills.

Apply to Join DQLabs Team



Best Practices

Data + AI Summit 2022 - DQLabs Events

EVENTS

Data + AI Summit 2022

Come meet us at Data + AI Summit 2022, the world’s largest data and AI conference happening in San Francisco and takes place virtually in a hybrid format on June 27-30, 2022. DQLabs is proud to be a sponsor for the grand event.  

Explore the latest trends and innovations, technical sessions, and networking opportunities from AI thought leaders and data professionals all around the world. 

If you want to explore or integrate a data management solution for your organization, you must meet DQLabs in the event. 

DQLabs’ AI augmented data quality platform gives organizations the ability to manage data smarter and leverage an immediate ROI in weeks, rather than months. With a Data Quality first approach powered by ML and self-learning capabilities organizations can connect, discover, measure, monitor, remediate and improve data quality across any type of data – all in one agile and innovative self-service platform. 

View More Arrow image