Remote Software and Data Engineer (JB1262)
Experience: Intermediate and Senior
Location: Fully remote for candidates living in South Africa
Duration: Permanent
Salary: R600 - 900 000 per annum
Using applied artificial intelligence and machine learning, our client has become a leader in the utilities space, providing comprehensive solutions. The environment is young, dynamic and fast-moving, with employees that are solution-driven, think like Scientists and build like Software Engineers.
This team creates robust and scalable computing environments which bring data-driven technologies to users. With a drive to constantly build more effective, data architectures, incorporate new technologies and expand computing power.
Ideal candidates would have a mix of Fluency in Python and some variety of SQL, a degree in computer science, or a quantitative discipline, experience in designing and setting up large data systems, with an ability to assess the tradeoffs in architectural decisions, Experience with distributed processing frameworks (Hadoop, Spark etc), Knowledge of large data architectures (data warehouses, lakes, meshes, etc) and experience working in cloud environments, especially Azure.
• Experience in building and optimizing big data pipelines
• Designing and implementing data warehouses, data lakes and data meshes
• Advanced knowledge of relational, non-relational databases with strong experience in SQL
• Working knowledge of streaming, message queuing and highly scalable data stores
• Understanding of DevOps processes, CI/CD, and agile development
• Experience in testing and observability in the context of big data
• Experience with distributed processing frameworks (Apache Spark, Hive, Kafka, etc.)
• Having worked in a cloud environment (Azure, AWS)
• Specific technologies: - Python, R, Scala - SQL (MS SQL Server, Postgres) - Apache Spark, Hadoop, Hive - Databricks, EMR