Senior Data Engineer

Job description

Capfin is a leading unsecured loan provider with a successful business model supported by great people with a strong culture of innovation and data driven decision making. The purpose of this role is to create business value by applying data engineering and data management disciplines to design and build data solutions that enable data driven decision support, in order to optimize business decisions and processes, as well as train and mentor more junior team members.

Requirements

Key Performance Areas:

Sourcing data
Identify data sources that can add value to decision making.
Work with source system owners and analysts to understand source data, e.g. data profiling, definition and mapping.

Loading data
Design and implement efficient data loads, using traditional structured data ETL techniques.
Design and implement real time and near real time data load solutions, using technologies like data streaming.
Design and implement unstructured data loads, e.g. text speech, images and video.
Design and implement load monitoring tools and procedures and perform continuous monitoring and optimising of loads.

Storing data
Work with analysts and architect to design and implement effective and efficient data models using appropriate modelling techniques.
Design and implement data warehouse data models.
Design and implement data pipelines for ad hoc, unstructured and other data models.

Enhancing data
Design and implement appropriate aggregation data structures that enhance usability of data, e.g. multi-dimensional OLAP structures, summary tables etc.
Design, implement and maintain appropriate indexing on tables to enhance speed of access.
Design and implement data models that support automated decision making and/or further analytics.
Continuously search for data elements from other sources to enhance existing data objects to supplement / enhance context.

Making data available
Design and implement interfaces for data access, e.g. batch exports, real-time decision API’s etc.
Design and implement interface monitoring and management solutions to ensure availability and accuracy.

Managing data
Monitor and maintain integrity of existing environment, troubleshoot technical and data issues and make appropriate changes where required.
Implement meta-data solutions that assist with understanding and managing data.
Work together with business owners, analysts and IT to maintain good data governance.
Provide technical and data related support to source system teams and external parties with whom we exchange data.
Manage data growth and usage by designing and implementing effective strategies, e.g. archiving and indexing.
Manage systems, technology and tools that enable data management and analytics.

Leadership responsibilities
Take ownership of own work by delivering high quality work on time.
Show initiative and be pre-active in finding opportunities to improve data and/or processes.
Take ownership of own career development by continuously improving skills, knowledge and the application thereof in designing and implementing solutions.
Positive engagement in team activities and actively contribute ideas to improve team dynamics and performance.
Stakeholder management (internal and external).
Assist in development of others, e.g. mentoring and knowledge share.
Quality control of other’s work.

Qualifications:
Degree in information technology / mathematics / engineering / actuarial science or related discipline.

Experience:
At least 5+ years in a technical data role, preferably in a formal data, data warehouse or business intelligence environment.

Functional Competencies:
SQL
Data analysis
Data visualisation
Data modelling
Microsoft business intelligence data technologies (SSIS, SSAS, SQL Server)
Data warehouse concepts and best practices
Financial services knowledge, specifically personal or unsecured loans
Business process monitoring and optimising
Microsoft business intelligence visualisation technologies (SSRS, Power BI)
IT infrastructure, e.g. storage, networking, servers, security
Unstructured data experience

Behavioural Competencies:
Information gathering and problem analysis
Applying professional / specialist / technical expertise
Creating and innovating
Quality & Detail orientation
Analysing