Data Quality Engineer - Johannesburg Verfied

Salary Negotiable Johannesburg, Gauteng Johannesburg, Gauteng more than 14 days ago 28-04-2021 11:47:10 AM
23-06-2021 11:47:10 AM
PBT Group has an opportunity for a Data Quality Engineer.

The Data Quality Engineer is responsible for designing, developing, documenting and performing data quality checks across all data assets. That includes ETL jobs, reports, dashboards and data pipelines. The primary goal for this role is to ensure high quality of data delivered to internal stakeholders and customers. Validation of data in data repositories against data from source systems and validation of metrics and data in reports/dashboards against data in the repositories is a key responsibility. Principle responsibilities are to making data assets consistently accurate for users.

Duties:
- Design, develop and maintain data quality assurance framework
- Work in conjunction with BI and Data Engineers to ensure high quality Data Deliverable
- Design and develop testing frameworks to test ETL jobs, BI reports and Dashboards and other data pipelines
- Write SQL scripts to validate data in the data repositories against the data in the source systems
- Write SQL scripts to validate data surfacing in BI assets against the data sources
- Ensure data quality by checking against our ODS and the front-end application
- Track, monitor and document testing results
- The development and maintenance of Extract Transform and Load (ETL) processes, database and performance administration, and dimensional design of the table structure. Work closely with Data Architect to understanding and operating data warehousing functionality, building the Unified Data Platform
- Write high-quality, well-structured code that is maintainable and extensible
- Design and build multi-tenancy Data Warehouse for business intelligence analytics solutions and other applications
-Analyze complex data systems to develop automated and reusable solutions for extracting requested information while assuring data validity and integrity
- Perform tasks spanning the full lifecycle of data management activities with minimal supervision
- Collaborate with Product Managers and business community to elicit, document, and translate business requirements for data needs into technical requirements while contributing to a unified system architecture and minimizing technical debt
- Promote and support the infrastructure of reporting, analytics, and data sourcing strategy; this includes developing and advancing standards and best practices and recommending infrastructure changes when appropriate
- Perform ongoing monitoring and refinement of data platform
- Design and implement solutions to improve data platform
- Collaborate on and learn new technologies
- Maintain, expand, and improve ETL processes
- Maintain data standards, enforce standard development protocols, and analyze requirements to ensure technical and standard operating procedure impacts are considered
- Develop and coordinate test plans
- Troubleshoot and performance tune data management code
- Limit non-standard solutions and escalate when used with documentation supporting exception requirements
- Manage code using GitHub and other version control approaches as applicable
- Perform other duties as assigned
- Work in a highly collaborative agile team
- Coordinate work with Database Administrators and System Engineers, as needed
- Assist report writers and data visualization team members with data sourcing
- Participate in technical reviews and provide detailed feedback for process improvement
- Collaborate with the Decision Services team, other members of Information Services, and cross-functional business stakeholders to translate business requirements into technical specifications

Skills & Experience Required:
- 7+ years relevant work experience working in a business intelligence developer role, preferably in the telco industry.
- Software/Hardware: Preferably 5+ years’ experience in Microsoft SQL Server 2005 and 2008 RDBMS, Microsoft SQL Server 2005 and 2008 Integration Services (SSIS), Microsoft SQL Server Reporting Services (SSRS) and Microsoft SQL Server Analysis Services (SSAS). Expert knowledge of SQL programming languages.
- Exceptional candidates will have proven experience with some or all the following technologies: web services, ASP.Net, C#, QlikView
- Effective communication skills with an ability to explain technical concepts to developers, product managers, and business partners
- Excellent problem solving and critical thinking skills
- Ability and desire to work with a team of people solving complex problems that often require independent research with minimal supervision
- Self-starter able to make an impact with little guidance
- Understanding of ETL methodologies and Data Warehousing principles, approaches, technologies, and architectures including the concepts, designs, and usage of data warehouses and data marts
- Experience working in virtualized cloud environment including cloud-based IaaS/SaaS/PaaS solutions.
- Experience in writing complex SQL queries and stored procedures
- Knowledge of data warehousing, OLAP, multi-dimensional, star and snowflake schemas
- Knowledge and experience with database design principles including referential integrity, normalization, and indexing to support application development
- Strong understanding and experience in development activities for all aspects of Software Development Life Cycle (SDLC)
- Understanding of agile processes
- Outstanding attention to detail and ability to meet deadlines
- Self-starter with the ability to multitask in a dynamic work environment
- Familiarity with Microsoft Visual Studio, SQL Server Reporting Services, and Team Foundation Server desirable

Education:
- A degree in Computer Science / Information Systems / Business Administration / Commerce or equivalent