Big Data Developer - Roodepoort
Salary Negotiable
Roodepoort, Gauteng
Roodepoort,
Gauteng
more than 14 days ago
07-05-2018 10:13:15 AM
02-07-2018 10:13:15 AM
PBT Group has an opportunity for a Big Data Developer with solid experience in Spark.
With over two decades of expert knowledge and experience as information management specialists, PBT is a recognised leader in Business Intelligence and data warehousing and has extended its competency base to cover data quality and advanced analytics.
We pride ourselves to be a company who attracts, recruits and retains consultants of an exceptional calibre, not only within BI, but also within focused areas of Application Development.
We have worked in more than 25 countries around the world and are currently implementing large-scale projects throughout Africa, the Middle East and Australia.
Duties:
- Support and Administration of existing Big Data solution – this include production, development and DR environments
- Analysis, Design, Development and maintenance of ETL processes – this will include development of NiFi flows for feed integration, Design of appropriate tables, development of Spark processes to load data and ongoing maintenance of these.
- Development of Linux scripts to automate support and maintenance tasks
Skills Required:
- 5+ years working in software development
- 2+ years developing, deploying and maintaining high volume production big data solutions
- 2+ years developing solutions in Spark using Scala & Python, Spark SQL, Spark Streaming
- 2+ years’ experience programming in Java
- Oozie workflow development and maintenance
- Experience using Ambari to administer a Big Data environment
- NiFi flow development, administration and maintenance
- HBase administration and table design
- Table design and DB administration in Apache Phoenix
- Analytics with Hive
- Security administration using Ranger
- Development of Kafka producers and consumers
- Strong database, SQL, ETL and data analysis skills
- Ability to work within a team
- Experience with basic Linux administration
Nice to have:
- Experience maintaining a Hortonworks environment
- HDP Certified Administrator qualification
- Master's degree in Computer Science or equivalent.
- Knowledge of EMC Isilon is an asset
- Experience with Bash Scripting
Qualifications/ Certification:
- Bachelor's degree in Computer Science or equivalent