Hadoop Big Data Engineer - £70,000 - £90.000 plus bonus and bens

  • Permanent
  • London
  • Posted on October 1st, 2018
This vacancy is now closed

Key Skills: Big Data, Hadoop, Clusters, Apache Spark, Apache Kafka, Python, Java, HA, SQL Server, Oracle, Security


Our client are the most prestigious Data Centre transformation consultancy in the UK market who have won multiple awards for their ability to consult the biggest names in the city. They are looking for a Lead Data Engineer to join their established team and deliver on these enterprise scale projects.


This position requires a versatile, multi-faceted technologist with a core focus on the Data Platform.  The individual should have detailed implementation and solution design experience in more than one data-platform technology based around Hadoop and commercial SQL databases.

As well as excellent technical skills, this individual needs good communication and documentation skills.

The applicant should be experienced in hands-on deployment of Hadoop Clusters and integrating associated Apache-project technologies into the Hadoop ecosystem.  Design and Implementation of a Security Architecture and Authentication Integration for this platform is a key requirement.

Additionally, the applicant should be confident operating in a commercial RDBMS environment (preferably MS SQL Server or Oracle RDBMS) and have experience Designing and Implementing standard enterprise database deployments covering topics such as High Availability, Disaster Recovery, Backup and Recovery and ETL strategies.

Experience of deploying a data-platform to at least one major cloud platform (i.e. Azure, AWS, Google) is preferred.

Knowledge and practical experience of Apache Spark, Apache Kafka, Python, SQL performance tuning is desired but not essential.

Primary Responsibilities

Technical consultancy covering the Design and Build of production grade Hadoop architecture data-platforms.  This includes integration into pre-existing identity management services (typically Active Directory).

Review, Design and Remediation engagements for SQL Database infrastructure deployments.  This includes top-level design meetings, documented responses and detailed technical implementation assistance.

Cross-platform data-integration design and implementation.  Design and implement a multi-platform data solution that incorporates structured and unstructured data using relational databases, distributed clusters using Apache technology as well as streaming technologies.

This person is expected to be the go-to Subject Matter Expert, acting as a source of knowledge for the wider technical team.  In addition, this person will work with the Sales team and the Practice Manager helping to build the Data Engineering consultancy business.


  • Demonstrable design and architecture skills
  • Should able to lead and manage different stages of projects such as POC/pilot/transition. 
  • Willing to travel when required 
  • Ability to act as technical expert and take ownership of the technical delivery
  • Good technical, problem solving, and communication skills 

Copyright © 2019 83Zero