Lead Data Engineer

Lead Data Engineer

Job Title: Lead Data Engineer
Contract Type: Permanent
Location: London, England
Salary: £70000 - £80000.00 per annum + benefits
Start Date: ASAP
Reference: BBBH17180_1625658731
Contact Name: Tony Smith
Contact Email:
Job Published: July 07, 2021 12:52

Job Description

Our client provides high-quality ethical data products for sensitive and mission-critical operations within the government, security and defence sectors. Their team of subject matter experts, infrastructure and data engineers is now looking to expand, to work on the rapid upscaling of their cutting-edge technical platform.

Lead Data Engineer (Remote working with occasional time in office)

The Lead Data Engineer will lead a small technical team including data engineers and infrastructure engineers to consolidate and scale the technical product. You will work within their secure-by-design AWS infrastructure, running daily stand-ups and working closely with the Client Servicing team to ensure the product continues to evolve, maintaining the precision and quality their customers expect.

Responsibilities will include:
*Setup and maintenance of secure environments for data management and access on AWS.
*Automation, monitoring and evaluation of data quality.
*Working with the Client Servicing team and PM to deliver data products to senior government clients.
*Working closely with senior stakeholders to outline project scope and requirements.
*With the technical PM, advise on feasibility of technical requirements in terms of data availability and appropriateness.
*Working on pre-sales to respond to immediate business opportunities and longer-term development projects.
*Overall responsibility for the setup of API and third-party data pipelines for batch and near real-time data collection across platforms such as Twitter and Youtube.
*Scale projects to ensure data is available as required.
*Scope out new data collection opportunities and manage access.
*Scope requirements and target architecture for project propositions.
*Manage and support the rest of the tech team in debugging, blockers etc.
*Oversee and run daily standups to distribute and prioritise tasks.
*Architect data solutions for new projects and new technology.
*Liaise with CEO and Business development on pitching new work, including costing technology, resource and timeframe.
*Comfortable with being 'hands-on' as well as manage the more strategic / technical aspects of code.
*Have experience in working with data scientists to productionise data science and AI models.

To be considered for this role, applicants will need to have previous experience in a similar Data Engineer role, with experience working in the private sector, building technical products for government, defence or security clients in the field of open-source intelligence data solutions. You will have an in-depth understanding of the open-source information environment, specifically the opportunities and challenges presented by such an environment.

Technical skills required:
Data warehousing
Serverless functions (AWS Lambda)
AWS Glue (PySpark)

In addition, the following experience would be desirable:
CI/CD pipelining

Lead Data Engineer
Remote, home-based (working with 1-2 days per month in Central London).
Salary: To £80k plus benefits

Key Skills: Data Engineer, Senior Data Engineer, Lead Data Engineer, Defence, Government, Security, AWS, Open Source, Open-Source, Data Solutions, Intelligence, Data Analytics, SQL, Python, Data Warehousing, Lambda, Docker, Glue, Terraform, CI/CD Pipelining, Data Pipelines.

PLEASE NOTE: Due to the nature of work undertaken at our client's site, incumbents of these positions are required to meet special nationality rules and therefore these vacancies are only open to British Citizens. Applicants who meet these criteria will also be required to undergo security clearance vetting, if not already security cleared to a minimum SC level.

Electus Recruitment Solutions provides specialist engineering and technical recruitment solutions to a number of high technology industries. We thank you for your interest in this vacancy. If you don't hear from us within 7 working days, please presume your application has been unsuccessful on this occasion. You are of course free to resubmit your CV/details in the future and we shall assess your suitability at that time.

This role is a permanent position.

Electus Recruitment Ltd is acting as an Employment Agency in relation to this vacancy.