Lead Data Engineer
|Job Title:||Lead Data Engineer|
|Salary:||£70000 - £80000.00 per annum + benefits|
|Contact Name:||Tony Smith|
|Job Published:||July 07, 2021 12:52|
Lead Data Engineer (Remote working with occasional time in office)
The Lead Data Engineer will lead a small technical team including data engineers and infrastructure engineers to consolidate and scale the technical product. You will work within their secure-by-design AWS infrastructure, running daily stand-ups and working closely with the Client Servicing team to ensure the product continues to evolve, maintaining the precision and quality their customers expect.
Responsibilities will include:
*Setup and maintenance of secure environments for data management and access on AWS.
*Automation, monitoring and evaluation of data quality.
*Working with the Client Servicing team and PM to deliver data products to senior government clients.
*Working closely with senior stakeholders to outline project scope and requirements.
*With the technical PM, advise on feasibility of technical requirements in terms of data availability and appropriateness.
*Working on pre-sales to respond to immediate business opportunities and longer-term development projects.
*Overall responsibility for the setup of API and third-party data pipelines for batch and near real-time data collection across platforms such as Twitter and Youtube.
*Scale projects to ensure data is available as required.
*Scope out new data collection opportunities and manage access.
*Scope requirements and target architecture for project propositions.
*Manage and support the rest of the tech team in debugging, blockers etc.
*Oversee and run daily standups to distribute and prioritise tasks.
*Architect data solutions for new projects and new technology.
*Liaise with CEO and Business development on pitching new work, including costing technology, resource and timeframe.
*Comfortable with being 'hands-on' as well as manage the more strategic / technical aspects of code.
*Have experience in working with data scientists to productionise data science and AI models.
To be considered for this role, applicants will need to have previous experience in a similar Data Engineer role, with experience working in the private sector, building technical products for government, defence or security clients in the field of open-source intelligence data solutions. You will have an in-depth understanding of the open-source information environment, specifically the opportunities and challenges presented by such an environment.
Technical skills required:
Serverless functions (AWS Lambda)
AWS Glue (PySpark)
In addition, the following experience would be desirable:
Lead Data Engineer
Remote, home-based (working with 1-2 days per month in Central London).
Salary: To £80k plus benefits
Key Skills: Data Engineer, Senior Data Engineer, Lead Data Engineer, Defence, Government, Security, AWS, Open Source, Open-Source, Data Solutions, Intelligence, Data Analytics, SQL, Python, Data Warehousing, Lambda, Docker, Glue, Terraform, CI/CD Pipelining, Data Pipelines.
PLEASE NOTE: Due to the nature of work undertaken at our client's site, incumbents of these positions are required to meet special nationality rules and therefore these vacancies are only open to British Citizens. Applicants who meet these criteria will also be required to undergo security clearance vetting, if not already security cleared to a minimum SC level.
Electus Recruitment Solutions provides specialist engineering and technical recruitment solutions to a number of high technology industries. We thank you for your interest in this vacancy. If you don't hear from us within 7 working days, please presume your application has been unsuccessful on this occasion. You are of course free to resubmit your CV/details in the future and we shall assess your suitability at that time.
This role is a permanent position.
Electus Recruitment Ltd is acting as an Employment Agency in relation to this vacancy.
Get similar jobs like these by email
By submitting your details you agree to our T&C's
Lead Cloud Solutions Engineer
Manchester, Greater Manchester
Senior Data analyst
Up to £55000.00 per annum + benefits
Software Test Engineer
West Yorkshire, England
Up to £0.00 per annum
£50000 - £60000.00 per annum + benefits