AI Data Engineer

AI/Data Engineer
Remote
€500 - €600 Outside
4 Month Contract

AI/Data Engineer needed for a product development project focused on an AI-powered Retrieval Augmented Generation (RAG) search application. The project is focused on the improvement of access to enterprise knowledge, documents, policies and operational data. The objective of this role is to build, design, and maintain the Data Ingestion, Data processing, indexing and RAG Pipelines.

Job Responsibilities/Objectives:

• Design and implement data ingestion pipelines from various enterprise data sources (SharePoint, Liferay, Web crawls etc) into equivalent search index
• Develop connectors and ingestion jobs capable of supporting batch, incremental and near-real-time indexing patterns
• Design the mechanisms to track document versions, source provenance, access permissions, update time stamps and deletion events for the accuracy and relevance of the search index
• Design and build the RAG pipeline that retrieves relevant enterprise content and uses large language models for grounded answers
• Implement agentic workflows for the AI application to answer user queries
• Implement Prompt engineering and orchestration patterns for reliable response generation (System prompts, Retrieval prompts, Guardrails, Context Assembly, Response Formatting)
• Implement Advanced retrieval techniques (Query writing, Query Expansion, Multi-query Retrieval, Metadata-aware Retrieval, Parent-child Retrieval, Contextual Compression)

Required Skills/Experience:

• Proficiency in Python
• Experience with data processing frameworks and libraries
• Hands-on experience with enterprise search platforms (Elasticsearch, OpenSearch, Azure AI Search)
• Experience designing index mappings, Metadata filters, Ranking profiles, synonyms and relevance tuning
• Experience with vector search, approximate nearest neighbour search, hybrid retrieval and semantic search
• Experience implementing semantic chunking strategies - Split content into contextually coherent sections (Heading, Structure, Metadata and parent document relationships to improve accuracy with the RAG Applications
• Experience in Ingestion pipelines that convert enterprise documents
• Experience in LLM orchestration frameworks (LangChain, LlamaIndex, Haystack or equivalent)

If you are interested in this opportunity, please apply now with your updated CV in Microsoft Word/PDF format.

Disclaimer
Notwithstanding any guidelines given to level of experience sought, we will consider candidates from outside this range if they can demonstrate the necessary competencies.
Square One is acting as both an employment agency and an employment business, and is an equal opportunities recruitment business. Square One embraces diversity and will treat everyone equally. Please see our website for our full diversity statement.