About UsZylon (https://www.zylon.ai/) transforms company knowledge into immediate productivity gains through a secure, private AI workspace. Our on-premise deployment ensures complete data sovereignty, cost control, and full customization by running local AI models with no vendor lock-in.As creators and maintainers of the popular open-source project PrivateGPT (https://privategpt.dev/) with more than 55K Github stars, we are committed to leveraging the latest AI technologies to drive innovation and deliver exceptional value to our customers.As an early-stage startup serving clients in financial, manufacturing, engineering, and other regulated industries, we're looking for talented individuals to help drive our growth in a company where we celebrate diversity and are committed to creating an inclusive environment for all employees.Role OverviewWe're seeking an experienced AI Engineer to join our team and play a crucial role in developing and enhancing our Private AI platform. You'll work on the whole AI stack, ranging from the GPU management and inference server, to leveraging and optimizing open-source AI models for on-premise bare metal deployments where we have complete control of the data pipeline and the agentic system running on top of it. All of this while ensuring high performance and security for our enterprise clients in regulated industries.Key ResponsibilitiesDesign, develop, and deploy AI systems that can run efficiently in on-premise environments with local models, without depending on 3rd parties like OpenAI, Amazon Bedrock, etc.Optimize Small Language Models (SLMs) for private deployments with attention to performance and resource constraints.Contribute to the architecture of our AI platform, ensuring scalability and security.Implement advanced prompt engineering techniques and secure data processing pipelines for knowledge extraction and transformation.Research, design and implement agentic strategies to improve AI model interaction and user experience.Stay current with the latest advancements in GenAI research and identify opportunities for bringing additional value to our product, as well as developing within the team a deep understanding of the capabilities, limitations, and availability of the different AI models.Work together with the Product & Engineering teams, actively participating not only in the implementation but also in the product definition, prioritization, scoping, and validation.Work closely with client success teams to troubleshoot and resolve technical issues.Requirements5+ years of experience in software or/and AI engineering roles with special focus on GenAI for the last 2 yearsExperience with on-premise deployment of AI systems.Strong background in integrating and fine-tuning open-source LLMsProficient in Python and relevant ML/AI frameworks (PyTorch, LlamaIndex, LangChain, etc.)Familiarity with Agents strategies and agentic frameworks, as well as different RAG implementationsExperience with vector databases and efficient retrieval systemsKnowledge of containerization and deployment technologies (Docker, Kubernetes)Strong problem-solving skills and attention to detailExcellent English communication skills and ability to collaborate effectively with cross-functional teamsEager to be up-to-date with new techniques, models and advances within the GenAI fieldLean and creative mindset, ready to iterate fast, learn and adaptCustomer-focused mindsetNice to HaveExperience working with vLLM + NVIDIA Triton architectureUnderstanding of NVIDIA drivers and CUDA platform to power NVIDIA GPUs for AIKnowledge of evaluation and observability frameworks (LangSmith, Opik, Arize Phoenix, Ragas, etc.) for GenAIKnowledge of Model Context Protocol (MCP) for building agentic systems.Experience with model quantization and optimization techniquesExperience in startups or early-stage companiesBackground in working with financial, manufacturing, or engineering domainsWhat We OfferOpportunity to shape the future of private AI in regulated industriesCompetitive salary and equity packageWork with cutting-edge AI technologiesDirect impact on product development and company growthCollaborative team environment focused on innovationFull-remote position, within EuropeFlexible schedule (40h/week)23 days of PTONecessary equipment to workPeriodic on-site team building events