Senior Data Scientist/ML Engineer: LLMs & Decision Models

Senior Data Scientist / ML Engineer – LLMs and Decision Models | Maternity Leave Replacement | 50% Position | Ramat Gan

A leading high-tech company is transforming how businesses drive revenue through advanced AI solutions, turning customer interactions into actionable insights and automated workflows. The company’s AI platform operates at a massive scale, powering intelligent agents used by thousands of businesses worldwide.
As the company continues to scale its AI capabilities, optimizing LLM usage and reducing operational costs while maintaining high-quality results has become a key priority. The role will focus on expanding the use of in-house decision models across product features, replacing expensive LLM calls wherever possible. In a later phase, the focus will expand to fine-tuning small open-source models to deliver cost-efficient alternatives without compromising performance or quality.

What you’ll do

Phase 1: Decision models in production

• Work with feature teams to identify tasks where decision models can replace expensive LLM calls. Start with use cases identified by the team, then find additional opportunities independently.
• Build evaluation datasets and Python benchmarks, analyze failures, and establish quality criteria with feature owners.
• Validate expected savings against production volumes and serving costs, and confirm that replacements maintain the quality required by each feature.
• Coordinate implementation and rollout with data scientists, feature engineers, and the ML serving team, contributing primarily through evaluations and technical guidance.

Phase 2: Fine-tuned small models

• After establishing initial adoption, fine-tune small open models for specific tasks or broader use cases.
• Validate and roll them out the same way as in Phase 1.

Success means more features moved off expensive LLMs and real cost savings, with no drop in quality.

What you bring

• Hands-on experience fine-tuning open language models, using methods such as supervised fine-tuning, distillation, or LoRA, with PyTorch and Hugging Face.
• Strong Python skills and experience designing ML or LLM evaluations and analyzing quality and cost tradeoffs.
• Experience collaborating with engineering teams to bring models into production, with an understanding of inference and serving constraints.
• Ability to drive work across teams, communicate findings clearly, and identify worthwhile model replacement opportunities.

Nice to Have

• Experience with KServe, vLLM, or similar model serving infrastructure is a strong plus.
• Experience with classification or decision models built on LLMs, including probability calibration.

The Details

• Contract for 6–12 months, about 50% time. Freelancers are welcome.
• We are also hiring for a Senior Applied AI Scientist role. If you fit both, we would love to hear from you.