← Zurück zur Liste
Stelle · Senior

Python Inference Engineer

Sonstige • Senior • Remote • Vollzeit • Serbien Serbien

Gcore is looking for a Python Inference Engineer to build and improve the inference layer of the Gcore Inference platform, integrating inference frameworks and bringing new language and multimodal models into production.

Responsibilities

  • ▹Build and improve the inference layer of the Gcore Inference platform.
  • ▹Integrate and operate inference frameworks such as vLLM, SGLang, NVIDIA Dynamo and TensorRT-LLM.
  • ▹Bring new language and multimodal models into production.
  • ▹Improve inference latency, throughput, memory use, GPU utilization and cost efficiency.
  • ▹Debug performance and reliability issues across model code, inference frameworks, GPU execution, networking and Kubernetes.
  • ▹Work with platform, infrastructure, product and customer-facing teams to turn inference improvements into reliable product features.
  • ▹Contribute improvements to open-source inference projects when appropriate.

Requirements

  • ▹5+ years of experience writing reliable, well-tested production code.
  • ▹Strong Python skills and experience designing production systems.
  • ▹Hands-on experience with PyTorch and deploying machine learning models.
  • ▹Experience with Linux, Docker and Kubernetes.
  • ▹Experience in at least one relevant area: distributed systems, GPU computing, ML runtimes, model optimization or cluster scheduling.
  • ▹Ability to debug complex problems across software, infrastructure and hardware.
  • ▹Strong sense of developer experience — thinking about how systems are used, not just how they work.
  • ▹Genuine interest in inference engineering; direct experience with inference engines is not required, but motivation to learn is.
  • ▹Good communication and collaboration skills.

Nice to have

  • ▹Experience with vLLM, SGLang, NVIDIA Dynamo, TensorRT-LLM or a similar inference framework.
  • ▹Experience running GPU workloads in production.
  • ▹Knowledge of inference techniques such as quantization, continuous batching, speculative decoding, prefix caching, chunked prefill or LoRA serving.
  • ▹Experience with CUDA, Triton, TensorRT or other GPU programming tools.
  • ▹Experience profiling and improving model latency, throughput, memory use or GPU utilization.
  • ▹Experience with distributed inference, multi-GPU systems, scheduling or autoscaling.
  • ▹Contributions to open-source ML, inference or infrastructure projects.

Soft skills

Sense of developer experienceGood communication and collaboration skillsGenuine interest in learning

What we offer

  • ▹Competitive compensation.
  • ▹Flexible working hours and hybrid or remote options, depending on role.
  • ▹Work from anywhere in the world for up to 45 days a year.
  • ▹Private medical insurance for you and your family.
  • ▹Extra paid vacation and sick leave days.
  • ▹Support for life's important moments and celebrations.
  • ▹Language courses.
  • ▹Modern, welcoming offices with snacks, drinks and entertainment.
  • ▹Team sports and social activities.

About the company

Gcore is a global provider of infrastructure and software solutions for AI, cloud, network and security, powering everything from real-time communication and streaming to enterprise AI.

Ähnliche Stellen