FPT Smart Cloud is hiring: AI Engineer – AI Inference (NCP) (Hanoi)
FPT Smart Cloud is hiring an AI engineer in Hanoi to build and tune model serving systems, LLMs above all, with tools such as vLLM, TensorRT-LLM or KServe. Hands-on experience deploying a serving engine, plus Python, Docker and Kubernetes, required.
The role
FPT Smart Cloud (FCI) wants an engineer to design, build and tune the systems that serve its AI models, large language models above all, so that they run fast, stay reliable and cost as little as possible. You would build serving back ends on frameworks such as vLLM, NVIDIA TensorRT-LLM, KServe, TorchServe or TensorFlow Serving, and load parts of a model only when they are needed, to save memory and shorten start-up. An important part of the work is speed: batching, caching, quantization, model parallelism, better use of GPUs and CPUs, and profiling. You would also expose models through REST and gRPC APIs for internal systems, gateways and client apps, set up deployment with logging, metrics, tracing and alerts, and agree on model formats such as ONNX with the research teams that hand models over.
What they ask for
- Has deployed at least one serving engine such as vLLM, TensorRT-LLM, KServe, TorchServe or TensorFlow Serving; production systems with clear SLAs are a plus
- A solid grasp of Transformer models and of loading a model in parts: sharded weights, memory mapping, offloading between CPU and GPU
- Python (preferred) and a web framework such as FastAPI, Flask or Django for inference services
- Backend basics: HTTP, networking, async programming, message queues, caching with Redis or in memory, microservices
- Docker, Kubernetes and CI/CD for AI services, plus monitoring with Prometheus, Grafana or OpenTelemetry
Good to know
- Location: Hanoi, in Vietnam: 7th floor, FPT Tower, 10 Pham Van Bach, Cau Giay. Full time, 8:30 to 17:30 with a lunch break from 12:00 to 13:00.
- Pay is agreed at the interview, based on ability.
- FPT Smart Cloud's website dates the job page, posted or last updated, 20 November 2026.
How the interview usually goes
Interviews for AI inference roles usually mix Python and backend coding with questions on how LLM serving works: the Transformer, the KV cache, batching, quantization and splitting a model across GPUs. An ML system design question is likely as well, for example serving a large model to many users within a latency and GPU budget, along with a deep dive into a serving system you have run.
