Industry Spotlight Talks

Dinil Mon Divakaran

Senior Principal Scientist at A*STAR Institute of Advanced Intelligence and Computing

Talk Title:

Rethinking AI for Network Security: Solving Architectural Rigidity Before Building Foundation Models

Abstract: Network traffic volume continues to scale, at the same time its nature is also evolving (e.g., shifting from TCP to QUIC and integrating agentic communications). As encryption protocols have become standard, traditional payload analysis has become obsolete. Despite the long-standing effort by the research community, the practical deployment of AI in Network Security and Privacy (NetS&P) continues to face significant hurdles. Many current approaches inherit architectures and tokenization schemes from NLP or computer vision, flattening traffic into raw byte sequences that fragment protocol fields, break field boundaries, and discard their underlying network semantics. While there is growing interest in developing `foundation models’ to address these challenges, this talk argues that we must first address a fundamental flaw: architectural rigidity in modeling traffic. Current models force a compromise, locking into either packet-level details or aggregated flow-level summaries. We introduce UniNet, a multi-granular traffic modeling framework that reduces reliance on any single representation and allows complementary context across levels. By utilizing a novel structured representation (T-Matrix ) and a lightweight attention architecture (T-Attent), UniNet seamlessly integrates packet, flow, and session contexts. Through empirical evaluations across diverse NetS&P tasks, we demonstrate how UniNet maintains performance, suggesting that resolving the core representational challenge is a necessary step toward developing robust foundation models.

Speaker Bio: Dr. Dinil Mon Divakaran is a Senior Principal Scientist at A*STAR Institute of Advanced Intelligence and Computing (A*STAR IAIC), where he focuses on research, development and technology translation of AI-based security solutions. In his current role, he leads R&D projects in partnership with public sector organizations and industry players, ranging from startups to established corporations. He is also an Adjunct Assistant Professor of the School of Computing in NUS (National University of Singapore).

Bingzhang Fu

Director of Alibaba High Performance (AI & Storage) Network

Talk Title:

Unlocking AI Inference: Mastering EP Load Balancing and Communication in Diverse AI Applications

Abstract: Diverse AI applications, including AI Agents, AI programming, and video generation, are reshaping the workload characteristics of inference systems. These range from the KV Cache pressure caused by long contexts and the need for real-time responses in multi-turn interactions, to the batch processing requirements of video generation and the low-latency preferences in coding scenarios. Such differentiated business models are placing increasingly stringent demands on inference architectures. This presentation focuses on performance optimization for inference systems. It will highlight dynamic load balancing techniques for Expert Parallelism in Mixture-of-Experts (MoE) models, addressing the performance bottlenecks caused by uneven Expert utilization. Additionally, it will introduce a communication optimization scheme tailored for super-node architectures. Drawing on practical deployment experience across multiple scenarios, we will explore how to build a next-generation LLM inference platform that is highly efficient, elastic, and cost-effective.

Speaker Bio: Dr. Binzhang Fu leads High-Performance (AI & Storage) Networking department Alibaba Cloud. He has directed the development of Alibaba Cloud's proprietary high-performance SmartNICs and communication libraries, alongside the end-to-end delivery and operations of hyperscale RDMA networks. Dr. Fu is also a recognized contributor to the field, with over 20 publications in top-tier conferences such as ISCA and SIGCOMM.

Dennis Gu

Huawei Fellow, Chief Architect of Huawei Cloud Service, Head of Cloud Innovation Lab

Talk Title:

FlexNPU: Transparent NPU Virtualization for Dynamic LLM Prefill-Decode Co-location

Abstract: Modern AI serving increasingly relies on NPUs for conventional inference and large language model serving. However, current NPU deployments commonly expose physical devices directly to applications, which limits runtime control over scheduling and makes it difficult to adapt execution to phase-level workload behavior. This limitation is particularly evident in LLM serving, where the prefill phase is compute-intensive while the decode phase is often constrained by memory bandwidth and KV-cache accesses. Static prefill-decode (PD) disaggregation reduces phase interference, but can introduce resource imbalance and unnecessary data movement. We present FlexNPU, a transparent user-space virtualization layer for Ascend NPUs. FlexNPU intercept on AscendCL APIs and routes NPU operations through per-device daemons, decoupling unmodified from physical NPU devices without modifying model inference framework like vLLM & SGLang, AI frameworks, or NPU drivers. This runtime boundary allows FlexNPU to virtualize NPU objects, control operator dispatch, and support phase-aware scheduling for LLM serving. In particular, FlexNPU enables dynamic PD colocation, which adapts scheduling between prefill and decode according to their complementary resource characteristics. We implement FlexNPU on Huawei Ascend NPUs and evaluate it with typical LLM workloads. Compared with direct NPU passthrough, FlexNPU introduces no measurable inference overhead and slightly improves throughput in some scenarios. Under a multi-turn, agentic LLM inference workload of GLM 5.1 with a context length exceeding 30K and a 90% KV cache hit rate, FlexNPU improves Token throughput per NPU card by 146.32% over static PD disaggregation on an Ascend A3 cluster with 128 Ascend 910C NPUs.

Speaker Bio: Dennis Gu led the planning and design of Huawei's unified architecture for public cloud, hybrid cloud, and partner cloud, as well as the FusionSphere cloud operating system. Drove disruptive cloud technology innovations including flexible computing and distributed software-defined storage. Defined the full-stack cloud services governance framework of Huawei Cloud, supporting the global-scale deployment and operation of Huawei's public cloud, hybrid cloud, and partner cloud. Author of Book “Cloud Computing Architecture and Technology Practice” , owning over 50 authorized domestic and international patents on the field of cloud & AI infrastructure.

Yibo Zhu

CTO, StepFun

Talk Title:

Challenges in LLM Inference Infrastructure and How Networks Can Help

Abstract: In this talk, I will begin with reviewing several major innovations in LLM inference infrastructure during the past few years. I will share behind-the-scenes stories and thoughts when we land those novel designs, which were started as research projects, in production systems. Some of them still face deployment challenges today. Then I will discuss how next-generation networks can help address those challenges. The talk will be concluded with one of our such attempts.

Speaker Bio: Dr. Yibo Zhu is a co-founder and CTO of StepFun. Before StepFun, Dr. Zhu had senior engineering leader roles at Google (2023) and ByteDance (2018-2023), where he led a wide range of AI infrastructure directions. He started as a researcher at Microsoft Research and was known for his work on RDMA network, including DCQCN (SIGCOMM'25 Test-of-Time Award). Dr. Zhu received his Computer Science Ph.D. degree from UC Santa Barbara and his bachelor's degree from Tsinghua University.