Jian Cai
AliCloud Observable Field Senior Expert
With many years of experience in Internet industry and business architecture design, currently focusing on AliCloud observable application observable field, mainly engaged in observable product ARMS and EagleEye development, design and evangelism, with rich observable field technology architecture and practical experience, successfully promote ARMS application performance monitoring and application security (RASP) convergence solution landing, attention to the latest developments in APM and OpenTelemetry open source community ecosystem. He has successfully promoted ARMS application performance monitoring and application security (RASP) convergence solutions, and is concerned about the latest development of APM and OpenTelemetry open source community ecosystem. Currently, he is focusing on the observable demand scenarios in the field of large language modeling, exploring the full-stack observable capability solutions and best practices to support the LLM application layer to the underlying infrastructure.
Topic
Exploration and Practice of Observable Solutions for LLM Applications
Introduction: With the concept of generative AI, a series of commercial or open-source big models have emerged in the market, represented by ChatGPT and Tongyi big model, and more and more applications and business scenarios have been built based on big language models and AI ecosystem technology stacks, and the large-scale model training and model reasoning scenarios have given rise to the demand for jobs related to MLOps, LLMOps, and so on. How to monitor and guarantee the performance and user experience of large model applications? How to support link visualization analysis and root cause localization for LLM applications in complex topology scenarios? How to obtain the actual online performance in terms of cost and effect, and assist in selecting, analyzing, evaluating, and optimizing the iteration of LLM models? Based on the above requirements and problem backgrounds, observable capability solutions for LLM application technology stacks are becoming increasingly important topics. Outline: 1、Why do we need LLM application observability? 2、What are the concerns of LLM application observability? 3, LLM application observable high quality data collection and reporting 4, Ali cloud LLM application observable solution introduction 5, LLM application observable customer practice cases 6、LLM Application Observability Outlook and Challenges
