Zhengfei Chen
Senior Technical Specialist, eBay AI Platform
He has worked for Ali and other Internet companies, and has many years of experience in basic platform, big data application and AI platform development. Currently working in eBay AI platform team, the work mainly involves feature platform, vector database and other areas, to help data scientists to quickly complete the development of features, on-line and retrieval. Currently, he is mainly responsible for the construction of enterprise-level data pipeline in AI field.
Topic
Constructing a Next-Generation Ray-Based Enterprise Data Pipeline
Introduction: With the rapid development of big data and machine learning technologies, eBay's various business units have an increasing demand for efficient and scalable data processing pipelines. Based on Ray's excellent parallel processing capability, support for heterogeneous computing and other features, our AI platform chose Ray to build the next generation of enterprise-class data pipeline. This presentation focuses on how eBay utilizes Ray to build data pipelines to support complex data processing and machine learning tasks. a) Background and issues b)Design principles and overall architecture c)Using Ray to build batch and near real-time data pipelines d)Effects and outlook
