Mega metric - how to healthcheck virtual assistant

The report will present the process of business quality monitoring of a virtual assistant. The importance of the task is due to the enormous influence of quality on the key business metric, the monthly audience (MAU). The requirements from the business and their implementation from the point of view of analytics will be considered. A methodology for evaluating the quality of work of each of the large ML systems of the virtual assistant will be presented, as well as a mechanism for combining these assessments into a single pipe to obtain the "metric of omnipotence" - e2e quality of a virtual assistant.