The main goal of the AI Model Monitor Agent is to collect the metrics that are related to the execution of pipelines in AI Inference Server and their environment. It is assumed that AI Inference Server executes valid pipelines.
Metrics contain:
Device level information from AI Edge Runtime
Pipeline level information from AI Inference Server
Metrics added with AI SDK Monitoring Extension
Custom metrics defined in AI SDK
The collected metrics are stored in a Prometheus database inside the AI Model Monitor Agent application, which provides Prometheus functionalities such as monitoring or alerting.
The AI Model Monitor application can be configured as a central metrics collector of multiple AI Model Monitor Agents installed on different IEDs. The application provides a Prometheus user interface where the received metric data can be queried. In addition, AI Model Monitor is prepared to be connected as data source to an externally hosted visualization application, for instance Grafana.
AI Model Monitor Agent****functionalities:
Obtain and store metrics
Analyze metrics directly on the IED via Prometheus
Configure alerting rules
Send alerts to configured receivers
Forward metrics to central AI Model Monitor application
AI Model Monitor****functionalities:
Receive and store metrics centrally