Metrics

AI Model Monitor User Manual

Portfolio
Industrial AI
Product
AI Model Monitor
Software version
2.0.0
Language
en-US

Custom metrics created with AI SDK are automatically collected from the topic

/siemens/edge/aiinference/{model-name}/{model-version}/metrics/{component-name}/{metric-name}.

If you wish to collect custom metrics, make sure that the topic is added to your configured Databus user.

The metrics are extended with labels identifying the running pipeline where it is originated from.

Metric name

Metric type

Metric origin

Metric description

host_box_availability

gauge

Edge device

Device availability. Represented by:

0 - not available and 1 - available.

host_box_memory_total

gauge

Edge device

Total available memory on the box in bytes.

host_box_memory_used

gauge

Edge device

Used memory on the box in bytes.

host_box_cpu_percentage

gauge

Edge device

Used CPU percentage.

host_box_edge_uptime

gauge

Edge device

Uptime of the box represented in minutes.

host_box_running_application_count

gauge

Edge device

Number of running applications on the box.

host_box_max_running_application_count

gauge

Edge device

Maximum number of applications allowed to run.

host_box_ApplicationName_status

gauge

Edge device

Application availability. Represented by:

0 - not available and 1 - available.

pipeline_global_inputs

gauge

AI Inference Server

Number of pipeline inputs generated since the active pipeline is running.

pipeline_global_outputs

gauge

AI Inference Server

Number of pipeline outputs generated since the active pipeline is running.

pipeline_NodeName_exec_min

gauge

AI Inference Server

Node execution time minimum.

pipeline_NodeName_exec_max

gauge

AI Inference Server

Node execution time maximum.

pipeline_NodeName_exec_avg

gauge

AI Inference Server

Node execution time average.

pipeline_NodeName_inputs

gauge

AI Inference Server

Number of node inputs.

pipeline_NodeName_outputs

gauge

AI Inference Server

Number of node outputs.

pipeline_status

gauge

AI Inference Server

The status of the AI Inference Server pipeline stored as label value. The metric always returns 0 value irrespective of the status of the pipeline.

pipeline_StepName_gpuruntime_inference_count

gauge

AI Inference Server

The cumulative count of successful inference requests made for this model (DOES NOT include cache hits).

pipeline_StepName_gpuruntime_execution_count

gauge

AI Inference Server

The cumulative count of successful inference executions performed for the model (DOES NOT include cache hits).

pipeline_StepName_gpuruntime_success_count

gauge

AI Inference Server

The cumulative count of all successful inference requests made for this model (INCLUDING cache hits).

pipeline_StepName_gpuruntime_success_duration

gauge

AI Inference Server

The cumulative duration for all successful inference requests in nanoseconds (INCLUDING cache hits).

pipeline_StepName_gpuruntime_fail_count

gauge

AI Inference Server

The cumulative count of all failed inference requests made for this model.

pipeline_StepName_gpuruntime_fail_duration

gauge

AI Inference Server

The cumulative duration for all failed inference requests in nanoseconds.

pipeline_StepName_gpuruntime_queue_count

gauge

AI Inference Server

The cumulative count of inference requests waited in scheduling or in other queues. (INCLUDING cache hits)

pipeline_StepName_gpuruntime_queue_duration

gauge

AI Inference Server

The cumulative duration that inference requests wait in scheduling (or in other queues) in nanoseconds (INCLUDING cache hits).

pipeline_StepName_gpuruntime_compute_input_count

gauge

AI Inference Server

The cumulative count of the prepared tensor data input required by the model (DOES NOT include cache hits).

pipeline_StepName_gpuruntime_compute_input_duration

gauge

AI Inference Server

The cumulative duration to prepare input tensor data as required by the model in nanoseconds (DOES NOT include cache hits).

pipeline_StepName_gpuruntime_compute_output_count

gauge

AI Inference Server

The cumulative count of the extracted tensor data output produced by the model (DOES NOT include cache hits).

pipeline_StepName_gpuruntime_compute_output_duration

gauge

AI Inference Server

The cumulative duration to extract output tensor data produced by the model in nanoseconds (DOES NOT include cache hits).

pipeline_StepName_gpuruntime_cache_hit_count

gauge

AI Inference Server

The count of response cache hits.

pipeline_StepName_gpuruntime_cache_hit_duration

gauge

AI Inference Server

The cumulative duration to look up and extract output tensor data from the Response Cache on a cache hit in nanoseconds.

pipeline_StepName_gpuruntime_cache_miss_count

gauge

AI Inference Server

The count of response cache misses.

pipeline_StepName_gpuruntime_cache_miss_duration

gauge

AI Inference Server

The cumulative duration to look up and insert output tensor data to the Response Cache on a cache miss in nanoseconds.

pipeline_StepName_gpuruntime_batch_compute_input_duration

gauge

AI Inference Server

The cumulative duration to prepare input tensor data as required by the model in nanoseconds with the given batch size.

pipeline_StepName_gpuruntime_batch_compute_input_count

gauge

AI Inference Server

The cumulative count of input tensor data with the given batch size.

pipeline_StepName_gpuruntime_batch_compute_output_duration

gauge

AI Inference Server

The cumulative duration to extract output tensor data as required by the model in nanoseconds with the given batch size.

pipeline_StepName_gpuruntime_batch_compute_output_count

gauge

AI Inference Server

The cumulative count of output tensor data with the given batch size.

pipeline_StepName_gpuruntime_batch_compute_infer_duration

gauge

AI Inference Server

The cumulative duration to execute the model in nanoseconds with the given batch size.

pipeline_StepName_gpuruntime_batch_compute_infer_count

gauge

AI Inference Server

The cumulative count of model executions with the given batch size.

is_drift

gauge

Monitoring Node

Whether a model drift is detected or not according to the collected buffer. Represented by:

-1 - buffer is being collected or error occurred,

0 - no model drift detected,

1 - model drift detected.

number_of_data_type_errors

gauge

Monitoring Node

Number of data type errors. The default threshold is set to 0.

number_of_missing_properties

gauge

Monitoring Node

Number of missing properties. The range of this metric is between 0 and the number of properties.

ratio_of_categorical_out_of_domain_features

gauge

Monitoring Node

Ratio of categorical out of domain features. This metric is bounded between 0 and 1.

ratio_of_numerical_out_of_domain_features

gauge

Monitoring Node

Ratio of numerical out of domain features. This metric is bounded between 0 and 1.

ratio_of_numerical_outlier_features

gauge

Monitoring Node

Ratio of numerical outlier features.

<custom metric name>

gauge

AI Inference Server

Custom metric defined in AI SDK at model creation.

error_metric_created

gauge

AI Model Monitor Agent

Error metric counter creation time.

error_metric_total

counter

AI Model Monitor Agent

Number of errors occurred during scraping.

scrape_duration_seconds

gauge

Prometheus

Duration of the scrape.

scrape_samples_post_metric_relabeling

gauge

Prometheus

Number of samples remaining after metric relabeling was applied.

scrape_samples_scraped

gauge

Prometheus

Number of samples the target exposed.

scrape_series_added

gauge

Prometheus

Approximate number of new series in this scrape.

up

gauge

Prometheus

Prometheus availability. Represented by:

0 - not available and 1 - available.