Model Distillation Report

Comparison of Best Student Models by Metric

About This Report

This report presents the best student models for each evaluation metric, comparing their performance with the teacher model. The report shows the optimal hyperparameters (model type, temperature, and alpha) for each metric.


Models were trained using knowledge distillation to transfer knowledge from a complex teacher model to simpler, more efficient student models.

Best Models by Metric

{% for metric_key, model in best_models.items() %} {% if metric_key in ['test_accuracy', 'test_kl_divergence', 'test_ks_statistic', 'test_r2_score', 'test_auc_roc', 'test_f1'] %}
Best by {{ model.display_name }} {% if model.minimize %} (lower is better) {% elif metric_key == 'test_r2_score' %} (higher is better) {% endif %}
{{ model.model_type }}
Temperature: {{ model.temperature }}
Alpha: {{ model.alpha }}

{{ model.display_name }}: {{ "%.3f"|format(model.value) }}
Teacher: {{ "%.3f"|format(model.teacher_value) if model.teacher_value is not none else 'N/A' }} {% if model.difference is defined and model.teacher_value is not none %} {% if model.minimize %} {{ "+%.3f"|format(model.difference) if model.difference > 0 else "%.3f"|format(model.difference) }} {% else %} {{ "%.3f"|format(model.difference) }} {% if model.retention is defined and model.retention is not none %} ({{ "%.1f"|format(model.retention) }}%) {% endif %} {% endif %} {% endif %}
{% endif %} {% endfor %}

Detailed Metrics Comparison

{% for metric_key, model in best_models.items() %} {% endfor %}
Metric Best Model Type Temp Alpha Teacher Value Student Value Difference Retention %
{{ model.display_name }}{% if model.minimize %} (lower is better){% endif %} {{ model.model_type }} {{ model.temperature }} {{ model.alpha }} {{ "%.3f"|format(model.teacher_value) if model.teacher_value is not none else 'N/A' }} {{ "%.3f"|format(model.value) }} {% if model.difference is defined and model.teacher_value is not none %} {% if model.minimize %} {{ "+%.3f"|format(model.difference) if model.difference > 0 else "%.3f"|format(model.difference) }} {% else %} {{ "%.3f"|format(model.difference) }} {% endif %} {% else %} N/A {% endif %} {% if model.retention is defined and model.retention is not none %} {{ "%.1f"|format(model.retention) }}% {% else %} N/A {% endif %}

Summary and Recommendations

Key Findings

  • {{ summary.best_overall_model }} models generally performed best overall
  • {{ summary.best_dist_model }} models performed best for distribution metrics (KL Divergence and R² Score)
  • Most models maintained at least 97% performance compared to the teacher
  • Higher temperatures ({{ "%.1f"|format(summary.avg_temperature) }} average) improved distribution similarity metrics
  • Alpha values around {{ "%.1f"|format(summary.avg_alpha) }} were optimal for most metrics

Recommended Configuration

Based on the overall performance, we recommend:

Model: {{ summary.recommended_model }}
Temperature: {{ summary.recommended_temp }}
Alpha: {{ summary.recommended_alpha }}

Rationale: This configuration provides the best balance between maintaining predictive accuracy and achieving excellent probability distribution matching.