Pareto Optimization

Pareto optimization deals with multi-objective optimization where you want to optimize multiple conflicting objectives simultaneously.

Key Concepts

Pareto Dominance

Point A dominates point B if:

• A is at least as good as B in all objectives
• A is strictly better than B in at least one objective

Pareto Frontier (Pareto Front)

The set of all non-dominated points. These represent optimal trade-offs where improving one objective requires sacrificing another.

Computing the Pareto Frontier

Using the paretoset Library

from paretoset import paretoset
import pandas as pd

# Data with two objectives (e.g., model accuracy vs inference time)
df = pd.DataFrame({
    'accuracy': [0.95, 0.92, 0.88, 0.85, 0.80],
    'latency_ms': [120, 95, 75, 60, 45],
    'model_size': [100, 80, 60, 40, 20],
    'learning_rate': [0.001, 0.005, 0.01, 0.05, 0.1]
})

# Compute Pareto mask
# sense: "max" for objectives to maximize, "min" for objectives to minimize
objectives = df[['accuracy', 'latency_ms']]
pareto_mask = paretoset(objectives, sense=["max", "min"])

# Get Pareto-optimal points
pareto_points = df[pareto_mask]

Manual Implementation

import numpy as np

def is_dominated(point, other_points, maximize_indices, minimize_indices):
    """Check if point is dominated by any point in other_points."""
    for other in other_points:
        dominated = True
        strictly_worse = False

        for i in maximize_indices:
            if point[i] > other[i]:
                dominated = False
                break
            if point[i] < other[i]:
                strictly_worse = True

        if dominated:
            for i in minimize_indices:
                if point[i] < other[i]:
                    dominated = False
                    break
                if point[i] > other[i]:
                    strictly_worse = True

        if dominated and strictly_worse:
            return True

    return False

def compute_pareto_frontier(points, maximize_indices=[0], minimize_indices=[1]):
    """Compute Pareto frontier from array of points."""
    pareto = []
    points_list = list(points)

    for i, point in enumerate(points_list):
        others = points_list[:i] + points_list[i+1:]
        if not is_dominated(point, others, maximize_indices, minimize_indices):
            pareto.append(point)

    return np.array(pareto)