Nonlinear Solvers

Goal

Provide a universal workflow to select a nonlinear solver, configure globalization strategies, and diagnose convergence for root-finding, optimization, and least-squares problems.

Requirements

• Python 3.8+
• NumPy (for Jacobian diagnostics)
• SciPy (optional, for advanced analysis)

Inputs to Gather

Input	Description	Example
Problem type	Root-finding, optimization, least-squares	root-finding
Problem size	Number of unknowns	n = 10000
Jacobian availability	Analytic, finite-diff, unavailable	analytic
Jacobian cost	Cheap or expensive to compute	expensive
Constraints	None, bounds, equality, inequality	none
Smoothness	Is objective/residual smooth?	yes
Residual history	Sequence of residual norms	1,0.1,0.01,...

Decision Guidance

Solver Selection Flowchart

Is Jacobian available and cheap?
├── YES → Problem size?
│   ├── Small (n < 1000) → Newton (full)
│   └── Large (n ≥ 1000) → Newton-Krylov
└── NO → Is objective smooth?
    ├── YES → Memory limited?
    │   ├── YES → L-BFGS or Broyden
    │   └── NO → BFGS
    └── NO → Anderson acceleration or Picard

Quick Reference

Problem Type	First Choice	Alternative