Overview Install Files SecuritySummary This source did not publish a separate summary. Review SKILL.md before using the skill.
Performance Profiling
Goal
Provide tools to analyze simulation performance, identify bottlenecks, and recommend optimization strategies for computational materials science simulations.
Requirements
Python 3.8+
No external dependencies (uses Python standard library only)
Works on Linux, macOS, and Windows
Inputs to Gather
Before running profiling scripts, collect from the user:
Input Description Example Simulation log Log file with timing information simulation.logScaling data JSON with multi-run performance data scaling_data.jsonSimulation parameters JSON with mesh, fields, solver config params.jsonAvailable memory System memory in GB (optional) 16.0
Decision Guidance
When to Use Each Script
Need to identify slow phases?
├── YES → Use timing_analyzer.py
│ └── Parse simulation logs for timing data
│
Need to understand parallel performance?
├── YES → Use scaling_analyzer.py
│ └── Analyze strong or weak scaling efficiency
│
Need to estimate memory requirements?
├── YES → Use memory_profiler.py
│ └── Estimate memory from problem parameters
│
Need optimization recommendations?
└── YES → Use bottleneck_detector.py
└── Combine analyses and get actionable advice
Choosing Analysis Thresholds
Metric Good Acceptable Poor Phase dominance <30% 30-50%
Parallel efficiency >0.80 0.70-0.80 <0.70
Memory usage <60% 60-80% >80%
Script Outputs (JSON Fields) Script Key Outputs timing_analyzer.pytiming_data.phases, timing_data.slowest_phase, timing_data.total_timescaling_analyzer.pyscaling_analysis.results, scaling_analysis.efficiency_threshold_processorsmemory_profiler.pymemory_profile.total_memory_gb, memory_profile.per_process_gb, memory_profile.warningsbottleneck_detector.pybottlenecks, recommendations
Workflow
Complete Profiling Workflow
Analyze timing from simulation logs
Analyze scaling from multi-run data (if available)
Profile memory from simulation parameters
Detect bottlenecks and get recommendations
Implement optimizations based on recommendations
Re-profile to verify improvements
Quick Profiling (Timing Only)
Run timing analyzer on simulation log
Identify dominant phases (>50% of runtime)
Apply targeted optimizations to dominant phases
CLI Examples
Timing Analysis # Basic timing analysis
python3 scripts/timing_analyzer.py \
--log simulation.log \
--json
# Custom timing pattern
python3 scripts/timing_analyzer.py \
--log simulation.log \
--pattern 'Step\s+(\w+)\s+took\s+([\d.]+)s' \
--json
Scaling Analysis # Strong scaling (fixed problem size)
python3 scripts/scaling_analyzer.py \
--data scaling_data.json \
--type strong \
--json
# Weak scaling (constant work per processor)
python3 scripts/scaling_analyzer.py \
--data scaling_data.json \
--type weak \
--json
Memory Profiling # Estimate memory requirements
python3 scripts/memory_profiler.py \
--params simulation_params.json \
--available-gb 16.0 \
--json
Bottleneck Detection # Detect bottlenecks from timing only
python3 scripts/bottleneck_detector.py \
--timing timing_results.json \
--json
# Comprehensive analysis with all inputs
python3 scripts/bottleneck_detector.py \
--timing timing_results.json \
--scaling scaling_results.json \
--memory memory_results.json \
--json
Conversational Workflow Example User : My simulation is taking too long. Can you help me identify what's slow?
Ask for simulation log file
Run timing analyzer:
python3 scripts/timing_analyzer.py --log simulation.log --json
Interpret results:
If solver dominates (>50%): Recommend preconditioner tuning
If assembly dominates: Recommend caching or vectorization
If I/O dominates: Recommend reducing output frequency
If user has multi-run data, analyze scaling:
python3 scripts/scaling_analyzer.py --data scaling.json --type strong --json
Generate comprehensive recommendations:
python3 scripts/bottleneck_detector.py --timing timing.json --scaling scaling.json --json
Interpretation Guidance
Timing Analysis Scenario Meaning Action Solver >70% Solver-dominated Tune preconditioner, check tolerance Assembly >50% Assembly-dominated Cache matrices, vectorize, parallelize I/O >30% I/O-dominated Reduce frequency, use parallel I/O Balanced (<30% each) Well-balanced Look for algorithmic improvements
Scaling Analysis Efficiency Meaning Action >0.80 Excellent scaling Continue scaling up 0.70-0.80 Good scaling Monitor at larger scales 0.50-0.70 Poor scaling Investigate communication/load balance <0.50 Very poor scaling Reduce processor count or redesign
Memory Profile Usage Meaning Action <60% available Safe No action needed 60-80% available Moderate Monitor, consider optimization >80% available High Reduce resolution or increase processors >100% available Exceeds capacity Must reduce problem size
Error Handling Error Cause Resolution Log file not foundInvalid path Verify log file path No timing data foundPattern mismatch Provide custom pattern with --pattern At least 2 runs requiredInsufficient data Provide more scaling runs Missing required parametersIncomplete params Add mesh and fields to params file
Optimization Strategies by Bottleneck Type
Solver Bottlenecks
Use algebraic multigrid (AMG) preconditioner
Tighten solver tolerance if over-solving
Consider direct solver for small problems
Profile matrix assembly vs solve time
Assembly Bottlenecks
Cache element matrices if geometry is static
Use vectorized assembly routines
Consider matrix-free methods
Parallelize assembly with coloring
I/O Bottlenecks
Reduce output frequency
Use parallel I/O (HDF5, MPI-IO)
Write to fast scratch storage
Compress output data
Scaling Bottlenecks
Investigate communication overhead
Check for load imbalance
Reduce synchronization points
Use asynchronous communication
Consider hybrid MPI+OpenMP
Memory Bottlenecks
Reduce mesh resolution
Use iterative solver (lower memory than direct)
Enable out-of-core computation
Increase number of processors
Use single precision where appropriate
Limitations
Log parsing : Depends on pattern matching; may miss unusual formats
Scaling analysis : Requires at least 2 runs for meaningful results
Memory estimation : Approximate; actual usage may vary
Recommendations : General guidance; may need domain-specific tuning
References
references/profiling_guide.md - Profiling concepts and interpretation
references/optimization_strategies.md - Detailed optimization approaches
Version History
v1.0.0 (2025-01-22): Initial release with 4 profiling scripts