Auto-Tuner Anatomy ①: Engine Overview and Design Philosophy

This document series dissects the internals of EXAWin's Auto-Tuner engine. We explain the meaning behind every line of code, the rationale for each statistical technique, and why each parameter must remain within its specific range — all in a lecture-style narrative.

By the time you finish this series, you will be able to explain why every Auto-Tuner recommendation is that specific value.

1. What is the Auto-Tuner?

1.1 One-Line Definition

Auto-Tuner = A system that "makes accurate" the Bayesian engine's parameters based on historical project outcomes (Won/Lost)

The key word here is "accurate." It does not raise P(Win) — rather, it adjusts parameters so that Won deals have high P(Win) and Lost deals have low P(Win) — aligning predictions with reality.

1.2 The Car Analogy

The engine (Bayesian formula) itself doesn't change. What the Auto-Tuner does is adjust the fuel mixture:

1.3 Five Design Principles

① Not fitting, but making accurate
② Preserve the impedance dual-structure
③ Provide recommendation + rationale together
④ Human approval mandatory — no automatic application
⑤ Stored data immutable — simulations are pure computation

Principle ⑤ is particularly important. The Auto-Tuner never modifies the database. When the analysis button is pressed, simulations run in memory, and only when the administrator clicks "Apply" does the database get updated.

2. Six Learning Targets

The Auto-Tuner analyzes and recommends exactly six parameters.

① Signal Lift — Discriminative Power Analysis

"When this signal appears, does the probability of winning actually increase?"

Calculates the Lift = (appearance rate in Won) / (appearance rate in Lost) for each signal. Lift > 1 indicates a positive indicator; Lift < 1 indicates a negative indicator. Validates whether the current classification (Positive/Negative) matches actual discriminative power.

📌 Details: ② Signal Lift Anatomy

② Impact Score — Optimal Weights

"Is 5.0 really the optimal value for Game Changer?"

Varies each ImpactType's score within a ± range to find the value that maximizes Separation (Won avg P(Win) − Lost avg P(Win)). Search range expands by Phase.

📌 Details: ③ Grid Search Engine Anatomy

③ T — Threshold Optimization

"Where should each stage's threshold be placed to best distinguish Won from Lost?"

Finds the T that maximizes Youden J statistic = Sensitivity + Specificity − 1. If J < 0.20, it means "the data cannot distinguish Won/Lost at this stage," so no recommendation is made.

📌 Details: ④ Threshold · k Anatomy

④ k — Slope (Velocity)

"How sharply should P(Win) react when crossing T?"

Previously used an empirical formula 1 + ln(ratio) based on evidence ratio (α+β), now switched to Grid Search-based optimization that directly maximizes separation. Upper bound is 12 per the theoretical reference.

📌 Details: ④ Threshold · k Anatomy

⑤ Dampening — Duplicate Signal Attenuation

"When three signals appear simultaneously in the same meeting, should they all receive equal weight?"

Compound Score = MAX(signals) + remaining × dampening. If dampening is 0, only the strongest signal counts; if 1, all signals are weighted equally. The current default of 0.25 is optimized via Grid Search.

⑥ Silence Penalty — Activity Gap Penalty

"How much penalty should accumulate when the customer hasn't been contacted for an extended period?"

Optimizes the penalty ratio added to β via Grid Search.

3. Five-Stage Data Maturity (Phase)

The Auto-Tuner prevents overfitting when data is scarce by assigning a 5-stage confidence level based on the lesser count of Won/Lost (min).

Why min?

If there are 100 Won projects but only 3 Lost, you cannot claim "this parameter distinguishes Lost well" based on just 3 cases. Statistical significance is always limited by the smaller sample.

What Changes by Phase

As the Phase increases, the Auto-Tuner's behavior progressively expands:

4. Core Metric: Separation

The Auto-Tuner's objective function is Separation.

• Separation > 0.40: Excellent (A) — Parameters closely reflect reality
• 0.25 – 0.40: Good (B) — Room for improvement
• 0.10 – 0.25: Needs Improvement (C)
• < 0.10: Urgent (D) — Parameter adjustment required

Limitations of Separation and AUC

Separation only measures the difference in means. It does not account for distribution overlap.

Example:

• Scenario A: Won avg 0.70, Lost avg 0.30 → Separation 0.40 → Excellent!
• Scenario B: Won range [0.20, 0.90], Lost range [0.10, 0.80] → Same average difference but heavy overlap

To compensate, ROC AUC is introduced. AUC represents "the probability that a randomly selected Won project has a higher P(Win) than a randomly selected Lost project." Overlap reduces AUC.

📌 Details: ⑤ Statistical Validation Anatomy

5. Simulation Engine

The core of the Auto-Tuner is memory-based simulation. Instead of using actual BayesianUpdate records stored in the database, it recalculates from scratch using raw data (activities, signals, Prior).

Why Recalculate?

To try different parameters, you need to calculate "what would P(Win) have been if Impact were 3.0?" This cannot be determined from stored historical results. Only by simulating from scratch with hypothetical parameters can you answer this.

One simulation cycle:
  α, β ← Prior initial values
  for each activity (chronological):
    → Calculate Compound Score from activity's signals
    → α += SWV × positive Compound
    → β += SWV × negative Compound
    → β += silence penalty (for activity gaps)
  P(Win) = α / (α + β)

Repeating this simulation for all Won/Lost projects reveals the separation for those parameters.

DB Queries = 0

During simulation, not a single DB query is executed. All data is preloaded into memory during initialization, and only pure computation follows. This is the implementation of Principle ⑤.

6. Document Series Guide