Auto-Tuner User Guide

The Auto-Tuner is a feature that learns and recommends optimal parameter values for the Bayesian engine based on result data from past Won/Lost projects. Administrators review these recommendations alongside data-driven evidence before deciding whether to apply them manually.

Location: Sidebar → Bayesian → Auto-Tune

⚠️ Auto-Tuner requires admin or super_user permissions.

1. Running an Analysis

Step 1: Navigating to the Auto-Tune Screen

Click the Bayesian → Auto-Tune menu in the sidebar. On your first visit, an introductory screen will be displayed in the center. If there was a previous analysis, it is automatically restored from sessionStorage.

Step 2: Starting the Analysis

Click the Start Analysis button located at the top or center of the screen.

After the analysis starts, a progress bar will display the real-time progress across 11 steps:

💡 You can cancel the analysis by clicking the Cancel button. Canceling will not affect your data.

⚠️ If MCMC is included, the full analysis takes about 15~40 seconds. Elapsed time will be displayed alongside the progress bar.

2. Result Screen Layout

Upon completion, the results are presented from top to bottom in the following sections:

① Summary Cards (4 Columns)

Four summary cards are displayed at the top.

Grading Criteria:

② Simulation Structure Panel

Below the summary cards is a collapsible 🔬 Auto-Tuner Simulation Structure panel. Clicking it reveals the overall structure and frequency of the simulations performed by Auto-Tuner.

Computation Distribution Bar: Visually depicts the computational ratio between Ruby and MCMC. Typically, MCMC constitutes over 99% of the total computations.

1. GRID SEARCH: Attempts 10 points within the current value ± range to maximize separation.
2. CROSS VALIDATION: Detects overfitting using 5-fold cross-validation.
3. MCMC Emcee: Ensemble sampling across 32 walkers × (500 warmup + 1,500 draws).

Where P=Projects, I=Impact Types, G=Grid points (10), D=Dampening attempts, S=Silence attempts, F=5 (folds), W=32 (walkers), Steps=2,000 (warmup+draws).

③ Signal Lift Analysis

Analyzes the Discrimination Power (Lift) of each signal.

• Lift > 1: Appears more frequently in Won projects → Positive indicator
• Lift < 1: Appears more frequently in Lost projects → Negative indicator
• ⚠ MISMATCH: A red warning is displayed if the current categorization (Positive/Negative) misaligns with the actual discrimination power.

If a signal has been assigned the wrong Impact classification, it will be detected here. If a mismatch is found, consider reviewing and reclassifying it in the Signal Master.

④ Prior α, β Recommendation

Recommends the optimal initial Priors based on historical data.

Phase 3 employs the Method of Moments, while Phase 4+ utilizes MLE (Newton-Raphson) fixed-point iteration for more precise estimation.

⑤ Impact Optimization

Results of the Grid Search recommendations for each Impact Type. Only Impacts displaying the adjust recommendation appear as cards.

Each card includes:

• Impact Type name
• Current value → Recommended value
• Separation improvement (+%p)
• Checkbox — Parameter selection to apply (Select All is possible)

💡 The search range varies by Phase: ±30% for Phase 3, ±40% for Phase 4, and ±50% for Phase 5.

⑥ Threshold (T) & Velocity (k)

Two tables are displayed side-by-side.

Rows with no recommended changes do not display a checkbox.

⑦ Impedance Impact

Simulates the effects on the Impedance function when applying the recommended T/k values.

If Impedance does not change due to T/k modification, a "No Changes ✅" message is displayed.

⑧ Dampening & Silence Penalty

Two cards are placed side-by-side.

Each card shows Current Value → Recommended Value + Improvement. If already optimal, it displays "✅ Optimal".

⚠️ Below Phase 4, the Dampening/Silence checkboxes are disabled, and a "🟢 Need Good or higher" notice appears.

3. MCMC Posterior Analysis

In Phase 3 or higher, the MCMC Posterior Estimation Results are displayed below the Grid Search results.

MCMC Header

Particle Storm Visualization

Clicking the ▶ Play Particle Storm button builds a real-time animated posterior distribution for each parameter:

• Density Histogram: The distribution shape emerges as MCMC samples accumulate sequentially.
• Solid Green Line: MCMC Estimated Mean.
• Dashed Red Line: Currently set value.
• Purple Box: 95% HDI (Highest Density Interval) span.

The Particle Storm visually aids understanding and doesn't represent the literal live MCMC processing. It reflects sequential trace data applied to histograms for each parameter.

MCMC Data Table

⚠️ A parameter with R̂ > 1.05 signals an incomplete convergence warning. If R̂ > 1.10, it is non-converged; do not apply it.

💡 Rows without checkboxes: If the difference between the MCMC mean and current value is less than 0.02 (ε threshold), the change is deemed insignificant and a checkbox is omitted. It displays a ≈ in the Δ column. For example, if current dampening is 0.25 and MCMC mean is 0.259 (difference 0.009 ≤ 0.02), it is excluded from application.

HDI Interpretation Guide

HDI [3.5, 6.2], Current 5.0
→ Current value is within HDI. Rational and acceptable; no change required.

HDI [2.0, 3.5], Current 5.0
→ Current value is outside HDI. Highly probable it's overvalued. Adjustment recommended.

HDI [0.8, 8.0] (Wide interval)
→ Lack of data causing uncertain estimation. Treat as reference only.

4. Applying Recommendations

Apply Bar

A fixed bar is located at the bottom of the result screen:

• Left: Number of selected parameters.
• Right: Apply Selected button.

Application Limits by Phase

Application Targets

⚠️ If you check both Grid Search and MCMC recommendations, the DB adopts the value selected last. Please select only one source per parameter.

Checks Before Applying

1. Check Grades: Assess the change between the current and projected grades.
2. Overfitting Risk: If Cross Validation issues warnings, deliberate carefully before applying.
3. MCMC R̂: If a parameter has R̂ > 1.05, only utilize MCMC recommendation as a reference.
4. HDI Spans: Exceptionally wide HDIs denote severe data deficiency.
5. Phase Restrictions: Dampening/Silence checkboxes only become active at Phase 4 or above.

After Applying

• A confirmation dialog appears, and upon approval, the DB updates.
• Following an update, sessionStorage resets, mandating a re-analysis.
• The quantity of successfully applied parameters will be displayed via a toast message.

5. Data Maturity (Phase) and Feature Limitations

Auto-Tuner grades reliability into 5 Phases based on the lesser count (min) between Won and Lost cases.

💡 Best way to upgrade Phases: Officially Close more projects as Won or Lost. Active (running) projects are not included in the analysis.

6. Frequently Asked Questions (FAQ)

Q. Can I run the analysis multiple times?

Yes. Auto-Tuner does not modify the DB at all during analysis. Analysis runs as an in-memory simulation, and zero changes are made to your data until you explicitly press "Apply Selected."

Q. What happens if I cancel midway?

Clicking the red Cancel button next to the progress bar instantly aborts the AJAX request. Your data remains perfectly intact.

Q. The analysis is taking a long time

If MCMC (Phase 3+) is triggered, it takes about 15~40 seconds because the Python MCMC engine (Emcee) conducts hundreds of thousands of simulations. Running just the Ruby Grid Search without MCMC (Phase 2) will finish within 1 second.

Q. What if Grid Search and MCMC recommend different values?

Q. There is a warning about elevated R̂

R̂ > 1.05 implies the MCMC chains have yet to fully converge. MCMC results for this parameter are sketchy at best. Default to the Grid Search recommendation instead, and retry MCMC once more actionable data is amassed.

Q. I can't see the MCMC section

MCMC won't run if:

• Currently below Phase 3 (min < 10 cases).
• The server lacks the Python Emcee environment framework.

In both iterations, Grid Search parameters will remain fully functional, meaning Auto-Tuner operates nominally without MCMC dependency.

Q. What is Particle Storm?

It's a real-time animated visualization mapping out MCMC sample data. It visually aids you in intuiting what the posterior distribution contours intuitively resemble. Hitting the Play button renders animations across parameter tiers sequentially.

Q. My previous analysis results are still showing

Auto-Tuner caches analysis states in sessionStorage. Your results remain preserved even if you navigate away and return. Applying parameters or hitting Re-run will naturally clear it out and restart fresh.

Related Documents

• Applying Bayesian: Prior Alpha & Beta
• Applying Bayesian: Parameter Calibration & Auto Optimization
• Auto-Tuner Theory ①: Engine Overview and Design Philosophy
• Auto-Tuner Theory ⑥: Dissecting MCMC Posteriors
• Auto-Tuner Technical Essay: Uncertainty Tamed via Simulation