Stats Explorer

Plan an A/B test: visitors needed per variant to detect the smallest lift that matters (mde) at a given power, and/or the smallest lift detectable with a given n_per_variant; days needed from daily_traffic. Conversion: baseline_rate. Continuous: baseline_sd (and baseline_mean for a relative mde). Use in an exploration (planning a future test); a running hypothesis test sizes itself in step 4.

Step 6 (or standalone in an exploration). For rare-event questions (fraud, defects, diagnoses, churn): what share of positive flags is real (PPV). Give prevalence, sensitivity, specificity (0–1), or file + condition_column + test_column (0/1 columns). If the question is not about rare events, use skip_step 6 instead.

Builds a report from results created in this session, only when the user asks for one. include = the result ids the user wants (default: the current analysis). The user confirms the sections. A hypothesis-test write-up (question, hypotheses, how it was tested, findings, verdict, base rates) appears only for completed hypothesis tests; explorations get no hypothesis section. format html (default) or markdown; save_to writes a file in the workspace.

Stops a background job started by a tool that answered STILL RUNNING.

Step 2. Checks shape (skew, kurtosis, small samples), group sizes, outliers, expected counts, ANCOVA slope equality and missing values for the test fixed in step 1. If an assumption fails, revising the test now (state_hypotheses again) is principled.

Step 3. Standard deviation, IQR, coefficient of variation, outliers and variance ratio for the data in the plan.

Recommends which test tool to run for a research question, with the assumptions to check. Call this first when unsure.

CI for a mean (column the user asked about) or a proportion. The user confirms the column first.

Strength and direction of the relationship between two numeric columns (r or ρ, CI, least-squares line), WITHOUT a p-value. To test whether a relationship is significant, use a hypothesis_test analysis with test "correlation". The user confirms the columns first.

Creates a chart for the columns the user asked to see: histogram (column), box_plot (column, optional group_column), scatter (x_column, y_column), bar (x_column categories, optional y_column mean), line (x_column, y_column), pie (x_column categories, optional y_column sum). The user confirms the chart type and columns first. Optional save_to (.svg). Charts are kept for build_report.

n, mean, SD, median, quartiles, range, skewness, kurtosis, shape and CI for the mean, for a column the user asked about (optionally by group). The user confirms the columns first.

For open analytics questions such as "what drives revenue?", "what is related to churn time?" or "which factors matter for score?": ranks every other column (or the ones given) by how strongly it relates to a numeric target column. Pearson r for numeric columns, correlation ratio η plus the highest/lowest group for categorical ones. Descriptive (no p-values); follow up on the top findings with group_summary, correlation, charts or a hypothesis test. Works on files of any size. Use it in an explore analysis. The user confirms the target and the columns first.

Analytics-style breakdown: count, sum, mean, median, min, max or SD of a numeric column for each category of another column (e.g. total revenue by region, average order value by segment, top 10 products by profit, rows per status), sorted, top N. Without value_column it counts rows per category. Descriptive only, no p-values. The user confirms the columns first.

Lists the columns of a local CSV/TSV/JSON file with their types, missing values, ranges and category values, plus a 5-row preview. Use before running tests on a file so you pass correct column and group names.

For a tool that answered STILL RUNNING: waits up to wait_seconds for the background job, then returns its result (the same result the tool would have returned) or its progress so far.

Only when a tool answered LARGE FILE, and only after asking the user: record their choice for that file. choice "full" reads every row (exact results, can take a long time, runs in the background); "sample" reads about 1% of the rows in blocks spread across the file (quick, approximate). save_columns keeps the columns that are read on disk so later questions on them are fast. Then call the original tool again.

Lists or deletes the columns kept on disk from large files (see large_file_plan save_columns). action "list" or "clear" (all, or one file).

In a hypothesis test (step 4): power of the planned test for the smallest effect the user cares about, using the data's sample size. smallest_effect is Cohen's d for means, r for correlations, or the true proportion p₁ for proportion tests. Ask the user for it; if they don't know, pass use_conventional: true. In an exploration: give two of effect_size, n_per_group, power to plan a two-group study.

Step 5. Runs exactly the test, columns and hypotheses the user confirmed in step 1 and returns the statistic, p-value, effect size, CI and verdict. Requires steps 1–4.

Only steps 4 (power) and 6 (base rates) can be skipped, with a short reason that will appear in the report.

Call this first for every new question. Read the user's request and choose mode: "hypothesis_test" when they want to know whether a difference, effect or relationship is real / significant / caused by something, or want to test a claim; "explore" when they want to describe, summarise, visualise, estimate or plan, without testing a claim. If the request is ambiguous, ask the user before calling. Returns the steps to follow.

Step 1 of a hypothesis test. Fix the test, the data selection and H₀/H₁ BEFORE looking at results. test_arguments are the same fields the test takes (e.g. t_test: {type, file, column, group_column, group1, group2, equal_variances, mu0}; anova: {file, column, group_column}; ab_test: {metric, file, variant_column, outcome_column, control, success_value, expected_split, segment_column}; chi_square: {type, file, row_column, col_column}; correlation: {file, x_column, y_column, method}). Only use columns the user asked about. The user confirms every column/group choice, the direction and α. Calling it again later revises the plan and restarts steps 2–6.

Shows the current analysis, which of the six steps are done, and the saved results (for the report).