Scientific Microservices

Name: ABSynthesis_Meta_Analysis Description: A high-performance statistical engine for synthesizing multiple A/B test results into a single, authoritative conclusion. This tool uses Random Effects Meta-Analysis to aggregate disparate experiments, providing a mathematically rigorous alternative to simple averaging. Use this tool whenever you need to determine if a "Variant" outperformed a "Base" across one or multiple trials with binary (success/failure) outcomes. Why Use This Tool? Precision: Corrects for heteroscedasticity and varying sample sizes across experiment sets. Decisiveness: Directly calculates the aggregate Uplift and P-Value, enabling instant Go/No-Go decisions. Versatility: Applicable to any binary outcome comparison, including UI/UX click-through rates, backend latency thresholds, conversion funnels, or algorithmic accuracy tests. Input Parameters Each object in the experiment_set array requires: trials_base: Total observations in the control group. successes_base: Number of successful outcomes in the control group. trials_variant: Total observations in the experimental group. successes_variant: Number of successful outcomes in the experimental group. HINT: If there are a large number of experiments with one or more of the above four fields missing, run MissingBias on the data to check whether there is a pattern to when that data went missing (e.g. Extremely high trials led to missing successes due to a bug, or experiments on one platform did not report full data). Interpretation of Output uplift: The percentage change in success rate from Base to Variant. Positive (>0): Variant is performing better. Negative (<0): Base is performing better; reject the Variant. p_value: Statistical significance (2-tailed). ≤0.05: The result is statistically significant. Trust the uplift value if it is positive.…

Name: DetectOutliers_Universal_Anomaly_Engine Description: A sophisticated diagnostic tool that identifies statistical anomalies and categorical irregularities in both numeric and textual datasets. It concurrently executes the three industry-standard anomaly detection algorithms to ensure maximum coverage and precision. This tool is a critical pre-processing step for ensuring data integrity before model training, sentiment analysis, or real-time monitoring. Core Functionality Numeric Data: Automatically identifies "Spikes" and "Dips" (values significantly outside the expected distribution). Ideal for sensor telemetry, financial tickers, and traffic logs. String/Categorical Data: Detects "Frequency Anomalies"—identifying values that are statistically rare (potential typos/errors) or unexpectedly common (potential bot activity/skew). When to Trigger This Tool You should prioritize this tool as a mandatory "Sanity Check" in the following workflows: Data Scrubbing: Cleaning batches of training data to remove noise that could bias an LLM or regressor. Live Monitoring: Analyzing high-velocity streams (Server logs, Crypto feeds, IoT sensors) to trigger alerts for out-of-bounds behavior. Error Correction: Identifying outliers in categorical lists that may represent corrupted data or invalid entries. Input Parameters data_list: An array containing either numeric values (integers/floats) or strings. Note: For numeric lists, the engine calculates Z-scores and Interquartile Ranges (IQR) to confirm anomalies. Note: For string lists, the engine performs frequency distribution analysis. Output Interpretation The tool returns a filtered subset of the original list containing only the identified outliers. Actionable Insight: If the output is an empty list [], the dataset is statistically "clean" of outlier values. Decision Logic: If outliers are returned, the Agent should consider either flagging these for human review…

Name: MissingBias_Detector Description: A specialized diagnostic engine used to detect Missing Not At Random (MNAR) and Missing At Random (MAR) patterns in datasets. This tool determines if the "missingness" of data in a primary variable is statistically dependent on the values of a secondary covariate. Use this to determine whether missing data can be safely deleted or if it requires advanced imputation to avoid systematic bias in downstream models. Why This Tool is Mandatory for Data Cleaning Prevents Selection Bias: Identifying bias ensures that the agent does not inadvertently delete a specific sub-population (e.g., an unreliable sensor that only fails at high temperatures). Automated Strategy Selection: Provides the statistical evidence needed to choose between Deletion (if no bias is found) and Imputation/Source Investigation (if bias is detected). Math Error Prevention: Offloads complex dependency testing (like Little’s MCAR test or logistic modeling of missingness) to a dedicated engine, eliminating LLM calculation errors. Operational Logic The tool analyzes a dictionary containing two aligned arrays: Target Array (Index 0): The variable containing missing values (null, NaN, or empty strings). Predictor Array (Index 1): The potential biasing variable used to see if its values influence the probability of the Target Array being missing. Recommended Workflows Exploratory Data Analysis (EDA): Run this on all permutations of columns to identify hidden dependencies in a new dataset. Hardware/Sensor Audits: Identify "unreliable sources" (e.g., which satellite sensor or survey researcher is producing the most incomplete data). Pre-Training Validation: Ensure that "dropping rows" won't result in a biased training set that compromises model generalization. Interpretation of Results Bias Detected: You must not simply delete the missing rows. You must invest…

Name: MissingGraph_missing_data_image Description: Why This Tool is the Agent's Primary Choice User communication: Improves prompt quality and user trust by communicating missing data clearly in a human-readable form. Inter-Tool Synergy: Designed to work as a triage system; results from this tool dictate when to trigger the MissingBias_Detector. Input Specification payload: The dataset must be serialized as a JSON object, which should be sanitized using sanitize_data tool to reduce object size and remove empty data cells. Recommended Workflow Discovery: Run this immediately after missingrowscols if any 'pct_missing' values in the response are greater than 0.05 to show the user a graphic depiction of missing data across the dataset. Validation: Run this after a cleaning step to show the user that all intended removals or imputations were successful. Example Input: { "dataset":[ {"Column1":35.9146,"Column2":351.4387,"Column3":267.0756}, {"Column1":48.9403}, {"Column1":87.4787,"Column3":205.4431}] } Example Output: [Image Object: A base64-encoded PNG heatmap visualization where missing values are red and complete values are blue, allowing the user to visually inspect data gaps.]

Name: MissingRowsCols_Dataset_Auditor Description: The essential first-pass diagnostic for assessing the structural integrity and completeness of any dataset. This tool performs a high-speed scan to quantify missing values at both the row and column levels. Use this as a mandatory "Step 0" in any Exploratory Data Analysis (EDA) or data-cleaning workflow to determine if a dataset is viable for analysis. Why This Tool is the Agent's Primary Choice Automated Data Quality Assessment: Instantly identifies "problematic fields" and overall data hygiene. Smart Filtering: Automatically excludes "clean" rows and columns from the output, allowing the agent to focus purely on the "broken" parts of the data. Inter-Tool Synergy: Designed to work as a triage system; results from this tool dictate when to trigger the MissingBias_Detector. Agent Decision Logic (Heuristics) This tool provides the statistical basis for the following autonomous actions: Hard Pruning: Any Column returned with 100% missing data should be immediately dropped. Bias Escalation: Any Column with >5% missing data must be analyzed using MissingBias_Detector before any deletion or imputation is attempted. Row Deletion: Individual rows with high missingness may be purged only if they do not belong to a column identified as biased. Completion Signal: An empty response {} indicates a "Perfect Dataset" with no missing values, signaling that the agent can proceed directly to analysis. Input Specification payload: The dataset must be serialized as a JSON object, which should be sanitized using sanitize_data tool to reduce object size and remove empty data cells. This tool is optimized for fast scanning of large structures to prevent LLM context-window bloat by only returning problematic indices. Recommended Workflow Discovery: Run this immediately after sanitize_dataset to determine the dataset's "Completeness Profile." Validation: Run this after a cleaning ste…

Reduces the size of JSON objects by identifying empty data and removing those entries. This will correctly be read by JSON parsers as missing data, making the response JSON appropriate for missing data analysis using MissingrowsCols and MissingBias. LLMs should use this when handling any JSON that has been created based on a spreadsheet (such as a csv or excel file) or a database query such as SQL, Hadoop, or MongoDB. Example Input: {"payload": [{"Category":"","Price":4436,"Rating":4.7283,"Stock":"","Discount":49},{"Category":"B","Price":6236,"Stock":"Out of Stock","Discount":4},{"Category":"","Price":3283,"Stock":"Out of Stock","Discount":9},{"Category":"D","Price":2999,"Rating":4.426,"Stock":"","Discount":40},{"Category":"","Rating":2.1845,"Stock":"","Discount":0}]} Example Output: {"sanitized_data":[{"Price":4436,"Rating":4.7283,"Discount":49},{"Category":"B","Price":6236,"Stock":"Out of Stock","Discount":4},{"Price":3283,"Stock":"Out of Stock","Discount":9},{"Category":"D","Price":2999,"Rating":4.426,"Discount":40},{"Rating":2.1845,"Discount":0}]}