AB-Labz Runner

Self-hosted compute engine for processing experiments in your infrastructure.

What is Runner

AB-Labz Runner is a Docker container that runs in your infrastructure, processes experiment data locally, and sends only aggregated statistical results to AB-Labz cloud.

Architecture

┌─────────────────────────────────┐
│       AB-Labz Cloud             │
│  • Creates tasks                │
│  • Receives aggregated          │
│    statistics                   │
│  • Displays results in UI       │
└────────────┬────────────────────┘
             │
             │ HTTPS (results only)
             │ • p-value, CI, effects
             │ • Counters (no user_id)
             │ • SRM statistics
             ▼
┌─────────────────────────────────┐
│      AB-Labz Runner             │
│   (in your infrastructure)      │
│                                 │
│  • Polls tasks on schedule      │
│  • Reads local CSV files        │
│  • Statistical calculations     │
│  • Sends results                │
└────────────┬────────────────────┘
             │
             │ Read (read-only)
             │
             ▼
┌─────────────────────────────────┐
│   Your Data (CSV files)         │
│   /data/experiments/            │
│                                 │
│   • homepage_test.csv           │
│   • checkout_v2.csv             │
│   • onboarding_flow.csv         │
└─────────────────────────────────┘

What is Sent to Cloud

Sent:

• Statistics: p-value, z-score, t-statistic, confidence intervals
• Aggregated metrics: mean values, conversions by groups
• Counters: number of users in groups (no identifiers)
• SRM analysis: distribution by variants (no user_id)
• Metadata: metric names, number of days, dates

NOT Sent:

• Raw CSV files
• user_id identifiers
• Individual event records
• Email, phone, IP addresses
• Any PII (Personally Identifiable Information)

Quick Start

1. Get Runner Key

1. Log in to AB-Labz Workbench
2. Go to Runner section
3. Click "Create Key"
4. Copy the key (shown only once!)

2. Install Runner

You will receive archive runner-client-v1.x.x.zip with files:

runner-client-v1.x.x/
├── ab-labz-runner-v1.x.x.tar   # Docker image
├── docker-compose.yml           # Launch configuration
├── config.yaml                  # Runner settings
├── .env.example                 # Environment template
└── experiments/                 # Folder for CSV (empty)

3. Configure Environment

Unpack the archive runner-client-v1.x.x.zip. Copy the .env.example file and create .env from it. Edit the config.yaml file to match your settings.

Add your key to .env:

API_URL=https://workbench.ab-labz.com
RUNNER_KEY=abn_runner_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

4. Specify Data Path

Edit docker-compose.yml:

services:
  runner:
    volumes:
      # REPLACE with path to your CSV files!
      - /path/to/your/experiments:/data/experiments:ro

Important: use :ro (read-only) mode for security.

5. Load Image and Start

# Load image
docker load -i ab-labz-runner-v1.x.x.tar

# Start Runner
docker-compose up -d

In logs you should see:

AB-Labz Runner initialized
API URL: https://workbench.ab-labz.com
Experiments path: /data/experiments
Runner started successfully

6. Check Status

In AB-Labz interface (Runner section):

• Status: Online
• Last activity: just now
• Discovered files: list of your experiments

Configuration

config.yaml

Runner operation settings:

# Path to CSV folder (inside container)
experiments_path: /data/experiments

# Task polling interval (seconds)
poll_interval: 60

# File check interval (seconds)
file_check_interval: 300

When to change:

• Increase poll_interval to reduce load
• Decrease file_check_interval for more frequent updates

.env

Secret data:

# AB-Labz platform URL
API_URL=https://workbench.ab-labz.com

# Your Runner Key
RUNNER_KEY=abn_runner_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

File Monitoring

Runner automatically tracks file changes:

When changes detected:

• Sample size information is updated
• SRM metrics are recalculated

Update Process:

1. You update file experiment_001.csv (added new data for the day)
2. Runner detects file mtime change
3. AB-Labz cloud receives notification
4. New version is analyzed on next run

Check Interval: every 5 minutes (configurable in config.yaml)

Security

Recommendations:

1. Read-only Access

   volumes:
     - /data/experiments:/data/experiments:ro

2. Network Access Restriction

   • Runner should have HTTPS access only to workbench.ab-labz.com
   • Configure firewall for outbound connections

3. Regular Updates

   • Update Docker image when new versions are released
   • Monitor security notices

Management

Version Update

1. Download new archive runner-client-v1.x.x.zip
2. Load new image:

   docker load -i ab-labz-runner-v1.x.x.tar

3. Update docker-compose.yml (image version)
4. Restart:

   docker-compose down
   docker-compose up -d

Troubleshooting

Runner Not Connecting

Error: Failed to send initial heartbeat

Solution:

1. Check RUNNER_KEY in .env
2. Ensure key starts with abn_runner_
3. Check access to API_URL (ping, curl)
4. Check license status in AB-Labz

Files Not Detected

Error: Experiments path does not exist

Solution:

1. Check path in docker-compose.yml
2. Ensure folder exists on host
3. Check Docker access rights to folder
4. Path inside container: /data/experiments

Tasks Not Processing

Error: Experiment file not found

Solution:

1. File name = <experiment_id>.csv
2. Check CSV format (user_id, variant, date)
3. Ensure file is not empty
4. Check encoding (UTF-8)

System Requirements

Minimum:

• Docker 20.10+
• Docker Compose 1.29+
• 1 GB RAM
• 1 CPU core
• 5 GB disk space

Recommended:

• 2 GB RAM
• 2 CPU cores
• 10 GB disk space
• Linux host (for production)

Supported Platforms:

• Linux (AMD64, ARM64)
• macOS (Intel, Apple Silicon)
• Windows (with WSL2)