Test Plan Generation

Planning algorithms combine change impact, risk signals, and policy priorities.

AI-Powered Test Planning

Plan generation starts with dependency graph analysis and historical defect patterns. The AI analyzes code changes, architecture context, and execution history to create optimized test plans.

Planning Algorithm

The Test Planning Agent follows a multi-stage process:

Change Analysis: Parse git diff to identify modified files, functions, and dependencies
Impact Assessment: Build dependency graph to find all affected components
Risk Scoring: Score each change based on historical failure rates and criticality
Coverage Selection: Select tests that cover affected code paths
Prioritization: Order tests by risk, impact, and execution cost
Optimization: Remove redundant tests and add missing coverage

Input Sources

The planner consumes multiple data sources to make informed decisions:

Code Changes

bash

# Git diff analysis
files:
  - path: "src/checkout/payment.ts"
    changes:
      - function: "processPayment"
        linesAdded: 12
        linesRemoved: 5
    impact: "high"

Architecture Knowledge

Service topology, API contracts, database schemas, and integration points from the Knowledge Layer.

Historical Data

bash

# Failure history for affected files
"src/checkout/payment.ts":
  failureRate: 0.08
  avgFixTime: "4.2 hours"
  lastFailure: "2026-03-10"
  criticalPath: true

Policy Constraints

Organization-defined policies for minimum coverage, required test types, and approval gates.

Test Plan Output

The planner outputs prioritized test suites for UI, API, and data workflows:

bash

testPlan:
  id: "plan_abc123"
  name: "Release v2.3.0"
  priority: "high"
  estimatedDuration: "12 minutes"
  coverage:
    expected: 87
    critical: 100
  
  tests:
    # High priority UI tests
    - id: "ui_checkout_flow"
      name: "Complete checkout with credit card"
      type: "ui"
      priority: "high"
      risk: "high"
      estimatedDuration: "45s"
      reason: "Covers modified payment.ts"
    
    # Medium priority API tests
    - id: "api_payment_contract"
      name: "Payment API contract validation"
      type: "api"
      priority: "medium"
      risk: "medium"
      estimatedDuration: "10s"
      reason: "Validates payment endpoint schema"
    
    # Low priority data tests
    - id: "data_order_persistence"
      name: "Order data persistence check"
      type: "data"
      priority: "low"
      risk: "low"
      estimatedDuration: "5s"
      reason: "Ensures order records are stored"

Coverage Goals

Plans target different coverage levels based on context:

Critical Paths: 100% coverage for payment, auth, and data integrity
High Impact Changes: 90% coverage with focus on integration tests
Medium Impact: 70% coverage with prioritized unit and API tests
Low Impact: 50% coverage with smoke tests only

Dynamic Adaptation

Plans adapt based on real-time execution feedback:

If early tests fail, planner adds related regression tests
If coverage is insufficient, planner generates additional tests
If execution time exceeds budget, planner removes low-priority tests
If flaky tests detected, planner marks for self-healing

Customization

Organizations can customize planning behavior:

bash

planningPolicy:
  # Minimum coverage requirements
  minimumCoverage: 80
  criticalPathCoverage: 100
  
  # Test selection strategy
  strategy: "risk-based"  # or "comprehensive", "fast"
  
  # Time budget constraints
  maxExecutionTime: "15 minutes"
  
  # Required test types
  requiredTypes:
    - "ui"
    - "api"
    - "integration"
  
  # Failure tolerance
  allowedFailureRate: 0.05

Integration with CI/CD

Plans can be triggered automatically from CI pipelines:

bash

# GitHub Actions example
- name: Generate Test Plan
  uses: ai-test-harness/plan-action@v1
  with:
    api-key: ${{ secrets.AI_TEST_HARNESS_API_KEY }}
    change-set: auto  # Automatically detect from PR diff
    environment: staging

- name: Execute Tests
  uses: ai-test-harness/execute-action@v1
  with:
    plan-id: ${{ steps.plan.outputs.id }}
    wait: true