Forgejo Runner Operator - Test Workflows

This repository contains test workflows for the Forgejo Runner Operator, designed to validate operator functionality through end-to-end workflow execution.

Purpose

Primary: E2E testing for Forgejo Runner Operator

• Validate runner pool management
• Test job assignment and execution
• Verify rate limiting and concurrency
• Test failure handling and recovery
• Validate metrics and observability

Secondary: API structure exploration (historical)

• Original purpose: Inspect Forgejo Actions API job structure
• See api-test.yaml and original INSTRUCTIONS.md

Test Workflows

Core Test Workflows

Workflow	Purpose	Jobs	Duration	Labels
test-basic.yaml	Basic functionality	1	~15s	kubernetes
test-resource-small.yaml	Small resource pool	3 concurrent + 1 verify	~5s	kubernetes,small
test-resource-large.yaml	Large resource pool	1	~15s	kubernetes,large
test-concurrent.yaml	Concurrency handling	8 concurrent + 1 verify	~20s	kubernetes
test-failure.yaml	Failure scenarios	3 failures + 2 verify	varies	kubernetes
test-long-running.yaml	Long execution	1 main + 1 followup	5min	kubernetes

Workflow Details

test-basic.yaml

Validates:

• Basic runner assignment from pool
• Simple command execution
• File system operations
• Environment variables

Triggers: Push to main/master, manual dispatch

Expected Pool: RunnerPool with label kubernetes, size 1+

test-resource-small.yaml

Validates:

• Label-based pool selection (kubernetes,small)
• Concurrent job execution
• Pool runner availability under light load

Triggers: Manual dispatch, push to workflow file

Expected Pool: RunnerPool with labels kubernetes,small, size 3+

test-resource-large.yaml

Validates:

• Large resource pool assignment
• Simulated CPU/memory intensive operations
• Long-running job handling

Triggers: Manual dispatch, push to workflow file

Expected Pool: RunnerPool with labels kubernetes,large, size 1+

test-concurrent.yaml

Validates:

• Multiple concurrent job handling (8 jobs)
• Runner assignment under load
• Runner release and reuse
• Queue behavior when pool exhausted
• Rate limiting (if configured)

Triggers: Manual dispatch only

Expected Pool: RunnerPool with label kubernetes, size 5-10 (can be smaller to test queuing)

test-failure.yaml

Validates:

• Failed job handling (exit code, command not found, timeout)
• Runner release after failure
• Pool remains operational after failures
• Metrics track failures correctly

Triggers: Manual dispatch only

Expected Pool: RunnerPool with label kubernetes, size 3+

test-long-running.yaml

Validates:

• Long job execution (5 minutes)
• Rate limiter doesn't interfere with running jobs
• Runner stays assigned for duration
• TTL cleanup doesn't affect running jobs

Triggers: Manual dispatch only

Expected Pool: RunnerPool with label kubernetes, size 1+

Quick Start

Prerequisites

1. Forgejo Runner Operator deployed to Kubernetes cluster
2. RunnerPools created with appropriate labels:
   • kubernetes (basic pool)
   • kubernetes,small (small resource pool)
   • kubernetes,large (large resource pool)
3. Forgejo instance with Actions enabled
4. Admin API token for Forgejo

Setup

1. Clone or fork this repository to your Forgejo instance
2. Ensure RunnerPools are created and in Ready state
3. Trigger workflows via push or manual dispatch

Running Tests

Method 1: Manual Dispatch (Recommended)

Navigate to Actions tab in Forgejo UI:

1. Select workflow (e.g., "Basic Operator Test")
2. Click "Run workflow"
3. Choose branch (master)
4. Click "Run"

Method 2: Push Trigger

git commit --allow-empty -m "Trigger test workflow"
git push

Method 3: API Trigger (Automated)

export FORGEJO_TOKEN="your-admin-token"
./trigger-workflow.sh test-basic.yaml

Monitoring Execution

View in Forgejo UI:

• Navigate to Actions tab
• Click on workflow run
• View job logs in real-time

Monitor via API:

export FORGEJO_TOKEN="your-admin-token"
./monitor-jobs.sh

Check operator status:

kubectl get forgejorunnerjobs -A
kubectl describe forgejorunnerjob <job-name>

Validating Operator Behavior

Expected Operator Behavior

For each workflow:

1. Job Creation:

   • External controller creates ForgejoRunnerJob CR for each job
   • CR contains jobID and label from workflow

2. Runner Assignment:

   • Operator finds matching RunnerPool by label
   • Assigns available runner from pool
   • Updates pool status (availableRunners--, inUseRunners++)

3. Job Execution:

   • Operator creates Kubernetes Job with runner credentials
   • Job executes workflow steps
   • Status tracked in ForgejoRunnerJob CR

4. Completion:

   • Job succeeds or fails
   • Runner released back to pool
   • Metrics recorded

Validation Checklist

After running workflows:

1. Check ForgejoRunnerJob CRs:

kubectl get forgejorunnerjobs -A
kubectl describe forgejorunnerjob <job-name>

Expected status progression:

• Pending → WaitingForRunner → CreatingJob → Running → Succeeded/Failed

2. Check RunnerPool status:

kubectl get runnerpools -A
kubectl describe runnerpool <pool-name>

Verify:

• availableRunners + inUseRunners = totalRunners
• Runners released after job completion

3. Check Kubernetes Jobs:

kubectl get jobs -l forgejo-job-id=<job-id>
kubectl logs <job-pod-name>

4. Check operator logs:

kubectl logs -n forgejo-runner-operator-system \
  deployment/forgejo-runner-operator-controller-manager -f

Look for:

• Runner assignment messages
• Rate limit hits (if applicable)
• Error handling

5. Check Prometheus metrics:

kubectl port-forward -n forgejo-runner-operator-system \
  deployment/forgejo-runner-operator-controller-manager 8080:8080

curl http://localhost:8080/metrics | grep forgejo_runner

Verify metrics:

• forgejo_runner_pool_size{state="available"} - decreases/increases correctly
• forgejo_runner_jobs_total{status="succeeded"} - increments on success
• forgejo_runner_jobs_total{status="failed"} - increments on failure
• forgejo_runner_jobs_current - tracks active jobs

Helper Scripts

trigger-workflow.sh

Trigger specific workflow via API:

./trigger-workflow.sh test-basic.yaml
./trigger-workflow.sh test-concurrent.yaml

monitor-jobs.sh

Continuously monitor job execution:

./monitor-jobs.sh

inspect-api.sh (original)

Inspect Forgejo API job structure:

./inspect-api.sh

validate-e2e.sh

Run complete E2E validation:

./validate-e2e.sh

Troubleshooting

Jobs Stuck in Pending

Cause: No matching RunnerPool or no available runners

Solution:

# Check pools
kubectl get runnerpools -A

# Create pool if missing
kubectl apply -f k8s/runnerpools/pool-basic.yaml

# Check pool capacity
kubectl describe runnerpool <pool-name>

Jobs Failing Immediately

Cause: Runner credentials invalid or Forgejo unreachable

Solution:

# Check runner secrets
kubectl get secrets -l app.kubernetes.io/created-by=forgejo-runner-operator

# Check operator logs
kubectl logs -n forgejo-runner-operator-system deployment/... --tail=100

Rate Limiting

Symptom: Jobs delayed despite available runners

Cause: Rate limiter protecting control plane

Solution: This is expected behavior. Adjust rate limits in operator if needed.

Pool Not Ready

Cause: Runner registration failed

Solution:

# Check pool status
kubectl describe runnerpool <pool-name>

# Check registration jobs
kubectl get jobs -l app.kubernetes.io/component=runner-registration

# View registration logs
kubectl logs job/<pool-name>-runner-0

Integration with Operator E2E Tests

These workflows can be used by the operator's E2E test suite (/app/test/e2e/):

// Trigger workflow via Forgejo API
helper.TriggerWorkflow(ctx, "mngrif/test-workflow", "test-basic.yaml")

// Monitor job creation
jobs := helper.WaitForJobs(ctx, expectedCount)

// Validate operator behavior
Expect(job.Status.Phase).To(Equal("Succeeded"))

See OPERATOR_TESTING.md for detailed integration guide.

Sample RunnerPool Configurations

See k8s/runnerpools/ directory for example RunnerPool YAMLs:

• pool-basic.yaml - Basic pool for standard tests
• pool-small.yaml - Small resource pool
• pool-large.yaml - Large resource pool
• pool-concurrent.yaml - Large pool for concurrency tests

Contributing

When adding new test workflows:

1. Follow existing naming convention: test-<purpose>.yaml
2. Include clear validation messages in workflow output
3. Document expected operator behavior in comments
4. Add entry to this README
5. Test with actual operator deployment before committing

References

• Forgejo Runner Operator: https://git.cleverlibre.org/mngrif/forgejo-runner-operator
• Operator User Guide: See operator repo docs/USER_GUIDE.md
• Operator Operations Guide: See operator repo docs/OPERATIONS.md
• Forgejo Actions: https://forgejo.org/docs/latest/user/actions/

License

Same as Forgejo Runner Operator: Apache License 2.0

Copyright 2026.