ADR-030: Smart Error Messages and User Feedback¶

Status: Accepted Date: 2025-11-09
Last Updated: 2026-01-25
Deciders: Development Team
Related: ADR-028 (Tekton Task Strategy), ADR-029 (Platform Version Dependency Review)

Status¶

Accepted

Context¶

During Tekton integration testing, we discovered critical UX issues with error reporting:

Current Problems¶

Silent Failures: Pipeline/PipelineRun creation fails but operator logs "Build created successfully"
Misleading Status: Status shows "Running" when nothing is actually running
No Root Cause: Errors don't explain WHY something failed (e.g., RBAC permissions, missing Tasks)
No Actionable Guidance: Users don't know what to do to fix the problem
Hidden Context: Critical information buried in logs instead of surfaced in status

Real Example from Testing¶

2025-11-09T17:26:50Z    INFO    Build created successfully  
2025-11-09T17:26:50Z    INFO    Build status updated successfully   {"status": "Running", "message": "Build created and started"}

Reality: No Pipeline or PipelineRun exists. Build is not running.

User sees: "Build created and started" ✅ (false positive)

User needs to know: - Pipeline creation failed due to missing RBAC permissions for Tasks - Need to add tasks resource to ClusterRole - Command to fix: oc patch clusterrole ...

Philosophical Foundation (Sophia Framework)¶

From "The Pragmatic Coders" - methodological pragmatism requires:

Explicit Fallibilism: Acknowledge limitations and errors clearly
Systematic Verification: Provide ways to verify what actually happened
Pragmatic Success Criteria: Focus on what works and how to achieve it
Error Architecture Awareness: Distinguish between different types of errors

Decision¶

Implement Smart Error Messages with three levels of intelligence:

Level 1: Accurate Status Reporting ⭐ CRITICAL¶

Principle: Never report success when operation failed

Implementation:

// BEFORE (WRONG):
logger.Info("Build created successfully", "buildName", buildName)
return &BuildInfo{Status: BuildStatusRunning}, nil

// AFTER (CORRECT):
buildInfo, err := t.createPipelineAndRun(ctx, job, buildName)
if err != nil {
    logger.Error(err, "Failed to create Pipeline/PipelineRun", "buildName", buildName)
    return nil, fmt.Errorf("failed to create Tekton build: %w", err)
}
logger.Info("Build created successfully", "buildName", buildName, "pipelineRun", buildInfo.Name)
return buildInfo, nil

Verification: Check that resource actually exists before reporting success

Level 2: Root Cause Analysis ⭐ IMPORTANT¶

Principle: Explain WHY something failed, not just THAT it failed

Implementation:

// Detect common failure patterns and provide specific messages
func analyzeError(err error) string {
    switch {
    case strings.Contains(err.Error(), "forbidden"):
        return "RBAC permission denied. Check ClusterRole has required permissions."
    case strings.Contains(err.Error(), "not found"):
        return "Resource not found. May need to be created first."
    case strings.Contains(err.Error(), "already exists"):
        return "Resource already exists. Consider using a different name or deleting the existing resource."
    default:
        return err.Error()
    }
}

Error Categories: - RBAC Errors: Permission denied, forbidden - Resource Errors: Not found, already exists - Configuration Errors: Invalid spec, missing required fields - Platform Errors: API not available, CRD not installed - Dependency Errors: Missing prerequisite resources

Level 3: Actionable Guidance ⭐ GAME CHANGER¶

Principle: Tell users HOW to fix the problem

Implementation:

      } //

href="#__codelineno-3-1">type SmartError struct { Category string // "RBAC", "Resource", "Configuration", etc. Message string // Human-readable error message RootCause string // Technical root cause Impact string // What this means for the user Actions []string // Specific steps to fix References []string // Links to docs, ADRs, examples pan> Example: &SmartError{ Category: "RBAC", Message: "Failed to create Tekton Pipeline: permission denied", RootCause: "ClusterRole 'notebook-validator-manager-role' missing 'tasks' resource permission", Impact: "Tekton builds cannot run. Operator cannot copy Tasks to user namespace.", Actions: []string{ "Add 'tasks' to ClusterRole resources", "Run: oc patch clusterrole notebook-validator-manager-role --type='json' -p='[{\"op\": \"add\", \"path\": \"/rules/-\", \"value\": {\"apiGroups\": [\"tekton.dev\"], \"resources\": [\"tasks\"], \"verbs\": [\"create\", \"delete\", \"get\", \"list\", \"patch\", \"update\", \"watch\"]}}]'", "Restart operator: oc rollout restart deployment/notebook-validator-controller-manager -n jupyter-notebook-validator-operator", }, References: []string{ "ADR-028: Tekton Task Strategy", "config/rbac/role.yaml", }, }

Level 4: Status Conditions (Kubernetes Best Practice)¶

Principle: Use Kubernetes Conditions for structured status reporting

Implementation:

// Add to NotebookValidationJob status
type NotebookValidationJobStatus struct {
    // ... existing fields ...

    Conditions []metav1.Condition `json:"conditions,omitempty"`
}

// Set conditions
meta.SetStatusCondition(&job.Status.Conditions, metav1.Condition{
    Type:    "BuildReady",
    Status:  metav1.ConditionFalse,
    Reason:  "RBACPermissionDenied",
    Message: "Failed to create Tekton Pipeline: ClusterRole missing 'tasks' resource permission. See status.buildStatus.error for fix instructions.",
})

meta.SetStatusCondition(&job.Status.Conditions, metav1.Condition{
    Type:    "ValidationReady",
    Status:  metav1.ConditionFalse,
    Reason:  "WaitingForBuild",
    Message: "Waiting for build to complete before starting validation",
})

Implementation Phases¶

Phase 1: Fix Silent Failures (IMMEDIATE) ⏰ ✅ COMPLETE¶

Priority: CRITICAL Timeline: Current sprint Status: ✅ IMPLEMENTED (2025-11-09)

Tasks: 1. ✅ Add error checking after Pipeline/PipelineRun creation 2. ✅ Verify resources exist before reporting success 3. ✅ Return errors instead of logging and continuing 4. ✅ Update tests to expect errors

Files Updated: - ✅ pkg/build/tekton_strategy.go - Added verification after creation - ✅ internal/controller/notebookvalidationjob_controller.go - Added RBAC marker for tasks - ✅ pkg/build/tekton_strategy_test.go - Tests pass

Commits: - cf1a735 - feat: Implement ADR-030 Phase 1 - Fix Silent Failures in Tekton Build - cd385ae - fix: Add tasks RBAC marker to controller for ADR-028 - e4ee92e - fix: Use uppercase param names for OpenShift Pipelines git-clone Task

Lessons Learned: 1. Verification is not enough: We verified Pipeline exists, but didn't verify it has correct spec 2. Parameter naming conventions: OpenShift Pipelines uses UPPERCASE param names 3. User feedback gap: PipelineRun error messages don't surface in NotebookValidationJob status clearly

Phase 1.5: Improve NotebookValidationJob Status Messages ✅¶

Priority: HIGH Timeline: Current sprint Status: ✅ IMPLEMENTED (2026-01-25)

Problem Identified (2025-11-09): User question: "Should we have better error messages or status based on notebookvalidationjob-tekton-sample-build?"

Current PipelineRun error:

Status: Failed
Message: Failure - check logs for details.
Log snippet: Pipeline default/notebookvalidationjob-tekton-sample-pipeline can't be Run;
it contains Tasks that don't exist: Couldn't retrieve Task "git-clone":
clustertasks.tekton.dev "git-clone" not found

Issues: 1. NotebookValidationJob status doesn't show the real error 2. User has to check PipelineRun logs manually 3. Error message is cryptic ("clustertasks.tekton.dev not found") 4. No guidance on how to fix

Solution: 1. Surface PipelineRun errors in NotebookValidationJob status:

// In build_integration_helper.go
if buildInfo.Status == BuildStatusFailed {
    // Extract detailed error from PipelineRun
    pipelineRun := &tektonv1.PipelineRun{}
    if err := r.Get(ctx, client.ObjectKey{Name: buildInfo.Name, Namespace: job.Namespace}, pipelineRun); err == nil {
        // Parse PipelineRun conditions for detailed error
        for _, condition := range pipelineRun.Status.Conditions {
            if condition.Type == "Succeeded" && condition.Status == corev1.ConditionFalse {
                // Analyze error message and provide smart feedback
                smartError := analyzeP ipelineError(condition.Message)
                job.Status.BuildStatus.Message = smartError.UserFriendlyMessage
                job.Status.BuildStatus.Details = smartError.TechnicalDetails
                job.Status.BuildStatus.Actions = smartError.SuggestedActions
            }
        }
    }
}

Add error analysis for common Tekton errors:

func analyzePipelineError(message string) *SmartError {
    switch {
    case strings.Contains(message, "clustertasks.tekton.dev"):
        return &SmartError{
            Category: "Configuration",
            UserFriendlyMessage: "Pipeline configuration error: Using ClusterTask instead of Task",
            TechnicalDetails: message,
            SuggestedActions: []string{
                "This usually means the Pipeline was created with old code",
                "Delete the Pipeline and let operator recreate it: oc delete pipeline <name>",
                "Or update operator to latest version with ADR-028 fix",
            },
        }
    case strings.Contains(message, "missing values for these params"):
        return &SmartError{
            Category: "Configuration",
            UserFriendlyMessage: "Parameter mismatch between Pipeline and Task",
            TechnicalDetails: message,
            SuggestedActions: []string{
                "Check parameter names match between Pipeline and Task",
                "OpenShift Pipelines Tasks use UPPERCASE parameter names",
                "Update Pipeline to use correct parameter names",
            },
        }
    default:
        return &SmartError{
            Category: "Unknown",
            UserFriendlyMessage: "Build failed - see details",
            TechnicalDetails: message,
            SuggestedActions: []string{
                "Check PipelineRun logs: oc logs <pipelinerun-pod>",
                "Check operator logs for more details",
            },
        }
    }
}

Update NotebookValidationJob CRD status:

type BuildStatus struct {
    Status   string   `json:"status"`
    Message  string   `json:"message"`
    Details  string   `json:"details,omitempty"`   // NEW: Technical details
    Actions  []string `json:"actions,omitempty"`   // NEW: Suggested actions
    Attempts int      `json:"attempts"`
}

Expected Outcome:

status:
  buildStatus:
    status: Failed
    message: "Pipeline configuration error: Using ClusterTask instead of Task"
    details: "Pipeline default/notebookvalidationjob-tekton-sample-pipeline can't be Run; it contains Tasks that don't exist: Couldn't retrieve Task \"git-clone\": clustertasks.tekton.dev \"git-clone\" not found"
    actions:
      - "This usually means the Pipeline was created with old code"
      - "Delete the Pipeline and let operator recreate it: oc delete pipeline notebookvalidationjob-tekton-sample-pipeline"
      - "Or update operator to latest version with ADR-028 fix"
    attempts: 1

Benefits: ✅ Users see error in NotebookValidationJob status (no need to check PipelineRun) ✅ Error messages are user-friendly, not cryptic ✅ Actionable guidance tells users exactly how to fix ✅ Technical details available for debugging

Phase 2: Root Cause Analysis ✅¶

Priority: HIGH Timeline: Next sprint Status: ✅ IMPLEMENTED (2026-01-25)

Tasks: 1. ✅ Create pkg/errors/smart_error.go with SmartError type 2. ✅ Add error categorization logic 3. ✅ Update error analysis to status messages 4. ✅ Integration with condition helpers 5. ✅ Implement Phase 1.5 (surface PipelineRun errors in NotebookValidationJob status)

Files Implemented: - pkg/errors/smart_error.go - SmartError struct with categories, severity, actions, references - pkg/errors/smart_error_test.go - Comprehensive unit tests - internal/controller/condition_helper.go - Status condition management

Error Categories to Implement: - RBAC errors (forbidden, unauthorized) - Resource errors (not found, already exists) - Configuration errors (invalid spec, validation failed) - Platform errors (API unavailable, CRD missing) - Dependency errors (prerequisite missing) - Tekton errors (parameter mismatch, Task not found, ClusterTask vs Task)

Phase 3: Actionable Guidance ✅¶

Priority: MEDIUM
Timeline: Sprint +2 Status: ✅ IMPLEMENTED (2026-01-25)

Tasks: 1. ✅ Build knowledge base of common errors and fixes 2. ✅ Add action recommendations to SmartError 3. ✅ Include relevant ADR/doc references 4. ✅ Add CLI command suggestions

Files Implemented: - pkg/errors/knowledge_base.yaml - Error knowledge base with patterns, categories, actions, references

Knowledge Base Structure (Implemented):

errors:
  - code: RBAC_TASKS_FORBIDDEN
    pattern: "forbidden.*tasks|tasks.*forbidden"
    category: RBAC
    message: "Permission denied: Cannot access Tekton Tasks"
    root_cause: "ClusterRole missing 'tasks' resource permission for tekton.dev API group"
    impact: "Tekton builds cannot run. Operator cannot copy Tasks to user namespace."
    actions:
      - "Add 'tasks' to ClusterRole resources in config/rbac/role.yaml"
      - "Apply updated RBAC: kubectl apply -f config/rbac/role.yaml"
    references:
      - "ADR-028: Tekton Task Strategy"
      - "config/rbac/role.yaml"
    retryable: false

Phase 4: Status Conditions ✅¶

Priority: LOW
Timeline: Sprint +3 Status: ✅ IMPLEMENTED (2026-01-25)

Tasks: 1. ✅ Add Conditions field to CRD status (already existed in NotebookValidationJobStatus.Conditions) 2. ✅ Implement condition management (internal/controller/condition_helper.go) 3. ✅ Condition types: BuildReady, ValidationReady, Progressing, Available 4. ✅ Integration with SmartError for automated condition setting

Files Implemented: - internal/controller/condition_helper.go - SetCondition, SetConditionFromSmartError, SetConditionsForPhase - internal/controller/condition_helper_test.go - Unit tests

Consequences¶

Positive ✅¶

Better UX: Users know exactly what's wrong and how to fix it
Faster Debugging: Root cause immediately visible
Self-Service: Users can fix common issues without support
Reduced Support Load: Fewer "why isn't this working?" questions
Operational Excellence: Aligns with Kubernetes best practices
Methodological Pragmatism: Explicit about failures and how to succeed

Negative ⚠️¶

More Code: Error handling logic increases codebase size
Maintenance: Knowledge base needs to be kept up-to-date
Testing Complexity: More error scenarios to test
Localization: Error messages harder to translate

Risks 🔴¶

Over-Engineering: Could spend too much time on error messages
Mitigation: Implement incrementally, focus on common errors first
Stale Guidance: Fix instructions become outdated
Mitigation: Link to ADRs and docs instead of hardcoding
Information Overload: Too much detail confuses users
Mitigation: Tiered approach - summary + details on demand

Examples¶

Example 1: RBAC Permission Error¶

Before:

Build created successfully
Status: Running
Message: Build created and started

After:

Build failed: Permission denied
Category: RBAC
Root Cause: ClusterRole 'notebook-validator-manager-role' missing 'tasks' resource permission
Impact: Tekton builds cannot run. Operator cannot copy Tasks to user namespace.

Actions to fix:
1. Add 'tasks' to ClusterRole resources in config/rbac/role.yaml
2. Apply updated RBAC: oc apply -f config/rbac/role.yaml
3. Or patch directly: oc patch clusterrole notebook-validator-manager-role --type='json' -p='[...]'
4. Restart operator: oc rollout restart deployment/notebook-validator-controller-manager

References:
- ADR-028: Tekton Task Strategy
- config/rbac/role.yaml

Example 2: Missing Task¶

Before:

Pipeline can't be Run; it contains Tasks that don't exist: Couldn't retrieve Task "git-clone"

After:

Build failed: Task not found
Category: Resource
Root Cause: Task 'git-clone' not found in namespace 'default'
Impact: Pipeline cannot start. Task needs to be copied from openshift-pipelines namespace.

Actions to fix:
1. Operator should automatically copy Tasks (ADR-028)
2. If automatic copy failed, check operator logs for RBAC errors
3. Manual copy: oc get task git-clone -n openshift-pipelines -o yaml | oc apply -n default -f -

References:
- ADR-028: Tekton Task Strategy (namespace copy approach)
- Operator logs: oc logs -n jupyter-notebook-validator-operator deployment/notebook-validator-controller-manager

Compliance¶

ADR-029: Platform Version Dependency Review¶

Smart error messages should include: - Platform version compatibility information - Links to PLATFORM-COMPATIBILITY.md - Warnings about deprecated APIs - Upgrade path suggestions

ADR-028: Tekton Task Strategy¶

Error messages for Tekton builds should: - Explain Task copying process - Provide RBAC fix instructions - Reference platform detection logic - Link to base image documentation

Monitoring and Metrics¶

Track error patterns to improve guidance:

type ErrorMetrics struct {
    Category      string
    Count         int
    LastOccurred  time.Time
    FixAttempts   int
    FixSuccesses  int
}

Metrics to Track: - Error frequency by category - Time to resolution - Self-service fix success rate - Support ticket reduction

References¶

Kubernetes API Conventions - Status
Kubernetes Conditions
"The Pragmatic Coders" - Methodological Pragmatism Framework
ADR-028: Tekton Task Strategy
ADR-029: Platform Version Dependency Review Process

Decision Outcome¶

Chosen Option: Implement all 4 levels incrementally

Rationale: 1. Level 1 (Accurate Status) is CRITICAL - fixes immediate false positives 2. Level 2 (Root Cause) is HIGH priority - dramatically improves debugging 3. Level 3 (Actionable Guidance) is MEDIUM priority - enables self-service 4. Level 4 (Status Conditions) is LOW priority - nice-to-have for advanced users

Next Steps: ~~1. Implement Phase 1 (Fix Silent Failures) immediately~~ ✅ DONE ~~2. Create pkg/errors/smart_error.go package~~ ✅ DONE ~~3. Update tekton_strategy.go to use SmartError~~ ✅ Integrated via condition_helper.go ~~4. Add error knowledge base YAML file~~ ✅ DONE ~~5. Update documentation with common errors and fixes~~ ✅ DONE

Implementation Summary (2026-01-25)¶

All phases of ADR-030 have been implemented:

Files Created/Modified:¶

pkg/errors/smart_error.go - SmartError struct with:
Error categories (RBAC, Resource, Configuration, Platform, Dependency, Tekton, Build, Network, Authentication)
Severity levels (Critical, Error, Warning, Info)
Root cause analysis via AnalyzeError() function
Actionable guidance with Actions and References
UserFriendlyMessage() and DetailedMessage() methods
pkg/errors/smart_error_test.go - Comprehensive unit tests
pkg/errors/knowledge_base.yaml - Error knowledge base with:
Error patterns and categories
Root causes and impacts
Suggested actions
Documentation references
Condition types for status reporting
internal/controller/condition_helper.go - Status condition management:
Condition types: BuildReady, ValidationReady, Progressing, Available
SetCondition(), SetConditionFromSmartError(), SetConditionsForPhase()
SetBuildFailedFromPipelineRun() - Surfaces PipelineRun errors
SetValidationFailedFromPod() - Surfaces pod failure analysis
ConvertPodFailureToSmartError() - Bridges PodFailureAnalysis with SmartError
internal/controller/condition_helper_test.go - Unit tests

Integration with Existing Code:¶

internal/controller/pod_failure_analyzer.go - Existing detailed pod failure analysis (PodFailureAnalysis) is bridged to SmartError via ConvertPodFailureToSmartError()
Status.Conditions - Already existed in CRD, now properly managed via condition helpers