---
title: "Check Automation Run Success"
description: "This document details the execution flow that calculates the compliance score for frameworks displayed on the dashboard, specifically focusing on how the success of automated evidence collection ru..."
last_updated: "2026-05-06T07:30:58.159818+00:00"
canonical_url: "https://www.doc0.dev/docs/49c89830-117b-4def-8edc-b5bcc50766e0/technical/how-it-works/check-automation-run-success"
---

<details>
<summary>Relevant source files</summary>

The following files were used as context for generating this wiki page:

- [apps/app/src/app/(app)/[orgId]/frameworks/page.tsx](https://github.com/blade47/comp/blob/main/apps/app/src/app/(app)/%5BorgId%5D/frameworks/page.tsx)
- [apps/app/src/app/(app)/[orgId]/frameworks/data/getFrameworkWithComplianceScores.ts](https://github.com/blade47/comp/blob/main/apps/app/src/app/(app)/%5BorgId%5D/frameworks/data/getFrameworkWithComplianceScores.ts)
- [apps/app/src/app/(app)/[orgId]/frameworks/lib/compute.ts](https://github.com/blade47/comp/blob/main/apps/app/src/app/(app)/%5BorgId%5D/frameworks/lib/compute.ts)
- [apps/app/src/app/(app)/[orgId]/frameworks/lib/taskEvidenceDocumentsScore.ts](https://github.com/blade47/comp/blob/main/apps/app/src/app/(app)/%5BorgId%5D/frameworks/lib/taskEvidenceDocumentsScore.ts)
</details>

This document details the execution flow that calculates the compliance score for frameworks displayed on the dashboard, specifically focusing on how the success of automated evidence collection runs is determined. The process begins when a user navigates to the dashboard page, triggering a series of data fetches and computations.

The primary goal of this flow is to assess the "strict completion" status of tasks associated with various compliance frameworks. A key part of this assessment involves verifying if any enabled automated evidence collection for a task has successfully completed its latest run. This ensures that compliance scores accurately reflect not only manual task completion but also the successful operation of integrated automation. The outcome directly impacts the compliance percentages shown to the user, providing an up-to-date view of their organization's adherence to various standards.

### Step-by-step Narrative

The execution flow begins with the rendering of the dashboard page and proceeds through several layers of data fetching and computation to determine task compliance.

<Steps>
<Step>
### DashboardPage Initialization
The `DashboardPage` component serves as the entry point for this flow. Upon loading, it asynchronously fetches all necessary data to populate the dashboard, including user session information, organization details, and various compliance-related scores. Crucially, it retrieves a comprehensive list of tasks, including their associated controls and any configured evidence automations. This raw data is then passed down to subsequent functions for processing.
</Step>
<Step>
### Fetching Framework Compliance Scores
The `getFrameworkWithComplianceScores` function is called by `DashboardPage` to process the fetched data. Its purpose is to take the raw framework instances and tasks, and enrich them with calculated compliance scores. It iterates through each `frameworkInstance` and delegates the core computation of compliance statistics to the `computeFrameworkStats` function. The function returns an array of frameworks, each augmented with its calculated compliance score.
Sources: [apps/app/src/app/(app)/[orgId]/frameworks/data/getFrameworkWithComplianceScores.ts:13-30](https://github.com/blade47/comp/blob/main/apps/app/src/app/(app)/%5BorgId%5D/frameworks/data/getFrameworkWithComplianceScores.ts#L13-L30)
</Step>
<Step>
### Computing Framework Statistics
The `computeFrameworkStats` function receives a single `frameworkInstance` and the full list of `tasks` relevant to the organization. It first identifies controls and policies pertinent to the given framework. It then filters the provided `tasks` to include only those associated with the current framework's controls. The function's critical role in this flow is to determine the number of "done tasks" by calling `countStrictlyCompletedTasks`, which directly contributes to the overall compliance score calculation.
Sources: [apps/app/src/app/(app)/[orgId]/frameworks/lib/compute.ts:16-51](https://github.com/blade47/comp/blob/main/apps/app/src/app/(app)/%5BorgId%5D/frameworks/lib/compute.ts#L16-L51)
</Step>
<Step>
### Counting Strictly Completed Tasks
The `countStrictlyCompletedTasks` function is responsible for iterating through a given array of tasks and determining how many of them meet the criteria for "strict completion." For each task, it calls `isTaskStrictlyComplete` to evaluate its status. The function then returns a count of all tasks that are deemed strictly complete.
Sources: [apps/app/src/app/(app)/[orgId]/frameworks/lib/taskEvidenceDocumentsScore.ts:101-103](https://github.com/blade47/comp/blob/main/apps/app/src/app/(app)/%5BorgId%5D/frameworks/lib/taskEvidenceDocumentsScore.ts#L101-L103)
</Step>
<Step>
### Determining Strict Task Completion
The `isTaskStrictlyComplete` function evaluates whether a single task is considered "strictly complete." It first checks if the task's `status` is either `'done'` or `'not_relevant'`. If this condition is met, it then proceeds to call `isTaskEvidenceComplete` to verify that all required evidence for the task is also complete. A task is only strictly complete if both its status indicates completion and its evidence requirements are satisfied.
Sources: [apps/app/src/app/(app)/[orgId]/frameworks/lib/taskEvidenceDocumentsScore.ts:96-99](https://github.com/blade47/comp/blob/main/apps/app/src/app/(app)/%5BorgId%5D/frameworks/lib/taskEvidenceDocumentsScore.ts#L96-L99)
</Step>
<Step>
### Checking Task Evidence Completion
The `isTaskEvidenceComplete` function focuses specifically on the evidence requirements for a given task. It filters the task's `evidenceAutomations` to identify only those that are currently `isEnabled`. If there are no enabled automations, the task is considered to have complete evidence by default. Otherwise, it iterates through each enabled automation and calls `isSuccessfulAutomationRun` on its latest run. The task's evidence is considered complete only if *all* enabled automations have a successful run.
Sources: [apps/app/src/app/(app)/[orgId]/frameworks/lib/taskEvidenceDocumentsScore.ts:86-94](https://github.com/blade47/comp/blob/main/apps/app/src/app/(app)/%5BorgId%5D/frameworks/lib/taskEvidenceDocumentsScore.ts#L86-L94)
</Step>
<Step>
### Verifying Successful Automation Run
The `isSuccessfulAutomationRun` function is the final step in this specific trace, directly evaluating the success of an individual evidence automation run. It takes an `EvidenceAutomationRunLite` object as input. The function returns `true` only if all three conditions are met: the run's `status` is `'completed'`, its `success` flag is `true`, and its `evaluationStatus` is not `'fail'`. If any of these conditions are not met, or if the run object itself is undefined, it returns `false`. This granular check ensures that only truly successful automation runs contribute to a task's evidence completion.
Sources: [apps/app/src/app/(app)/[orgId]/frameworks/lib/taskEvidenceDocumentsScore.ts:81-84](https://github.com/blade47/comp/blob/main/apps/app/src/app/(app)/%5BorgId%5D/frameworks/lib/taskEvidenceDocumentsScore.ts#L81-L84)
</Step>
</Steps>

### Sequence Diagram

```mermaid
sequenceDiagram
    participant P as apps/app/src/app/(app)/[orgId]/frameworks/page.tsx
    participant G as apps/app/src/app/(app)/[orgId]/frameworks/data/getFrameworkWithComplianceScores.ts
    participant C as apps/app/src/app/(app)/[orgId]/frameworks/lib/compute.ts
    participant T as apps/app/src/app/(app)/[orgId]/frameworks/lib/taskEvidenceDocumentsScore.ts

    P->>G: getFrameworkWithComplianceScores(frameworksWithControls, tasks)
    G->>C: computeFrameworkStats(frameworkInstance, tasks)
    C->>T: countStrictlyCompletedTasks(uniqueTasks)
    loop For each task
        T->>T: isTaskStrictlyComplete(task)
        alt Task status is 'done' or 'not_relevant'
            T->>T: isTaskEvidenceComplete(task)
            alt Enabled automations exist
                loop For each enabled automation
                    T->>T: isSuccessfulAutomationRun(automation.runs[0])
                    T-->>T: true/false (run success)
                end
                T-->>T: true/false (all automations successful)
            else No enabled automations
                T-->>T: true (evidence complete)
            end
            T-->>T: true/false (task strictly complete)
        else Task status not 'done' or 'not_relevant'
            T-->>T: false (task not strictly complete)
        end
    end
    T-->>C: count (strictly completed tasks)
    C-->>G: FrameworkStats (including doneTasks)
    G-->>P: FrameworkInstanceWithComplianceScore[]
```

### Flowchart



### Key Observations

*   **Cross-module Boundaries**: This flow demonstrates a clear separation of concerns across multiple modules. `page.tsx` handles initial data fetching and orchestration, `data/getFrameworkWithComplianceScores.ts` aggregates and prepares data for computation, and `lib/compute.ts` and `lib/taskEvidenceDocumentsScore.ts` contain the core business logic for calculating compliance and task completion. This modularity enhances maintainability and testability.
*   **Potential Failure Points and Handling**:
    *   **Authentication/Authorization**: `DashboardPage` explicitly checks for a valid session and redirects to `/login` if not present. It also verifies `onboardingCompleted` status, redirecting if onboarding is still pending.
    *   **Data Availability**: `getScores` and `getControlTasks` within `DashboardPage` handle cases where `organizationId` might be missing from the session, returning default empty values.
    *   **Automation Run Status**: `isSuccessfulAutomationRun` meticulously checks three conditions (`status`, `success`, `evaluationStatus`) to determine success, preventing partially or incorrectly completed automations from being counted as successful. If an automation run is `undefined`, it defaults to `false`.
    *   **No Enabled Automations**: `isTaskEvidenceComplete` gracefully handles tasks with no enabled evidence automations, considering their evidence complete by default, preventing false negatives in compliance scores.
*   **Performance Considerations**:
    *   **`cache` usage**: `getScores` and `getControlTasks` in `page.tsx` utilize `cache` from `react`, indicating that their results are memoized for the duration of the request, preventing redundant database calls within the same server-side render cycle.
    *   **Database Queries**: The initial data fetching in `DashboardPage` involves several database queries (e.g., `db.organization.findUnique`, `db.onboarding.findUnique`, `db.member.findFirst`, `db.task.findMany`, `db.frameworkEditorFramework.findMany`, `db.finding.findMany`). These are optimized with `select` and `include` clauses to fetch only necessary data.
    *   **Looping and Filtering**: The core compliance calculation involves iterating over frameworks, controls, and tasks. While efficient, for very large datasets (thousands of tasks/controls), the nested loops could become a performance bottleneck. The use of `Map` for deduplication in `computeFrameworkStats` is a good optimization to avoid redundant processing of tasks and policies.

## Sitemap

See the full [sitemap](https://www.doc0.dev/docs/49c89830-117b-4def-8edc-b5bcc50766e0/llms.txt) for all pages in this wiki.