A/B Testing

Overview

Dub provides built-in A/B testing capabilities that allow links to split incoming traffic across multiple destination URLs based on percentage weights. When configured, edge routing inspects user cookies or performs a weighted random allocation to select a variant URL, ensuring sticky routing for returning visitors during active testing windows. Automated test evaluation and expiration are coordinated via QStash cron jobs that trigger when a test concludes. Upon test completion, analytics data from Tinybird is queried to evaluate conversion metrics, automatically select the winning destination URL, rewrite the link, and dispatch workspace webhooks.

Sources: apps/web/lib/middleware/link.ts:163-174, apps/web/lib/middleware/utils/resolve-ab-test-url.ts:8-58, apps/web/lib/api/links/ab-test-scheduler.ts:6-23, apps/web/app/ee/api/cron/links/linkId/complete-tests/route.ts:8-45, apps/web/lib/api/links/complete-ab-tests.ts:12-89

A/B Testing Variant Schema

Validation Constraints and Thresholds

Variant configurations undergo verification before any weight calculations occur. A link must supply a valid testVariants collection and an active testCompletedAt expiration timestamp that remains in the future (new Date(testCompletedAt) > new Date()). If either condition fails, variant resolution immediately returns null.

Sources: apps/web/lib/middleware/utils/resolve-ab-test-url.ts:16-27

Percentage Weight and Selection Rules

Once structural checks pass, the selection engine checks for an existing sticky routing cookie named dub_test_url. If present and its value matches one of the candidate variant URLs in testVariants.map((t) => t.url), that exact URL is returned without executing random selection.

When no valid tracking cookie exists, the system computes percentage weights dynamically from the variant array. The first variant's percentage establishes the initial weight, and subsequent entries accumulate weight additively through a cumulative summation loop. A random floating-point value is generated between 0 and the final cumulative weight using Math.random() * weights[weights.length - 1]. The resolver iterates through the cumulative weights array, selecting the first variant index where the cumulative weight exceeds the generated random value.

Sources: apps/web/lib/middleware/utils/resolve-ab-test-url.ts:16-58

Edge Variant Selection

Cookie Inspection and Destination Resolution

When link processing resolves a short URL in LinkMiddleware, it invokes resolveABTestURL using the cached link's testVariants and testCompletedAt properties. If a test is active, the resolver checks for a sticky session cookie named dub_test_url via Next.js headers cookies API. If the cookie exists and its URL value matches one of the candidate variant URLs, that exact destination is returned to maintain user session consistency. When no cookie is present, weighted random allocation computes a variant, and the selected URL is stored in cookie data parameters alongside tracking cookies.

Sources: apps/web/lib/middleware/link.ts:168-173, apps/web/lib/middleware/link.ts:213-218, apps/web/lib/middleware/utils/resolve-ab-test-url.ts:8-36

Middleware Integration

Copyconst testUrl = await resolveABTestURL({
  testVariants,
  testCompletedAt,
});
 
const url = testUrl || cachedLink.url;
 

Sources: apps/web/lib/middleware/link.ts:168-174

Sources: apps/web/lib/middleware/link.ts:173, apps/web/lib/middleware/link.ts:217

Automated Test Scheduling

QStash Job Registration

Copyexport async function scheduleABTestCompletion(
  link: Pick<ExpandedLink, "id" | "testVariants" | "testCompletedAt">,
) {
  if (!link.testVariants || !link.testCompletedAt) {
    return;
  }
 
  const url = `${APP_DOMAIN_WITH_NGROK}/api/cron/links/${link.id}/complete-tests`;
  const testCompletedAt = new Date(link.testCompletedAt);
 
  // Tests are not complete yet, schedule a job for completion
  if (testCompletedAt > new Date()) {
    await qstash.publishJSON({
      url,
      delay: (testCompletedAt.getTime() - new Date().getTime()) / 1000,
      deduplicationId: link.id,
    });
  }
}

Sources: apps/web/lib/api/links/ab-test-scheduler.ts:6-24

The scheduleABTestCompletion function accepts a link object containing id, testVariants, and testCompletedAt. It verifies that both testVariants and testCompletedAt are defined on the link before proceeding; otherwise, it returns immediately without scheduling.

+------------------------------------+
| scheduleABTestCompletion(link)     |
+------------------------------------+
                  |
                  v
       { testVariants & testCompletedAt present? }
                  |
         +--------+--------+
         | Yes             | No
         v                 v
 { testCompletedAt > now? }  Return
         |
    +----+----+
    | Yes     | No
    v         v
 Publish JSON   Return
 via QStash

Sources: apps/web/lib/api/links/ab-test-scheduler.ts:6-17

Delay and Deduplication Configuration

When the expiration date stored in link.testCompletedAt is in the future compared to the current system time, the function constructs a target webhook URL using APP_DOMAIN_WITH_NGROK pointing to /api/cron/links/${link.id}/complete-tests. It invokes qstash.publishJSON with three parameters:

• url: The constructed cron endpoint destination URL.
• delay: The difference in seconds between testCompletedAt and the current timestamp, computed as (testCompletedAt.getTime() - new Date().getTime()) / 1000.
• deduplicationId: Set directly to link.id to prevent duplicate schedule entries for the same link test window.

Sources: apps/web/lib/api/links/ab-test-scheduler.ts:13-23

Test Completion Trigger Endpoint

Request Signature Verification and Link Lookup

The API route handler exposes a POST method at /api/cron/links/[linkId]/complete-tests that receives callbacks scheduled by QStash when an A/B test expiration window is reached. Upon receiving incoming request payloads, the endpoint reads the raw request text and validates authenticity by calling verifyQstashSignature. If verification fails, an error response is returned through handleAndReturnErrorResponse. After successful signature checks, the handler queries the database using Prisma to locate the link record matching the linkId dynamic route parameter. If no matching link record is found, the endpoint exits early and returns a plain text response indicating the link was not found and skipping further execution.

Copytry {
  const rawBody = await req.text();
  await verifyQstashSignature({ req, rawBody });
 
  const link = await prisma.link.findUnique({
    where: {
      id: linkId,
    },
  });
 
  if (!link) {
    return new Response(`Link ${linkId} not found. Skipping...`);
  }
 

Sources: apps/web/app/ee/api/cron/links/linkId/complete-tests/route.ts:18-31

Test Window Validation and Completion Dispatch

Before triggering the test resolution flow, the endpoint evaluates three conditions to determine if the test is eligible for finalization. It confirms that link.testVariants and link.testCompletedAt are populated, verifies that the completion timestamp is in the past (link.testCompletedAt < new Date()), and ensures that the execution falls within a 15-minute window from the target completion time (Date.now() - link.testCompletedAt.getTime() < 15 * 60 * 1000). When all criteria are satisfied, the handler invokes completeABTests(link) and returns a confirmation string. Otherwise, it responds indicating no test completion is necessary.

Copy// only complete tests if:
// - there are test variants
// - the tests completion time is in the past
// - the tests completion time is within the last 15 minutes
if (
  link.testVariants &&
  link.testCompletedAt &&
  link.testCompletedAt < new Date() &&
  Date.now() - link.testCompletedAt.getTime() < 15 * 60 * 1000 // Limit to a 15-minute window
) {
  await completeABTests(link);
 
  return new Response(`Tests completed for link ${linkId}.`);
}
 
return new Response(`No test completion necessary for link ${linkId}.`);

Sources: apps/web/app/ee/api/cron/links/linkId/complete-tests/route.ts:32-47

Sources: apps/web/app/ee/api/cron/links/linkId/complete-tests/route.ts:35-41

Analytics Evaluation and Winner Selection

Tinybird Analytics Querying and Max Lead Calculation

The completeABTests function first verifies that link.testVariants, link.testCompletedAt, and link.projectId are all present; otherwise, it exits early. It extracts the variant configurations from link.testVariants and queries Tinybird analytics by invoking getAnalytics with the leads event, grouping by top_base_urls, and constraining the date range between link.testStartedAt (if available) and link.testCompletedAt for the given workspace and link ID.

Copyexport async function completeABTests(link: Link) {
  if (!link.testVariants || !link.testCompletedAt || !link.projectId) {
    return;
  }
 
  const testVariants = ABTestVariantsSchema.parse(link.testVariants);
 
  const analytics: { url: string; leads: number }[] = await getAnalytics({
    event: "leads",
    groupBy: "top_base_urls",
    linkId: link.id,
    workspaceId: link.projectId,
    start: link.testStartedAt ? new Date(link.testStartedAt) : undefined,
    end: link.testCompletedAt,
  });

Sources: apps/web/lib/api/links/complete-ab-tests.ts:12-26

The function calculates the maximum lead count obtained across all test variants by mapping each variant to its corresponding lead count from the analytics dataset and finding the peak value via Math.max. If the maximum lead count equals zero, no variant generated any leads during the test window, and the function logs a message and returns without modifying the link.

Copyconst max = Math.max(
  ...testVariants.map(
    (test) => analytics.find(({ url }) => url === test.url)?.leads || 0,
  ),
);
 
// There are no leads generated for any test variant, do nothing
if (max === 0) {
  console.log(
    `AB Test completed but all results are zero for ${link.id}, doing nothing.`,
  );
  return;
}

Sources: apps/web/lib/api/links/complete-ab-tests.ts:28-40

Winning Destination Selection and Tie-Breaking

Once a non-zero maximum lead count is established, the function filters all test variants to identify those whose lead count matches the maximum value. If the filtered winners array is empty, a fallback log check handles the edge case. When multiple variants achieve the exact same highest lead count, a single winner is selected randomly among them using Math.floor(Math.random() * winners.length). If the chosen winner's URL matches the current link.url, no database update is required and execution terminates.

Copyconst winners = testVariants.filter(
  (test) =>
    (analytics.find(({ url }) => url === test.url)?.leads || 0) === max,
);
 
// this should NEVER happen, but just in case
if (winners.length === 0) {
  console.log(
    `AB Test completed but failed to find winners based on max leads for link ${link.id}, doing nothing.`,
  );
  return;
}
 
const winner = winners[Math.floor(Math.random() * winners.length)];
 
if (winner.url === link.url) {
  return;
}

Sources: apps/web/lib/api/links/complete-ab-tests.ts:42-59

Sources: apps/web/lib/api/links/complete-ab-tests.ts:55-55

Link Rewriting and Background Operations

When a winning variant with a distinct URL is identified, the link record is updated in Prisma to set its primary destination URL to winner.url, including related tag, program enrollment, and project relations in the response. Using Vercel's waitUntil, the function asynchronously executes three cleanup and notification tasks via Promise.allSettled: updating the edge link cache via linkCache.set(response), recording the updated link metadata in Tinybird via recordLink(response), and dispatching a workspace webhook (link.updated) if the project relation exists.

Copy 
  const response = await prisma.link.update({
    where: {
      id: link.id,
    },
    data: {
      url: winner.url,
    },
    include: {
      ...includeTags,
      ...includeProgramEnrollment,
      project: true,
    },
  });
 
  waitUntil(
    Promise.allSettled([
      // update the link cache
      linkCache.set(response),
      // record the link
      recordLink(response),
      // send a link.updated webhook to the workspace
      response.project &&
        sendWorkspaceWebhook({
          trigger: "link.updated",
          workspace: response.project,
          data: linkEventSchema.parse(response),
        }),
    ]),
  );
}

Sources: apps/web/lib/api/links/complete-ab-tests.ts:60-90

Related

• Link Dispatch
• Tinybird Analytics