A/B Testing Tableau UX Analytics Digital Marketing Conversion Optimization

A/B Test Analysis: Webpage Performance

A Tableau-powered analysis of a two-week A/B test for DigitalPath Innovations — comparing two webpage versions across engagement, conversion, feedback, video interaction, and referral source dimensions to determine which design better serves users and the business.

Versions Tested

2 Weeks

Test Duration

Variables Tracked

Analysis Dimensions

Objective & Problem

The Objective

DigitalPath Innovations, a leader in digital marketing solutions, sought to enhance its website's user experience to improve engagement metrics, conversion rates, and customer satisfaction. A two-week A/B test presented two distinct webpage versions (Version A and Version B) to a randomized group of users.

The task was to perform a comprehensive analysis of user behavior across both versions using Tableau — identifying which design performs better and delivering data-backed recommendations to guide future product and UX decisions.

The Business Problem

Without a structured analytical framework, DigitalPath could not reliably determine which webpage version was driving better engagement, conversions, or user satisfaction. Intuition-based design decisions risked leaving significant conversion and engagement value on the table.

The company needed a Tableau dashboard that would visualize the behavioral differences between Version A and Version B across every key dimension — from time spent on site to purchase completion — enabling confident, evidence-backed design decisions.

Five analytical objectives guiding the project:

Engagement Metrics Analysis

Assess how each version impacts user engagement — time spent on site, pages viewed, and actions taken.

Conversion Rate Optimization

Compare the full conversion funnel — from product views through cart additions to purchase completion.

User Feedback Analysis

Evaluate the correlation between feedback submission rates and webpage version as a proxy for satisfaction.

Content Interaction

Investigate how each version influences video watching behaviour and overall content engagement.

Referral Source Impact

Analyse how different referral sources (organic, paid, social, direct) affect user behaviour and version preference.

Data Description

One structured dataset capturing 14 variables per user session across both webpage versions.

User Interaction Dataset — Field Reference

UserID — unique user identifier

SessionID — unique session identifier

Version — A or B webpage variant

Date — session date

TimeSpent — seconds on page

PagesViewed — pages per session

ActionsTaken — clicks & form submissions

ProductViewed — products browsed

ProductAdded — items added to cart

CheckoutInitiated — checkout started (Y/N)

PurchaseMade — purchase completed (Y/N)

FeedbackSubmitted — feedback given (Y/N)

VideoWatched — video viewed (Y/N)

ReferralSource — traffic origin channel

My Approach

A structured Tableau analytics workflow — from raw session data to an executive-ready A/B test dashboard.

1
Data Preparation — Cleaned and validated the user interaction dataset, confirming session-level completeness across all 14 variables. Verified randomization integrity — ensuring the Version A and Version B user groups were comparable in composition before drawing any behavioral comparisons.
2
Engagement Metrics Analysis — Compared TimeSpent, PagesViewed, and ActionsTaken between Version A and Version B users in Tableau — using bar charts, box plots, and average comparison views to surface statistically meaningful engagement differences between the two designs.
3
Conversion Funnel Analysis — Built a step-by-step funnel visualization in Tableau tracking users from ProductViewed → ProductAdded → CheckoutInitiated → PurchaseMade for both versions — identifying where each version gains or loses users in the conversion journey.
4
Feedback & Content Interaction Analysis — Analysed FeedbackSubmitted and VideoWatched rates by version — using cross-tabulations and proportion comparisons in Tableau to determine whether one design more effectively drives user satisfaction signals and content engagement.
5
Referral Source Segmentation — Segmented all behavioral metrics by ReferralSource to determine whether version performance varied by traffic channel — revealing whether Version A or B was better suited to specific audience segments arriving from organic search, paid ads, social media, or direct traffic.
6
Dashboard Design & Recommendations — Assembled a comprehensive Tableau dashboard consolidating all five analysis dimensions into a single, stakeholder-ready interface. Translated visual findings into clear, actionable design recommendations for DigitalPath's product and marketing teams.

Metrics Tracked

Six core performance dimensions compared between Version A and Version B.

Time on Site

Average seconds spent per session — a proxy for content relevance and design engagement depth

Pages Viewed

Average pages browsed per session — indicating navigation clarity and content discoverability

Conversion Rate

Purchase completion rate — the primary business outcome metric, tracked across the full funnel

Feedback Rate

Proportion of sessions where feedback was submitted — used as a satisfaction and engagement proxy

Video Watch Rate

Proportion of users who watched video content — indicating content design effectiveness per version

Referral Performance

Version A vs B engagement rates segmented by traffic source — organic, paid, social, and direct

Version Comparison

A head-to-head summary of how Version A and Version B performed across key behavioral dimensions.

Metric	Version A	Version B	Winner
Avg. Time on Site	Analysed via Tableau	Analysed via Tableau	Determined by dashboard
Avg. Pages Viewed	Analysed via Tableau	Analysed via Tableau	Determined by dashboard
Actions Taken	Analysed via Tableau	Analysed via Tableau	Determined by dashboard
Purchase Conversion Rate	Analysed via Tableau	Analysed via Tableau	Determined by dashboard
Feedback Submission Rate	Analysed via Tableau	Analysed via Tableau	Determined by dashboard
Video Watch Rate	Analysed via Tableau	Analysed via Tableau	Determined by dashboard

Exact metric values are drawn from the Tableau dashboard analysis of the user interaction dataset. The table above reflects the comparative framework applied — version winners are determined by statistically significant differences visualized in the dashboard.

Key Insights

What the A/B test analysis revealed — and the design decisions it informs.

Not all metrics point in the same direction. A/B tests rarely produce a clear, universal winner — one version may drive higher time-on-site while the other converts better. The Tableau dashboard isolates each dimension so DigitalPath can make nuanced decisions rather than adopting one version wholesale, potentially borrowing winning elements from each.

Funnel drop-off points differ by version. Conversion funnel analysis reveals where each version loses users — whether at product view, cart addition, or checkout initiation. Identifying the specific funnel stage where one version outperforms the other gives the design team a precise intervention point rather than a vague "improve the page" directive.

Video content engagement signals deeper design effectiveness. Differences in VideoWatched rates between versions reflect how well each design surfaces and promotes content — a version that drives higher video engagement is likely structuring its layout and information hierarchy more effectively, which tends to correlate with broader engagement improvements.

Version performance varies by referral source. Segmenting results by ReferralSource reveals that the winning version for paid traffic may not be the winner for organic or social traffic. This audience-level insight means DigitalPath can consider serving different versions to different traffic sources — a more sophisticated strategy than a single universal winner.

Feedback submission rate is a leading indicator of satisfaction. The version with a higher FeedbackSubmitted rate signals stronger user engagement and willingness to interact — even when conversion rates are similar. This qualitative signal complements the quantitative conversion data and provides a more complete picture of which design builds stronger user relationships.

Outcome & Impact

From Two Designs to One Clear Direction

The Tableau dashboard delivered a comprehensive, visual A/B test analysis that gave DigitalPath Innovations the evidence needed to make confident webpage design decisions. Rather than relying on gut instinct or aggregate metrics, the product and marketing teams gained dimension-by-dimension visibility into how each version performs — by engagement, conversion funnel stage, content interaction, user satisfaction, and referral channel. The analysis replaced opinion-driven design debates with data-backed recommendations, directly supporting DigitalPath's goal of improving user experience, conversion rates, and customer satisfaction at scale.

Tech Stack

Tableau A/B Test Analysis Funnel Visualization Engagement Analytics Referral Segmentation Dashboard Design Behavioral Analytics UX Data Analysis Conversion Optimization

Key Learning Points

Exploratory Data Analysis — uncovering behavioral patterns across session-level user interaction data
A/B Test Methodology — structuring valid comparisons between randomized user groups across multiple metrics simultaneously
Tableau Dashboard Design — building multi-dimensional, stakeholder-ready visual analytics from behavioral datasets
Insight Communication — translating statistical test results into plain-language design and marketing recommendations