Alternative Terms for Home Life

Control (A): Home Life

_{Figure 1: Control - Home Life}

Variation 1: Army Services

_{Figure 2: Variation 1 - Army Services}

Variation 2: Family & Community

_{Figure 3: Variation 2 - Family & Community}

Variation 3: Community Services

_{Figure 4: Variation 3 - Community Services}

Week 1 (August 05 - August 10, 2024)

• Test setup completed and variations implemented across the site.
• Initial data collection began on August 05.

Action Items

• A/B Test New Variations Based on Insights
• Reason: The current top performers ("Home Life" and "Army Services") have significantly higher CTRs, suggesting they resonate more with users. There may be an opportunity to test additional terms.

Analysis

• Home Life has the highest CTR at 47.10%, indicating that it is very effective at converting visits into clicks.
• Army Services is close behind with a 45.30% CTR, showing strong performance.
• Community Services has a much lower CTR at 14.01%, suggesting that it may not be as engaging or intuitive for users.
• Family & Community also has a lower CTR at 17.77%, but it performs better than Community Services.

Week 2 (August 11 - August 18, 2024)

• Traffic has been evenly distributed across the test variants, ensuring fair comparison and reliable results.
• As of today, we’ve completed two weeks of data collection, and initial results show significant differences in user engagement across the variations.

Action Items

• Consider Replacing or Adjusting "Community Services": With its low CTR, "Community Services" may need to be re-evaluated. Consider testing alternative terms or making adjustments to improve its intuitiveness and relevance.

Analysis

• Home Life continues to lead with the highest click-through rate (CTR) of 31.92%. This indicates that the "Home Life" variation is highly effective at engaging users and driving clicks within the navigation menu.
• Army Services is performing strongly as well, with a CTR of 28.77%. Although slightly lower than "Home Life," it still shows significant user engagement, making it a competitive alternative.
• Community Services has a significantly lower CTR of 9.44%, suggesting that this term is less intuitive or engaging for users. The higher bounce rate also indicates that users may be struggling to find relevant content under this variation.
• Family & Community has a CTR of 10.18%, which is better than "Community Services" but still notably lower than "Home Life" and "Army Services." This suggests that while it may resonate better than "Community Services," it is not as effective in driving user engagement compared to the top-performing terms.

Week 3 Update (August 19 - August 25, 2024)

Action Items:

• Consider Replacing "Community Services": With its consistently low CTR and lower user engagement, this term is proving to be less effective. Testing an alternative term or reorganizing the content under a different label may yield better results.
• Monitor Long-Term Trends for "Home Life" and "Army Services": These terms continue to perform well and are almost neck-and-neck in effectiveness. We should keep monitoring them to see if one clearly outperforms the other as more data rolls in.
• Investigate Potential Adjustments for "Family & Community": This term shows some promise but needs further refinement to boost engagement. Additional tweaks, content updates, or slight changes in wording could help improve its performance.

Analysis:

• Home Life continues to lead the way with a CTR of 44.14%, indicating that this term remains the most effective at driving user engagement. However, it’s important to consider that returning users may prefer "Home Life" simply because it’s the original term they’re familiar with. This familiarity could be contributing to its high click-through rate and consistent performance.
• Army Services follows closely with a CTR of 43.22% and a slightly lower bounce rate of 1.69%, indicating it’s also a strong contender. The term remains intuitive and engaging, making it a competitive alternative to "Home Life."
• Community Services continues to underperform, with a CTR of 14.62% and the lowest average clicks at 1.42. The lower engagement and slightly better bounce rate (1.49%) suggest users may still struggle to find relevant content under this label.
• Family & Community shows marginal improvement with a CTR of 14.80%, outperforming "Community Services" but still trailing behind the top two variations. With an average click count of 1.57, it’s clear that while this term has potential, it isn’t resonating as strongly as the others.

Week 4 Update (August 26 - September 1, 2024)

Action Items

• Remove "Family & Community" and "Community Services": Given the consistently low performance of both "Community Services" and "Family & Community," we recommend phasing out these terms entirely. The data suggests that neither term resonates well with users, resulting in lower engagement and CTRs. Instead of continuing to test these underperforming labels, it would be more effective to focus the comparison between the top-performing terms.
• Test "Home Life" vs. "Army Services": With "Home Life" and "Army Services" emerging as the clear leaders in user engagement, we suggest narrowing the scope of the A/B test to just these two terms. By concentrating the testing on these top-performing labels, we can gather more detailed insights and make a more informed decision on which term should be adopted across the platform.

Analysis:

• Home Life continues to lead with a high CTR, reinforcing its position as the most effective term for driving user engagement. However, we should remain cautious about attributing its success solely to familiarity, as returning users might be favoring the original term they know well.
• Army Services remains a close second, with a strong CTR and close bounce rate compared to "Home Life." Its performance suggests that it is a well-understood and engaging alternative, appealing to users seeking relevant content.
• Community Services continues to lag, with a CTR that remains below expectations. The lower engagement and bounce rate suggest that users are still finding it difficult to locate relevant content under this label, indicating that it may not be resonating as intended.
• Family & Community shows slight improvement but still trails behind the top two terms. While its CTR has increased marginally, it is clear that further refinement or testing of alternative terms may be necessary to boost its effectiveness.

Week 5 Update (September 2, 2024 - September 7, 2024)

Progress:

• A/B testing continued with steady traffic across all variations, ensuring a fair comparison between the navigation terms.
• Focused on confirming trends seen in previous weeks as the testing phase nears its conclusion.

Recommendations:

• Remove "Family & Community" and "Community Services": Both terms continue to underperform, and it may be more efficient to phase them out in favor of a more focused test.
• Test "Home Life" vs. "Army Services": Since these two terms are clearly outperforming the others, we recommend narrowing the scope of the A/B test to compare only these top-performing labels, allowing for more detailed insights and an informed decision on which term to implement.

Analysis:

• Home Life continues to lead with the highest CTR of 38.22%, confirming that it remains the most engaging term for users. However, the high engagement might be due to the familiarity of the label with returning users.
• Army Services holds a strong second place with a CTR of 35.25% and a low bounce rate, showing that it is an intuitive and effective alternative.
• Community Services is still underperforming, with a low CTR of 12.28%, suggesting that the term is not resonating with users and may be causing confusion.
• Family & Community shows marginal improvement with a CTR of 12.94%, but it still lags behind the top two contenders.

Progress:

The A/B test has concluded, evaluating the effectiveness of four navigation terms: Army Services, Community Services, Home Life (original term), and Family & Community. The goal was to measure user engagement through key metrics such as total visits, average clicks, click-through rates (CTR), and bounce rates.

Analysis:

• Army Services performed exceptionally well, achieving a CTR of 46.48%, indicating a high level of engagement. Its average clicks (1.72) and low bounce rate (2.03%) suggest users were not only attracted to the term but also found the content relevant and engaging.
• Home Life, the original term, had the highest CTR (50.22%), showing strong familiarity and user recognition. However, its bounce rate (2.2%) was slightly higher, which could indicate that the content under "Home Life" may need to be fine-tuned to reduce drop-offs.
• Community Services and Family & Community had lower CTRs at 17.04% and 17.63%, respectively, meaning fewer users clicked on these terms. Despite the lower engagement, Family & Community had the lowest bounce rate (0.97%), suggesting that while fewer users clicked, those who did stayed longer on the site.