Scaling distribution for higher quality Spaces
Led design on the creator experience team, focusing on the producer-side strategy (e.g. writers) and Spaces from 2020-2021.
Team of 1-2 PMs, 1-2 designers, 4-6 engineers, 2-3 data scientists.
Team of 1-2 PMs, 1-2 designers, 4-6 engineers, 2-3 data scientists.
Launching Spaces (aka user-defined channels and communities)
In late-2018, Quora focused on scaling content generation by launching a communities product with user defined boundaries, called Spaces. Spaces allowed people to form communities around shared interests and tastes. Users were able to concisely ask questions and receive appropriate answers, without needing to provide additional context. Content distribution began delivering content more accurately to its intended audience.
The anatomy of a Space
The goal of a Space was to set up explicit boundaries to address the lack of predicatable audience expectations,
or "context-collapse".
or "context-collapse".
When I joined the creator experience team in 2020, some Spaces grew to >1 million followers, increasing content generation by 50x since launch. Following this rapid growth, Spaces failed to address on how it would appropriately scale within the current distribution network system. Below, I highlight some key projects and product areas I worked on that spanned over multiple quarters and teams, to help address this issue.
Overall Impact: Scaled Spaces engagement by over 40% in the span of a year, contributing towards an overall 15% in engagement and 1% increase in PHUVs (~3M unique visitors,~$1M in revenue).
Some examples of Spaces that exist today –
Various use cases exist for Spaces today – ranging from shared interests, content curation, building a newsletter or audience, engaging with experts, learning about a niche topic to more.
The problem: (Over-)scaling Spaces
When Spaces initially launched, the initial focus was on accelerating user adoption by maximizing follows through various distribution channels, resulting in significant growth, with Spaces reaching >1M followers and nearly 1/3 of active users following a Space.
Upon this rapid Space follows growth, both Space engagement and content generation and quality stagnated, creating a bottleneck in distribution and growth where Spaces with large followings overshadowed potentially high-quality Spaces. Readers lacked understanding the purpose of Spaces and how they differed from other content, while writers struggled with low engagement rates despite investing heavily.
The strategy's emphasis on Space followers led to ML ranking models assuming that popular Spaces equated to "high-quality", increasing their distribution and exacerbating the growth of "low-quality" Spaces, making it difficult for new, potentially better Spaces to compete for traction.
"How can Quora leverage its distribution network to promote high-quality Spaces and minimize distibuting low-quality ones, while addressing issues in producer content generation and consumer engagement?"
Left: Users prompted to follow Spaces when signing up for Quora without having context of them or their interests
Right: Old design of suggested Spaces shown in feed that users would follow without seeing its content
Solution #1: Reconfiguring Space follow mechanisms
Limiting "low-quality" Space follows
In order to start promoting high-quality Spaces within the product, low-quality ones needed to stop gaining traction and have limited distribution within the product. User should only be able to follow a Space after having context of what a Space is, the implications of following one, and seeing the type of content a Space curates.
This meant removing the ability to follow a Space directly from the two highest follow streams – the onboarding step when a user is signing up for Quora and seeing suggested Spaces to follow within feed. After signing up, the Spaces step was replaced by a banner to encourage users to discover Spaces that might interest them.
Left: Discover Spaces banner displayed on top of feed after signing up,replacing the follow Spaces step shown when a user is signing
Right: Suggested Spaces force users to visit the Space before following it
Impact: Naturally, this led to nearly a -50% drop in overall Space follow metrics, partially mitigated with the new "Discover Spaces" banner. In order to align leadership with tolerating this drop, we identified projects with metric estimations to re-coup these losses by increasing "high-quality" Space follows, with greater, long-term impact on overall engagement metrics.
This was also partially validated through new users who followed Spaces after discovering them from the banner having higher engagement rates with Space content.
Leveraging feed to increase "high-quality" Space follows
Discover New Spaces Section
Users perusing feed were considered low-intent, but had the highest rates of visiting unfollowed Spaces. These visits would primarly come from the "Discover New Spaces" section, featuring popular Spaces tailored to individual interests.
- Revamped the UI to display a more detailed list view and added metadata information to help users gauge relevance
- Implemented a filtering system with tags like "New", "Popular," and "Top" to facilitate better Space categorization and ranking
Impact: >40% increase in Space click-throughs from the “Discover Spaces” section in feed, >5% increase to the Spaces discovery page, collectively leading to a ~20% increase in "high-quality" Space follows and ~1% increase in active feed engagement metrics
Space header on feed stories
Content from Spaces users do not follow, but the system infers they would be interested in, is shown within their main feed, as another low-intent Spaces dicovery channel. Users are encouraged to follow that Space if they liked its content and unfollow Spaces with irrevelent content for a more tailored feed experience.
Impact: >3% increase in Space follows, >5% increase in Space unfollows, but ~0.05% increase in feed engagement
Intentionally discovering new Spaces
Spaces discovery page
Space suggestions in feed allow users with low-intent to find and follow Spaces the system thinks aligns with their interests. For users with medium-intent, where a user is looking for Spaces to follow, or high-intent, where a user has a specific topic for a Space, a centralized Spaces discovery page was built.
This page failed to inform users about what Spaces are, lacked filtering or intentional discovery flows, and used inconsistent UI patterns, leading to high drop-off and low usage and return rates.
Proposed Spaces discovery page
The Spaces discovery page proposal addressed issues for consumers, producers, and the system, while serving as a centralized place for Spaces discoverability. Consumers (i.e. readers) would benefit from user-driven discoverability, through categorization and filtering features. Producers (i.e. writers) would be encouraged to "compete" to create better Spaces, rewarded by increased visibility. The system would be able to use this competition to promote higher quality Spaces more accurately.
Impact: Initial A/B tests and usability feedback suggested postive results, but stakeholder feedback priotized the development of a "Spaces leaderboard", to foster compeition among Spaces and derisk the investment of building this page.
Building a leaderboard to recognize great Spaces
A Spaces leaderboard was developed to highlight the best Spaces for a specific topic. This recognition encouraged prducers to build better Spaces, minimizing churn. It helped new writers learn from successful Spaces, serving as a template. Featured Spaces on this leaderboard received badges and more visibility in distribution. It facilitated consumers with finding the best Spaces for a topic more easily.
Each Space was tagged with a topic, which was leveraged to display a Spaces leaderboard on each topic page to test viability. Developing a ranking system posed challenges, especially ensuring consistency across different topics. Initial heuristics based on "Top Writers" proved inadequate, so various metrics like engagement rates, content types, and topic relevancy were tested.
Impact: Despite multiple iterations, creating a universal metric and ranking heuristic algorithmically, without manual auditing, for all types of Spaces proved difficult. Consequently, a "Most Viewed Writers" leaderboard was built, showcasing influential writers who owned Spaces within those topics.
Follow channels on the Space page
Follow banner on Space page
When visiting a Space a user does not follow, they would have to scroll back to the Space header section, in order to follow it. On scroll, a follow Spaces prompt was added encourging users to follow a Space they liked.
Impact: >5% increase in Space follows
Showing related Spaces on Space page
Users may visit a Space for a topic they might be interested in, but the Space might not be exactly what they're looking for. "Spaces you may like" allows for users to see related Spaces to that topic, to find one that may better suit their interests.
Impact: >2% increase in Space follows
Recommended Spaces notification
Leveraged notifications to create another channel to drive traffic to Spaces. These notifications were sent when the system identifies a Space it thinks you might like or Spaces that have some social affinity to the user, such as Spaces created or followed by users they follow.
Impact: >2% increase in Space follows
Solution #2: Increase Spaces content consumpation
Building a dedicated "following feed"
A new way to see stories from people and Spaces users follow
As Quora scaled Spaces, feed was becoming convoluted from content generated from recommendation systems and content the user followed. Both readers and writers would get frustrated for not being able to filter their feed's content to only show them stories from sources they explicitly followed. Metrics showed this led to a huge drop-off in user follow rates for writers and Spaces after the first couple weeks, as they weren't able to easily engage with that content specifically in their feeds. By building a separate feed for followed content from writers and Spaces, readers can have a more curated feed reading experience and Space producers (e.g. writers) are motivated through more engagement with their content.
Impact: +5% increase in feed usage, resulting in a +0.02% increase in PHUVs, +5% increase in user follows for active users, +8% increase in Space follows
Space notification settings
Adding the ability for Space digest emails
Leveraging email would be a significant traffic driver to Spaces, but the current Space notification settings did not support an email option. Emails would unlock the ability for Spaces producers to send newsletters and allow consumers to engage further with content they are invested in. Adding an "email" option required a redesign to address existing notification system complexities.
Usability tests revealed user frustration with managing notification settings, leading them to turn off all notifications and a huge loss in engagement. Key issues included the need to choose notification channels individually for each Space, unclear organization of settings, and inconsistencies across platforms with managing notifications.
A comprehensive solution would be to allow each notification channel (push, email) to set its own frequency (all posts, personalized, off), but this required a much larger engineering effort. A simpler solution to allow for only one frequency per Space and choosing which channels to receive those notifications from, was not as optimal, but gave users the flexibility needed to receive Space emails.
Right: Launched direction setting a global Space frequency and then its channels, with the addition of emails
Impact: Reduced notification disabled by 10%, Increased Space visits by 15% by unlocking the ability for users to send/receive Space emails
Solution #3: Empower producers to be able to create successful Spaces
Spaces Producer Onboarding
High quality Spaces for readers to consume and for the system to distribute are dependent on the ability for Space producers to curate successful Spaces. While the platform implicitly teaches writers how to create good Spaces, through its distibutio channels, explicity teaching producers is more impactful.
While migrating this flow to React, A/B tests were conducted to refine the user interface and interactions. Research revealed Spaces that posted 8 or more pieces of content within the first week had lower churn rates by 80%. Similary, inviting 5 followers and reaching a minimum of 250 followers within the month also reduced churn by 80%. Notifications were strategically utilized to prompt users to complete these tasks systematically.
Above: Carousel variant, focusing on one task at a timeBottom: Launched variant as a list view with expandable tasks for details
Impact: Increased Spaces onboarding tasks completion rate from 40% to +85%, resulting in a -30% decrease in Space producers churn, increased Spaces content distributed in feed by 15%, resulting in a +0.05% increase in overall feed engagement