February 5, 2013 Back to All Blogs

Say What?! Web Design for Continuous Experimentation

ImageMost organizations looking to redesign their existing website or build a new one want their technology vendor to ask the right questions, fully understand the goals for the website and set about to create a stunning masterpiece.  The final product is “launched” with great fanfare and everyone oohs and aahs at how great it looks.

The problem with this method is that you never learn whether the site is effectively engaging those who visit it until it’s too late. While the site looks exactly the way you wanted it to, you may not be attracting and keeping visitors. You may wonder why conference registration numbers are down, social media activity is weak and no one is staying to read your press releases.

To avoid eventual disappointment and delusion about your web projects, regard them as a way to test assumptions and understand your audience.

‘What!?” you may say. “Such experimentation sounds frivolous, loosey goosey, and expensive … I am paying experts to give us the best design based on their knowledge and expertise.”

Consider the cautionary tale of Etsy – one of the most popular shopping websites in the US, growing at a rate that outpaces EBay. Nearly 700,000 new members join Etsy each month to browse and purchase millions of hand-made items from 875,000 shops.

To ensure those new members were able to view as many items as possible, Etsy’s IT staff built an endless scroll architecture that loaded a batch of new items every time a shopper reached the bottom of a page.

“Seeing more items faster was presumed to be a better experience,” said Dan McKinley, Principal Engineer at Etsy. Therefore, Etsy spent months developing and designing infinite scroll to their search listings, only to find that it wasn’t working and actually turned people off. Both the faster loading images and the number of images did not cause people to stay and buy.

“It involved a lot of work upfront… and all of that work was pointless,” said McKinley in a presentation to web designers. McKinley stressed that “it’s not that infinite scroll is stupid. It may be great on your website. But we should have done a better job of understanding the people using our website.”

As a result, any updates to the Etsy site are now the products of incremental, measureable adjustments. Developers isolate changes, see how their website visitors respond and tweak to ensure the site truly works for users. The lesson of the “infinite scroll fail” was the continuous imperative to create and test assumptions and hypotheses. What do you want users to do? What mechanisms do you have in place to accomplish your goals? What does the data show is working (or not)? Test that dropdown menu. Measure any changes in response to the location of a navigation button.

In the “Top Ten Nonprofit Tech Trends to Watch in 2013,” we called it the Whole Foods-ification of website and technology endeavors. It’s organic! Nonprofits are slowly learning not to treat their website and technology as they do their annual reports—projects that are perfected and completed. Incremental changes and improvements based on feedback and experience is a vastly better approach than issuing an RFP every five years to do an expensive website redesign. Facebook often makes multiple tweaks or code changes in a day. They know that routine testing, learning and improving continuously provides new value to users.

Does this mean that projects take longer to create and are more expensive? No. It simply means that we put some of our assumptions about what will make a great website to the test. A fellow blogger recently stressed “Why Big Testing Will Be Bigger Than Big Data.” Pantheon agrees.

As we build each website, every step along the way is an opportunity to get feedback from the client’s target audience that shapes the most effective architecture and navigation tools.

Sure we trust our own expertise and your experience and vast knowledge of your field, but as Etsy’s McKinley says – incremental and continuous testing “keeps us honest and stops us from breaking things.” In the high-stakes world our clients live in, that’s well worth the effort.

Mark Tobias (@PanthTech) is president of Pantheon, which combines technology expertise and a deep knowledge of health care, education, and social impact markets to provide online technology solutions for nonprofits, associations, and government.