comScore has published some numbers on the evolution of the smartphone market in the UK over the past year.

They include market share percentages for a few smartphone platforms:

• 27.1% of UK smartphones are iPhones
• 26.7% of UK smartphones are Android phones
• 22.5% of UK smartphones are Symbian phones

According to Engadget citing comScore

42 percent of all mobile users in the UK used a smartphone in May of this year (2011)

What does this mean for all phones?

Although I love my smartphone to pieces, if I was trying to choose a platform / channel for my mobile service I would be interested on the mobile market as a whole, and not only on the smartphone chunk.

So I decided to calculate what the comScore figures actually mean for the overall UK mobile market:

• 11.382% of all UK mobile phones are iPhones
• 11.214% of all UK mobile phones are Android smartphones
• 9.45% of all UK mobile phones are Symbian smartphones

So, if you choose a native application as the channel to deliver your mobile service, and you release it for iPhone and Android phones, you are targeting 22.596% of the overall UK mobile phone market.

PS: I am assuming the comScore numbers are smartphone only, and therefore do not include tablets.

Huge disclaimer: I am no statistician, so I sincerely hope I made my calculations right. If you spot a mistake, please let me know!

On February 28th I sent an email to 3 mobile-related discussion groups (Mobile Web, WURFL and Mobile Monday London) about user research activities applied to mobile applications and websites. This is what I wrote:

I am doing some work around user research for mobile apps and websites, and it would be really interesting to know how many of you do some kind of user research as part of your development cycle (usability testing, interviews, etc).

Do you validate designs, prototypes or early builds with prospective users in any way? Do you do user research before starting development to explore different design options? Or maybe you don't do it at all. If you don't, why not? Is it lack of time and money, or are there other reasons? You can answer on or off list. I'll compile the answers (if any) and distribute the outcome.

Thanks!!

7 wonderful people replied, and what follows is a compilation of their responses.

The wonderful respondents

Replies came from the UK, US, India, Australia and New Zealand.

• 1 came from a software company with successful desktop products who have developed mobile versions
• 1 came from a software company with a successful mobile product
• 2 came from software development agencies
• 1 came from a mobile design agency
• 2 came from independent user centered design consultants

The user research techniques

All 7 respondents carry out user research activities during design and development of their mobile software. The most popular activity seems to be usability testing, which was mentioned by 6 out of 7 respondents. Other user engagement activities that came up are:

• focus groups
• field studies
• A/B testing
• surveys
• and interviews, although not everybody is a fan

We tend not to do interviews though, as we have found it to be largely counter productive. As mobile is so new as a medium customers either do not 'get' what we are trying to do, or just recycle experiences that they like from other apps.

Respondents also mentioned internal brainstorming, and peer reviews as a back-up activity whenever project constraints prevented them from involving end users.

I cannot say it happens all the time though. When time is tight, we may 'just' do peer reviews instead.

Usability testing

Assessment, not exploration

Usability testing is mainly used to evaluate possible design solutions:

We don't hold-off development if we have a strong use-case but we bring in users/customers very early-on and make adjustments to our design as needed.

Once we had designs that we were happy with (based on our experience of what does/doesn't work for mobile), it's time to take it to users for validation (...) All in all, this was a fairly tactical exercise, the main priority being to validate the proposed designs.

We test (…) normally part way through the development process when we've got some kind of prototype we can use.

Methodologies widely used for testing desktop software seem to translate well to mobile. Respondents mentioned:

• task-based assessment
• the think aloud protocol
• the System Usability Scale (pdf),
• Rapid Iterative Testing and Evaluation (RITE)
• and Rapid Desirability Testing with the Microsoft product reaction cards.

Prototyping

Only one respondent mentioned paper prototypes: most seem to use interactive ones.

One respondent reported using storyboards for exploration:

For newer ideas, we do storyboards and get feedback.

One respondent mentioned that no-code-required prototyping tools frequently used for desktop software do not work well for mobile.

a quick prototype was built. Well, it wasn't that quick… since couldn't use Axure etc. since not suitable for mobile, so it was necessary to code up the pages (…) it's hard to build a prototype (because you can't use the standard toolkit like Axure/Balsamiq or whatever).

An additional factor impacting prototype build is the importance of animations and transitions in touch interfaces:

Something you didn't mention was around animations and transitions, which can massively influence the experience. We do a lot of work around these too, especially looking at how haptic elements can be included.

Prototypes built by linking static mockups are obviously unsuitable for testing animation frameworks.

Devices

One respondent observed how the introduction of device types makes participant profiling and recruitment more complicated than in desktop software:

Recruitment was really important (…) we wanted to see a range of different devices… some iPhones, some non-iPhone smartphones and some feature phones. (…) It costs more money to recruit.

The other issue related to mobile devices is which one to use during the test itself. Respondents who mentioned the matter agreed on using participants' own phones:

For these interviews, it was important to have participants use their own devices - which removes any bias/difficulty that might be caused by unfamiliarity with the device, or of course, the loss of context if you were to test with an entirely different form-factor (like an emulator on a laptop or PC).

Using participants' own phones is easy to do with web-based functionality, but more complicated when testing resident applications (native or not), which require installation:

If it's something we have public we ask them to bring their own phones and run the test on their own handset. In the case of the iPhone development it's more difficult as we don't want to get into sideloading dev versions of apps onto their phones, so in that case we just use our own handsets for the test, but first ask them to show us how they use [our application] on their handset (to check which version they are on etc - and make sure they really are an iPhone user).

Recording

Capturing video of usability testing sessions is a bit of an issue, so much so that one of the respondents has decided not to do it at all:

We have used Morae in the past also for web site testing, but could not easily use the system for mobile (…) Obviously it's difficult to video something like that so we just sit next to them and observe.

Another respondent seems to have built his own infrastructure for video recording:

You need some equipment to record (…) We got by with a webcam attached to a piece of curved metal which the participants phone can be strapped into and held in-hand.

If you are stuck about how to record your usability tests of mobile software, our DIY kit might help.

Surveys

They deserve their own section, because they are carried out in quite innovative and unorthodox ways. Online and mobile surveys were of course mentioned, but one of the respondents does them on the street too:

One other thing we've found helpful recently is impromptu street surveys - not so much for usability, but for early design and colour feedback, including for icons and logo work. A really quick way to get some input and an impartial 'outside view'. We just look for people in our target age group, ask them if they can help briefly and double check their age, then show them some designs and ask them what they think. Using an iPad is great for this, partly because people want to stop to take a look at the iPad itself.

It is also remarkable the use of remote services initially created for software testing, not for user testing:

We utilize services of Mob4hire users for testing across geography, network condition, etc. I have used Deviceanywhere too.

Is it worth it?

Despite the time, effort and investment required, respondents agreed that involving users in the design process is as worthwhile in mobile software as it is in desktop software:

(Research)'s been very important in new product/platform launches especially. Every time we've done it we've learnt something and caught 2-3 interaction issues that we would not have realised were a problem without testing.

In mobile we are learning that a lot of people have strong opinions on how something should work based on the app or platform (iOS/Android) they are most familiar with. Also, we are surprised (and delighted) at the solutions they suggest based on leveraging the smartphone.

PS: my apologies to all list members for the delay in posting this summary, and my gratitude to the seven people who took the time to share their experiences with us all.

Mobile Monday London came back yesterday with a jam-packed session about tablets. Rumor has it there were people from LG, Blackberry and Google showing non-iPad versions of the thing.

This might sound a bit daft, but the main point of the night was the very bold statement: a tablet is a tablet. It's not a phone, it's not a netbook, it's not a laptop. We therefore must try and figure out what tablets are for, since they must be for something different, right?

The books

David Roth-Ey, Group Digital Director at Harper Collins, had a few interesting things to say about how book publishers are dealing with the digitalisation of their content.

• Book lovers feel guilty about abandoning the physical book. He thinks they'll need a powerful reason, a truly wondrous digital book-reading way if they are to leave the paper incarnation. I have to agree with him: I will not migrate easily.
• There is life beyond the book reader. Harper Collins are experimenting with new ways of delivering content, and apps is one of those ways. Check out the SAS Survival Guide for iOS
• All-you-can-it subscriptions for books are a no-go. When asked about a monthly subscription model for books (pay a monthly fee, read as much as you want), Mr. Roth-Ey replied that's one of his worst nightmares. He explained digital business models must compensate for the value of the content book publishers provide.
• Buy once - read anywhere. For Harper Collins, it's all about accessing the content you buy from any device. Once you pay for the book, it's yours to read from your iPhone, your iPad, your Kindle, your IdeaPad, your Inspiron Duo and whatever next. This poses a few interesting content adaptation and multiple user interface questions: what's the digital equivalent of folding the page?
• The ubiquitous promise of the web. HTML5 vs. native came up of course. Mr. Roth-Ey was quite bold about the whole thing. He explained how fragmentation is not a big problem for publishers, since they can easily piggyback on existing readers for multiple platforms. But their moto is maximum reach, and the web is, from all platforms, the best positioned to deliver it.

The telly

David Gibbs, Director of Mobile Applications and Services at BSkyB, always strikes me as an unusually focused person. These Sky guys know what they are about:

• Consumption. They don't care about people producing their own content. They only care about people consuming Sky's content.
• and TV. They are years ahead of the other TV providers in the UK when it comes to incorporating portable devices into their offering, but their strategy keeps television as the centre of gravity.

For Sky, portable devices have limitations: they cannot offer access to all TV content through phones and tablets due to rights issues. They have however discovered their strengths in 2 areas:

1. Self-service, with the web as the preferred platform to deliver it.
2. Remote control capabilities, with native applications as the preferred platform to deliver them.

Sky's clarity is incredibly rare in the cross-device world, and it's probably paying off handsomely.

The promise

But for the rest of us earthlings, what are tablets for? Someone from the audience confessed himself underwhelmed by what has been offered through tablets so far: clones of smartphone and desktop applications, mere scans of print magazines and newspapers. Sure we can do better, but how?

Harper Collins replied: developers should be working more closely with our writers. Add your readers to the mix, Mr. Roth-Ey, and you might be onto something.

To commemorate the upcoming Global Service Jam a few people at LBi London have organised a series of lunch sessions with external speakers.

I finally managed to catch one yesterday.

The happy marriage of video and ethnographic research

Geke Van Dijk is the soul behind a company called STBY. I think they pronounce it stand by. They specialise on ethnographic research for product and service innovation.

Geke focused on how they are using video for research activities, armed with just a few Flip cameras.

In my head, all the examples Geke shared fit neatly into 2 groups:

1. Video as a research tool
2. Video as a communication tool

1. Video as a research tool

• Self-interviews: participants record themselves answering questions set by the researches. Geke explained how they handed an envelope with questions and a Flip camera to the participants, brought them to a park nearby and set them loose to record their answers. The main advantage of this method is that answers are free from group bias. To me this also seems a good way of running interviews in very little time.
• Video safari: during contextual enquiries, record a journey and the stories that surface. For example, if you are researching water usage in participants' homes, ask them to guide you through the different water dispensing points in the house. Water-usage related stories are likely to surface at each point, and they'll be captured in your recording.
• Video-diaries: use video as the main logging mechanism for diary studies. Geke advised that it's important to provide clear instructions on how to create video diary entries. Otherwise you'll end up with widely different formats, which will make analysis much harder and time-consuming. She gave us an example of such instructions:
  • Step 1: point the video camera outwards and rotate 360 degrees to show us your surroundings.
  • Step 2: point the video camera to yourself and respond to the question we sent you via text message.
  • Step 3: point to a specific thing in your location you'd like to show us.

2. Video as a communication tool

• Video to document personas: each persona is illustrated by video clips from the research that exemplify goals and behaviours.
• Video to immerse design workshop participants: use video fragments from the research to set the tone and drive inspiration during ideation and co-design activities.
• Videos as customer icons: Geke explained that once they release these video clips from research to their clients, they acquire a life of their own. They go around the organisation, they are shared across disciplines and departments, they become "customer icons", a kind of shortcut to end users. I really liked this one.

Managing hand off

I asked Geke how to make sure research insights make it into design. She gave me 2 tricks:

1. Design workshops using research outcomes: they help reinforce research results in designers' heads, and also provide an example of how research insights can be used in design activities.
2. Inverted engagement: I just made up the name, by the way. Basically, designers are lightly involved from the beginning of the research process. Sometimes they attend research activities, but they also help prepare them. Apparently, designers are very good at producing research props. Over time, design involvement increases and research involvement decreases. Once in the design phase, the research team stays lightly involved. Normally they analyse the research material to look for answers to new questions that invariably rise during the design process. I've made a little drawing to illustrate this trick.

This was a truly fascinating talk. Well done to the LBi guys who organised it, and thanks to Geke for sharing her research experience. I am off to buy a Flip camera!

These are the slides for my 10-minute presentation at the UK UPA February event. Well done to the speakers, attendees and specially to the organisers: I found the 10-minute presentation format really pleasant.

This is what my Gmail account told me this morning.

Look at what happens when you click the green window control in the Calculator application (Mac OS X 10.6.5).

I have in front of me a paper titled Cross-Platform Service User Experience: A Field Study and an Initial Framework, by a group of 4 researchers from Finland. It is about how people use the desktop and mobile versions of 3 web-based applications: Facebook, Dopplr and Nokia Sports Tracker.

What's most remarkable in this study is the mismatch between the amount of functionality expected and actually provided by the mobile incarnations of the apps. Look at these quotes:

Many users wished for more functionality in the mobile UI.

The strongly reduced amount of functionality on the Dopplr mobile UI was disappointing to may users. It was considered as excessively simplified compared to its PC counterpart.

With the Nokia Sports Tracker (...) some users hoped to be able to browse training data with their mobile device, as they wanted to keep updated while on the road.

The users of Facebook and Dopplr did not feel that they were able to do everything they wanted on the mobile.

Are we dumbing down our mobile apps a bit too much?

I am back in old London from EuroIA in Paris, and there is a lot of awesomeness in the air:

• The Eurostar is awesome
• Paris is awesome
• Good Bordeaux wine is awesome
• La Galoche d'Aurillac is awesome

But the most awesome thing of all were the people who attended the DIY Mobile Usability Testing session. Look at these tweets:

We are flabbergasted

By your tweets, your comments and your response in general. To show our gratitude, the slides are up.

We also have a video that shows you how to build the whole thing.

And now, what?

Brian from iQ Content, who used to be sort of my boss, told me I should make an article out of this. And you know what? I think he is right.

So there is an article in the offing. I'll keep you posted.

Hold on, just one more thing

A HUGE thank you to Theba Islam for becoming our usability testing participant during the session. It wouldn't have been the same without her.

And if you have any questions, need some help building the damn thing, or you've used it and want to share your pain, give me a shout (should I say a chirp?). My Twitter incarnation is called @belenpena

When we say Apple is all about the user experience...

...I guess we mean they don't force you to click through the 55 pages to reach the Agree button.