Goals, Metrics and Data
Data allows for more effective decision making, but data is only as useful as your ability to use it. Metrics allow you to step back from your ideas and time and ensure you are meeting the needs of users. Analytics focused on users' behaviour can be a good source for explaining why you made a decision.
Metrics are often not useful on their own. All metrics need to be situated in context and focused around outcomes (e.g. the number of users to the site isn’t useful, but if you know the size of your potential audience, then knowing how many people you are reaching vs. could be reaching is helpful.)
Try to stay away from vanity metrics. These are numbers that don’t tell you anything in isolation. (e.g. number of page views). They tell you nice/bad stuff but not what to do next or why it is happening.
Mix methods approach to evaluation can often help to fill in the gaps in your knowledge.
- Social media analytics: are not transparent analytics. They are biased towards marketing and they don’t show all data. This is problematic for social media analysis on what people are talking about e.g. the sample isn’t random and the data is biased on who is talking about X topic. Therefore it isn’t helpful and you can’t talk about significance.
- A/B testing: different content and/or layouts to see which are preferred by users.
- Digital ethnography: watch what people are doing online
- Eye tracking: not that helpful. People usually read from the left.
- Usability testing: remotely using a shared screen or recording users interacting.
- Usability Heuristic review article: Jakob Nielsen's 10 general principles for interaction design. They are called "heuristics" because they are broad rules of thumb and not specific usability guidelines.