Here's a crazy idea: Don't use analytics at all but focus on your product. If your success relies solely on "improving conversions" by tracking your users and then changing the position and color of your "Checkout" button then maybe try setting yourself apart such that customers want to buy your product even despite an obnoxious purchasing flow. Only then start optimizing it.
More serious thoughts: Google Analytics introduces performance overhead for your website and now you have to explain to your users which third party is responsible for processing their data on top of yourself. Why introduce those headaches? Are the insights from Analytics really valuable enough to justify the cost? I personally haven't seen it.
> Don't use analytics at all but focus on your product.
Huh? Analytics is how you focus on your product.
By instrumenting your product with analytics, you can find out if customers are using your new useful feature or if they can't find it. If they're performing a task quickly because it's easy, or slowly because they're struggling with the UX. And you find out that customers on a certain mobile device are suffering huge performance issues, for example.
You don't know these things until you measure them. That's analytics.
Obviously analytics are only one piece of product improvement -- there's sitting down with users for 30 minutes to watch them use the product, interviews, surveys, etc.
But analytics are a critical piece. You can't focus on the product without analytics.
Analytics isn't just about conversion. Analytics is about the entire product experience.
> Huh? Analytics is how you focus on your product.
For the first decade of my career, I really believed this, and spent a lot of time doing split tests, and studying analytics, and trying to "make things better" by understanding the numbers as they were given to me.
This was a mistake.
I've since learned that these numbers will rarely help me make meaningful improvements to my product, and my business. Sure, they can be useful so that I'm not "running blind," but they simply aren't going to show me how to create an ingenious idea that takes things to the next level.
Analytics will help you optimize things to a "local maximum", but they'll blind you to the real possibilities of creating something new that can completely transform a business. As soon as I understood this distinction, I've been quite a lot more effective.
There's a similar problem with things like "user interviews." A common pitfall is to ask people which features they want. That has limited use. The real work you need to do is the "creative thinking" that others haven't done. Figure out what people don't know they want; learn what the numbers can't tell you. Then go and build it. Yes, understand the numbers, choose a good business model, and optimize based on those numbers, but don't let the numbers create the product. It's a dead end.
> but they simply aren't going to show me how to create an ingenious idea that takes things to the next level.
Of course not. There's no substitute for straight-up creativity and deep thinking.
But once you have your ingenious idea, you still have to design it, make sure it's clear to users, that they find it and can use it effectively. Your "ingenious idea" may turn out to be largely sabotaged if a button you thought had an intuitive label is misunderstood by 90% of users, or a link you thought was highly visible is being scrolled past by nearly everyone.
Yes, analytics is all about optimizing things to a local maximum. But you might not be anywhere near your local maximum. It's astonishingly easy for the first version of your ingenious idea to only be achieving 5% or 10% of the actual local maximum potential. We shouldn't downplay the difficulty or achievement involved in getting even close to a local maxima.
And you're correct that in user interviews, if you only ask what features they want, you're drastically limiting the value you might uncover. On the other hand, you'd better not ignore the features users are frequently requesting either. A lot of users are pretty smart and know exactly what they need, at least to get to that local maxima.
If you’re just using analytics to look at how effective UI designs are in making business conversions, couldn’t you still measure that by checking the backend and looking for a spike in activity towards the API endpoint that the UI invokes? Couldn’t you measure effectivity with a spike or drop in sales? I mean, good UX doesn’t so much rely on Google Analytics but on a UX engineer’s depth of knowledge about human psychology.
> couldn’t you still measure that by checking the backend and looking for a spike in activity towards the API endpoint that the UI invokes?
You could have multiple UIs hitting the same endpoint. Also, why limit yourself with such crude metrics?
> good UX doesn’t so much rely on Google Analytics but on a UX engineer’s depth of knowledge about human psychology
UX in theory, and UX in application are two different things. You could have the best models of how users will interact with your site, but until you deploy and measure, you have no idea what will happen.
You can still parameterizethe API calls if you want to attribute user activity to a specific flow, and that way you wouldn’t be “feeding the beast” that is GA.
How you mark/report the events is different from where you report them. You could use any one of self-hosted solutions on your own domain instead of GA without changing much the way you report back.
There's plenty of alternatives to Google Analytics, including open-source software you can self-host so it doesn't share your users' private data with a third party. You don't need to roll your own just to avoid GA.
If you are unable to find out that 90 percent of people can't use a major feature that you thought would differentiate your product, you are missing all forms of feedback, including much more important sources besides analytics.
I disagree completely because I've seen it in practice.
A button turns out to be below the fold on common small screen sizes that a new designer forgot to consider. A bad translation results in 90% of users in a particular country misunderstanding something. A JavaScript library doesn't work on a common Android phone you don't test with. Latency issues make something virtually unusable from the other side of the country because of a single badly written function that's easy to rewrite -- but you still have to catch it!
You need ALL forms of feedback. It's not a question of some being more important than others. They're all important and play their own unique roles. Analytics is for catching AND debugging all the things that go wrong at scale in the real world, as opposed to the artificial and limited environments used for user testing.
One goalpost at a time please. 90% of users in "a given country" is not the same things as saying 90 percent of all users. Unless that country is so important that it's everyone, and again you needed to test that your product is usable in the first place.
And even so, while there are a lot of countries and languages, a collection of tests in all of them, which is all it takes for these examples, is not "big data". Again you are crediting analytics for things that people were fully capable of and responsible for doing in the days without traffic data. I realize it is great for the resume and sounds sexier to do "analytics" as opposed to just basic software testing (you should see what people are calling "AI" in other industries these days). And I'm certainly happy to agree that analytics has some value, say in improving wording and stuff that a single instance won't tell you, but again these aren't cases of that.
If you see in your backend metrics, that a button isn't clicked by a huge number of visitors you can start there and analyze. Common screen sizes are helpful, but more helpful is to look at the page since maybe other optimisations can be made, if that is the important button.
Similarly with languages, that you cans er from backend metrics.
More data seems always nice, but analyzing more data isn't making analysis simpler and there is a big privacy impact in analytics, especially when outsourcing to data collectors, who can gather data across sites. (Which then also gives Google information which services are interesting to user and can be integrated into search etc.)
> if you are unable to find out that 90 percent of people can't use a major feature
How would the users know they can't use a feature they don't know it exists?
Let's say you add a brilliant new feature X, but due to a bug the users can't load the code for that feature so they never see it. How would they know to submit a form feedback for a feature they don't know it's there?
Because when you're talking to them directly you ask them about it. This is what I mean by all forms of feedback.
I'm getting the impression that people are just ignoring all advice about communicating with customers in their startups and just throwing stuff out there to see what sticks. Besides being wasteful in doing stuff no one wants anyway, what if that bad feature crippled your product and your paying customers have permanently switched to a competitor the instant your change frustrated them? And now you're bankrupt and can't afford analytics. Relying on analytics as a crutch to catch these things was the mistake in the first place.
Fun story I actually had forgotten about till now: I briefly worked at a tiny startup out of my school in the ending days of the internet bubble, trying to sell a "data mining" software product. Way too soon before it was cool, sadly. It was really hard to make the case that people needed to pay us $100k for the benefits we could get from their data. And companies certainly weren't going to go for a pitch like "if you launch a broken product we will catch that fact". They spend a lot of money to be sure that doesn't happen already. We even had one major customer figure out that they could just have an engineer perform a simple counter over incoming communications that would catch all they needed to know, and hence they didn't need our product anymore. That was kind of the end for us, in fact.
You're essentially arguing for qualitative data instead of quantitative, but both together is usually where the money is. I agree that qualitative analytics are underestimatd because they're hard to do, but I also think that having quantitative analytics together with qualitative allows you to contextualize your numbers in ways that lead to insights you wouldn't have otherwise.
Also, after you've already reached product-market fit, it's important to take your product to its "local maximum".
I think you may be right; after all, thinking critically about my story above, I did spend quite a lot of time learning about analytics and quantitative numbers. It could be this gave me an intuitive sense of what works, which I could then apply to the more creative thinking. I don't know. Either way, I'm grateful to make a living the way I do.
Qualitative data has another challenge - representativeness. It's very easy to do 10 user interviews and feel comfortable that you understand the market. Our brains lie to us all the time.
Quantitative data lets you drill down into different dimensions. Because it is much easier to collect at scale (it's the sum of your users' interaction w/ your product, after all!), it's much easier to make representative decisions.
No it isn’t because the data will never tell you “why” people are doing something or not doing something. You can guess but you’ll never know why until you a) talk to users and b) watch them use your product. Qualitative research isn’t about statistical significance, it’s about deep insights. 10 user interviews will undercover 100 insights.
What makes you think the 100 insights are actually insights, and not simply deriving from confirmation bias?
To be clear, I support qual + quant user research (+ split testing). They are all tools that help influence product direction. I have, however, seen a lot of cases where qualitative research uncovers an insight that doesn't actually match real world behavior when we bake it into the product.
Compare that to a split test, where I don't know why something is happening, but I am able to optimize against goals. It gives less insight but is more foolproof-actionable. With insight, you have to go insight -> improved_action. With testing, you just have the improved action.
To me, qualitative research is crucial to build a model of the environment. These can be used to generate hypotheses, which you then split test. Any time I see someone take direct insight and build a product feature from it, I have a lot of questions about what alternate ideas they tested. And how to know the one that was developed is as close to optimal as you can get, given the fixed resources available.
Apple famously "ignores" its users, partly because users usually can't see far beyond what is in front of them, often because they don't know about impending advances in technology or clever new designs. They'll ask for faster, cheaper versions of what they already have (faster horses, cheaper buggy whips as they say) rather than the next big thing. Faster/cheaper weren't the primary draws of the Mac, iPod, iPhone, iPad, etc. (though price/performance is a big draw of the M1, the big breakthrough is performance/watt which leads to all-day battery life and better thermals.) Instead it was a quantum improvement in design, usability, and functionality combined.
As another example, consider that in 2007 Apple developers were begging for an iPhone SDK, and Steve Jobs crushed their hopes by telling them to just make web apps. A year later Apple came out not just with an iPhone SDK, but with an entire App Store. (Though I suppose some developers [Epic] and users [HN] wish they had just come out with an SDK, and that the iPhone wasn't locked down.)
That being said, they do a lot of user testing of the next big thing before it is revealed publicly.
It's really not that useful for Apple to collect analytics because they make physical products, where the potential of analytics is limited. When it comes to a SaaS or a web page, the possibilities of analytics are much greater.
And yet, Macs will still send usage and performance data to Apple so they can incorporate that information into future product versions and find out about system software issues.
We're pretty deep in a single thread here, but the original article isn't saying "don't use analytics", it's saying "don't use Google Analytics".
Your SaaS or webpage probably gathers 80% of what Google Analytics does in your log files. It could without doubt provide deeper insights that GA is capable of by adding your own behaviour tracking code (which can be written with the understanding of your specific problem dom ain, rather than be an "everything to everybody" generalised solution).
Nobody is suggesting we shouldn't collect usage and performance data, the thesis here is that we shouldn't send all that directly to the worlds most profitable advertising agency, just because they'll draw us some pretty graphs for free.
Well, you can't really opt-out of all analytics. It's really hard to completely stop all server logs, or them tracking how many purchases you make (eg. invoices).
Where exactly do we draw the line to what's core for a company to track in order to run their business and what can be opt-out?
The issue I have with analytics, additional to your (imho) valid points is that lots of practical users seem to always measure the wrong endpoints and don't include negative outcomes.
Leads are useless without measuring how many people bounced off your shitty online shop, because they couldn't understand your UI.
Staying time is useless without measuring closed tabs, because your website is unreadable with ads.
The irony behind it is that ad metrics are used to buy/sell a website's worth for ads. Sometimes I feel like it's like a bubble that's invented on purpose to not have a measurable outcome of anything.
I agree with you in theory but I'll point out you do measure bounce rate and reoccuring vs new visitors and staying time at the same time to give you that exact view.
But I agree they measure the wrong end points or rather they measure too many end points for me to make sense of which one is better. The goal of overall traffic sometimes works against keeping bounce rates low or staying time high.
In the end you would think products sold matters. But an over promising website will over hype and under deliver causing returns or bad reviews which may affect future sales. The message has to describe the product but still meet the market's demand.
> There's a similar problem with things like "user interviews." A common pitfall is to ask people which features they want.
Asking what users want is not the right way to do interviews. I advise you read "Just enough research" by Erika Hall. The goal of interviews is to gather enough data points to understand the user needs and struggle, and to know their journey through the product with accuracy, and also understanding why they use or don't use a function. It is not a way to get a "wish list".
Then, designers usually have tons of tools and methods to process this data, take decisions and try creative solutions (usually more than one), and play them back to the users through prototypes to see which ones work better. You can check design thinking as a starter, but there are many more.
A similar book is The Mom Test. You're not supposed to let the user find how they would've solved the problem (as had they known, they would've solved it themselves), you're supposed to understand their pain points and then design the most effective solution for them.
I've done so many projects that revolved around instrumenting every little click and creating A and B versions of an experience to run tests against and in all that time I can't recall a single useful product decision that was actually informed by the collected data.
I would disagree on user research. It works on certain types of broad questions like messaging and branding. It's probably overused though.
I'm with you in having come around on not trying to cram everything into that hole of testability. But I do think there's room, specifically when it comes to testing your ideas. It's rare to look at data and see what the problem is, but it's common to come up with a hypothesis for what the problem is and a way to experimentally measure whether you new solution has actually made a dent.
Why is analytics only numbers / quantitative data to you? I've used services like HotJar that record the interactions of the user on the site or app which is qualitative analytics at scale, and it helped me identify how the user was actually using the product. I wouldn't not call this analytics simply because it's not numbers driven.
Why would you need consent to track anonymous data from users, without sharing the data to 3rd parties? What's the difference between recording an event "clicked_purchase" and displaying the click event in a graphical way, as a recording or heatmap?
Do you know of any legal document mentioning this, that storing the clicked element is allowed, but not the click position? They can be both used to run the business and improve the product, thus being in the "necessary" category.
Of course not. And unless I'm in the EU, I don't need it (and even then, not really, HotJar can be made GDPR compliant). HN will definitely balk at me saying that, but understanding UX is much more important than whatever philosophies one has about not tracking users. Because if you don't understand UX, there's no point to the debate about tracking vs not tracking users, because you won't have any users in the first place.
Only on HN would you find people seriously arguing that focusing on product development is a good reason to remove analytics from a site. I've worked almost exclusively at companies where the product is the website, and initially this idea struck me as laughably naive. But let me be fair and think through this.
Tech startups do definitely have this problem of focusing on website analytics where the product is NOT a website or app. If we're generous we can assume many people here develop for these kinds of companies. Some waste a lot of time looking for up-and-to-the-right arrows for investors or trying to be data-focused when data about the website isn't actually all that important. Many of these companies might actually be better off with no analytics to waste time on. I'd still argue it's better to check in every once in a while to look for problems and ask yourself some questions.
The idea of removing analytics where the product is an app or a website is silly. This would be like arguing a grocery store shouldn't track what people are buying from their stores, and instead just source good products. You need to do both. What are you going to do when I ask what is or is not working? Tell me your feelings? Shake an 8-ball? Aside from detecting problems, analytics can be a jumping off point for innovation if you're smart about it. What can we do that's more like what's working? How can we improve this page type?
There are for sure people who over-focus on analytics (often on the wrong data points) instead of creativity, but these are not mutually exclusive. If I were to list the millions of dollars I've earned and saved via analytics this would be a very long post. Sadly, most of those millions were for other people, but it's a very valuable tool for optimizing and creating if you use it correctly.
You could talk to your users... (i.e. UX research), observe them using your website or product and ask them non leading probing questions to see what the intent is behind the behaviors affecting your bottom line. Qualitative research methods are a rich source of insight that is typically underinvested and underutilized.
Analytics (quantitative data) can help you find areas of bottlenecks to explore further by doing qualitative user research and getting to the ‘why’ behind the people problems standing in the way of your metrics you are tracking (retention, adoption, etc.). This is called ‘triangulation’ using quant and qual research methods to understand your users more deeply than just looking at data can achieve.
This is a very important point, thanks for saying this. It’s amazing how many people think they can start an indie SaaS and think that all they have to do is build it, deploy it, and buy AdWords or whatever. Talking to your customers is more important than any of that, and it’s fun.
UX research participants are usually compensated, and so it's usually done to drill down deeper into problems that analytics found and test possible explainations. I don't see what's unscientific about that.
Could be that, it could be design, it could be how something is worded, it could be that what you want the client to see, isn’t being seen. Analytics helps to identify this problem. For instance I noticed recently a huge drop-off in visitors from Ipads, and a redesign apparently made some parts of the site dysfunctional for some ipad users, something we didn’t catch earlier.
> You can't focus on the product without analytics
This is provably false. You do not need intrusive analytics to develop fantastic products.
Have people somehow forgotten about good old-fashioned user testing? It is expensive, time consuming, and amazingly effective. Most importantly you can actually talk to your users because they are people instead of data points.
But then surely, adding a prayer to St. Isidore of Seville [0] before every release will be better than just the both of them, so three > both > one.
I've recently interacted with a startup where the amount of resources they spend on trying to get them both, is making them blind to the power of one. It's not a pretty sight when all the numbers are tracked, plotted and planned on, yet nothing seems to work.
You can misuse everything. As you mentioned: "all the numbers are tracked, plotted and planned on, yet nothing seems to work." Sounds like there's a plan but no execution / follow up. That's not a problem with either analytics or user feedback.
Out-of-the-box, no. With custom work, yes. The amount of effort is not large, and there are off-the-shelf solutions available. The name commonly used is "engagement timer."
By default, GA only sends one hit on page load. If there's no second hit, there's no way to tell if someone was looking at the page for a second or a minute or an hour.
In theory the correct way to send analytics when leaving the page is via navigator.sendBeacon, although it's not clear how reliable it is: https://volument.com/blog/sendbeacon-is-broken (note: read the comments for some rebuttals to the article)
But only if the user goes on to visit another page. You don't see how much time they spent on last page they visited, even if they only visit one page, by default.
This is not the same kind of analytics. You are talking about something like HotJar with heat maps to find out how customers use the product for example.
No, I'm talking about Google Analytics which is the topic of the article and parent comment.
Heat maps are great too but Google Analytics is still used as the foundation for figuring out which types of users are clicking and not clicking on what, both in isolation and as part of a pathway between elements/pages/etc.
Plug: If you're interested I have built a tool to solve this need of having both qualitative data (heatmaps, recordings) and quantitative data (stats, segments), plus it's self-hosted for highest privacy: https://usertrack.net
I worked for a tech company popular with enthusiasts when GDPR was first rolled out. We had a lot of requests from users who wanted us to provide their data per the GDPR allowance. We also had an influx of tech journalists filing GDPR requests in hopes of catching us doing something wrong or tracking too much personal data.
When we sent users their "data", many of them were in disbelief at how little data they received. Many had come to believe that all tech companies are secretly building inventories of user data to sell to 3rd parties, when really most of us just want to know if our heavy users of Feature A are also heavy users of Feature B, or if Feature C is more popular with new users but not old users.
The strange part is that tech companies are taking the brunt of the bad PR for things like gathering customer feedback and serving relevant ads, while traditional companies like cell phone providers and credit card companies are actually selling customer data. The latter doesn't get enough attention despite being a much more widespread issue.
Facebook doesn't sell your data, but your phone provider and credit card company probably do. But ask the average person who's selling their data, and Facebook will get all the blame.
I think Facebook gets the blame because they are at the end of the chain. They don't sell your data but they do sell the access to your data (from themselves through the website or tracking and what they bought from said other companies). Being that major player in the service it makes sense that they get the heat, but at the same time most people are still tech illiterate. I mean look at how people think Amazon is only a retail company.
I have never been inside a newsroom, so I can't know for sure, but I suspect facebook also gets a fair amount of the blame from news media because of their fraught relationship as pseudo-competitors.
Huh, thank you. Obviously the people on the ground don't, but there's no truth to the idea of what stories are greenlit by editors?
Could be as simple as occasionally removing mentions of companies that advertise with the paper in peices about customer data creating the effect, not J. J. Jamison telling people to get him pictures of zuckerberg :D
Relevant ads are the kind you get on DuckDuckGo: relevant to the content you're looking at. E.g. if you look at a site about origami, you get ads from arts&crafts supply stores.
What you've probably meant are called predatory ads: they chase you around the Web wherever you go, like a predator chases its prey.
Statistically speaking, you're more likely to buy a second object right after you bought one than someone who has not shown interest in the product. Things break, you might want to return it for a slightly different version, you might buy one for a friend.
For you, it might be wrong, but when the advertiser is buying millions of ad impressions and is looking for a 0.01% hit rate, the math shows that you're one of the more likely future customers.
Which makes at least a little sense when I'm on another site. But just this morning, Amazon started "recommending" a product to me that I'd actually bought from Amazon two months ago. How many printers does Amazon think I need?
Of course, as you note, printer companies realized people were actually doing this (due to the perverse incentives created by their ridiculous razors/blades business model) so now most printers come with pathetic "starter" ink cartridges that run out after about 10 pages.
The irritating bit though is that many cartridges don't last long (intentionally and unintentionally) once you open/install them, regardless of whether you actually print anything. So, if you don't print a lot, it might still make (financial) sense to buy a new printer... ;-[
I am tempted though to get a model with refillable ink tanks, but most of what little printing I do is on an old b/w laser printer which I've had forever. Still looking for the holy grail color laser printer that is cheap, networked, duplexing, compact, and can print photos.
If "the externalities are not fully priced in", isn't that just good for me? I hate to say it, but my financial situation hasn't been great recently; I might just go for it.
Before the days of the web, we had a solution for this that was better and easier for users and didn't invade privacy: UI design guidelines and UI consistency.
Apple is the only company that still occasionally does this, though even they have driven off into the realm of every application having its own entirely novel interface. Go back in time to MacOS or Windows in the 1990s and you'll find an entirely different paradigm: every application has the same interface... or at least the same interface paradigms. Learn the computer once and you've learned the computer.
Features were remarkably discoverable. They were organized logically in menus. Keyboard shortcuts were always available and usually intuitive. I remember opening a new app I'd never used before on Windows 95 and just unthinkingly hitting a keyboard combo and it doing the general thing I expected, or mousing to where I expected to find a feature I imagined should be there only to find that it actually was there. I never used Mac Classic much but I heard it was similar.
The web is what really broke this. Web UIs overtook desktop UIs due to the difficulty of installing local software and the power of trendiness. Web UIs were never uniform and couldn't be since the web was anarchistic and wild and often driven by designers who wanted to make their product look a specific distinct way.
I remember in the days immediately before the web there being talk of algorithmic generation of UIs from data schema. If the UI is thoroughly standardized then it becomes at least thinkable to examine data structures and generate user interfaces from them, even UIs that aren't horribly ugly or hard to use. This was building on a previous generation of incredible WYSIWYG UI design tools. Then the web came and all that stuff was completely abandoned.
Today's UI design tools in things like Xcode and Android Studio are horrible by comparison to what people were using in 1995. Go back and try Visual Basic on Windows 95. The VB language sucked but the UI designer was aeons ahead of anything we use today.
> Analytics isn't just about conversion. Analytics is about the entire product experience.
... which you can entirely do by analyzing web server logs.
You dont need google for this, pushing your users (with violation of GDPR consent) into google monstrosity. Dont do that, I block every freaking google domain from cdns, fonts to analytics.
But I don't block 1st party analytics.
I have no reason to. I have visited your site, I dont have anything against YOU following what I read. It is your site. But you will ask me for consent for giving those data to google. And I will say 'no'. And I am not the only one.
Just to inform you, that the person/entity that allows Google to gain access to PII data is directly responsible for this - if google is fined due to GDPR violation, you can get fined to by providing it the way to get users data. They will survive. You might not.
Have your analytics, but you will not sell my soul (which GDPR explicitly forbids you - you are handing over my PII data to 3rd party that is known for violating it and that makes you accomplice) for you having your graphs.
You can get those data from web server logs. You will have all the data that you need. Actually more data, as no one will block them.
Needing "google analytics" is just a huge, giant, hype driven, lie. You don't need them to analyze what you already have in YOUR logs.
chaos_emergent: please do explain, what data google analytics is offering to you than what is already in your server logs? Without violating GDPR even more? Yes, you can surely track something more, again "on your side". Dont give it to google as it WILL get blocked and you will have a distorted picture of how your site is being used. If you want real data, skip 3rd party analytics. Found a way to require to be unblocked? I will skip your site, you have just lost a user. A paying user if the content is worth the money. And sites with selling my data for a graph or two are not worth it.
I agree that people shouldn't be using Google Analytics. I disagree that people should just rely on their web server logs. Products are more than the data that is being accessed on them - copy and design make a product usable and don't show up on web servers. Am I missing something?
FWIW, there are many well known techniques for shipping data to GA regardless if you block it or not. Many integrate server-side for this reason (as you say - server logs are very rich), and client side is used supplementally. Some are using sneaky techniques to move requests through 1P domains. Adblockers make zero difference.
> Some are using sneaky techniques to move requests through 1P domains
Just for an info, I have written a mitming proxy that takes care of those (cname cloaking) and a lot of other things, including fingerprinting, supercookies etc., with support for various blocklist (domain to adblock), js injection where the injected scripts are handled in same manner as blocklist (you can stockpile them and make rules), changing validity of cookies (to session cookies for instance), saves your data trough highly effective caching and even helps spying cdns save some bandwidth as they are mostly no longer visited. And yes, works as transparent proxy too (for router enforced usage). Imagine fully armored firefox for whole network, regardless of browser, freebsd,linux,windows,arm8. Working on cross platform gui, release soon.
Yes, but analytics is also used to trick folks into buying a paper back full of shit.
A good product, that does what it advertises, does not need analytics. But a bad product, that somebody desperately wants to make successful, or at least successful enough to sell to a PE and exit, needs analytics.
The problem is that when making a product you're often wrong, what you think is a good product is often a bad product, or it's a nearly good product with a couple of fatal flaws that can only be seen in hindsight. S
While some people have an uncanny sense of vision, and seem hit on the right ingredients more often than seems fair, but most companies aren't led by this kind of person.
You need things that tell you when and how to course-correct, this is what analytics gives you. Now, of course, this needs to be balanced against privacy concerns. I push back on things that track literally everything (the tools that record every click and cursor movement are fascinating, but undeniably creep), and I try to avoid sending any PII to 3rd-parties. The amount of stuff Google Analytics phones home about by default is also pretty troubling.
I'm on board with basically every privacy-based criticism of tracking, but I don't buy this argument that only bad products benefit from it.
> the tools that record every click and cursor movement are fascinating, but undeniably creep
That may not be so for a game developer who wants to modify gameplay that is heavy or reliant on things like particular mouse cursor movement and mouse click usage. Especially if a meticulous gameplay goal is the objective.
"If they're performing a task quickly because it's easy, or slowly because they're struggling with the UX. And you find out that customers on a certain mobile device are suffering huge performance issues, for example."
...
"You don't know these things until you measure them."
If you don't build big, bloated tools using ultra-high-level frameworks and if your product is a simple tool that performs a single, useful task ... then you do know these things and you don't need analytics to tell you.
>Here's a crazy idea: Don't use analytics at all but focus on your product. If your success relies solely on "improving conversions" by tracking your users and then changing the position and color of your "Checkout" button then maybe try setting yourself apart such that customers want to buy your product even despite an obnoxious purchasing flow. Only then start optimizing it.
This is part of why costco is so successful. When I buy a kirkland brand item, I know with like 95% confidence that I'm getting a quality product. Not only that, but they go through heroic lengths to vet the other products that they put out. If you buy extra virgin olive oil at costco, it's very likely that it's pure extra virgin olive oil.
They turned retail on its head after 30 years of abuses by people focused on quarterly earnings and selling their brand into the ground. Instead of making the product as shitty as possible and charging as much as possible, costco hard caps their margins and will then invest money to optimize their suppliers manufacturing process to pass the savings along to their customers.
The big thing with costco is trust. I trust them and their products because they've earned it. In the rare event something is wrong with their product, they'll make it right with basically no questions asked. They used to do this to an absurd degree until people started abusing it.
Compare that to the amount of vetting I have to do for almost every amazon purchase now. It's a huge headache, and there's a lot of stuff I just won't buy off of amazon anymore.
Absolutely. Even though I find the Costco in-store experience stressful and the online experience lacklustre, I put them at the top of my list for all of my non-fashion shopping (unless I'm looking for dad clothes). I can always trust their products will be as high or better quality than anywhere else, and at the best or close to best price. And I have no worries about returns.
Compare that to Amazon, which I have zero trust in. I absolutely can't trust the reviews, and I can't trust any products are genuine (even for minor things: my last purchase several months ago were steel wool dish scrubs - name branded, but I'm certain they were fake). I also don't trust them to do anything about it, because I've reported fake reviews, and fake products several times, and all of those sellers are still selling with thousands of 5 star reviews. The only thing it has going for it is price and convenience.
The latest trend on Amazon seems to be selling items packaged for non-us markets and shipping them to us customers. I bought a two pack Duracell lr44 battery and I think the entire packaging is in Turkish? Or maybe some Eastern European language. Regardless, the battery is definitely not packaged in a way that’s legal to sell in America and the Amazon page did not say it would be packaged for another country. Super shady. And I’m not even sure how to tell if it’s not 100% fake and not just the packing for another country.
It would never occur to me to care, at all, whether something as trivial as a dish scrubber was a genuine name-brand item. What difference does it make? I suppose this is why I have experienced none of the trust issues people have begun talking about with Amazon, recently.
I think you're missing the point. The brand is irrelevant. The point is if they are faking the brand, you can no longer trust the integrity of the product at all.
In this case, it began deteriorating with steel 'hairs' coming undone immediately after first use. Should I be concerned about that? Is it even steel wool or another material? If the latter, is it safe and tested against items that humans will be consuming food from? Is it sterile? Were some other chemicals used to treat it for something as trivial as attempting to match the colour of the brand?
I echo your sentiments on Costco but this is an argument for different metrics, not an argument against analytics.
There is no chance that Costco doesn't use analytics. They might not A/B test their online button colors for highest conversion rate, but I'd bet they have their own set of analytics to determine product quality, sales, returns, viability, etc.
This, thank you. I feel like people are making big assumptions about what "analytics" are that don't even begin to cover the whole spectrum of analytics.
> Here's a crazy idea: Don't use analytics at all but focus on your product. If your success relies solely on "improving conversions" by tracking your users and then changing the position and color of your "Checkout" button then maybe try setting yourself apart
I am very much onboard with this idea. The notion that it is a requirement that businesses track individual customers’ actions in order to succeed is pervasive in this newly-connected world we inhabit. It feels too early in my life to be a grumpy old man but I do certainly feel like it sometimes. Brick and mortar retailers manage to reach an acceptable level of business without watching exactly how every customer looks at shelves by just selling things people want or need and I don’t see why the internet should be much different. Surveillance just because we can is not something I like.
Disclaimer: I hate Google Analytics because I hate Google, I hate monopolies, and I hate excessive hoarding of user data. I think the necessary aspects of tracking can be maintained without having all the bad aspects, but we need to break up the ad companies first.
> without watching exactly how every customer looks at shelves
This is absolutely not true. Do you have experience with managing brick and mortar stores? Ever been to a grocery store with a "discount"/rewards program? That's their tracking of individual behavior.
They also use credit card data for the same purpose, although CC data is less reliable than rewards cards.
Coupons accomplish the same thing. You put out a specific coupon code for each newspaper/circular/TV ad, and then you see how they convert.
Things like Google Analytics are just the web version of things that have been done for almost 100 years.
Rewards programs are opt-in. Stores have even stopped harassing me to sign up.
Analytics are the equivalent of a computer following you around the store, watching what you look at, how long you look at it, what you pick up, what you pull out of your cart and put back...
I’ve never in my life gotten message from the grocery store saying, “Hey, we saw you were looking at grapefruit. Here are some other citrus fruits we think you might like.”
I’ve never in my life had a store send me a message offering to sell me all the items I abandon in my cart on my last trip.
There is certainly data brick and mortar are looking at and it is often intrusive and creepy. Still, they weren’t doing this a few decades ago and they were fine. They’re pointing at online stores and saying “But it’s the only way we can survive!” It’s almost bizarre to point back at the more expensive, often first-party controlled, less accurate solutions of brick and mortars and say, “But they’re doing it too!”
(This problem is compounded when this data is given freely to a third party who now has much more data than even the individual stores or websites.)
All this adds no benefit to me as a customer. Lower prices are not a benefit if I’m also being psychologically influenced to spend those savings and more on something else.
Online or offline, I don’t care. Stop doing it. It’s creepy and unethical. It’s a waste of resources that could go towards giving me better products or a better experience. It’s the equivalent of cops asking for back doors in encryption schemes because it makes their jobs easier. If businesses can’t find a way to stop doing it themselves than I think maybe we need some regulation.
> Analytics are the equivalent of a computer following you around the store, watching what you look at, how long you look at it, what you pick up, what you pull out of your cart and put back...
I hate to break it to you, but that's already happening too. Many retailers are using NFC/RFID + door scanners combined with CCTV and computer vision to track exactly these sorts of things, as well as patterns of flow. Retail store layout is a critical part of product placement optimization and is used to create particular flows through the store. The most blatant example of store layout controlling flow is how an IKEA is designed, however these things are used very heavily in grocery and mixed retail spaces (Walmart, Target).
> Still, they weren’t doing this a few decades ago and they were fine.
This part at least isn't really true. Since grocery stores have been around, there have been people working full time in effectively analytics - observing customer behavior, modifying products/layouts/UX, doing a/b testings etc. Digitizing everything gives new tools and approaches but overall the game hasn't really changed.
Every part of your experience in an grocery store has been analyzed and tweaked since at least the 60s.
I disagree, and I would prefer the entirety of my phrasing be quoted:
> Brick and mortar retailers manage to reach an acceptable level of business without watching exactly how every customer looks at shelves by just selling things people want or need [...]
I am referring to the enormous number of independent retailers, not just the small number of ultra-wealthy retail giants. Note I was careful to say "acceptable level of business" rather than "absolutely maximised profits", because I don't necessarily agree that it is a requirement that all people make as much money as they can possibly manage.
If you were to make the argument that monitoring customers' actions is a means to make more money, I wouldn't disagree. I just don't think that all businesses will fail miserably without it.
> Do you have experience with managing brick and mortar stores?
Yes. They're small shops that serve a well-understood need for the local population and produce a sustainable income for everyone involved. No need to do much more than that for me.
I think classifying grocery store operations as frontend/backend is too far from reality to be a useful analogy. That’s not to say that I don’t agree that google analytics is more insidious than just about anything grocery stores do.
Tracking individual users is how Google Analytics works. Every hit has a "client ID" to tie together hits that came from the same browser. You can trace the actions of an individual in the "User Explorer Report." Although in practice, that's only useful for debugging.
The terms of service prevent you from putting PII into Google Analytics data. It is perfectly acceptable (and even encouraged) to put in an opaque identifier, which connects to PII stored in a different system. That is, for example, how you implement the official integration between Google Analytics and Salesforce CRM.
When I first put my personal website up on Github Pages last summer, I didn't include any analytics. I figured it was unnecessary—why should I care who was looking?
I quickly realized the obvious—without any analytics, I had no idea whether or not I was just screaming into the ether. Even for a simple noncommercial site, it's discomforting!
I now have Cloudflare Analytics and I'm much more satisfied. I feel as though I'm respecting my users's privacy, while also getting a basic sense of traffic.
You definitely don't need client-side anaylitics to know if you content is being viewed. I don't know if Cloudflare Analytics is that but a simple view counter on the backend (ie a log analyzer) is enough for that. Or just search for your site to see if people link to it.
> a simple view counter on the backend (ie a log analyzer) is enough for that.
I can't do that on Github Pages.
Cloudflare Analytics gives me a tad more information than raw page views—for instance, I can see what sites people came from. But there's zero individual user tracking.
A log counter on the backend will report the 1000 bots that visited your site as 1000 page views. But that won't give you any information on how well your posts did.
Bots are the reason I switched back from a free backend db based solution to google analytics.
> If your success relies solely on "improving conversions" by tracking your users and then changing the position and color of your "Checkout" button
The problem is typically that you don't know if the color of your checkout button is a problem. Without some level of analytics you are only guessing as to what is driving your customers away and if your customers aren't educated engineers with comfortable incomes, you are probably going to guess wrong.
I've worked with analytics and I've often been surprised at where customers run into trouble.
I mostly run into problems in webstores because they just can't run without pulling javascripts and iframes from 50 different domains. uMatrix blocks them all and I can't be bothered to figure out the absolute minimum to allow to get to order so I just leave. Do I even show up in your google analytics?
I'm totally on board with not obsessing over user behavior or tweaking unrelated things in hopes of a revenue boost, but I prefer the scientific approach, which means I still want metrics. I'd like to know when I make a product change whether that improves the number of people who see my product and stick around versus seeing and leaving.
I don't have anything live that uses Google Analytics but I've used it once or twice in the past, and the primary thing they got right is that it's just so dang easy, and I'm almost guaranteed to have the data I want. I'd so much rather support an open source product that does the same, though.
Tracking is not just about improving conversions. It's primarily about understanding if your ads worked. It doesn't matter how good your product is if you can't tell anyone about it, and you can't spend a bunch of money telling people about it if you don't know which ad networks are giving you bang for your buck.
Yeah, as a bootstrapper on a shoestring budget I'm not gonna dump a load of money into eg LinkedIn ads and just hope and pray that something happened. I need to know if they worked.
>As long as your product is good enough, no one needs a marketing budget or to measure if marketing is working.
Let's say you make an incredible mousetrap, better than any before. Then you tell some people about it, they buy it, and stop thinking about it because it isn't central to their existence.
You've now saturated your market and have no ability to expand easily without putting effort into marketing or advertising. How do you go from there to 10 million units sold without a marketing budget of any kind?
What about if the package is extremely off-putting to people outside your culture or if the language on it is confusing. How do you know without measuring?
I used to think that way. My customers liked my products, but I didn't really get many new customers. I doubled down on improving the technology. Then I ran out of money.
"If you build it, they will come" is only very rarely true, and chances are there was some kind of submarine marketing going on anyway that you just didn't know about.
I wonder what makes nerds like us end up with that opinion.
Surely anyone that reaches adulthood ought to know that selling yourself well and having some damn good looks will bring your farther than just being the real deal?
It takes some huge lack of awareness of one's surrounding not to notice it
While developing your own analytics seems like overkill, you definitely can host your own analytics, using one of the many solutions where all the analytics data are kept on your own servers.
You do realize that SEO to rank in search results and posting on social media (or hacker news) about your product is marketing? Without marketing how do people find your product?
I used to be a very "analytics focused" product manager until I joined an enterprise software company that hardly uses them and is wildly successful.
We're succesful because we talk to our users about everything. I spend most of my time talking to customers and watching them use the software. We occasionally use analytics to help us validate hypotheses or assumptions, but that's always complimented with a full range of qualitative methods.
Analytics can help with observation, but it'll never give you the "why". In my experience only observation and a lot of conversations will get you there.
Enterprise-focused companies are often some of the most user-hostile, opaque organizations in the world. I would rather a company post prices and documentation openly (with analytics) rather than requiring me to sign up for at least an account, and more often, a one-hour sales call to get any information at all.
Analytics revealed to me that a product I'd built (well, a free side project, but that's beside the point) was succeeding in the exact opposite of the market I'd built it for.
I used to run an online Japanese-English dictionary. It was geared towards people learning Japanese. I built it when I was studying Japanese and thought it would be useful for other students.
Turns out the overwhelming majority of the traffic to that site was from... Japan! I had it completely backwards! People weren't using it to aid their Japanese studies; they were using it to aid their English studies. (You might be tempted to suggest that it was in fact being used by people learning Japanese, and that those people were taking part in exchange/immersion programs, but there were other data that disproved this hypothesis.)
I never would've realized that if I hadn't had analytics on there.
And? What does this trivia change about how you interact with that project? Is it not possible that there were other ways you could learn about this eventually?
> Here's a crazy idea: Don't use analytics at all but focus on your product. If your success relies solely on "improving conversions" by tracking your users and then changing the position and color of your "Checkout" button then maybe try setting yourself apart such that customers want to buy your product even despite an obnoxious purchasing flow. Only then start optimizing it.
Here’s crazy idea for early humans - don’t use fire to cook your food. If your health depends on cooking meat, instead of hunting only for the healthy, bacteria and parasite free ones, then you should first focus on getting only highest quality meat, and only then figure out what to do with it.
Your analogy is not applicable. A better example would be... Well better. Perhaps:
"Here's a crazy idea for early humans - don't map out the location and habits of all the food that you hunt to maximize the number you kill, instead focus on improving the tools that you hunt with so you can reliably kill what you need. Only focus on maximizing kills when you start needing more food than you can find."
One sentence has decimated my digital marketing career and I honestly couldn't agree more. I think this has something to do with more than just Google itself though and good old capitalism.
There are heaps of SaaS platforms out there (from my last point of reference there were 2000+ MarTech companies, I would guess double that now) that focuses on; A/B testing, email marketing automation, customer success tools, heat mapping and much more. They have funders who want their returns, one way or another. Which then leads the marketing team of the SaaS to develop growth hacking articles which startups tend to absorb.
As a marketer you are backed into a corner of having to test everything because there are so many articles out there showing us how A/B testing a button from 17px to 18px increased sales by 50%. Or this genius new AI content tool that can swap things around for each and every user to match up with their purchase intent. It's gambling. There is data and some poor calculations that lead you astray hoping for that quick win. You will also find that one 'unicorn' SaaS will also dictate the UI/UX for the vast majority of others out there, look at Intercom which basically has been cloned in design across the board.
Focusing so hardcore on metrics can also lead to the loss of a curated product's edge. If you just keep following what the lemmings do you would end up falling off a cliff eventually, it often takes domain knowledge and experience to make educated opinions about how a product should move forward.
Selfishly, I hope you do stay in digital marketing, and be the change I want to see. Ad-tech needs some sanity and reality checks, and I hear a rumbling in the deep around ethical advertising practices.
Ad-tech is not just feeling more manipulative by the year, it's also feeling more and more like snake oil to your average business. I feel like there's a niche opening up for honest feeling, more simple online advertising networks.
Exactly, I've worked for a couple of start-ups that focused completely on growth hacking. Product was 2nd simply because of tech debt build up. A lot of people assume reading GA data you can unearth these hidden gems, it's nothing more than gambling.
The issue with me being the reality checker is that I am considered 'negative', 2 years down the line after I'm long gone and reality bites, I look like some sort of messiah. Damned if I do, damned if I don't.
Doing my SEO volunteer work with small shop owners in the UK during lockdown opened my eyes. As you said the average business is hammered with SEO, PPC, growth tips (snake oil) to the point you have to try it. But they don't have the time so it's half baked mostly and they feel disheartened. Honestly, ad-tech is nothing more than going to the casino and thinking you can win against the house.
You are also right on the niche ad-tech. I want to discover new products and brands again, not get shown the same product over and over again because I used a specific keyword or bought that product 3 months ago.
This is something I will ponder on as it can potentially lead me down the path of improving digital marketing without banging my head against a wall! Less talking more doing as they say.
This is exactly the same thought I came to with browserless.io. There simply wasn’t enough traffic to make informed decisions, and when there was it was really silly things (small copy changes and the like).
Eventually we just tore it all out, and never looked back. Improving the product and blogging about our findings are a win-win for us and the ecosystem at large, versus agonizing over traffic and data
Another headache induced by analytics is how one will sometimes need to discard an efficient and effective feature design and build it differently in order to be able to track use of the feature properly. It wouldn't surprise me in the least if the amount of time and energy burned on implementing analytics equals or exceeds that of the work on the feature it's tracking.
Let's think about it. Are games doing analytics? Games are essentially UIs that people pay for the privilege to use.
Although I am sure that contemporary game makers use analytics to understand user behaviour and optimize for in game spending and engagement, at least in the bigger games probably the core experience comes from creative human processes.
The more the analytics the further optimised the game would be towards KPI.
Also, I suspect that Netflix is creating it's materials based on analytics rather than creative human input.
Maybe the problem is not analytics but greed and ill chosen KPI? Pre-total-tracking world, creatives still needed to test ideas and to test ideas you need to be able to measure. They would pay attention to what sells, how people react to a specific line etc.
Maybe it was more fun because it was less optimised for profit?
A prime example of analytics driving games is Slay the Spire, where they would record every decision you made and outcomes in the game to better balance the experience and give the player interesting decisions.
Slay the Spire does not have subscriptions, loot boxes, etc. so it can optimize for balance, instead of engagement or play time. It's very much the exception.
The risk of analytics is an erosion of your design and vision. You just do whatever makes the numbers go up in the short term.
I'm sure Portal was heavily playtested but it hasn't significantly changed after release so there is no reason to believe that they are making changes based on player analytics (even if they are collecting at least some via achievements).
What do you think half of the achievements are for in a game - a very primitive form of analytics. There is usually an achievement for making it past the first level or prologue, there's another one for finishing the game, and probably many more for passing stages of the game. These are all to see how many people progress that far in the game.
Apart from that there's also crash telemetry and other event based tracking included in games.
I think for companies like Activision, EA, analytics drive a massive amount of their decision making.
I would even go so far as to posit that Blizzard (a master of psychological manipulation), most decisions are driven by analytics based optimization for engagement FIRST, then mechanics and creative design get to come play.
I have no evidence, nor am I an insider, just an observer and scholar of games.
I agree. I dropped all analytics for my blog and sideprojects, thoufh I found out that for my personal motivation I really like some stats about user engagement.
I could use the access logs from the webserver. Can anyone recommend a tool that post-processes those logs in a "analytics" like way?
> Here's a crazy idea: Don't use analytics at all but focus on your product.
Here's even crazier one: Just do analytics on the backend instead?!
You don't need GA, nor 3rd party tracking cookies, just a simple session ID and a proper web-server's log analyzer, and you can get almost all of the same metrics.
You can nowadays even sniff on clients' screen resolutions and other browser details using just img srcset, css and log analyzers.
The choice isn't GA or no analytics. You can do local server side analytics yourself if you wanted to, and even buy an on-prem product from someone out there that does the same.
They’d probably steal your data and peek into it regardless.
My fatalistic outlook is justified by the amount of documented abuses they’ve committed over the years. Nothing is sacred to them, except the idea that “more data (in our servers) is good”.
This is so reductionist I really can't believe this is the top comment. What if your "product" is a web or app service that depends on user interactions? How can you actually know what your users are doing or how they're using your product if you can't track it?
I removed Google Analytics from https://ooer.com because it was taking my lighthouse performance score from 100 down to 99, and I wasn't using the data for anything anyway.
Agreed. I'm building https://increment.me and we specifically avoid web analytics and 3rd party cookies entirely. We don't keep data that doesn't serve a purpose and don't sell our data to anyone. For feedback, we use Increment itself to gather feedback directly from customers.
Having customers directly give feedback is one great signal that really works, particularly when you demonstrate your commitment to action it. When you combine this with a 1st party view of how the product is used like from ephemeral log data, you can get a great pulse on how well you're helping your customers get the most value from you - and how you can adapt to help them more.
I wish every business I interacted with had the same philosophy in creating value for customers and building trust in every interaction.
>>Here's a crazy idea: Don't use analytics at all but focus on your product. If your success relies solely on "improving conversions" by tracking your users and then changing the position and color of your "Checkout" button then maybe try setting yourself apart such that customers want to buy your product even despite an obnoxious purchasing flow. Only then start optimizing it.
YUP!!
Any company, especially startups, should treat analytics they way they should treat MBAs - as a plausibly useful sub-function *after everything else in the product/service is running well at scale*.
Before that, the entire focus should be on the product and how it gets smoothly to the customer.
Only when there is lots of extra sales and production capacity, and lots of extra cash piling up, THEN is the time to start adding financial guys to efficiently manage it, and analytics to optimize your channels, etc.
Plus, NEVER let either of those tails wag the dog. Once a company's financial numbers start to rely on the finance department, or the sales numbers start to rely on channel optimization, the death spiral has started. It may take a while and look like an improvement at first (e.g., see GE), but it is still a death spiral.
More serious thoughts: Google Analytics introduces performance overhead for your website and now you have to explain to your users which third party is responsible for processing their data on top of yourself. Why introduce those headaches? Are the insights from Analytics really valuable enough to justify the cost? I personally haven't seen it.