January 2024 archive

Hey Google! Why is your support so broken? Fix it!

Happy New Year to All. As one of the IT leaders of an institution that was an early adopter, I have thoroughly enjoyed our relationship with Google. There was a time in the past when Google paid special attention to Higher Ed and we developed some strong professional connections, some of which continue to date. Most of the folks we knew have moved on and privately they would tell me that the reason for the move is that the current Google is not what it used to be. Totally understandable, because this is what happens to corporations and they have constant financial pressures and they have to make changes. But, frankly, this should not be at the expense of loyal and long term customers like Higher Eds and K-12s!

Our current issue with Google support has to do with Gmail and Photos showing the total storage usage of the domain on each user’s account, instead of their individual storage usage. Photos also displays a red warning bar at the top due to our domain having less than 20% of storage left in our newly enforced domain storage limit. It has taken six days of numerous techs and emails to finally get them to understand that it is a bug and not a user, account, or browser display issue. They have finally escalated it to their engineering team but with no ETA on resolution. But to get to this, I would like to share some back story (pretty long), so, please hang in with me.

As you know, Google initially provided free services to K-12 and Higher Eds. We were told that the founders strongly believed in giving back to educational institutions that had a strong role to play in making them who they were. We would repeatedly ask if this is a sustainable model and we were repeatedly assured yes, nothing to worry, because, look, even after giving you all these services free, the company is still financially flourishing. And part of this deal was that we didn’t have to worry about how much data we are storing in Google.

The thing about Google was, pretty much everything just worked! Initially at least, there was no concept of “Google support”. Services were free and we basically were on our own and consulted their documentation when we needed help. But there were multiple groups we network with who are Google customers and we relied on each other for assistance. When there were some catastrophic failures, we contacted our Higher Ed contact in Google directly for assistance and they internally escalated these. Obviously, we all know now that this was not a sustainable model.

But in the past 5 years or so, we were directed to seek help from Google support. Google has been rolling out a lot of new features and our need to rely on trusted sources has been increasing. So, I would say, our reliance on Google support has been steadily increasing over the past 5 years. Our overall experience with Google support has been dismal.

Why? Because many support folks responding to our tickets simply do not understand what the issues are and provide canned answers. I get the feeling that they do not understand the Higher Ed environment and issues. We need to escalate many of our tickets multiple times to get to someone who understands and answers our question and pray that they will be reachable if needed. Many times you are so frustrated that you simply give up, if it is some new feature that you want to adopt! We revisit such a list of unfinished new features adoption often! This is NOT how this is supposed to work.

In 2021, Google dropped a big bomb! Specifically read the section “Introducing a new storage policy”.

  • “we’ve grown to serve more schools and universities each year, storage consumption has also rapidly accelerated” – Understandable
  • “Storage is not being consumed equitably across — nor within — institutions, and school leaders often don’t have the tools they need to manage this” – Well, when you had told us that everyone has unlimited storage, it is natural for this to happen, so why is anyone surprised? And we didn’t have any tools to manage it because you, Google, never provided any!
  • “… we will be implementing a new pooled storage model and helping admins and school leaders manage their storage. Nothing is changing today and we expect that more than 99% of institutions will be within the pooled storage provided by the new policy.”  – We all appreciated that we had some time to deal with this! Thank you Google for being somewhat considerate.
  • “The new storage model will provide schools and universities with a baseline of 100TB of pooled cloud storage shared across all of your users” – This is the kicker because we have a LOT more. I guess we are in the 1%!!!

Why is this a problem for us?

  • Coming up with a single number to apply across everyone is easy and convenient for Google, but for a company that takes pride in data analysis, this is SO flawed.
  • We argued that Google should set these limits based on additional criteria, one of which is how long a customer has been using their unlimited storage offer.
  • When you gave away unlimited storage (and marketed it heavily) and did not provide any tools to monitor usage, you basically encouraged certain behavior on the part of our users. Google folks used to proudly say “You don’t have to delete anything! Use search to find stuff!”
  • Secondly, we understand that this has resulted in an uncontrolled use of resources and something needs to be done and I am truly thankful that they gave us a couple of years to deal with it. But 12 years of accumulated content is way too much to trim down to 100 TB.
  • Every single method we came up with was met with use cases where our plans to delete would be highly disruptive. For example several graduated students still had data from research they did as an undergraduate that their faculty mentor and their group continues to access. As a result, we cannot simply delete a departed student’s drive files. Same thing with employees.
  • So we asked Google if we can archive most of the older files that can be retrieved when needed. Long story short… they paraded a bunch of vendors, but no one had a solution. Because Google Docs, Sheets and the associated metadata (such as who it is shared with and version history) are proprietary, you cannot use other systems to backup and restore and Google itself has no solution to this. Believe me, Google Vault is Not a solution to this since it does not store sharing or folder structure data.

Understandably, we are too small to win any argument with Google, so we launched a campaign to try to cut down storage, but we were not that successful. And we will continue this. But, we ponied up money and purchased additional storage from Google recently. Whereas it may appear reasonable, it really is not. As I mentioned earlier, I would much rather archive a lot of what we have and not have them in expensive active storage. Amazon Glacier or Google Cloud storage would be perfect, but I cannot move them there in their native format! And this is NOT my fault and Google is asking me to pay for this.

Google also has been rolling out new tools to help us manage the storage. So, here we are, we have purchased additional storage.  Since we only purchased enough for what we predict to use this year, we are using over 80% of our storage, which creates a warning message on some apps. This is useful for admins, but the current bug shows this warning for all our users in Photos.

We created a ticket 6 days ago and received replies from a different support tech each time, none of whom understood the issue and gave us steps to try that did not address the actual bug!! After bringing this up with our account rep and finally getting a tech that understood it, six days later it has finally been accepted as a bug. And there is no ETA for a fix. In the meantime, we are getting cases from our users worried that they are over quota and confused because they know they aren’t. This is so frustrating on many fronts.

If this had happened when the services were free, my approach to this would have been very different from now, when we are paying for services! And the frustration is compounded by another episode with the Google Account team and reseller of Google services recently that I will write about later.

As I began this, this is not the Google we were used to. Period.