UniSuper down for 3 days

[bringchristintoeveryhour]

Posting this as a disappointed member. UniSuper has been down for three days without a peep.

It’s obviously not planned maintenance, as it would have a defined outage window.

If it was technical, then I think they would have reassured us.

So then are we to assume it’s a data breach?

Even if it’s not, as a large financial firm managing people’s retirement funds, it feels totally unacceptable to lock people out of their accounts with no acknowledgment for this amount of time.

Optus and Medibank as bad as they were, at least we heard something.

 

[striveforclarity]

@bringchristintoeveryhour They sent an email to me as I am a member giving more info they said it wasn’t a hack or data breach but a prob with a third party provider

 

[mercyofthenight]

@striveforclarity Been nearly a week now. Honestly if it’s not fixed soon I’m gunna seriously consider changing funds, pretty bad that they can have a weeklong outage without any timeframe it may be fixed by - makes it hard when you’re trying to finalise mortgage approval !

 

[beverly05]

@mercyofthenight I need to open a superfund as I turn 18 soon. They better get on quickly before my workplace opens me with their shitty default super

 

[mercyofthenight]

@beverly05 There’s plenty of good alternatives out there with cheap fees and good returns - HostPlus, Vanguard etc. I’m probs gunna move after this experience.

 

[cliquely]

@striveforclarity I am a member too but I don't know if I believe them. I am kind of concerned that it was actually a data breach. Even worse, if they lost members' monies because there was fraud, etc. It's getting more concerning by the minute.

 

[sherrykay1]

@cliquely I'm thinking the same thing. We just received an email from the CEO with an assurance that no data was exposed to an unauthorised party, and seemingly shifting the blame to Google Cloud. Terms like "isolated one-of-a-kind issue" reek of BS to me.

I wouldn't be surprised if the language changes over the next week or so to "maybe", then followed by an actual admission of a breach. I hope I'm wrong (obviously) but the rhetoric being used here is suspicious.

 

[sandpatch]

@striveforclarity Google Cloud specifically.

 

[futurelife]

@bringchristintoeveryhour Yeah, it's not great. I would expect regular updates at the very least, but they have provided exactly 0 updates or transparency. I understand outages happen, but the extended timeframe and poor communication during it really makes me question staying with them long term given the many other competetive options out there like REST and Hostplus.

 

[futurelife]

They just sent an email out right after I hit post:

You may be aware of a service disruption affecting UniSuper’s systems.

UniSuper is working through this issue which originated from one of our third-party service providers, and we are actively partnering with them on a resolution.

This service provider has confirmed to UniSuper that the disruption was not a result of malicious action or cyber attack, and no UniSuper data has been exposed to unauthorised parties as a result of this issue.

Unfortunately, this has caused disruption across the business. We are working around the clock to get systems back online swiftly, safely and securely.

We are unable to confirm a timeline for restoration at this stage. We appreciate that not being able to give a time is frustrating, and we apologise for this experience.

Importantly, we have teams dedicated to assisting members who may be experiencing difficulty. They can be contacted via our contact centre on
1800 331 685.

We are in the process of finalising how we are able to process member requests during this time in a way that is fair and equitable for members, as you expect and deserve.

The mechanism for this may vary with the type of request, and we will provide further details when we can.

As always, our members are front of mind, and we are prioritising the restoration of systems required to provide services to members.

Again, UniSuper has been assured by the third-party provider that this disruption is not the result of a malicious act or cyber attack, and no UniSuper data has been exposed to unauthorised parties as a result of this issue.

We will continue to advise on developments as we work to restore these systems as swiftly as possible.

Thank you for your patience and understanding, and apologies for the inconvenience.

 

[futurelife]

More updates: https://www.unisuper.com.au/contact-us/outage-update

They are blaming Google Cloud. As a cloud engineer myself, I can confidently say this is complete bullshit, it is 100% UniSuper's fault. A 4+ day outage of a cloud system is absolutely bonkers. The techniques cloud providers give you to ensure zero downtime is unparalleled, even in the event of a full region failure. UniSuper would have had to ignore many best practices and have had a very very poor implementation to start with, probably hand rolled or lift-and-shifted from on-premises with very little automation, and missing or untested DR procedures.

I'm guessing they've suffered data loss and are struggling to restore and reconcile their databases from old backups.

 

[zal56wes]

@futurelife Their message states

UniSuper is experiencing a service disruption, which originated with one of UniSuper’s third-party cloud service providers, Google Cloud.

So they are not really blaming Google Cloud just saying it's happened there. I bet they got their lawyers to approve that statement because Google wouldn't take kindly to being blamed if it was due to poor DR practices on Unisuper's behalf.

I agree with everything you said though it's pretty unbelievable that an outage of this scale could occur. More so when it only affects Unisuper and not any other Google Cloud customers.

 

[futurelife]

@zal56wes I don't know, it definitely sounds like blaming to me. Every time they go to explain what the problem is, they answer "Google Cloud".

From their emails to members:

To provide clarity, earlier today we were able to issue a joint statement with Google Cloud, the third-party cloud service provider from whom this outage originated.

"from whom the outage originated" is very blamey.

And from the FAQ in the link I sent:

What's happened? Due to an issue with our third party provider, Google Cloud, UniSuper’s essential services are experiencing an ongoing outage.

Directly pointing the finger.

I can't imagine Google is happy with the "joint statement". I hope they release their own root cause analysis.

 

[moodymoose77]

@futurelife Confidently, huh

https://blocksandfiles.com/2024/05/14/google-cloud-unisuper/

 

[hototot]

@futurelife Software engineer using cloud too. Even if Sydney received a nuclear bomb, Google Cloud would not take so long to come back.

 

[bringchristintoeveryhour]

@futurelife Thanks for that
Still nothing in my inbox, but good to know it’s not a data breach.

 

[randybroekema]

@bringchristintoeveryhour With how much they stressed that it isn't, it makes me think it is lol