Kyoto University recently reported that as a result of a defective software update, 77 TBs of research data was wiped from its supercomputer. According to the university, 34 million files from 14 research groups were deleted and a third of the lost data will not be recovered due to lack of additional backup copies.
There is a common misconception that backups stored locally are somehow safer than backups that are stored in the cloud, and I assume that Kyoto University was storing their backups locally. Had those backups been stored in the cloud, and particularly in a cloud that offered immutable storage – which prevents data from being tampered with or deleted, even by an administrator – it is likely that backups would have been unaffected by the software upgrade and would have allowed the University to recover nearly all the lost data.
The modern threat landscape
The data loss at Kyoto University must be devastating for the scientists and researchers who may have lost years of work. It underscores the need for universities to modernize their IT infrastructures to protect against such massive data losses like in Kyoto, and other ongoing threats like ransomware, natural disasters that can destroy on-prem servers, and human error.
Ransomware is still the number one threat facing university data globally. Education institutions are now being routinely targeted, with the number of attacks against universities increasing by 100 percent from 2019 to 2020. Something that will always be true is that ransomware infections, more frequently than not, occur because of user error or carelessness. People are human and they make mistakes. They fall for scams, they get tricked into giving out their credentials, or they get tricked into clicking on and installing malware. Most of the security industry focuses on intrusion prevention and detection. But it’s a losing battle because the vulnerabilities are not just technical – they depend on people never making a mistake. And that’s not likely to ever be the case.
So rather than obsessing over the latest firewall technology or intrusion detection software, it’s often better just to have everything fully backed up. Restoring data is one thing, but in many cases the only way to get rid of ransomware is to wipe the computer’s disks and start afresh. That means IT teams have to reinstall operating systems and other foundational software, plus all of the applications in order to restore the data. This transition should involve moving away from on-premises data storage and towards the cloud.
Backing up data in the cloud
The vast volumes of research data and multiple departments requiring access to data, as well as sensitive personal information on students and faculty, make the data storage requirements for higher education institutions vastly different from other industries. And in the event of a massive data loss incident, universities need to be back on their feet in a matter of minutes, rather than days. With this in mind, the cloud has become a lifeline for many higher ed institutions as it promises the scale and capacity needed to archive, protect, and easily access data at a moment’s notice.
The cloud storage market for the education industry is now projected to reach US$3.986 billion by 2023 and many industry leaders have expressed their plans to increase cloud usage by more than 50 percent over the next three years.
In the case of Kyoto University, catastrophic data loss could have been prevented if there was a multigenerational backup in place to restore deleted files. Here are some steps institutions can take to implement an effective cloud backup plan and avoid similar incidents:
1. Test your recovery process:
The recovery costs following an infrastructure failure or human error can be significant. For example, Baltimore County Public Schools spent more than $8.1 million on recovery after an attack in 2019. Recovery testing must be done in anticipation of any potential threat to determine and eliminate any errors in the process. It also must be done frequently and on a fixed schedule. Keeping records of the results will not only help you find gaps in protection but also will be useful for decision-making when adjustments need to be made. This testing can be time-consuming and technical, but with a flexible cloud ecosystem, IT teams can easily access their data and test their recovery process in advance.
2. Implement a 3-2-1 backup approach:
Another key lesson learned from Kyoto University’s data loss incident is to avoid having all data backed up in one place. It’s good practice to keep at least three copies of data, with two on different media formats, and one of those being off-site – also known as the “3-2-1” backup approach coined by Veeam, one of the world’s leaders in cloud backup. Different media formats could be a hard drive, tape, or the cloud, depending on your school’s budget and the importance of your data given some media formats are less secure than others. Keeping one copy off-site means diversifying the storage locations, which could be another building sent through the WAN or Sneakernet, shipping the tapes off to a storage facility, or using public or private cloud.
3. Leverage object-level immutability:
Data in the cloud can still be affected by threats, and some operators will try to extort money from educational organizations by targeting cloud backups. In these instances, attacks are often started on-premises (by way of an infected USB flash drive, attached file, URL download, or other) and uploaded to the cloud through a backup. In some cases, cybercriminals can access the networks of victims via exposed remote desktop services, gain access to their cloud credentials, and then proceed to delete their cloud backups, before deploying the ransomware.
With object-level immutable storage in the cloud, data cannot be deleted or altered by anyone, not even a systems administrator, during a specified retention lifetime. This feature minimizes human error and ransomware threats, and protects critical data like research data and students’ and faculty’s personal information from being tampered with or deleted. Another key advantage of immutable storage is the enhanced audibility it provides your team and/or outside auditors to make sure your IT systems and practices are FERPA-compliant.
The evolving threats universities face daily will likely be around for the foreseeable future, so it’s critical that IT departments are prepared for such eventualities – it’s not a matter of if, but when. The move to the cloud will be a pivotal step for the industry in guarding against future cyber threats.
David Friend is a cloud storage veteran, serial entrepreneur, six-time founder, and the current co-founder and CEO of hot cloud storage company, Wasabi Technologies.
- Using real-world tools to prepare students for the workforce - April 18, 2024
- 8 top trends in higher education to watch in 2024 - April 16, 2024
- Defining a path to equitable AI in higher education - April 12, 2024