On May 17th, Salesforce announced a significant outage to its service, resulting in customers losing access to one of the most critical applications being used daily. The issue was acknowledged by Parker Harris, Salesforce’s chief technology officer and a co-founder, while the company worked together to try to resolve the critical outage as soon as possible.
At the center of the disaster was a faulty database script that was deployed in the production environment. Salesforce announced that “a database script deployment inadvertently gave users broader data access than intended.” This affected Salesforce customers who use Salesforce Pardot, a b2b marketing CRM, as well as any customers who have used Pardot in the past. The inadvertent access allowed users to both read and write permissions to restricted data.
Salesforce took initial steps to mitigate the problem by blocking access to all instances that contained impacted customers, and by shutting down other Salesforce services. This heat map below shows the extent of the blackout for Salesforce customers.
The essential nature of the Salesforce application is self-evident, so these outages were extremely significant. Users who need Salesforce on a daily basis as part of their job found themselves idle, forcing many businesses to simply send them home.
As a data center company, focused on protecting the most critical applications, here are our essential four insights following the crisis:
- Think Further than Cyber-Attacks
Always remember that cyber-attacks are not the only threats on your data center. When evaluating your data-center risks, it is important to take into account internal “threats” and implement the right controls that will protect your “digital crown jewels” – the most critical business applications and processes. For example, separating your production and development environments is foundational for strong security, ensuring that testing scripts cannot run in your production environment, even in the case of human error.
- Always Consider the Cloud
Companies are increasing their presence on the cloud, for reasons such as a positive impact on cost, maintenance efforts, and flexibility. However, security needs to be considered from the outset of your cloud strategy. Some companies are unaware that cloud apps have a greater exposure to different threats due to lack of visibility and the difficulty to introduce policy and controls. On the cloud, your business is at greater risk in the case of a breach or an outage.
- Zero Trust
You cannot trust your single point of configuration to control and isolate your environment. Best practice is to criticize your controls and simulate the situation of failures. Zero Trust, the approach of “never trust, always verify,” can be focused on lateral movement and breach detection attempts in internal vs. external networks. However, it can also be relevant for any security controls that are being used or updated. In many cases, your business is in danger from internal threats, misconfigurations, and innocent mistakes, all of which can be as catastrophic as a malicious cyber-attack. The zero trust approach helps to limit the damage.
- Be Ready for a Crisis
Distributed controls are your strongest weapon to ensure that you are prepared for any eventuality. These will allow you to act quickly against the unexpected, especially in hybrid cloud environments where you need to manage multiple clusters and control planes. Make sure that you have the visibility and control of your entire environment that allows you to instantly isolate any affected environments. This will give you time to put your incident response plan into place, and protect your critical assets until a solution has been found.
The Salesforce outage shows that mistakes can happen to anyone, and the best protection is always going to be preparation. Start by separating your environments, limiting the exposed surface, and then move on to using the zero trust model to keep your most critical assets safe from harm, even in a hybrid-cloud infrastructure. Remember that without adequate segmentation, you are exposing your applications to internal threats as well as external ones. With strong data center security, you are one step ahead at all times.
Want to learn more about micro-segmentation in the cloud? Read our white paper on how to secure today’s modern data centers.