joan-pepin

As CSO, Joan is responsible for the holistic security and compliance of Auth0's platform, products, and corporate environment. She brings 20 years of experience to the role, with a career that has spanned a wide variety of industries, including healthcare, manufacturing, defense, ISPs, and MSSPs. <br /><br />Previously, Joan served as Business Information Security Officer (BISO) at Nike, Inc, CISO, and VP of Security at Sumo Logic, and held different positions at Guardent/Verisign/Secureworks organization. Joan holds a patent for developing the methodology to assess whether a communication contains an attack. <br /><br />She is also is a well-recognized thought leader and has spoken at major events, such as RSA, WhiteHat Security Summit, and Forrester Security Summit, and is frequently called upon for her expertise and commentary in Cloud Security and Compliance in large-scale and DevOps/CI environments.

Chief Security Officer (CSO)

Joan Pepin

Insights from giving our business continuity plan a COVID-19-induced update



What are the critical things your company needs to keep its doors open? 

Many companies have a Business Continuity Plan (BCP) in place that covers loss of facilities during things like floods and earthquakes, but it's primarily the really big companies (or those who provide critical services for health care) who’ve done more than download a pandemic plan template off the internet and call it good.

[Only 62% of 500 U.S. companies surveyed](https://www.techrepublic.com/article/business-continuity-plans-and-tech-are-lacking-during-the-coronavirus-pandemic/) by SaaS-based accounts payable vendor _AvidXchange_ at the beginning of March had business continuity plans – nearly half of those said they covered contingencies for only two to three weeks – and 10% have no plan at all. At Auth0, we’ve had a BCP in place for years, but [COVID-19](https://auth0.com/blog/business-continuity-at-auth0-our-response-to-covid-19/) prompted an immediate review. The process proved both necessary and revealing. And because we believe it can help other companies, we’re sharing our approach here.

## Ask Yourselves How Do You ‘Degrade Gracefully’?

The first time I heard of a Pandemic Plan was during the [Bird Flu Outbreak](https://en.wikipedia.org/wiki/Avian_influenza) in 1997, while working for the main security company. A large pharmaceutical company was a major vendor, so they required a plan. Unlike our current situation with COVID-19, the plan remained largely theoretical. That’s not the case now. Even if you have a theoretical pandemic response plan, now is the time to make it real. What are you actually going to do?

In computer science, if hardware or a network has been partially destroyed or rendered inoperative, there’s a limited function necessary to keep it going. That process of how the machine or system shuts down is called “graceful degradation.” So, in a pandemic where your people could be out due to illness or permanently gone, you can apply the same question to your business: How do you degrade gracefully? 

This is not what anyone wants to think about. What if half my staff is sick or permanently gone? But it is much easier to think about it now than when you’re in the middle of it rather than when it’s too late to remediate. Think about it now and you will be doing your future self a big favor. Your future self will be so thankful that your past self documented all these things and you don’t have to think about it while you’re worried about your team. You can just follow the plan that you’ve made.

![Pandemic Plan](https://images.ctfassets.net/23aumh6u8s0i/6eWOAZSW8CT2gSGUHYWjix/af815dc9c1ab6e6e4b4036099e423c9b/plan_sep1)

## How to Create a Pandemic Business Continuity Plan
You’re going to start by assessing the most critical functions that need to continue to happen for you to remain in business. Or the way I’ve been thinking about it: what is the absolutely last thing that gets shut off? Then you’re going to work backward. What are the things that are required to make sure that thing keeps working? What are the dependencies?

In our case, maintaining service for our existing customers is our most critical function. (Yes, it’s going to be important to add new customers, but if we’re just talking about the absolute most critical function, that’s it.)

In order to keep our service running for existing customers, we need to continue to pay AWS and the employees who are directly responsible for keeping it running. So for us, that would be the very last thing to go before the end of the world when we put all the chairs on the table, lock the doors, and head for the hills. 

If you’re a consulting company, you’d end up in a different critical place. You need to keep shipping deliverables to your clients. The ability to email or transfer files would be a critical function. The key is to go to the end of the world, find the most critical business process, and then work back. 

At Auth0, we worked ourselves backward to an absolute worst-case scenario where our staff numbers became more and more constrained, taking us from our global staff of 660 to just 10% remaining and created a plan where we could continue serving our customers. You should review your specific needs to understand your minimum number of employees for critical functions.

### Defining ‘Mission-Critical’ Functions
This process isn’t going to be easy for everyone, but it is an opportunity to think about the bigger picture. This is an opportunity to think about what’s best for the whole company and your customers. Try and set any ego aside. It’s not an insult if your department is not a business-critical function. Just because something isn’t “mission-critical” doesn’t mean it’s something the company will stop doing even in a pandemic. 

As CSO, I know that my team's ability to monitor and respond to security incidents is by definition a mission-critical role even in a pandemic. But, I also know that if we're really down to 10% of staff across the company, I'm going to take whatever resources I have to help keep our core service running for our customers. That's how the BCP helps me, my team, and my peers prioritize. In a sense, even as CSO, I know my place in this chain.

Since creating a BCP plan can bring a lot of different priorities and perspectives into conflict across the company, our entire executive leadership team worked closely to define a set of business-critical priorities for their teams to work from when fleshing out the remainder of the BCP update. 

![Pandemic Plan](https://images.ctfassets.net/23aumh6u8s0i/N21EflGpHem6FVAnIRzNa/9596f571f0021f037ab387287aac831a/plan_sep2)

Because human beings have egos and there may be disagreement about what your most essential business processes are, this is an exercise that the most senior level of leadership should do and agree on. It should not be up to the C-level to put the plan together, but it is their responsibility to have a unified voice because these are business-critical functions. We broke our functions into four categories:

* **Mission-Critical:** Loss of these functions would result in wide-spread loss of reputation, damage to the business, as well as potential damage to the general public.
* **Business Essential:** Loss of these functions would make it difficult, but not impossible to continue the business. Things like generating revenue, customers, following compliance regulations, and maintaining our external help desk fall under this category for Auth0.
* **Business Core:** Loss of these functions would have an indirect revenue impact. In the COVID-era that includes not attending conferences, which impacts both leads and indirectly, customer satisfaction.
* **Business Supporting:** These functions impact employees, not employee productivity. Auth0 examples include employee reviews, our internal help desk, and business support functions.

As things fail, how do we work from the bottom of the list to protect our mission-critical functions? 

If half of your people could be out at any given time and only two people know how to do a given thing that’s critical, that’s not enough.

Here are three questions to ask:

* What are the tools necessary for each business function?
* Who are the people who know-how and are authorized to perform that function?
* Is the process documented with screengrabs so that a smart person could figure out how to make it happen?

### The Hard-Drive Tolerance Approach to Cross-Training
There isn’t great guidance on how many people are needed for cross-training. If all your finance people are in Seattle and everyone in Seattle gets sick, well, that’s not going to work. So I based our approach on hard drive tolerance.

For a hard drive to be considered fault-tolerant, typically the data is backed up in five copies in different places. That’s what we’re doing with people — five people who understand how to perform the process and have access to simple documentation but aren’t all in the same place. As a globally-distributed company, multiple locations are an easily achieved standard for us. If you’re not able to rely on geographic distance, you might want to cross-train additional staff.

![Pandemic Plan](https://images.ctfassets.net/23aumh6u8s0i/6pUwP72d4UnvgotPIDO5YP/954716e2549c226b4994e733066131fe/plan_sep3)

## Update 2x a Year, More in Crisis
We took five working days to review our plan and put this streamlined approach in place. Compared to other pandemic plans I’ve seen, ours is super-actionable — the kind of thing that leaders can actually lean on to help them make decisions.

Essentially it’s:

* A stack rank of priorities by team
* Documented instructions
* A Confluence page with who does what, down to five levels of people and their status
* And the instructions to print it out and store it

We are going to keep an eye on our plan and update it quite regularly through the evolution of this pandemic, likely on a monthly basis. Once we’re through the pandemic, we’ll dial that back to twice a year.

## Your Plan Will Make You Stronger
Your plan doesn’t have to take weeks. As I mentioned, we got our review done in five working days. Even if you’re starting with no plan at all, the effort is worth it for what you’ll gain. You’ll discover things you didn’t realize about your business like:

* Departments you didn’t realize were critical.
* Areas where you are under-staffed.
* Processes that need to be documented.
* People who need to be cross-trained.


Your teams will gain:

* A clearer understanding of how they function AND how they relate to the company’s overall mission.
* Likely every department will have an “aha” moment by going through this process.
* Clear direction on communication AND actions.
* Relief that they won’t have to make it up as they go along.

Like the 40 pounds of pasta I have in my garage, if this all blows over, it will get eaten, it’s not going to go to waste. And going through this process will make you a stronger team even when this situation disappears.

<include src="asides/AboutAuth0" />


"business-continuity-plan"