pawan bhardwaj

Pawan is a seasoned business and technology enthusiast who welcomes challenges and pushes boundaries for value creation. In addition to his passion for technology, he also enjoys learning about music, art, architecture, and philosophy as lifelong pursuits.

In his career, he has worked in the domains of product management, solutions architecture, cybersecurity, cloud, telecom, and software engineering. He has led SaaS scaling initiatives in Cybersecurity (IAM), portfolio evolutions to cloud and telecom APIs, and deep partner engagements.

He has an MBA from Sonoma State University and an MS from the University of Delhi.

Principal Product Manager

Pawan Bhardwaj

jc tierney

JC Tierney was born and raised in the suburbs of Chicago, and is a life-long Blackhawks, White Sox and Bears fan.

He graduated from Benet Academy in Lisle, Illinois. He went on to attend the University of Dayton, where he graduated with a degree in Management Information Systems, and loved every minute of the experience.

Director, Engineering

JC Tierney

Efficiency on a robust foundation: Exploring multi-subscriber (Public Cloud) deployment on Auth0's converged platform architecture.

**TL;DR:** Auth0’s SaaS services are available via two modes of cloud deployment:

* Multi-subscriber (shared infrastructure, a.k.a. Public Cloud) 
* Single subscriber (dedicated infrastructure, a.k.a. Private Cloud). 

These offerings now have a converged platform architecture that increases business agility, developer productivity, scalability, reliability, and security by using industry-leading tools and fully automated operations.

This article continues a series of earlier ones focused on Private Cloud deployments. Unlike the single-subscriber Private Cloud, the Public Cloud hosts thousands of subscribers. In multi-subscriber deployments, data isolation, security, and elimination of noisy neighbor problems are paramount.

Public Cloud leverages the converged architecture composed of three tiers - the Central tier, Points of Presence (PoPs), and Spaces. The Central tier and PoPs are for control, operations, and shared services. Spaces are environments that provide customer identity services as per customers’ needs.

We leverage Kubernetes for lifecycle and runtime management of Auth0’s cloud-native containerized solution. Auth0 is composed of stateless services that are easier to scale. We use MongoDB, PostgreSQL, and Redis as required by the situation. We ensure a robust and performant solution through network isolation and extensive use of edge networks for DDoS protection, rate limiting, and low-latency DNS resolution (including custom domains). The converged architecture's intra-region High Availability (HA) and inter-region geo-failover characteristics help us meet the availability SLAs of Public Cloud deployments.

## Auth0 Services on Multi-Subscriber (Public Cloud) and Single-Subscriber (Private Cloud) Deployments

Auth0 is a Software-as-a-Service (SaaS) solution with two deployment models - Public Cloud and Private Cloud. The Public Cloud is a multi-subscriber environment where resources are shared between customers and is available in the United States, Europe, Australia, Japan, and now in the United Kingdom. Private Cloud deployments, on the other hand, are single-subscriber environments that provide customers with dedicated infrastructure and can be deployed in most regions across Amazon Web Services (AWS) and Microsoft Azure.

The Public Cloud offering is deployed on AWS and addresses the needs of a vast majority of customers. However, customers may choose Private Cloud for a variety of reasons, such as: 

* Geographical requirements that dictate deploying in an area not served by Public Cloud (e.g. for data residency and/or latency reduction)
* Industry-specific needs (e.g., PCI compliance)
* Preference for Microsoft Azure instead of AWS
* Higher transaction volume than allowed on Public Cloud

![Auth0 Services on Multi-Subscriber (Public Cloud)](https://images.ctfassets.net/23aumh6u8s0i/6ex2w8DgyfbKhdJLx1lIPi/60a5ba0c641c9059317da35e9f6aab1c/multi-subscriber-public-cloud-01.png)

## Why Converged Architecture?

The converged architecture allows us to deploy and manage both Public and Private Cloud offerings on the same underpinning. This architecture is built on the following premises:

* Multi-cloud: AWS and Azure cloud support through a common interface
* Container orchestrated: use of container images instead of installing packages
* Stateless: scalable and replaceable by design
* Immutable: run only trusted code and configuration
* Fully automated: provisioning, upgrades, and decommissioning are done by machines
* GitOps: enhanced DevOps, push code and safely reproduce it thousands of times

![Architecture](https://images.ctfassets.net/23aumh6u8s0i/57nIODdJEv5clLYYswHEl5/12fabdd26c7eddf4db40754196ef1659/multi-subscriber-public-cloud-02.png)

The benefits of a single underlying architecture for developing, deploying, and operating services across different environments are manifold. The primary reason is the ability to offer new capabilities to customers faster and at a lower expense. The architecture also allows us to ensure feature parity across different deployment environments seamlessly.

The converged architecture helps in focusing the organization’s energy on a common infrastructure that yields higher developer productivity/velocity. It results in improved platform scalability to address various scaling demands from both multi-subscriber and single-subscriber deployments. It provides a constructive space for assessment and adoption of security practices that might be perceived as unique to one deployment environment type, but have practical implications on the other as well. It also helps with reliability refinements and enhancements across the board. 

Besides optimizing engineering resources, a converged architecture has several other internal benefits through unified automation, toolsets, and processes. It has expedited identification and realization of infrastructure and operational cost optimizations across the board, which helps deliver good value to our customers.

## Deployment Characteristics of Multi-Subscriber Services 

As a reminder, the Public Cloud is designed to host thousands of subscribers over shared resources, whereas a Private Cloud deployment is designed for a single subscriber. 

Ensuring optimal performance and security for the different Public Cloud subscribers is paramount. While the underlying compute, storage, database, networking, domain name, and other resources are shared in a multi-subscriber deployment, this solution ensures each subscriber is isolated within this shared environment.

The database and application workloads are partitioned to ensure an appropriate level of data isolation and security. The Auth0 Public Cloud combines pooled partitioning (i.e., partitioning of data by subscriber identifier) and on-demand subscriber-specific workload instances. The architecture is refined continuously to eliminate “noisy neighbor” problems.

## Public Cloud Platform Infrastructure Architecture

### Tiers - Central, PoPs, and Spaces

Once again, the converged architecture has three tiers - Central, Points of Presence (PoPs), and Spaces. Besides these, the Edge network for DNS and DDoS protection, and Auth0 HQ based systems for analytics and operations complete the picture.

![Tiers - Central, PoPs, and Spaces](https://images.ctfassets.net/23aumh6u8s0i/5wVpjNsZNeyRnxGmRIHLwM/deee78839a615f2ebc1679b65eafc873/multi-subscriber-public-cloud-03.jpg)

It’s important to note that the solution has been designed with isolated fault domains; the control plane and data plane separation allows the central control plane to be offline/under maintenance without impacting customers’ authentication flows.

* **Central**: A globally unique component that contains the platform control plane and other centralized global infrastructure. The control plane manages the creation of Public Cloud environments and keeps them updated with continuous (daily) releases.
* **PoPs**: An intermediate point of infrastructure pinned to a specific cloud region (e.g., AWS us-east-1). The PoP hosts internal visibility tools and securely proxies messages from the central down to the regional Public Cloud environments.
* **Space**: The Public Cloud environment is located ‘beneath’ a PoP in a particular AWS region. These spaces have cross-region geo-failover capabilities to ensure resiliency.

### Aspects - control and runtime, scalability, datastores, networking, and reliability

When Auth0 took on the converged platform project, it started as a small team, initially focused on our single-subscriber (Private Cloud) offerings. This team comprised a cross-section of our platform teams with expertise ranging from Kubernetes to API, tooling, infrastructure, and datastores. They were focused on developing a proof of concept for the converged platform.

As we learned from our experiments — and as the converged platform evolved and matured — we launched single-subscriber environments for general availability. You may read more about this in a series of posts we shared outlining the technical, security, and architectural decisions we made:

* [A Technical Primer of Auth0’s New Private Cloud Platform](https://auth0.com/blog/a-technical-primer-of-auth0-s-new-private-cloud-platform/)
* [The Architect’s View of Auth0’s New Private Cloud Platform](https://auth0.com/blog/the-architect-s-view-of-auth0-s-new-private-cloud-platform/)
* [Securing Auth0's New Private Cloud Platform](https://auth0.com/blog/securing-auth0s-new-private-cloud-platform/)
* [Data Pipelines on Auth0’s New Private Cloud Platform](https://auth0.com/blog/data-pipelines-on-auth0-s-new-private-cloud-platform/)
* [How We Do Releases in Auth0’s New Private Platform](https://auth0.com/blog/how-we-do-releases-in-auth0-s-new-private-platform/)
* [Optimizing Cost and Performance of Auth0's Private Cloud](https://auth0.com/blog/optimizing-cost-and-performance-of-auth0s-private-cloud/)

We continued the logical progression with the converged platform, and adopted it for Public Cloud. Behind the scenes, the team continued to grow, and eventually, as we began to gain more and more traction from the converged platform, we organized the team into specialties to best support growth.

* **Runtime -** Heavily focused on our Terraform and Kubernetes layers. Ensuring that the core of the platform functions and that automation has the flexibility to meet the needs of product delivery teams.
* **Control -** Owners of our controlling API and platform CLI. The tooling that is used for operating the platform in all cases both normal and incidents.
* **Datastores -** Focused on our data storage technologies and interacting with our product delivery teams and the most efficient way to utilize the available stores.
* **Networking -** Owners of our edge technologies and supporting the myriad of ways in which our customers leverage the edge.
* **Sizing -** Partners with other internal teams to establish the performance characteristics of our offerings and additionally adds improvements to our platform including auto-scaling.
* **Services -** Owners of tooling and services leveraged by product delivery in their development. This includes abstractions that allow for easier access to the underlying platform.
* **Data -** Responsible for ensuring the data flow from the individual environments to the regional Points of Presence (PoPs) and beyond. This includes application logs for support as well as abstracted metrics. 

In order to get the technology behind a solution right, it's critical to fully enable the people behind the technology to co-create innovative solutions. At Okta, we cultivate a BLOC culture, which stands for “Builder, Learner, Owner, and Collaborator.” These values foster a high-functioning, well-oiled machinery that can take on big and small challenges effectively and efficiently.

![BLOC culture](https://images.ctfassets.net/23aumh6u8s0i/1uDGWPkHZzzH5CKHlrvEbJ/114d89fab2474e4cb2aec9272dea795f/multi-subscriber-public-cloud-04.png)

With the different team specializations, collaboration becomes all the more important. At Auth0, we value documentation not only for informing but also for decision-making. Our Engineering teams follow a simple mantra for collaboration: "Write. It. Down." We use Request For Comment (RFC) documents to record architectural designs and Decision Record documents to discuss and align ideas across teams. These documents also provide important context for new team members about what we're building and how we arrived at the decisions we've made and served as an important knowledge base as we began onboarding service owners to the new converged platform. These core tenets have become critical for sustaining the growth of the platform and introducing new capabilities over time.

### Reliability

The converged architecture provides a highly reliable and available solution leveraging not only the High Availability (HA) constructs across three Availability Zones within an AWS region, but also geo-failover capability between Active and Failover regions.

![Reliability](https://images.ctfassets.net/23aumh6u8s0i/2jbDdi3fCuZ1hm6D0qyWCI/ffe5decea1a0c72156636d38073d1975/multi-subscriber-public-cloud-05.png)

The geo-failover regions are always synchronized from the datastores’ perspective, with demanding RPO/RTO and 99.99% availability SLA targets. Once the failover decision is made, a fully automated process is initiated for the failover to take effect.

![Reliability](https://images.ctfassets.net/23aumh6u8s0i/5HQ6xBqVyquDFEK7ewW4kC/299cf5878d8cee63c706c633cc0b1abf/multi-subscriber-public-cloud-06.jpg)

In the event of a multi-region failure (an extremely rare event), we can perform an automated restore from the last backup.

The historical availability metrics are a testament to the robustness of the Auth0 solution. We continue to meet our targets in Private Cloud with the converged architecture and expect to continue meeting or exceeding these targets with Public Cloud deployments on this architecture.

![Private Cloud](https://images.ctfassets.net/23aumh6u8s0i/3xjskcZcaHUYxFuePah6r4/fdd40e7755462d852c29d16293a3bc2e/multi-subscriber-public-cloud-07.png)

## What Next?

Auth0’s Public Cloud offering continues to be boosted across different facets. The availability of Public Cloud across geographies will continue to expand. The launch of the UK Public Cloud region in April 2023 is the latest example, providing yet another regional point of presence for customers with data residency and latency sensitivity needs. Also, support for a Public Performance tier in H2 2023 caters to customers’ higher performance needs on Public Cloud.


We look forward to continuing to deliver new and enhanced customer identity capabilities, ease of use, platform resilience, scalability, and security in our Private Cloud and Public Cloud offerings through the new converged architecture.

Multi-Subscriber (Public Cloud) Deployment on Converged Architecture

bernardo

Bernardo is an Engineering Manager at Auth0 focusing on improving its Performance and Resilience posture. He is a technical leader who is passionate about building quality software that is scalable and resilient to failure, so we can meet the rising expectations of our customers.

Engineering Manager (Performance and Resilience Testing)

Bernardo Guerreiro

How Auth0 uses Performance Engineering to balance infrastructure cost and performance.

Auth0 is providing a world-class service empowering the critical path of millions of logins around the world, so it’s important that we ensure our services are performing well while keeping our infrastructure efficient. The Performance and Resilience Testing team was established in late 2021, as Auth0 made a lot of improvements in its resilience posture. Together with Platform and Product teams, we work toward upholding that performance, reliability, and resilience posture that is such an important pillar of Auth0. 

During 2022, the [development of Auth0’s new Kubernetes-based platform ramped up](https://auth0.com/blog/the-architect-s-view-of-auth0-s-new-private-cloud-platform/), which allows for a more repeatable and automated deployment of public and private cloud environments. Since then, our team began a natural partnership with the Platform teams and started working on defining the Private Cloud offerings for our customers. 

One of our key responsibilities has been to define performance tiers for private cloud customers based on different RPS (requests per second) thresholds - 100 RPS, 500 RPS, and 1500 RPS (with more to come). This initiative, called Tier Right-Sizing, is essential to ensuring Auth0 is able to provide our customers with offerings whose infrastructure costs are efficient for their needs, allowing our sales teams to more accurately sell to customers based on their capacity and performance needs. This blog post outlines the process we followed to define these tiers.

## Our Performance Testing Tooling

As you’ll find out as you read the blog post, in order to define these tiers, our team (and many other individuals at Auth0) ran countless performance tests of varying durations. One of the key things our team works on is the tooling and infrastructure for Performance Testing for the whole Engineering organization.

For the last three years, we’ve used [Artillery](https://www.artillery.io/) to run Performance Tests at Auth0, and it has allowed us to scale easily as usage of our framework organically grew within the company. This was important for a few reasons:

- Firstly, because in the last year and a half, the adoption of our framework went from a few people running Performance Tests at Auth0 to almost every engineering team;
- Secondly, because the scope of our tests also grew. For example, in December 2022, as part of the Tier Sizing for a larger customer, we ran 6,000+ RPS tests, which Artillery was able to easily handle thanks to leveraging AWS Fargate’s horizontal scaling.

We’ve also written our own CLI (a wrapper around the Artillery CLI called Airstrike), which has allowed us to standardize Artillery usage across the organization and cater to Auth0-specific use cases. Developers can now run authentication flows they’re familiar with by name, point it at an environment of their choice and simply type a target RPS, all without having to use an Artillery config. This abstracts even further the work of running a Load Test, which is especially helpful for teams that might not be product experts but still need to understand the performance impact of their work, such as platform teams. For example, anyone can run the `authorizationCode` scenario in a specific internal environment at any given RPS, for the duration they need, and with their desired load profile. All they need to do is run this fully configurable command:

```
airstrike run authorizationCodeNew --env tus1 --targetRPS 100 --targetDuration 3600 --loadProfile spike
```

Additionally, the latest Artillery offering, Artillery Dashboard, was also an important part of this Tier Right Sizing effort. Since Airstrike programmatically adds the relevant tags to tests by default, we were able to leverage the dashboard as a centralized location to view all our Performance Test reports for this effort, by filtering by tags to get the relevant test executions. Any load test report is available there for easy historical reference with all its results, logs, and graphs, and we found it easy to consume for stakeholders and developers alike.


![Auth0 Artillery Dashboard](https://images.ctfassets.net/23aumh6u8s0i/1ZpGlYTzV4cNJShIKMk9eq/f1d22eb93f77fe897cd2d75f91cda7bd/auth0-artillery-dashboard.png)


## Tier Right Sizing

My team followed a thorough four-step process which I’ll outline in this section. This process was followed for each of the tiers we’ve defined (5 private cloud tiers total). For all phases, a subset of core authentication scenarios was chosen by our Product team, which represents key product patterns our customers use. 

### Phase 1 - Exploratory Investigations

During this phase, the team worked together with Platform experts on experimenting with several infrastructure sizings and configurations, including:

- Number of resources needed for main services;
- Database Instance Sizes;
- Burstable Instance types;
- Kubernetes Nodes Instance Sizes;
- Kubernetes resource allocation;
- etc.

Since there were a lot of permutations to try, each experiment was only a 10-15 minute load test. This phase can almost be compared to “Exploratory Testing”. Apply an infrastructure change, run a load test, analyze results, rinse and repeat. It gave my team a good sense of the “gotchas” of the new platform, and served to develop an instinct for what would work and what wouldn’t. 

Once the team had arrived at what they believed to be an optimal configuration, it was time to prove that.

### Phase 2 - 1-Hour Baseline Load

This phase was the main acceptance criteria given to us by Product. Once we reached a stable configuration in Phase 1, the intent was to prove that it could hold the Performance SLA for 1 Hour at the defined Max RPS for the tier, for each of the core scenarios (ran sequentially).

If it failed the SLA, it was then necessary to go back to Phase 1 with more experiments until the team narrowed down a working configuration again.

### Phase 3 - 6-Hour Synthetic Load 

The previous phase (1-Hour Baseline Load) gave us confidence in the performance of each individual scenario at peak load, but that’s not always a typical traffic pattern. Realizing this, we decided to borrow from a concept we have in our Staging environments we call “Synthetic Load”, which emulates production-like traffic by running multiple weighted scenarios at the same time. 

For this effort, we added to our existing core scenarios a few more scenarios that customers are likely to have in their environments (Management API, Extensibility, etc.), creating a 6-hour Synthetic Load test, with a total RPS of the Max RPS for the Tier. This gave us confidence that the Tier would perform under more normal production-like conditions, when a lot more services are being hit.

### Phase 4 - 72-Hour Soak Test

The final piece of due diligence was to ensure that we didn’t see a performance impact from any memory leaks, cron jobs, database background operations, etc. For that reason, after consulting with a few experts in the system, we decided to be safe and run long soak tests. These gave us confidence that we wouldn’t see a problem manifest itself over time.

This ended up being a crucial part of the process and actually revealed a few important performance improvements that had to be done in our platform.

### Rollout

After the new tier definitions were live, all new customers used the new tier definitions. However, in a few cases, we had customers already migrated to our new Private Cloud platform, but not yet in our new optimized tiers. In order to guarantee a seamless transition to the newly defined tiers, a dedicated Platform team worked on a release plan. 

All infrastructure changes were gradually rolled out to customers through our [automated release pipeline](https://auth0.com/blog/how-we-do-releases-in-auth0-s-new-private-platform/). Changes were broken down into small batches, and thanks to our automated testing and the soak time built into our release pipeline, Platform teams were able to monitor each change going in to ensure no issues had been introduced, therefore minimizing any risk during this transition. 

## Conclusion

Right Sizing infrastructure is a game of balancing infrastructure cost and performance. There is no right final answer, and while this effort has taken us quite a bit of time, it will likely never be fully done - we will continue to optimize and improve on this over time. In a few cases, we have also had to do this exercise for specific customers with more unique and large-scale use cases (adapting which scenarios were used), so we will be working on generalizing some of these customer patterns.

I want to leave a final note that we were very excited to support this effort on the road to providing a modern platform for our customers. On a personal note, I am incredibly proud of my team for their contributions to this project, from the work in the tooling that eventually supported this effort to the people who directly worked on the project. It’s exciting to take our new platform to higher limits and take on bigger and bigger customers!

Optimizing Cost and Performance of Auth0's Private Cloud

Cyril David

Cyril is a Platform System Architect at Auth0. He is a seasoned engineer with over a decade of experience in building distributed systems.

Platform System Architect

Designing for quality, reproducibility, and determinism.

**TL;DR**: Releasing individual services today is largely a solved problem. The complexity arises when we need to factor in the combinations of services, the configs and secrets necessary for a given service version, and the underlying infrastructure supporting those services while ensuring quality, reproducibility, and determinism.

## What’s in a Service?

When we talk about services, we largely focus on the code — but other things we might need to account for are:

* Config and Secrets
* Database related migrations
* Related infrastructure dependencies for a particular service version
* Implicit dependencies with other services

## Introducing the Concept of a Release Manifest

Given these problem constraints, we introduced a **release manifest** concept, which we define as a versioned JSON file describing a point-in-time representation of the entire Auth0 product stack, which includes: all service versions and the infrastructure version it was deployed on.

Configs and secrets are also versioned for every release manifest, which are snapshotted in our config/secrets storage.

Just to make it more concrete, here’s a hypothetical snippet:

```json
{
	"version": "v202221.99.0",
	"services": {
		"productpage": {
			"entity": {
				"name": "productpage",
				"default_vcs_branch": "main",
				"stacks": ["STACK_AUTH0"]
			},
			"artifact": {
				"version": "1.464.0",
				"vcs_revision": "dc62808d3a75a814a0748827d74e0936669dfec9",
				"vcs_url": "https://github.com/auth0/productpage",
				"reference": "sample.jfrog.io/docker/productpage:1.464.0",
				"metadata": {
					"build_number": "464",
					"build_started": "2022-05-24T18:52:34.753+0000",
					"build_url": "https://samplebuilds.auth0.net/job/productpage/job/main/464/"
				}
			}
		},
		"reviews": {
			"entity": {
				"name": "reviews",
				"default_vcs_branch": "main",
				"stacks": ["STACK_AUTH0"]
			},
			"artifact": {
				"version": "1.31.0",
				"vcs_revision": "dc62808d3a75a814a0748827d74e0936669dfec9",
				"vcs_url": "https://github.com/auth0/reviews",
				"reference": "sample.jfrog.io/docker/reviews:1.31.0",
				"metadata": {
					"build_number": "31",
					"build_started": "2022-05-24T18:52:34.753+0000",
					"build_url": "https://samplebuilds.auth0.net/job/reviews/job/main/31/"
				}
			}
		}
	}
}
```

A couple of notes:

* The version `v202221.99.0` is based on a year/week scheme (e.g., year`2022`, week `21`) and just a monotonic minor version which represents an atomic change.
* In this example, we only have two services called `productpage` (versioned `1.464.0`) and `reviews` (versioned `1.31.0`).
* This file is committed within our central git ops repository, with a matching tag `v202221.99.0` where the underlying infrastructure as code is also versioned for that specific point in time.
* Config/secrets aren’t versioned in git — but instead are stored separately in our config/secrets storage where we have the ability to snapshot them based on a specific version (in this case, all secrets/config have the same version `v202221.99.0`attached as metadata).

With this construct defined, we now have a baseline primitive for ensuring the properties we set out to solve initially:

* **Quality** — we can easily run our suites of system tests against a given version to ensure that our features work as expected.
* **Reproducibility** — we can create as many different spaces for testing or for onboarding new customers.
* **Determinism** — we’re guaranteed we have the same combination of service versions, config/secrets, database migrations, and infrastructure.

## A Note about Database Migrations

Database migrations themselves are a vast topic that is beyond the scope of this blog post, but the core properties we need to ensure are:

* **Forwards and backwards compatibility** – that a new version can be read by old code, and new code can work with the old schema.
* **Idempotency** – that any transient failures are wrapped in a transaction so we can retry the entire job if it fails. In addition, being able to run the same job once completed should be a no-op.

We utilize kubernetes jobs as the underlying primitive to run these database migrations and write them in such a way that retries during transient failures are ok — and that if we have to roll back, the previous service version will work with the new database migration.

Rollback is only within a reasonable constraint – in our case, the constraint is one major version away, which means a weekly version. So we only need to ensure backward/forward compatibility one major version away.

## Deploying a Release Manifest-Version

Given that we have all the necessary information codified in a release manifest version, the next step is to deploy older versions to a new version. Taking our hypothetical example:

* We have the current version `v202221.99.0`
* We need to deploy to a new version `v202222.105.0`

### Maintenance windows

Our private cloud customers can define a maintenance window where their space gets deployed every week. This means that these versions can only go out during that maintenance window — and that if anything potentially goes wrong, we have the ability to stop the release and Rollback to the prior version.

### Promotion pipeline

Like any release pipeline, we define a hierarchical set of channels; let’s imagine the following hypothetical example:

```
stage
+- internal
   +- free
      +- enterprise
         +- private cloud dev
            +- private cloud prod
```

So, in this case, if the stage had the version `v202221.99.0`, it means we need to deploy it _first_ to `v202222.105.0` before we can deploy `internal` to `v202222.105.0` and so forth.

### Rolling back

If, for any reason, a deployment encounters an error, we can roll back by shifting traffic back to the previous stable cluster.

## What’s Next?

The new release pipeline we’ve developed is only the beginning – and we’re planning to build more innovative tools for our platform in order to consistently deliver excellent service and reliability to our customers.

We are publishing a series of technical posts on how and why we replatformed. They’ll be linked at the bottom of this post as they publish and are also findable on our blog. If you have technical questions or ideas for future posts, please reach out to us via the [Auth0 Community](https://community.auth0.com/). (Our marketing team also tells us we would be remiss if we didn’t include a link to talk to Sales, so that is [here](https://auth0.com/contact-us), too.)

How We Do Releases in Auth0’s New Private Platform

MatthewHughes

I'm a Liverpool-based technology writer with a passion for usability and security. When I'm not churning out copy, you'll find me walking my two dogs: Luna and Apollo

Business Content Writer

Matthew Hughes

When it comes to SaaS applications, performance matters. Private Cloud SaaS helps you scale the performance of your growing SasS applications.

Performance matters. You know this, of course. Every second of additional latency translates into [lower conversions and tougher customer retention](https://businessmondays.co.uk/tackling-cart-abandonment-with-improved-login-experiences/). It's why you agonize over every individual detail, why your cloud infrastructure vibrates with the well-oiled efficiency of a Formula One engine.

As your business scales, staying fast becomes much harder. Essential SaaS (software-as-a-service) components that worked well when you were a smaller company now struggle to contend with your higher traffic volumes. 

Fortunately, there's an alternative. Private cloud SaaS offers the flexibility of a self-hosted application with none of the downsides. And, for performance-sensitive applications, it offers the boost necessary to deliver an incredible, slick user experience. 


## The Need for Speed 

Virtually every company and public sector organization uses SaaS in some form. Why? It's often the most practical and affordable option. Hiring and training someone to operate and provision a system takes time, money, and resources. With SaaS, the vendor handles long-term maintenance and upgrades. As most use a subscription model, there are no major upfront costs.

Most SaaS applications, by design, are multi-tenant environments. Multiple customers — whether individuals or organizations — share the same computational resources. This model is cheaper but has several trade-offs. Customers have little choice in where the application is deployed. They lose control. They can't make the subtle tweaks that ultimately result in faster, more responsive applications. 

Private cloud SaaS applications are different. They combine the benefits of a SaaS app (low costs, low maintenance, faster time-to-market) with the flexibility and autonomy of a self-hosted application. Customers can choose where, how, and with whom to deploy their application. They are the sole tenants, and they set the rules.

This control helps organizations comply with the ever-increasing number of data residency laws worldwide. But it can also deliver meaningful performance benefits, resulting in more responsive applications and better user experience outcomes.


## Reducing Latency One Millisecond at a Time

Have you ever wondered why Google only shows you ten results for each page? It seems counterintuitive, doesn't it? Google is a search engine. Its job is to show you relevant results for your queries. More is better, surely?

Not necessarily. In 2006, Marissa Mayer, then the company's Vice President of Search, [shared the results of an internal experiment](http://glinden.blogspot.com/2006/11/marissa-mayer-at-web-20.html) at the Web 2.0 Conference. Google asked a select group of users if they wanted to see more results with their queries. Most said yes. 

In response, Google provided random users (not just those in the focus group) with 30 results per page — triple the normal amount. 

What happened next? Frustration. Load times more than doubled for those in the test cohort, soaring from 0.4 seconds to 0.9 seconds. That 0.5-second delay had a measurable effect on user activity and satisfaction. Traffic and revenue from the test group dropped 20%. As Mayer said at the time: "Users really respond to speed." 

How does this relate to private cloud SaaS? Well, there are limits to how far you can optimize an application for speed with efficient coding and generous resourcing alone. 

If your primary application is located in Amsterdam but relies upon a component hosted in Los Angeles, you'll find a measurable delay as packets make their long journey through a network of submarine cables. You add 135ms to your load times – give or take 10ms.

You can't speed this process up. You are constrained by the speed of light. That's fast, but when you are trying to deliver an incredible experience to your customers, it feels like an eternity. 

You have only one solution: bring that component physically closer to your core application. 

The private cloud SaaS model lets you deploy resources in the same continent, country, and even data center. By eliminating performance bottlenecks in your product, your customers will benefit from a faster, more responsive experience. 


## How Private Cloud SaaS Can Help You Scale 

As your business grows, you may feel constrained by the limitations of multi-tenant, public cloud SaaS applications. Many vendors limit the number of requests that can be made per second (RPS) in order to preserve the quality of the service. While most SaaS companies are prepared to oblige larger customers with bespoke plans, this isn't always a given. 

And you cannot always predict when demand will spike. Take lab-testing and diagnostic services provider (and Auth0 customer) [BioReference Labs](https://auth0.com/case-studies/bioreference), for example. BioReference had begun transitioning to a patient-centric model, where customers could access their lab results without going through an intermediary — like their physician. As a result, it invested in Auth0's multi-tenant customer identity and access management (CIAM) platform. 

At the start of the COVID-19 pandemic, demand for its services jumped 25 times. Its monthly active users increased from 8,000 to 148,000. In response, the company migrated its identity infrastructure to a private instance of Auth0's Identity Platform, giving it essential breathing room to meet existing demand and add more customers in the future.

By opting for a private cloud SaaS application, you can provision your environments with the resources necessary to meet existing demand. And you have the flexibility to change your environment as circumstances dictate. 


## Becoming Resilient with Private Cloud SaaS 

Nobody likes downtime. Not your customers, who rely on your product. And certainly not you. 

Performance issues can devastate your bottom line. As load times and instability increase, conversion rates drop. Unfortunately, these problems are often beyond your control. In a multi-tenant SaaS environment, seasonal demand spikes — like those during the holiday shopping season — can result in longer wait times and instability for your customers. 

The private cloud SaaS model can insulate your business from any disruption in the public cloud SaaS application. With control over the environment, you can structure your SaaS infrastructure to be highly resilient, with custom disaster recovery protocols and redundant failover systems. 

You can establish custom backup and restore procedures. Mitigating against data center-level disruption becomes easy. Should the underlying cloud infrastructure fail, you can automatically switch operations to an unaffected environment, keeping downtime to a minimum. And with bespoke update rules, you can deploy patches and new features at a time when your customers are least active. 


## Nimble SaaS, Happy Customers

For most companies, it makes sense to use a multi-tenant public cloud SaaS product. But as your needs increase, you may find it sensible to switch to a dedicated environment. 

Private Cloud SaaS offers the best of both worlds. It requires little maintenance but offers the flexibility of a self-hosted application. You become able to make the choices necessary to deliver a lightning-fast, highly resilient product.

The Auth0 Identity Platform is available in both multi-tenant and private cloud versions. When you opt for a private environment, you get all the benefits of our industry-leading CIAM platform, but with the flexibility to choose how and where it's deployed. To find out how it can benefit your company, reach out to our team.

How Private Cloud SaaS Can Boost Application Performance

Dan McCorriston

Senior Product Marketing Manager

Now, all Auth0 customers - including those on our private cloud platform - have access to all the offerings on the Auth0 Marketplace.

Auth0 is excited to announce a newly expanded offering for [our new private cloud platform](https://auth0.com/blog/a-technical-primer-of-auth0-s-new-private-cloud-platform/) customers - Actions Integrations are now available from the Auth0 Marketplace. What that means is all Auth0 customers that utilize our new private cloud platform have access to all the offerings on the Auth0 Marketplace - SSO, Social Connections, Log Streaming Integrations, and now, Actions Integrations.

Actions Integrations will start to appear in the Development/Staging space now and will be ready in the Production environment in the coming weeks dependent on various release cadences.

### What are Actions Integrations and the Auth0 Marketplace, and why should I care?

At Auth0, we realize that many customers have unique needs, and as a result, they want the ability to customize their identity product. In order to serve these additional customer needs, Auth0 built a variety of tools to extend the functionality of identity. This concept is called **extensibility.**

Auth0 Actions and Actions Integrations is the next generation of **extensibility**. Actions is the low-code offering, and Actions Integrations allows customers to use pre-written no-code, drag and drop options to add other functionality such as ID proofing and consent management. Auth0 provides this additional functionality through integrations with **our Ecosystem partners** offered in the Marketplace.

The Auth0 Marketplace and Auth0 Actions are the two complementary ways we extend Auth0. Customers and their end users need the ability to resolve their unique identity requirements — which may include payments, subscription management, signing terms of services, etc. — to complete a successful user access journey into their application or service.

### What is Private Cloud?

Auth0’s [Private Cloud offering](https://auth0.com/cloud) provides the flexibility and agility offered by cloud deployments but also allows a customer to maintain the control and security they need to protect their customers and their businesses. Many customers that have high security and regulatory concerns - those in the financial services industry, for example - may require certain standards of uptimes, updates, failovers, and support for their deployments. The Auth0 Private Cloud deployment models come in both Standard and Managed levels, depending on the customer’s requirements.

Auth0 [recently invested in retooling the Private Cloud platform](https://auth0.com/blog/a-technical-primer-of-auth0-s-new-private-cloud-platform/) to support more automation, adopt current technologies and help scale the Private Cloud to thousands of environments.

### Benefit of Private Cloud and Actions Integrations

Private Cloud has a strong appeal for customers who have high volume or who are from certain verticals. For example, verticals like banking, where shared databases are a concern. Or customers who have data residency requirements or even those who prefer to deploy in a specific region close to their customers. Private Cloud customers do not have to be concerned with sharing space, public cloud request limits, and multiple customer’s data being present in the same deployment. 

Many of the same customers that have a need for Private Cloud also have extensive use cases for extensibility and the no-code integrations available in the Auth0 Marketplace. Easy-to-use integrations such as Identity Proofing and Customer Data Management. With Actions Integrations, developers no longer need to build or maintain custom code or make changes when we alter programming models or release new versions of Node. These Actions Integrations are built and maintained by our Ecosystem partners and are available right in the Auth0 Marketplace. 

A couple of examples of Actions Integrations available in the Auth0 Marketplace:

* [**Deduce Intelligent MFA**](https://marketplace.auth0.com/integrations/deduce-intelligent-mfa): Deduce prevents consumer-facing Account Takeover (ATO) and registration fraud via the constantly growing Deduce Identity Network, with over 400M profiles and in excess of 1.2B daily activities. Deduce risk signals significantly reduces false positive MFA challenges.
* [**Tealium User Login**](https://marketplace.auth0.com/integrations/tealium): Tealium connects customer data - web, mobile, offline, Internet of Things (IoT) - so brands can connect with their customers. Tealium empowers companies to create a unified, real-time customer data infrastructure and makes customer data more valuable, actionable, and secure.

### Is this an increased opportunity for Ecosystem Partners?

Now that Private Cloud customers have access to Actions Integrations in the Auth0 Marketplace, our Ecosystem Partners have a new and expanded pool of potential customers to reach through building and publishing Actions Integrations into the Marketplace. There’s never been a better time to become an Ecosystem Partner and market your Actions Integrations through the Auth0 Marketplace.

Make sure to visit the [Auth0 Marketplace](https://marketplace.auth0.com/) to see if there is an integration that optimizes your customer’s identity experience. To find out more about becoming a [Private Cloud](https://auth0.com/docs/deploy-monitor/deploy-private-cloud) customer, please [send an inquiry](https://auth0.com/get-started?place=documentation%20post&type=link&text=contact%20auth0%20sales) and someone from Auth0 will contact you about whether it’s a good fit for your business.

Actions Integrations Now Available in Private Cloud

Jon May

Jon May is a Principal Security Engineer at Auth0 where he is responsible for securing the platform and infrastructure of the Auth0 Identity Platform. He has over 10 years experience in cybersecurity and technology and holds a bachelor’s degree in economics from Brown University. Prior to joining Auth0, Jon was a Security Architect at Bridgewater Associates where he was responsible for designing and implementing cloud and enterprise security controls.

Principal Security Engineer, Platform Security

Architecting and building a new platform that is secure by design.

**TL;DR:** Auth0 replatformed our private cloud offering into a multi-cloud Kubernetes architecture. This large architectural change brings with it new threats and new security challenges. We took the opportunity to design and integrate security into the platform from the start, as opposed to "bolting it on" at the end.

In this post, we'll discuss how we approached securely replatforming the Auth0 Identity Platform into a large, multi-cloud architecture. We'll walk through the risks that most concern Auth0's Security team and the threats that could result in those risks becoming real, tangible, negative outcomes. After that, we'll discuss the challenges our team encountered while addressing those threats in a multi-cloud environment alongside the principles we applied to address them. Finally, we'll review the controls Auth0 put in place to mitigate the identified threats.

If you haven't read the previous posts yet, [A Technical Primer of Auth0's New Private Cloud Platform](https://auth0.com/blog/a-technical-primer-of-auth0-s-new-private-cloud-platform/) and [The Architect's View of Auth0's New Private Cloud Platform](https://auth0.com/blog/the-architect-s-view-of-auth0-s-new-private-cloud-platform/), I encourage you to check them out first. They provide foundational knowledge and context which will help you better understand this post. 

## Understanding What Risks Affect Us

At the highest level, Auth0's Security team is responsible for identifying, assessing, and mitigating risks to our business and, in effect, our customers. It's our job to reduce the impact and/or likelihood of undesirable business outcomes caused by security events. Specifically, we're interested in mitigating security and privacy risks, such as those involving:

1. **Product/platform compromises**: bad actors gaining control of a product or platform component
2. **Customer or Auth0 data loss**: unintended deletion or unauthorized access to customer or Auth0 data
3. **Service disruptions and outages**: disrupting the Auth0 service and causing an outage for our customers

There are more risks than the three listed above and more nuance as well, but we'll use them for brevity. These high-level risks help us enumerate our list of threats.

## Identifying How the Platform Will be Attacked

This new Kubernetes-native, multi-cloud architecture is large and complex, which makes it challenging to secure. Our approach is to start by enumerating the areas of the architecture most likely to be targeted and how they can be attacked. This process, called threat modeling, helps us determine where to focus our resources and implement countermeasures (also known as security controls). These "areas to focus on," or threats, are circumstances or events that could cause one of the risks highlighted above to occur.

![Platform](https://images.ctfassets.net/23aumh6u8s0i/3kEf7N3gMeLfxUtIOqqEZP/5a7ee4be2c16df0549cdf2528eb1749e/01_private_cloud.png)

The threat modeling process results in a long list of threats of varying severity. For the purposes of this post, we'll focus on a short list of significant ones:

1. **Employee account takeover**: an attacker compromises or gains access to employee credentials and utilizes them to access Auth0 systems
2. **Malicious insiders**: an internal employee abuses existing authorized access to Auth0 systems 
3. **Service or application compromise**: an attacker gains control of an Auth0 product or platform service, third-party managed service, or a cloud provider service
4. **Software supply chain compromise**: an attacker tampers with or exploits misconfigurations/vulnerabilities in software dependencies used by the Auth0 product or platform

This list of threats informs what secure design patterns and security controls we deploy into the platform's architecture.

## Securing Multi-Cloud Architecture is Hard

Now that we have defined the list of threats we're protecting against (the "what"), we can start determining the design changes, secure design patterns, and security controls that will address those threats (the "how"). But first, before jumping into that, let's take a step back and understand the challenges with securing a platform that utilizes multiple cloud providers. In Auth0's case, we use both Amazon Web Services (AWS) and Microsoft Azure.

### Increased complexity

Using multiple cloud providers increases the architecture's overall complexity. Each cloud provider has similar general capabilities such as computing, networking, storage, and logging. However, each cloud provider implements these capabilities in their own ways. What we end up with are multiple implementations of the same core capability, like virtual machines or blob storage. These nuances between cloud providers introduce complexities that are challenging to secure.

Let's use an example to illustrate these differences. The Auth0 product requires [blob storage](https://auth0.com/blog/azure-storage-introduction/), so let's compare AWS S3 and Azure Storage Accounts and only focus on the different access methods.

![AWS S3 and Azure](https://images.ctfassets.net/23aumh6u8s0i/6DKlibDUAMicDy2oQaUKez/a66838c5086a3e1203d513dc7c3d1eef/table.png)

Another example is AWS Security Groups (SG) vs. Azure Network Security Groups (NSG) and how each handles rule evaluation. AWS SG rules are additive (i.e., allow the superset of all applicable rules), while Azure NSG rules are assigned a priority value and evaluated in that order. These are only two examples of the many differences between AWS and Azure we accounted for in the new platform's architectural design. 

### Larger attack surface

Using multiple clouds means potentially securing multiple, similar services for each cloud provider, e.g., AWS S3 _and_ Azure Storage Accounts. The complexity mentioned above and an increased number of services result in a larger attack surface. Today, for instance, we need to secure two cloud service providers (AWS and Azure), and that means securing two sets of each cloud-specific service required to run Auth0. It also means our Detection and Response team needs to ingest and monitor multiple sets of cloud-specific log sources to properly monitor and investigate the activity. That's a large surface area to secure.

### Cross-cloud connectivity

Communication between services within the same cloud is straightforward. We simply need to use the cloud provider's own mechanisms, and it should "just work." However, there are different points in our new platform's architecture that require some form of communication between AWS and Azure. Network connectivity, for example, needs to be established between Azure Points-of-Presence (PoPs) and Auth0 Central (which runs in AWS). We don't have the ability to use either cloud's PrivateLink service in this scenario (which would be preferred). In another example, Azure Spaces need to upload usage metrics into Auth0 Central (AWS). In single cloud architecture, we could simply use Azure RBAC configured with the proper role assignments. However, that's not possible in this scenario; the Azure Kubernetes Service (AKS) cluster/pod will need to authenticate using AWS S3 authentication methods.

## Addressing Multi-Cloud Challenges

So far, we reviewed the risks Auth0 is most worried about, enumerated a short list of threats to help guide our security decision-making, and discussed some of the challenges Auth0 experienced securing a multi-cloud architecture. Next, let's walk through the principles we applied to determine how we address those threats in the context of those multi-cloud challenges.

### View all security-related decisions through a risk-based and threat-centric lens.

That means all secure design decisions or security controls implemented were directly related to the risk and threats we enumerated during the previous sections. Prioritization is key, and using this lens influences where we need to spend available resources and which countermeasures would be most effective.

### Apply secure design principles and integrate security into the platform early and often. 

The Auth0 Security team is involved throughout the product life cycle, from ideation to design to build to operation and finally to deprecation. We are heavily involved in the design and build phases. This is important because it's easiest to make architectural changes before the technology is built.

### Establish clear, enforceable trust boundaries.

These trust boundaries inform how we implement various countermeasures such as access controls. They also encapsulate areas of risk, making them easier to manage. For example, our development and production environments are completely isolated and separate from one another. Operators cannot use development access to affect any part of the production environment.

### Balance cloud-native and cloud-agnostic capabilities.

This is a tricky one because each has its own benefits and challenges. Cloud-native capabilities (e.g., AWS managed databases) are tightly integrated with the cloud provider, which means you'll get those integrations — like GuardDuty or Microsoft Defender — out-of-the-box. On the other hand, cloud-native services are tightly coupled to their cloud, and deploying them for _both_ AWS and Azure increases the complexity and attack surface (as we discussed above). Compare that to cloud-agnostic capabilities, which can provide a single pattern and enable a simpler architecture with less attack surface. However, integrating with cloud provider services tends to require more work or "glue" to stitch them together. For example, our Auth0 product consists of Kubernetes workloads (cloud agnostic). We use managed Kubernetes services (AKS and EKS), and it's very easy to use the same services across both clouds. That means less complexity (i.e., one change to the Auth0 product will be applied to both clouds) and less attack surface (i.e., one service to secure).

## Selecting the Right Set of Security Capabilities

Finally, let's review the various security capabilities Auth0 implemented into the new platform to address the threats and multi-cloud challenges mentioned above. In addition to our own security capabilities, Auth0 takes advantage of the cloud's shared responsibility model. The shared responsibility model allows us to focus on securing other areas of the platform instead of worrying about the underlying cloud service provider infrastructure.

### Cloud Governance

All Auth0 Spaces, PoPs, Central, and Control Plane technology stacks are isolated into their own AWS Account or Azure Subscription. This provides clean and clear resource isolation, especially for single-tenant Spaces. We used AWS Organizations to create an Organization Unit (OU) hierarchy based on environment and functionality and add each account to a specific OU. We used Azure Management Groups to create a similar hierarchy for Azure Subscriptions. The hierarchies we created enable scalable, granular access controls and policy enforcement; we grant access or apply policies (Azure Policy and AWS Service Control Policies) to specific Management Groups / OUs or Spaces. For AWS, we grant access based on OU conditions and assign SCPs to OUs. For Azure, we assign roles and policies scoped to specific Management Groups. The access and policies are then inherited by the intended accounts or subscriptions.

![Cloud Governance](https://images.ctfassets.net/23aumh6u8s0i/ovvm1JDueXMwiN8ZKQThA/8f1684125fcc76f224b5e9329d5aa0fb/02_private_cloud.png)

### Identity and Access Management

Auth0 was built to secure identities, so we leverage Auth0 + Okta to secure authentication and authorization to our platform tooling. We manage all cloud providers and support application access using a least privileged RBAC model. We also require Multi-factor authentication (MFA) for all operator access to the platform; this includes cloud providers and all supporting applications. Operators are required to use Azure Privileged Identity Management (PIM) - which provides just-in-time access - for any administrative activities. Azure AD Identity Protection identifies and blocks any risky Azure tenant sign-ins.

### Network Security

All single-tenant and multi-tenant Spaces are logically isolated from one another with no interconnectivity between them. This strong isolation drastically reduces an attacker's ability to pivot into other Spaces or PoPs and further their attack. We further restrict inbound connectivity to spaces by using AWS and Azure PrivateLink to connect outbound from Spaces to a PoP. All traffic is encrypted in transit using TLS 1.2+. This includes network traffic between all cloud services, Kubernetes pod-to-pod traffic, and Kubernetes cluster-to-cluster traffic. For some workloads, we utilize mutual TLS (mTLS). On the network edge, we protect against DDoS attacks on our infrastructure, identify and block bots, enforce rate limiting, and block malicious traffic with web application firewall (WAF) rules. Operators must use VPN to access platform infrastructure, adding another layer to our defense-in-depth strategy.

![Network Security](https://images.ctfassets.net/23aumh6u8s0i/2DQcFgwaIkylGFaN479rmz/4a28c57f312f42a25052f84ae788b7b1/03_private_cloud.jpeg)

### Data Protection

All data is encrypted at rest using AES-256 or better, and encryption occurs at multiple layers (e.g., disk, file, database, and application layers). All data in transit is encrypted using TLS 1.2+. Auth0 encrypts platform secrets using HashiCorp Vault and takes full advantage of Azure Key Vault and AWS KMS for encrypting data in cloud services.

![Data Protection](https://images.ctfassets.net/23aumh6u8s0i/1XOTfQ1STNV4y9rTo7FcHf/3fd90c1fd4c3317f63804562e892d5e5/04_private_cloud.png)

### Detection and Response

Auth0 monitors the platform using a combination of cloud services and our own monitoring tools. We take advantage of AWS GuardDuty and Microsoft Defender for Cloud in addition to our own tooling. Auth0 ingests a vast amount of AWS and Azure platform logs for all of the cloud services used by the platform. These logs are ingested into a security data lake and monitored with detections defined as code.

![Detection and Response](https://images.ctfassets.net/23aumh6u8s0i/3aH0JsipzpNkdNubtfYwWN/6123b09fe51ae344fd4537e777081a22/05_private_cloud.png)

### Kubernetes Security

The [4C's of Cloud Native security](https://kubernetes.io/docs/concepts/security/overview/) is a good framework to use when thinking through Kubernetes security. We are, in varying aspects, responsible for security in the clouds we're using, the clusters we're running, the containers running on those clusters, and the code running within the containers. Auth0 uses the managed Kubernetes services Azure Kubernetes Service (AKS) and AWS' Elastic Kubernetes Service (EKS). These offerings from Azure and AWS are security feature-rich. Our worker nodes meet Kubernetes CIS Level 1 Benchmarks. Cluster access is restricted to operators, and least privilege is enforced using specific RBAC roles. We use Kubernetes NetworkPolicy objects to restrict pod network ingress and egress to authorized sources and destinations.

![Kubernetes Security](https://images.ctfassets.net/23aumh6u8s0i/48mIINcAPiqFzHe39D2I53/eeb1592c93f41b01ed465d6abd7bf349/06_private_cloud.png)

### Vulnerability Management

Our build and deploy pipelines continuously scan our cluster, containers, and code for vulnerabilities. Our security capabilities include the usual controls: static application security testing (SAST) on both code and infrastructure-as-code, dynamic application security testing (DAST) on our application and API endpoints, and software composition analysis (SCA) on our software dependencies and containers. Cloud Security Posture Management (CSPM) tooling identifies and responds to cloud misconfigurations (e.g., publicly accessible S3 buckets). Patching is a foundational security control; we continuously upgrade our containers and software dependencies. In the shared responsibility model, Azure/AWS are responsible for building upgraded node images, and Auth0 is responsible for deploying the images to our nodes. We accomplish this by rebuilding our Kubernetes nodes with the latest image during each Auth0 product release cycle.

### Automation is a must

We are dealing with a scale that makes it impossible to configure all of these security capabilities manually. To address his challenge, the Auth0 Security team created an in-house service that automates AWS account and Azure subscription creation and configuration. Similar to a production line, when new Spaces and PoPs are needed, this service creates AWS accounts and Azure subscriptions to meet our strict security standards using predefined templates. It effectively "stamps out" completely configured new accounts/subscriptions on-demand. When the account or subscription is no longer needed, the service will remove all resources, revoke all access, and decommission it.

## Continuously Improving Auth0's Security Posture

All of this work has culminated into a more secure, scalable platform that continues to meet or exceed the many requirements needed to achieve ISO 27001/27018, SOC 2, PCI, STAR, and HIPAA compliance.

The threat landscape is ever-changing and constantly evolving. The Auth0 Security team's job is never done, and a core principle is a continuous improvement. There are always ways to mitigate more risk and enhance our security posture, and we continually investigate additional security capabilities to make our platform and product even more secure for our customers.

We are publishing a series of technical posts on how and why we re-platformed. They'll be linked at the bottom of this post as they publish and are also findable on our blog. If you have technical questions or ideas for future posts, please reach out to us via the [Auth0 Community](https://community.auth0.com/). (Our marketing team also tells us we would be remiss if we didn't include a link to talk to Sales, so that is [here](https://auth0.com/contact-us), too.)

<include src="asides/AboutAuth0" />

Securing Auth0's New Private Cloud Platform

df6c864847fba9687d962cb80b482764?size=200 avatar

df6c864847fba9687d962cb80b482764?size=200

Martin Gontovnikas, a.k.a Gonto, is a software engineer at heart who moved to the ‘dark side’ and became VP of Marketing at Auth0. He considers himself lucky to have found a way to combine his two passions and apply his engineering thinking model to marketing. Gonto’s analytical thinking is a huge driver of his data-driven approach to marketing strategy and experimental design. He is based in the Bay area, and in his spare time, can be found eating gourmet food at the best new restaurants, visiting every local brewery he can find, or traveling the globe in search of new experiences.

Former SVP of Marketing and Growth at Auth0

Martin Gontovnikas

Streaming companies must adapt to unprecedented network strain

Everyone is coping with the Coronavirus pandemic in their own ways, yet people around the world seem to share a few rituals: washing our hands, wearing masks, and of course, catching up on our Netflix queue. 

With much of the world’s population stuck at home under social distancing, quarantine, and mandatory lockdowns, consumer spending on streaming has increased [anywhere from 20% to 50%](https://www.nytimes.com/interactive/2020/04/11/business/economy/coronavirus-us-economy-spending.html) depending on the channel. Given this unprecedented surge in demand, it’s not hyperbolic to say the internet is going through the most intense stress test in its history.

The global desire for entertainment and escapism is [creating a windfall for streaming services](https://www.cnn.com/2020/03/02/investing/coronavirus-stay-at-home-stocks/index.html). Netflix, for example, [more than doubled](https://www.theguardian.com/media/2020/apr/21/netflix-new-subscribers-covid-19-lockdown) its projected number of new subscribers since the pandemic began, and [Disney+ added](https://www.theguardian.com/media/2020/apr/09/disney-netflix-rival-doubles-subscriptions-in-virus-lockdown) 22 million subscribers as well. However, a combination of network slowdowns (and failures) and streaming models that don’t monetize increased traffic means that the situation for streaming media companies is more complicated than it first appears.

Media companies are facing new challenges and new opportunities and working to develop technological solutions that help them weather this historic moment.

## Quarantined Nations Increase Their Streaming Time

We can see the effects of COVID-19 on streaming services by watching how currently quarantined countries (those hits hardest by the virus) have been behaving. Even without a full-blown quarantine, the rise of voluntary social distancing and enforced isolation creates at least a comparable situation.

Though China [is showing signs of recovery](https://www.cnn.com/2020/03/19/asia/coronavirus-covid-19-update-china-intl-hnk/index.html), they implemented mass quarantine first and can give us useful data on streaming behavior. Chinese Nielsen [data shows a significant rise](https://mp.weixin.qq.com/s/vxKFJOFeXLWuasuvtu7kBw) in TV viewing, but people are also streaming less conventional content. Quarantined Chinese citizens are getting creative with streaming entertainment, [streaming concerts](https://www.nytimes.com/2020/03/04/arts/music/arts-coronavirus.html), and [setting up digital clubbing and digital gyms](https://www.bbc.com/news/world-asia-china-51628693) in their living rooms.

Italy, still in quarantine, has seen a similar rise in streaming content, with an average viewing-time increase of [11% nationally with highs of 17%](https://www.corriere.it/spettacoli/20_febbraio_29/effetto-coronavirus-volano-ascolti-telegiornali-talk-9399464c-5afe-11ea-8b1a-b76251361796.shtml?refresh_ce-cp) in the Lombardy region. During the nationwide quarantine, the Italian telecoms giant Telecom Italian SpA has reported an overall [70% increase in internet traffic](https://www.bloomberg.com/news/articles/2020-03-12/housebound-italian-kids-strain-network-with-fortnite-marathon) from work-from-home traffic and daytime online gaming.

**The spread of the novel coronavirus is also forcing studios and streaming services to rethink how they create and distribute content**, jumping release dates forward, skipping the box office, and closing down production. This will all lead to greater strain on streaming services and the larger internet.

The outbreak is closing theaters [across the United States and across the world](https://www.npr.org/2020/03/17/817136846/as-coronavirus-shuts-down-theaters-universal-shifts-some-films-to-on-demand), which has been linked to a [13% rise in streaming in the US that could rise up to 60%](https://time.com/5806060/coronavirus-movie-theaters-streaming/) based on prior crises. Production companies are fighting this by streaming [first-run movies straight to home services](https://www.vulture.com/2020/03/coronavirus-is-pushing-movies-online-faster-than-ever-before.html): Universal released _Emma_ and other new movies to streaming services, the new Pixar movie _Onward_ is going right to Disney+, and huge films like _Frozen 2_, _Birds of Prey_, and _The Invisible Man_ are coming much sooner than their prior release date.

While these first-run movies popping up on streaming services does create a large viewership base at home, it may not lead to the massive streaming profits being predicted.

<include src="TweetQuote" quoteText="Consumer spending on streaming media is up 20-50%. What streaming media companies can do to make sure they can withstand the quarantine-driven stress test."/>

## Streaming Services Are Doing Better Than Expected

It seems like a simple, logical equation: more time at home equals more streaming, which means greater profits for streaming services. But many streaming services, like Netflix, aren’t modeled to reap more profits from an increase in individual streaming time - which is why some were predicting that Netflix might not see a huge boost in profits.

 “If you have to decide between food and your Netflix subscription, it seems to me like you disconnect your Netflix. You’re probably not going to add another $13 a month if you don’t know when your next paycheck’s going to come in.” — Laura Martin, the senior entertainment analyst at Needham & Company, [told the Guardian](https://www.theguardian.com/film/2020/mar/12/coronavirus-binge-watching-netflix-streamers-viral).

**Netflix doesn’t monetize volume**. They make their money [from subscriptions](https://www.vox.com/2019/12/17/21025154/netflix-global-growth-numbers-sec-streaming-investors) ([and some product placement](https://www.cnbc.com/2017/07/27/bill-gates-ben-does-product-placement-in-netflix-and-amazon-shows.html)). The stock market reflects this uncertainty — Netflix has seen [less than a quarter of a percent of growth](https://www.cnbc.com/quotes/?symbol=NFLX) in the last month.

However, despite a [majority of consumers being aware of Netflix](https://morningconsult.com/2019/09/19/in-tvs-content-war-netflixs-shows-capture-biggest-share-of-viewer-awareness/) before lockdown and quarantine conditions, Netflix has seen a surprising surge in subscriptions. In fact, they’ve doubled their estimates: from an estimated 7 million new subscribers [to 15.77 million new subscribers](https://www.theguardian.com/media/2020/apr/21/netflix-new-subscribers-covid-19-lockdown). 

This may be a temporary situation however: considering the shutdown of television and movie production across the board, time will tell if new subscribers will stick around for old content.

**A la carte streaming services like Amazon Prime may see a boost in revenue**. An Amazon Prime subscription [nets users a variety of shows and movies](https://www.businessinsider.com/amazon-prime-benefits-what-is-included) for “free,” like Netflix, but the newest releases must be rented or purchased.

Then again, despite the [rush of online ordering](https://blog.aboutamazon.com/company-news/amazons-actions-to-help-employees-communities-and-customers-affected-by-covid-19) and their strong grip on streaming media, [Amazon stock isn’t seeing much of a surge either](https://www.cnbc.com/quotes/?symbol=AMZN). Though the entire stock market has taken a hit, [early predictions that streaming company stock would be safe refuges](https://nypost.com/2020/02/28/how-coronavirus-benefits-netflix-and-other-in-home-services/) may have been premature.

Meanwhile, Twitch — the [most popular streaming service](https://www.nytimes.com/2019/12/15/business/tech-video-game-streaming-twitch.html) for gaming videos and livestreams — [has seen a 10% growth](https://www.theverge.com/2020/3/18/21185114/twitch-youtube-livestreaming-streamelements-coronavirus-quarantine-viewership-numbers) in viewership as of March 18. Unlike Netflix or Amazon Prime, Twitch monetizes through ads and will likely see a commensurate rise in revenue.

But differing monetization methods don’t tell the whole story. These potential revenue boosts or profit stagnation trends are dependent on network architecture surviving the sudden swell of usage, which may not be a guarantee.

## Strained Network Architecture Will Need Relief

Social distancing and quarantine [are closing schools](https://www.npr.org/sections/health-shots/2020/03/16/816428320/coronavirus-u-s-enters-quarantine-life-as-many-schools-and-businesses-close) and [workplaces](https://www.washingtonpost.com/national/coronavirus-outbreak-shutdown-america/2020/03/13/d8589434-6550-11ea-acca-80c22bbee96f_story.html), which means a glut of people at home streaming media for entertainment, work, and educational purposes placing unprecedented strain on the internet. Some websites are [even taking bets](https://www.sportsbettingdime.com/news/odds-global-netflix-outage-playstation-network-xbox-live-crash/) on which streaming or web service will go down next.

[Microsoft Teams crashed](https://www.independent.co.uk/life-style/gadgets-and-tech/news/microsoft-teams-service-status-crashes-outage-issues-today-work-home-down-a9404771.html) under the increased load of remote video conferences, and YouTube and Netflix have started [throttling quality on their videos](https://edition.cnn.com/2020/03/19/tech/netflix-internet-overload-eu/index.html) in order to reduce bandwidth in the face of the surge. Hulu [users reported a widespread outage](https://www.cnet.com/news/hulu-is-down-as-users-complain-of-videos-not-loading/) on March 20, one that seemed to affect multiple devices:

![hulu-activity-03-20-2020](https://images.ctfassets.net/23aumh6u8s0i/6Gp4ntk7M1cxNGwTHZxD3k/049ae9a357782e60896684ff437b7297/hulu-activity-march-20-2020)
[DownDetector.com](http://DownDetector.com) - Hulu - March 20, 2020

Many companies [suspended their data caps](https://www.cnet.com/news/carriers-are-suspending-internet-data-caps-during-coronavirus/) and [increased streaming limits](https://www.pcgamer.com/discord-ups-streaming-viewer-limit-while-coronavirus-keeps-people-at-home/) in order to help people stay entertained and in contact during isolation and quarantine. However, these acts of generosity may increase the amount of simultaneous traffic online.

Streaming services may need to try new network solutions to stay ahead of this bloom of potentially network-crashing traffic.

## Private Cloud Boosts Streaming Infrastructure

In the face of this inflating audience and inflating screen time, streaming services using a public cloud may want to consider other options to bolster their network architecture.

There’s nothing wrong with choosing a public cloud; it’s a perfectly reasonable option for a smaller streaming service. Larger streaming services, however, can increase their bandwidth and reduce their downtime and slowdowns with a private cloud solution.

A private cloud service like Auth0 offers scalability and can provide a [greater request-per-second](https://auth0.com/blog/how-finservs-can-reduce-risk-increase-innovation-with-private-cloud/) (rps) baseline — one that can add capacity as needed. It also has enhanced uptime over a public cloud, with a guaranteed 99.99% uptime.

<include src="TweetQuote" quoteText="How private cloud can help streaming companies withstand quarantine streaming pressures."/>

A private cloud doesn’t share resources or bandwidth with other companies, which is where a public cloud can sometimes struggle. Public cloud services like AWS do create server instances as needed, but they’re doing that for every business or individual using the service. COVID-19 isolation causes internet traffic to increase through every channel — streaming media, [communication](https://www.voicesage.com/blog/communication-best-practices-for-organisations-working-to-contain-covid-19/), [education](https://nypost.com/2020/03/17/coronavirus-in-ny-teachers-prepare-for-unprecedented-challenges-of-remote-learning/), and more — public cloud services are going to be hit from multiple angles at increasing speeds.

A private cloud can help streaming companies with servers dedicated to their businesses. Moreover, a private cloud can be managed or unmanaged, depending on whether you have the time and dedicated staff to maintain it.

## Getting Through This Together, Apart

The COVID-19 pandemic is a situation without precedent, creating entire nations of at-home workers, students, and entertainment-seekers all jumping on the internet at the same time. And with isolation and [lockdown orders](https://www.wsj.com/articles/china-reports-no-new-domestic-coronavirus-infections-for-the-first-time-since-outbreak-started-11584611233) coming [faster every day](https://www.washingtonpost.com/world/2020/03/20/coronavirus-latest-news/), this isn’t a situation [likely to change in the immediate future](https://www.washingtonpost.com/health/2020/03/16/social-distancing-coronavirus/).

Keeping streaming services going is a necessity for education, entertainment, and the economy. After a discussion with Netflix CEO Reed Hastings, the Internal Market Commissioner of Brussels, Thierry Breton, had this to say in [a statement to Politico](https://www.politico.eu/article/brussels-in-talks-with-netflix-about-reducing-internet-congestion/):

 “We all have a joint responsibility to take steps to ensure the smooth functioning of the internet during the battle against the virus propagation.”

With scalable network solutions such as a private cloud, we can deploy and use more servers to keep everyone entertained, educated, in shape, and in contact with one another until the situation is resolved. And, hopefully, we can all get back to business — and life — as usual.

<include src="asides/AboutAuth0" />

The (Surprisingly Complex) Impact of COVID-19 on Streaming Media

jenny-obrien

I'm a former business journalist who's fascinated by the intersection of tech and business strategy — in particular how the right solution can transform an idea into reality. I specialize in ROI, data privacy, and identity.

Business Content Manager

Jenny O'Brien

Gain cloud flexibility and agility while maintaining security and control

Amazon famously brought [book publishing “to its knees”](https://mashable.com/2014/07/30/amazon-has-killed-publishers-they-just-dont-know-it-yet/) by dominating distribution, creating eBooks, and transforming the publishing world with its own titles and opportunity for authors to self publish. 

In 2018, [Amazon entered the insurance market in India](https://www.insurancebusinessmag.com/us/news/breaking-news/amazon-set-to-file-for-insurance-licence-111581.aspx). [And in 2019, the U.S.](https://www.geekwire.com/2019/amazon-jpmorgan-roll-new-health-insurance-plans-part-haven-joint-venture/) 

After killing off [car insurance comparison tools](https://www.businessinsider.com/google-kills-auto-insurance-comparison-tool-2016-2), Google’s invested in [insurance agency software](https://www.insurancejournal.com/news/national/2019/01/02/513324.htm) and signaled further interest in the market. 

In other words, digital transformation is already here. 

While industry complexities afford some protection, they offer greater opportunity for insiders, says TechCrunch. 

A recent [J.D. Power survey](https://www.jdpower.com/business/press-releases/2019-us-home-insurance-study) on customer satisfaction with home insurance noted that the home insurance business model was “primed for change” and that “weak points in customer experience offered an opportunity for startups.” 

The other side of the desire for customer experiences that are real-time and online is the fact that FinServs are tasked with protecting hackers’ favorite data targets. And the [global rise in data privacy regulation](https://auth0.com/explore/data-privacy/) makes stronger protections important from customer, legal, and compliance perspectives.

While digital transformation can seem daunting for incumbents relying on decades of legacy code, the right identity solution can create seamless customer experiences while increasing your data protection — even if you’re an incumbent partnering with (or acquiring an) up and coming Insurtech startup.

Forbes expects [XaaS (everything as a service), UX, CX (User/Customer Experience), and digital privacy](https://www.forbes.com/sites/danielnewman/2019/07/14/top-10-digital-transformation-trends-for-2020/#3dfc8bdc76be) to “take center stage” in as a digital transformation trend 2020. Like other industries, the need to streamline services, reduce capital expenditures, and be increasingly agile and innovative is fueling FinServ’s change. But FinServ’s need for security, stability, and high availability kept many systems in On-Prem solutions that have proven expensive to scale and maintain — putting the ability to keep up with evolving customer expectations at risk.

For FinServ’s looking to innovate (or even just keep pace), Public Cloud is often touted as an option, but it isn’t the best choice. Public Cloud outages and uptimes provide a strong reason to go private. Auth0 Private Cloud helps (or eliminate) that risk, by providing a secure, flexible, extensible solution. 

![Deployment Options Diagram](https://images.ctfassets.net/23aumh6u8s0i/23TVeU9N9cKlxxrenpT6hA/ae7991f61be42885833856bed91a74f7/deployment_options_diagram)

## Why Is Private Cloud Particularly Suited for Finserv Environments

For many years, incumbent FinServs have been buffered by deep compliance capabilities and an established customer base. But FinTech disruptors have eroded that edge says a recent PWC [report](https://www.pwc.com/gx/en/financial-services/assets/pdf/technology2020-and-beyond.pdf), in part because successful disruptors are able to offer a stronger customer experience at a lower cost to customers and with less investment. 

To solidify standing and increase the customer base in this environment, FinServ’s need to:

* Improve customer experience
* Keep ahead of evolving data privacy regulations
* Reduce costs without sacrificing cybersecurity

The data that FinServs hold makes them a prime target for hackers — and the sheer company scale usually means millions of breached accounts, rather than thousands. Globally, the increase in data privacy laws like GDPR and CCPA demonstrates that regulators have lowered tolerance for breaches and expect greater protections to be in place. When not implementing features like multi-factor authentication (MFA) can hit an insurer with a large fine, like what happened to [Dutch public insurance company UWV](https://auth0.com/blog/gdpr-fines-hit-these-companies-hard-heres-how-to-avoid-them/), the need for greater security while still ensuring flexibility becomes more appealing.

<include src="TweetQuote" quoteText="The data that FinServs hold makes them a prime target for hackers — how Private Cloud can help."/>

From an identity perspective, let’s take a closer look at what Auth0 Private Cloud can mean for key digital transformation considerations.

### Performance at Scale
The most innovative solution in the world does zero good if it doesn’t offer you the performance you need. Auth0 Private Cloud offers request-per-second (rps) baseline is higher than public cloud, with the option of adding on capacity up to 1500 rps.

### Reliable High Availability
FinServ is more aware than most industries the dramatic impact of service interruptions, especially given the pressures of changing global conditions. High availability helps guard against those challenges, especially since Private Cloud can be deployed between two regions to provide failover.

### Control & Segregation
Managed Private Cloud offers a fully-isolated development environment that is updated independently from the production environment. Standard Private Cloud is a single instance production environment. Both Standard Private Cloud and Managed Private Cloud meet standards that require all data to be housed within specific regions.

### Free Developers to Improve Customer Experience
For a developer coming up with an innovative solution while working only On-Prem can feel like trying to code while wearing handcuffs — it’s hard to even get your hands on the keyboard. 

To keep up with the pace of customer’s hunger for real-time innovations, streams of data from connected vehicles and smart homes, and the data privacy compliance requirements from laws like GDPR and CCPA, developers need not only the flexibility but also the scale of a [cloud deployment](https://auth0.com/cloud). And they need that deployment to be secure and reliable enough to handle the demands of enterprises whose customers are often interacting during moments of high stress or need.

### Unify Legacy and New Data Stores
Unifying legacy and new data stores can be a giant headache, as well as a source of weeks of downtime without a plug-n-play identity solution that allows bulk imports, but can also streamline user migration without expensive forced password resets – whether unifying stores at a single organization or when an incumbent partners or acquires an insurance startup.

### Support for Different Languages/Architectures (SDK)
Aside from unifying data stores, incumbent companies often rely on a maze of programming languages and architectures. Shutting down one part of the maze can have unexpected consequences if all the interrelationships aren’t fully mapped out or even understood due to poorly commented code or loss of institutional knowledge due to attrition or staffing turnover.

Your devs will be a lot happier (and more productive) when working with an identity solution that can handle multiple programming languages and architectures and already has SDKs (software development kits) that can be dropped into place to provide rapid access to functionality. And the right solution gives you the flexibility to map the maze at your own pace, reducing costs by removing unnecessary middleware, maintenance, and security as you deem appropriate.

### Extensibility
<iframe width="560" height="315" src="https://www.youtube.com/embed/kCmU4EMMPf0" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

The real world is a messy place where you might have an out-of-the-box tool that needs to be integrated with custom code. The ability to inject code into the Auth0 Identity solution means that developers can handle the last-mile customizations that execute on the C-suite’s business strategy without weeks of downtime or missed opportunities. To delve into this topic in greater technical detail, you might also check out Auth0 Principal Architect Vittorio Bertocci’s whitepaper “[The Tao of Extensibility.](https://auth0.com/resources/whitepapers/tao-of-extensibility)”

## Support and Failover
Uptimes, updates, failovers, and support can have a big impact on the success of your deployment. 

With Public Cloud coming in with an uptime SLA of 99.90%, Private Cloud offers a stronger uptime SLA of 99.95% with the option to upgrade to maxim uptime guarantees of 99.99%. Geo redundancy can provide resiliency against catastrophic events. As mentioned above, Private Cloud offers failover between two geographic regions. 

For teams that would appreciate additional support, Auth0 offers a support SLA as an add-on, as well as the add-on option to work with the skilled identity experts on our Professional Services team.

## Managed Private Cloud vs. Standard Private Cloud
Let’s take a look at the different deployment Private Cloud models and benefits, including a checklist to help you assess deployment needs.

**Standard Private Cloud** is a low touch engagement model that offers additional power, security and compliance* benefits through an isolated tenant just for a single customer. Uptime SLA is 99.95% with an upgrade possible to 99.99%. PCI compliance is available as an add-on.

**Managed Private Cloud** is a high-touch engagement model where customers are much more involved with all aspects of this deployment model. Our team will work closely with the customer to ensure all aspects of their Auth0 environment are tuned for what best suits their business needs. Uptime is 99.99%. PCI compliance is available as an add-on.

**Reasons to choose Managed Private Cloud** * **Choice of update frequency** Coordinated with Auth0 (required once per 3 months). The update cycle begins with the Pre-Production Environment.
* **A separate DEV (Pre-Production) environment** Fully-isolated and independently updated instance for development and testing (Single Node)
* **On-demand scaling** WIll scale customer environment for any spikes * **Load Testing** 4x Load Testing per year with dedicated Auth0 MSE Support * **Customer-hosted AWS deployment** More **control** over the environment * **GEO-HA deployment option** Datacenter **redundancy** and automatic **failure handling**. 
 
**Reasons to choose Standard Private Cloud**
* ***Doesn’t require any of the Managed Private Cloud reasons***
* **More cost-effective**
* **Less involved with managing the deployment** The customer is looking for an offering that feels similar to a standard Public Cloud SaaS but has isolation and the other advantages of our Private SaaS Deployments. |

<include src="TweetQuote" quoteText="Legacy code can make #digitaltransformation feel like a high hurdle for FinServs. Find out how Private Cloud can help securely smooth the jum"/>

## Reduce Risk, Increase Innovation With Private Cloud
Like identity, digital transformation is not a one-and-done solution, but an ongoing need to keep pace with the evolution of technological change. Auth0 Private Cloud provides the flexibility, agility offered by cloud deployments but allows FinServ’s to maintain the control and security they need to protect their customers and their businesses. Your IAM solution is a critical component of your security strategy, but even if you’re dealing with decades of legacy code, you don’t need to sit down and build it in-house. The more secure option is to let identity experts protect those critical credentials, freeing your team (and resources) to focus on what you do best. Identity no longer needs to be a roadblock or an afterthought. Identity Access Management (IAM) is an essential component of customer security and privacy. If you’d like to learn more about how Auth0 Private Cloud might work for your particular situation, please reach out to an [Auth0 resource](https://auth0.com/).

"private-cloud"