Previous post: [Yada Yada Cloud] Choosing from Exchange On-prem vs Exchange Online: The never ending story? – Part I

Continuing this two set article about Exchange on-prem vs Exchange Online, I’m going to cover the most common conversation points from #5 to #8 and one bonus track.

As in my previous post, I’ll review the Exchange on-prem, Exchange Online and my personal perspective about each topic.

Let’s start.

5. The data security concerns about cloud and Microsoft being the “big brother”

This is a recurrent topic when people from security, legal and auditing are present in the discussion. There are organizations with sensitive and confidential information that are fairly concern about their data being stored and managed by people outside their company.

The Zero Wing reference for “All your base are belong to us“

Let’s review about the Exchange On-Prem view:

• At any moment, Microsoft can say yes to a higher institution and provide our company’s private data.

Microsoft and the Ireland case represents one of the most important affairs regarding digital security. The case began in December 2013 when a judge in New York issued a warrant asking Microsoft to produce all emails and other private information associated with a customer account. This account was involved in a criminal narcotics investigation.

This user data was being stored in one of Microsoft datacenters in Ireland. Microsoft produced account information kept on servers in the United States, but refused to produce any data stored on servers in Ireland, arguing the government’s warrant does not apply in foreign countries.

There was a lot of back and forward from 2013 to 2016, but ultimately Second Circuit Appeals Court ruled in favor of Microsoft and against the US Government, confirming that the Redmond Company did not have to provide the data (here’s the full order available in PDF).

• Microsoft could be selling our private data to other companies, for example data they can obtain from Delve to target users from their preferences to advertise a product.
• We can use the capabilities for e-mail security using Exchange Online Protection (EOP) service, without the need to take all of our data to the cloud.
• There are some scenarios, due to the sensitive information, the emails need to be isolated as much as possible for security reasons.
• If Hillary decided to go on-prem and worked for her, why not us?!

During election times in 2016, as anyone can imagine, this was a recurrent topic to discuss when an Office 365 migration meeting occurred. The full and detailed explanation about this affaire can be reviewed in Wikipedia: “Hillary Clinton email controversy”.

As a quick summary of the incident, Hillary Clinton allegedly used a personal email server (Exchange 2010 platform located in a datacenter in their hometown in Chappaqua, New York) to send and receive classified information from the Secretary of the State, when she was acting as the leading role in that department.

During the several investigations over the years, Hillary Clinton handed to the authorities over 30,000 emails as printed versions to confirm the content was not classified. They also turned over the servers (two Dell PowerEdge 1950 Servers, one for Exchange 2010 and one server for the BlackBerry Enterprise Server) and their drives, which the authorities later mentioned that those disk drives were erased before properly received.

The FBI investigation concluded stating that Clinton was “extremely careless” in handling her email system but recommended that no charges be filed against her.

This is a screenshot of the still active (to this date) Exchange 2010 OWA for Clinton’s email: https://mail.clintonemail.com/owa

Taking a look to the Exchange Online approach:

• Microsoft can only provide information to the government in special exceptions, when there’s a risk of national security.

Have you ever wondered how many times did Microsoft disclose customers’ information to the government? There’s the Microsoft Law Enforcement Requests Report, which is updated twice a year with a complete detail on how many cases and type of responses to the government Microsoft had to provide.

In this global support dashboard, only 2.65% of the cases content was disclosed (majority of disclosed data represents just information about the subscriber). You will also see in the detailed XLS than can be exported, almost 95% of the countries involved had 0 cases where they needed to disclose content.

Microsoft also created the concept of “customer lockbox”, which includes the customer in the decision workflow, if possible by the country’s law, when a request for disclosing information is placed. This way the customer has a vote to decide if they want the information to be disclosed or not.

• The security standards and processes implemented by Microsoft to safeguard the servers and information is usually much higher than most of the companies available.

Providing the high level of security that Microsoft provides by default in Office 365 can be extremely expensive for most organizations and challenging to maintain. And if a security breach occurs, the entire organization loses faith in IT.

Here are some of the links and references available from Microsoft where you can deep dive on the security and legal concerns about data in the cloud, regulatory compliance, etc:

Office 365 Trust Center

https://www.microsoft.com/en-us/TrustCenter/CloudServices/Office-365

Compliant with Global Standards

Standards Conformance & Regulatory Compliance
Security, Audits, and Certifications
Use Office 365 to help comply with legal, regulatory, and organizational compliance requirements
Top 10 compliance areas of Office 365
FISMA/FedRAMP: Frequently asked questions
EU Model Clauses—frequently asked questions
Office 365 & Microsoft Dynamics CRM Online HIPAA/HITECH frequently asked questions

Protected by Lawyers

Law Enforcement Requests Report
U.S. National Security Orders Report
Content Removal Requests Report
Office 365 Privacy Statement

Tested by Security Experts

CodeChat 003 – Ben Godard (Office 365 Red Team)
From Inside the Cloud: What does Microsoft do to prepare for emerging security threats to Office 365?
Security, Audits, and Certifications (including details on Office 365 ISO certifications)

Trust through Transparency

Auditing and Reporting in Office 365
​Controlling Access to Office 365 and Protecting Content on Devices
Data Encryption Technologies in Office 365
​Data Resiliency in Office 365
​Defending Office 365 Against Denial of Service Attacks
​Financial Services Compliance in Microsoft’s Cloud Services
Office 365 Administrative Access Controls
Office 365 Customer Security Considerations
Office 365 Security Incident Management
Privacy in Office 365
Tenant Isolation in Office 365

As my personal opinion on the Office 365 security matter:

The security conversation can also be difficult to have when we have organizations or just a few people with very sensitive information. It can also transform into a philosophical discussion on trying to define who owns the information used by the people and their productivity; or even going further, what are the boundaries of “owning” the information.

Mostly the discussions go around “How secure I feel by allowing Microsoft to store my information?” and “How much do I trust in Microsoft and the use they can have with my data?”

The regulations and standards established as baselines for the cloud services are clear, and the probability on having Microsoft or the government trying to obtain maliciously our private data will always be lower than the risk on having any malicious internal or external party trying to access our information. Considering also the large investment a company needs to make to obtain a similar type of security from what Microsoft offers in an on-premises environment.

6. The challenges of remote locations and inconsistent Internet connections.

The remote locations and slow internet connections discussion can appear in those organizations distributed in large regions. It may not seem all that common in the US since the connections available in most of the country are fairly acceptable, but there are other countries and continents that can experience these issues.

Let’s review about the Exchange on-premises point of view:

• Staying with Exchange on-premises, in case of troubleshooting, we remove the Internet connection as a variable.
• The information handled within email it’s highly confidential, we cannot expose that platform to public connections.
• We have sensitive users that are highly dependent on Outlook and sometimes they don’t have good connections. Outlook connected through Internet requires a good amount of bandwidth.

As we know, Outlook Clients can be an intensive application for CPU, memory and network bandwidth. For the latter, there are several variables than can affect the experience of a “slow network response”. Microsoft developed, several years back, the “Exchange Client Network Bandwidth Calculator” to size properly the necessary bandwidth in one or more locations.

Uses as reference Outlook 2007 and Outlook 2010, they haven’t updated that yet, but the network behavior hasn’t changed all that much in the later versions.

Let’s take a look about an Office 365 example using this bandwidth calculator. Here are the parameters for a “Very Heavy” Outlook client user:

The one item to observe is the “Avg Mailbox Size (GB)”. I’m using the 100GB scenario to include the worst case on all of these users with a full mailbox for O365, which won’t be the case in almost any company. This parameter is actually looking for the OST resyncs to be performed from these clients.

As an exercise, I’m also using the “Medium” profile (average mailbox usage of 20GB, the rest stays the same) and “Heavy” profile (average mailbox usage of 50GB, same for the rest).

Here’s the bandwidth calculation of a mixture scenario: 300 Outlook 2010 (cached mode enabled); 50 Mac users; 50 Outlook 2007; 50 users with OWA; and 300 mobile clients (we can fairly assume most of the Outlook users will be using a mobile at the same time).

For the “Very Heavy” scenario, which is the worst case, we have the 37.51Mbits/sec bandwidth requirement for download.

Going to the other side and reviewing the Exchange Online perspective:

• For the remote places, unless you have the mailbox server in that remote location, your problem will stay the same. The Outlook client will use Internet (Outlook Anywhere or VPN) to connect. The internet connection will always be part of the variables.
• Outlook connectivity might be a problem in some scenarios, but OWA and ActiveSync for mobiles can definitely work with slow connections.
• Microsoft offers several tools and guidelines to assess and optimize client connectivity for Office 365. One of the most valuable ones is “Office 365 Network Analysis Tool” (link included for North America, but there are also available for Europe and one for Asia-Pacific).

Here are some other tools and guidance material available to consider:

• “Best practices for using Office 365 on a slow network”
• “Network and migration planning for Office 365”
• “Network planning and performance tuning for Office 365”
• “Generate Message Profile for the Exchange Calculators”: PowerShell script (updated frequently) to obtain detailed information about the Exchange environment and usage within the organization, works for Exchange 2010, 2013 and 2016. For more information about using this script: “Generating user message profiles for use with the Exchange Calculators”.

But what happens if you get the “My network connection is suitable and I already checked all the best practices for Office 365, but my Outlook connection experience is still poor”, here are a couple of tips to troubleshoot and validate Outlook issues and network problems.

• If you are suspicious on inconsistencies within the applications, “Microsoft Office Configuration Analyzer Tool” (OffCAT) will provide you on any problem related about application configuration, patch level or even data corruption.
• Use “Remote Connectivity Analyzer” to validate issues within an existing Exchange environment, for troubleshooting Outlook issues, certificates inconsistencies, account setup, etc.
• There’s a long history of Outlook add-ons causing issues within the application (iCloud add-on with Outlook 2013 for example). Try to disable all of them for proper testing.
• Graphics acceleration for Outlook 2013 and 2016 can cause problems, you can disable it from Options > Advanced > Disable Graphics Acceleration.
• Limiting the number of calendars in the “My Calendars” folder will improve Outlook performance.
• Having over 5,000 items in any of Outlook folders (Inbox, Sent, etc.) usually cause slow responsive behavior in Outlook.
• It is well known for any iOS or Android users that the native mail applications do not work very well with Exchange (online or on-premises). Try to avoid those for troubleshooting.

As my personal opinion in this matter:

We are already in a world where decent Internet connections represent a basic and common service, like electricity. It’s hard to believe most companies will put a stop into innovation because there are still users who are exclusively Outlook dependent and they also do not have a decent Internet connection.

There are very few cases where they are using this scenario of no public connections for email connections. With smartphones, tablets and so many technologies and devices; user productivity will always go beyond a LAN connection for the messaging service.

Companies should try, if possible, to isolate those particular use cases and offer workarounds to the cloud service, like ActiveSync or OWA (which has great enhancements to make the web experience very similar to Outlook). And of course the hybrid scenario is useful again for this case.

7. Collaboration platform constructed and integrated already on-prem, re-building this scenario might be impossible in Office 365.

There are some customized scenarios customers have in their environment, where several collaborative and productivity platforms are highly integrated, for SAP platforms integrated with Exchange and printers so invoicing for a company runs completely automated. Let’s take a look about each of the perspectives.

Reviewing the approach of Exchange on-premise:

• We already invested time and money to complete these integrations, best case scenario we will have to make a new investment to achieve this migration.
• We don’t even know if our SAP system –or any other- will be able to support Office 365. Microsoft will not provide support for 3^rd parties.
• We cannot install anything on servers if we need a particular customization.
• Office 365 is offering some ways to connect with their systems using APIs, but that integration is limited.

For the Exchange online perspective:

• The systems provide native support for the vast majority of collaboration and productivity platform, like SAP and thousands of others.
• For the case of in-house developments of applications integrated with Exchange, most of companies have developed components integrated with AutoDiscover and EWS (Exchange Web Services) already; which can be easily integrated with Office 365.

Microsoft offers complete guidance on how to achieve these integrations. Here are some links:

Autodiscover for Exchange from MSDN
How to: Use Autodiscover to find connection points
Get started with EWS client applications
Get user settings with Autodiscover, sample application and code (C#)

• There are hundreds of applications available in Azure Market Place, where there are a big number of apps for Azure Active Directory, which is the identity backbone for Office 365. Here’s the link of all the Azure Active Directory applications available and ready to use.

As my personal opinion regarding this subject:

Companies that are constantly requiring in-house customizations on their collaboration platforms might not be suited for Office 365. Personally, I’ve never found an organization that had a large amount of customizations that cannot be implemented within O365 using Azure Market Place apps or taking advantage of the built-in features within the cloud platform. Data Loss Prevention (DLP) capabilities and Right Management Services (RMS) are the services I’ve seen the most as used by companies with 3^rd party products; and those services are already included in Office 365.

If we are talking about a scenario where there was a large customization investment from a company and they don’t want to change their platform, it is also a dangerous scenario because it implies that upgrading the Email platform (servers and/or clients) represents a high risk of losing functionalities. Therefore, in this case, the company is condemned to keep outdated versions of products for a long period of time.

8. Office 365 service availability is hard to track, Exchange on-prem monitoring can provide a clear diagnostic for the problem.

Those responsible about the monitoring of the platform can go either way: Or they can feel relieve that they don’t have to put much effort in monitoring a cloud solution; or can be really hectic about not knowing the service’s components health at all time.

Let’s take a look for the Exchange on-prem approach:

• In case we need to troubleshoot, we can find the solution to the problem and also apply it.
• With Exchange on-prem, we control the complete scenario: Clients and users, network and servers, external connectivity, etc.

One great feature in System Center Operations Manager (SCOM) are Distributed Applications, from which in one simple and clear diagram we can obtain a detailed view about the health status on all the components from our application. Distributed Applications are highly customizable, so your diagram can include anything that SCOM can monitor (which can even include, among other things, the UPS where your servers are connected).

This is an example of an Exchange Distributed Application in SCOM:

Not shown in the diagram but available in SCOM are the Watcher Nodes. You can install an agent (in a VM for example) in remote sites that will constantly monitor connections from the agent to your Exchange server. The diagram will show the ability for the agent to connect to the server from OWA or trying to use MAPI TCP protocol as an Outlook client.

For the Exchange Online perspective:

• Office 365, as most of the relevant cloud services, includes Service Level Agreements (SLAs) and service credits if the service’s uptime is not delivered.

Office 365 has very detailed documentation about SLAs which is described per workload.

Taking Exchange Online as an example, a downtime is described as “Any period of time when users are unable to send or receive email with Outlook Web Access”. Outlook app is not considered as a downtime factor if OWA is available.

Here’s the monthly uptime percentage, the credit associated and how is calculated:

This example refers to the documentation updated in January 2017.

• Office 365 includes the Service Health Dashboard, which can be monitored constantly using System Center Operations Manager (SCOM). Includes also a mobile app.

In this link you’ll find the Microsoft System Center Management Pack for Office 365. This is an SCOM dashboard example monitoring Office 365, using online watchers:

• You can monitor Office 365 in any way you want by using the Service Communication API. Including: Real-time service health; message center communications; and planned maintenance notifications.
• Office 365 includes 24×7 support. Here’s the contact information in case of support needed: “Contact Office 365 for business support – Admin Help”.

Microsoft offers differences from Business and Enterprise plans regarding support. Both include 24×7 support for critical issues (preventing to access your data) and a response time within the hour.

Business plans for high severity issues (events that affect the productivity of users but have moderate business impact) and non-critical (events that have minimal service or productivity impact on the business) are available only on business hours and it does not have a time frame committed to respond.

For Enterprise Plans the support is always 24×7.

As my personal take in this area:

A thorough, detailed and proactive monitoring of an Exchange on-prem platform can be challenging and with a high cost. There are several variables to consider around a server: Disks, memory, network ports, services running, performance counters, hardware components (including the UPS we can monitor with SCOM), etc. In most cases, at the end of the day, the final user is the one that is telling us that the service is not available.

All that cost associated in monitoring is greatly reduced with Exchange Online, moving that operational responsibility to Microsoft. Human error can also appear in cloud services, let’s take the AWS downtime where a S3 admin entered incorrectly a command line and generated a major meltdown; but these scenarios are rarely encountered in cloud platforms like AWS or Office 365.

[Bonus Track] Public Folders dependency within an organization and the challenge to migrate those to the cloud

Public folders is definitely a hot topic when I had the Exchange on-prem vs Exchange Online conversations. Surprisingly I had a large number of customers that assumed public folders is a feature no longer available in Office 365 (a subconscious desire maybe?).

No, Public Folders still exist in Office 365 and there are supported methods available to migrate them from Exchange on-prem. Surprisingly, to this date, there’s no supported method to migrate Public Folders from Exchange 2016 to Exchange Online.

What is Microsoft official stand about Public Folders being a feature that is going to be deprecated? From “FAQ: Public folders”:

Are public folders going away?
No. Public folders are great for Outlook integration, simple sharing scenarios, and for allowing large audiences to access the same data.

For any that participated in Microsoft events with Exchange experts or had personal reviews with any of them, you probably heard a different message than that. Microsoft has been trying to get rid of Public Folders for a long time. They’ve been trying since Exchange 2007 offering to customers migrate their content to SharePoint and collaborative group workspaces.

It is also true that a lot of Exchange admins suffer considerably with Public Folders, based sometimes on people using them for content that is not suited for this platform. That is something we should never negotiate, we know Public Folders has some limitations and considerations regarding sizing and the impact it has on our Exchange platform.

Microsoft offers several features and capabilities that facilitate Public Folders management within SharePoint, OneDrive, Office 365 Teams and even Yammer in some extent. These workloads will offer simpler and more scalable solutions to some of the Public Folders content we see.

Final Thoughts Regarding Exchange On-Prem vs Exchange Online

To summarize this set of articles, the “no silver bullet” statement maintains. Office 365 will not be automatically the solution to your problems, there are going to be different scenarios to consider but for sure it is a very valid and useful option. At the end of the day, Exchange on-prem will give you the ability to control exactly what goes on with your data and environment, but it also makes you fully accountable for it. That is something that not all IT departments are able to do.

Maintaining a complete, complex, secure, highly available and reliable collaboration platform represents a large investment of time, resources and of course money. Would you prefer shifting that spending on more strategic definitions for the business? Instead of dedicating resources on “keeping the lights on” for the messaging platform, start dedicating those resources for enhancing organizations capabilities with new solutions?

We all suffered at least once in our professional lifetimes with organizations that just see IT as an operational cost, with no real value to the business. We are also responsible on changing that mindset.

The cloud path has started for most of us and for the platforms we support, the features and capabilities are getting bigger by the minute. Most of the constraints and limitations we may have today, most likely will not be in the near future. Exchange is embracing that, the hybrid scenario has transformed from a recommended approach to highly necessary for most organizations.

Exchange 2016 might not be the last Exchange on-prem release, but we can assume the on-prem components could start reducing significantly in the next version, focusing on platforms that will be integrated with the cloud.

Are you sure you want to be the last organization with a 100% Exchange on-prem platform?

Previous post: [Yada Yada Cloud] Choosing from Exchange On-prem vs Exchange Online: The never ending story? – Part I