Chat with us, powered by LiveChat

Blog

Back

Who moved my millions of morsels of cheese?

1 May 2018 by Jason Jacobo

Part 1: An industry-accepted workaround.

As we all know, if you don’t address a problem properly, it just manifests later, differently, and often in a more complicated form than before.

Email, and the systems that serve it, has its own story to tell when it comes to problem avoidance causing convolution; one that continues to shape the entire industry. This four-part blog series begins with the history of email storage, the ways in which the limitations have been perpetuated, rather than eliminated through technology – and the industry-wide impact that this has had.

A brief(-ish) history of email

When email was first introduced, its importance was never fully conceived. Although somewhat debated, it could be argued that the precursor to modern email systems was introduced in the 60’s prior to ARPANet or the Internet. Regardless of whom you feel the inventor of email was, the earliest forms of modern email were introduced in the 70’s and quickly experienced exponential growth, reaching as many as one million users in 1989. Each year since its inception, new solutions to serve and receive email were introduced, message sizes grew, and the repositories of these systems grew along with it.

With ever increasing popularity of email systems, organizations soon began to utilize them as business-critical or ‘line of business’ solutions. Soon, the use of these systems superseded their design and capabilities. The number of users, sizes of messages, sizes of mailboxes and scalability of storage all became issues, impacting the stability and recovery of email systems. Data loss and long recovery times were just the precursors to loss of revenue and productivity for the organizations that were dependent upon this now critical component of daily business life.

Storage models for email servers were often insufficient, with costs that prohibited businesses from accommodating growth. In addition to this, initial attempts to mitigate these issues were not well received. Messaging administrators began to limit the size of messages and mailboxes in an effort to slow the tremendous growth occurring in their environment. This method of remediation was both unpopular and seen to limit the ability to use email as a viable business solution.

The business community already had a taste, and email would be here to stay, despite any challenges faced by the servers responsible for managing it.

Personal storage and de-centralization

Accepting that the solutions of the time were simply unable to handle the demand placed on them, an answer outside of the mail server was sought. Local personal archives were introduced as a workaround to offload the data from the server without impacting the business users’ reliance upon the medium. In Microsoft’s case, the PST file (historically only used for server-based operations) was the answer to their overburdened servers. These locally stored databases could hold a user’s messages and be opened directly from their mail client, providing a notably similar user experience while removing the load off of the server, at the cost of decentralization. Users were happy, administrators kept their jobs, but many organizations needed more. PST files proved that there was no need to align users with the intended design of email solutions, and that they could happily work within the strict limits seen necessary to keep these systems stable. Users could send and keep as much email as they desired because the data did not need to be stored on the server.

As the problem of use vs. design shifted from mail servers to local archives such as PST or NSF files, data growth continued to go unchecked. Soon these local archives began to reach the same limitations as the servers where the data originated from. PST files would reach size limitations, become corrupt and inaccessible, and this would result in data loss. Although the capacity of these repositories would be increased over time, decentralization of this data became an issue for several organizations. Email was being used in nearly every aspect of business. In some instances, trade secrets, internal information, or scandals were contained inside these local databases.

It became apparent to many organizations that the need to re-centralize the data was critical to the success, management, security, and even competitiveness of their operations.

What happened next? Keep an eye out for our next blog to find out.

Jason has been at Quadrotech since 2014, moving from the Delivery team to Product Owner for Archive Shuttle. He has been in the industry for decades, in a variety of different roles. Jason would describe himself as an “exploratory tinkerer” and a technical veteran. The main thing that drew him to technology was a passion for solving problems wherever he sees them.