11 Dec 2019 by Mike Weaver
Integration: The Final Step in Change Management
The final step in successful change management is the Integration stage. Here’s how to bring everything together. Watch now.
Twenty years ago, offline PST files were the solution to a problem.
Exchange Server deployments typically had very small mailbox quotas, and organizations actively encouraged users to either delete emails and/or attachments, or to save data offline in PSTs. Nobody foresaw the rise in compliance, retention and discoverability requirements that would eventually make the content of every PST file potentially useful in court.
Today the default view is often to hang on to all content ‘just in case’. As a result, PSTs have often become a bloated, inefficient way of storing critical historic mailbox data. They’re not centrally controlled, are difficult to discover, and are frequently at the center of high-profile data breaches.[vc_column width=”1/4″][vc_column width=”3/4″]The unlimited archive mailbox feature of Office 365/Exchange Online provides a great opportunity to eliminate PSTs and to meet modern compliance and security standards. Organizations can relieve themselves of the burden of storing, backing up, and searching through that muddle of PSTs. But what are the steps involved in finding them all, attributing data to a user, uploading them to an archive location, and then purging them for good?
Paul Cunningham, a Microsoft most valued professional (MVP) specializing in Exchange Server and Office 365, and the publisher of Exchange Server Pro, recently went hands-on with Quadrotech’s PST FlightDeck migration solution. You can read his full report here, and in this blog we’re going to highlight some of his main observations of the migration process.
There are plenty of individual solutions to parts of the PST puzzle, but smaller organizations may manage perfectly well with a greater degree of manual intervention. It’s only when you have to manage complex scenarios that enterprise software like PST FlightDeck – “with a lot of switches and knobs,” as Paul puts it – comes into its own.
Microsoft offers an Office 365 Import Service, but it has no discovery capability or staging area and extensive PowerShell scripting would be normally be needed to make it viable. Microsoft also offers a free PST Capture Tool, but does not guarantee to maintain it as new versions of Exchange Server are released, or to adjust for changes in the Exchange Online cloud service. Its PST Capture Agent must be deployed to computers before you can scan them for PSTs, but it does not provide this deployment capability itself, forcing you to use more separate tools or processes. Custom scripts are needed to scan network file shares, or to deploy the PST Capture Agent to the server.
Against that, the attractions of a fully-automated approach are somewhat clearer. With an automated solution a full SQL instance is required, but most enterprise customers already have SQL available on their network. Alternatively, cloud migration services can reduce or even remove the on-premises infrastructure required, including removing the need to run a SQL instance. The most sophisticated solutions allow you to install all of their components on a single server, or to distribute them over multiple servers or sites as required.
Paul went with a single server deployment for his evaluation, with PST files placed on some client computers as well as in a file share on a server.
There is an important difference between running an evaluation and setting up a production system to manage PST migration, in that users tend to use default policies in evaluation mode. For example, Quadrotech provides a ready-made script for evaluations that uses fairly aggressive settings. Scheduled events and workflow steps fire more often, helping users move through a small evaluation or pilot quickly.
In a production environment these options wouldn’t be suitable. Paul says: “The defaults for most items seem quite reasonable, but as with any enterprise software if you need special tuning performed you’re likely going to engage the vendor or their partner to stand up the server for you and get it to an operational state.”
Ease of use is something that should be considered, according to Paul. Paul says he was presented with “a simple, easy to navigate management console. Once I was past the initial period of familiarization, I was able to get around pretty easily.” Just as useful is a web interface, which can be provided to project managers so that they can monitor the progress of the eradication project themselves, without having to chase you for status reports.
For real production environments, Quadrotech provides a PST Snapshot service that can quickly identify the extent of PST use throughout your organisation ahead of a PST FlightDeck purchase decision.
For Paul’s evaluation, however he scattered a few PST files around his test lab and deployed the PST FlightDeck migration agent. Packaged as an MSI, he says you can “easily deploy the agent across your network using Group Policy, SCCM, or your preferred software deployment platform. At next login the agent scanned my client computers and quickly located the PST files.” Paul’s evaluation was performed on Windows clients, but he notes the recent announcement by Quadrotech that a client for Apple Macs is now available too.
Of particular use is a scan module for shared folders, which searches file servers and archive storage locations on the network. These could, for example, hold all the PST files exported from the mailboxes of departed users. But offline files can lurk anywhere, and their portability is an issue. “If you’ve got a bunch of DVDs or removable hard drives full of PSTs in your cupboard, dump them into a file share and let the shared folder scanner discover them,” advises Paul.
Discovered PST files can be either:
Paul decided to migrate some PST files into archive mailboxes hosted by his on-premises Exchange server. First, he marked the discovered PST files for migration, and then set a migration priority for the user. For the evaluation he set all users to the same value, but he points out that in other scenarios – such as prioritizing VIP users, or users whose desktops are about to be replaced (risking the total loss of the old PSTs) – the ability to prioritize would be very useful.
You might have various PST FlightDeck components deployed to different physical locations, in which case you could specify that agents in one location will upload their PST files to a particular server for processing, while users in another location can upload to a different server. This makes particular sense with Exchange servers on different continents. As Paul says: “Why copy PST files from New York to London if they’re going to be ingested into archive mailboxes back in New York?”
Rather than having to keep users updated on progress manually, Paul found that his evaluation automatically triggered notifications from the migration agent. These notifications were customizable, and included popups and dialogs from the agent, as well as emails sent by the server to keep users informed. Paul observed: “Having been involved in PST migration projects before where a small team of dedicated communications people handles all of this using a big spreadsheet and lots of broadcast emails, this definitely seems like a better idea to me.”
First, the user receives a notification to restart Outlook. Next, they’re prompted to start the migration now, or postpone it for up to 14 days. When they start their migration, the PST files discovered by the agent and marked for migration by the administrator are presented to the user. This gives the user a great deal of control – they can choose to migrate a file, delete it, or mark it as ‘Not mine’, in which case it gets parked for further analysis.
Network connectivity can prove an obstacle during migration, so it’s best that files get uploaded in the background in a way that can handle interruptions. Paul says “most companies will have something that interrupts connectivity such as desktops shutting down at night, or remote workers who connect and disconnect throughout the day.”
If you’re using a manual approach, ownership of PST files is usually determined by the name of the PST file, where it is located, and whether it is attached to an Outlook profile. Paul explains that in many cases ownership is nearly impossible to determine without further manual analysis, “which is time consuming and will still be inaccurate.”
In comparison, PST FlightDeck’s Six Factor Analysis (6FA) determines ownership using weighted criteria: file owner, scanned owner, user name in path, most common sender, most common recipient, and which Outlook profile it is attached to. If the weighted criteria add up to a 90% certainty, then ownership is automatically attributed to a user. Paul notes that “in a large PST migration project, this type of automated and accurate analysis is critical.”
Once ownership was established, the de-duplication and ingesting steps of the workflow continued, and the PST migration for Paul’s test users finished successfully. “The content was where I expected to see it in the archive mailboxes of my users, and one more popup appeared on the desktops to notify the that the migration was complete. The migration agent even removed the PST files from the original client machines.”
At one point, Paul’s migration project stalled because the documentation he was using wasn’t clear on a particular point. After the PST files are uploaded to the server, another copy is supposed to made on the server as a backup before anything else is done; in Paul’s case the process wouldn’t move past the backup stage.
“I actually recommend that during any software evaluation you try out the vendor’s support process, but sometimes an actual problem occurs and I need to seek assistance anyway,” says Paul. If he had been attempting to use a free solution such as Microsoft’s PST Capture Tool, for example, there would have been no support from Microsoft if he’d hit a problem.
As it was “a little over a day later I’d spent an hour on a Webex desktop sharing session with Quadrotech support. In the end the problems were simple, but they were things that seemed to be missing from the documentation I was working from. Documentation updates are easy fixes though, and I suspect most Quadrotech customers have their initial configuration done by a consultant or partner anyway, and wouldn’t run into the same issues that I did.”
Paul notes that: “it’s reasonable to expect a low percentage of PST files will require some manual intervention in any size project,” but suggests automation can reduce this need. For example, there’s PST FlightDeck’s ability to automate the process of discovering, reconciling and migrating the mailboxes of inactive users – generally those who have left the company. “If you have a big stash of PST files for departed users to deal with as part of your overall project, then this is a very useful capability,” says Paul.
Is a manual, semi-automated or automated solution right for you? Paul concludes: “Having dealt with the process of manually discovering, analyzing, and importing large quantities of PST files in the past, the PST FlightDeck solution has many benefits that are obvious to me. You can always consider deploying it as an evaluation and using it to scan your environment to see just how big your PST problem is. Armed with that information, you might just find that PST FlightDeck will fit your scenario.”