How to Migrate PST Files to Office 365
This guide is also available as a downloadable PDF.
In this article, we’ll discuss how to import PST files to Office 365, but to begin, let’s reflect on the history of the humble PST.
What are Outlook Data Files?
An Outlook Data File (.pst) contains messages and other Outlook items, like tasks, calendar details, and so on. Originally called a personal storage table to differentiate itself with the tables in the server-based Exchange online database, the PST is now often referred to as a personal storage file or personal folders, which is a local repository saved on the client itself.
Concerns with Outlook Data Files
In these days of regulation and compliance, it seems unthinkable that a mailbox format could become so widely used without central management, backup and control. Anyone with a rudimentary knowledge of their personal computer’s file structure can copy, duplicate, move and create additional PSTs at will. Also, reliable network connections and fast speeds mean today’s users expect “always on” connectivity across all devices; with the exception of the synchronized cache there is no need to store local copies of mail in order to retain productivity.
Many customers I work with have lots of PST files in their environment. PST files can be found on local devices, network shares, home shares, and removable media. This sprawl can make a centralization project difficult. In the course of the migration to Office 365, organizations want to get rid of PST files and migrate them to the Exchange Online mailbox or Exchange Online archive to preserve the data. This helps resolve many of the issues with PST files, such as:
- Not centrally managed: There is no way of confirming how many PST files exist in the infrastructure, where they are, or who they belong to.
- Inefficient: Organizations attempting to centralize PSTs using NAS and/or file servers risk data loss, file corruption and Outlook speed reduction – not to mention expensive!
- Inhibit business-aligned change: PSTs are a blocker to modern client architectures involving multiple devices, VDI and BYOD.
- Major security risk: Portability of PSTs exposes organizations to data leakage. Susceptible to malware.
- Legal and compliance obstacle: Organizations cannot enforce retention policies, and legal departments struggle to ensure they have complete discoveries to defend cases.
- Security: Password protected PSTs are not encrypted. Passwords are easily circumvented by even a moderately skilled attacker using well-known tools and techniques. Once a PST is unlocked, it can be loaded into Outlook by any user.
Note: The above list is taken from the Complete Guide to Eradicating PST Files, which discusses many technical, security and compliance-related issues, compares various ways of locating and eliminating PST files, and the complex challenges involved. It is highly recommended reading.
Following high-profile cases such as the Sony hack in November 2014, when attackers penetrated and retrieved 179 PSTs containing some 73,000 messages from the mailbox of Sony Chairman Amy Pascal along with other data, it’s easy to understand why companies are reconsidering the use of PSTs.
Migrating PST Files to Office 365
Microsoft offers an Office 365 built-in tool to import PST files to Exchange Online. This could be done in two different ways:
- Network upload: Upload the PST files over the network to a temporary Azure storage location. Then you use the Office 365 Import service to import the PST data to mailboxes in your Office 365 organization (free of charge).
- Drive shipping: Copy the PST files to a BitLocker-encrypted hard drive and then physically ship the drive to Microsoft. Microsoft will then upload the data to a temporary Azure storage location. Then you use the Office 365 Import service to import the data to mailboxes in your Office 365 organization ($2 USD per GB of data).
Important: Microsoft offers with both methods only the technical way of getting your data to Office 365. This technique doesn’t cover all the required planning and preparation, like locating all PST files within your environment, checking the PST files for corruption, removing passwords, identifying which PST files belongs to which mailbox, etc. All of these important steps must be done manually prior to migrating data to Office 365 and are covered in the Complete Guide to Eradicating PST Files eBook as well.
If you have sufficient internet bandwidth or very few PSTs to import, you can consider using the network transfer method to move the data across to Microsoft’s datacentre.
First, you need to assign the Mailbox Import Export role in Exchange Online to import PST files to Office 365 mailboxes. It is recommended to create a new role group in Exchange Online that’s specifically intended for importing PST files to Office 365. Further, you have to be assigned the Mail Recipients role in Exchange Online. By default, this role is assigned to the Organization Management and Recipient Management role groups. Or you have to be a global administrator in your Office 365 organization.
Migrate PST to Office 365: Step by Step
- After creating the new role group, log in to the Security & Compliance Center admin portal in Office 365 (https://protection.office.com) and click Data governance – Import – New import job.
Figure 1: Office 365 import job in the Security & Compliance Center
- Name your job and click Next.
Figure 2: Name your job
- In the next step, you can choose between “Upload your data” and “Ship hard drives to one of our physical locations”. Choose Upload your data and click Next.
Figure 3: Upload your data
- On the Import data page, do the following two things:
Figure 4: Import data
- In step 2, click Show network upload SAS URL. After the SAS URL is displayed, click Copy to clipboard and then paste it and save it to a file so you can access it later. This key is very important and should be protected and not published.
Figure 5: SAS URL for network upload
- In step 3, click Download Azure AzCopy to download and install the Azure AzCopy tool. You can leave the Import data page open or click Cancel to close it.
Once you have the AzCopy tool connected, and the key and secure upload URL have been retrieved, you can start uploading PST files across the network to Microsoft.
AzCopy is a command line tool and if you installed it in the default location, go to %ProgramFiles(x86)%\Microsoft SDKs\Azure\AzCopy
AzCopy has a lot of parameters that can be found in the help file:
C:\Program Files (x86)\Microsoft SDKs\Azure\AzCopy>AzCopy.exe -?
AzCopy 8.1.0-netcore Copyright (c) 2018 Microsoft Corp. All Rights Reserved.
AzCopy </Source:> </Dest:> [/SourceKey:] [/DestKey:] [/SourceSAS:] [/DestSAS:]
[/V:] [/Z:] [/@:] [/Y] [/NC:] [/SourceType:] [/DestType:] [/S]
[/Pattern:] [/CheckMD5] [/L] [/MT] [/XN] [/XO] [/A] [/IA] [/XA]
[/SyncCopy] [/SetContentType] [/BlobType:] [/BlockSizeInMB:] [/Delimiter:]
The important parameters for uploading PST files are:
- /Source:, the UNC path to the folder containing the PSTs
- /Dest:, the secure upload URL provided by Microsoft
- /V:, specifies the file name to output logging information
- /S, tells AzCopy to also upload any files contained in subfolders of the source path
- /Y, allows the use of write-only SAS tokens when you upload the PST files to a storage location
The AzCopy tool is run from CMD.exe. For example:
C:\Program Files (x86)\Microsoft SDKs\Azure\AzCopy>AzCopy.exe /Source:”\\localhost\pst\” /Dest:”SAS_URL” /V:C:\Users\domin\AzCopy.log” /Y
The uploaded PST data is held in Azure for 30 days if there are no import jobs in progress. When you have completed your uploads you can create the PST mapping file. You can download a template of a mapping file here. The Office 365 Import Service allows you to specify a target folder in the mailbox to import the PST file data to, and also specify whether the data should be imported to the archive mailbox.
The CSV mapping file for our PST upload file looks like follows:
Figure 6: CSV mapping file
- Back in the Security & Compliance Center, check both radio buttons and click Next
Figure 7: Start import job
- Select the mapping file and validate the file. Click on Save.
Figure 8: Upload and validate the CSV mapping file
- If the upload was successful, click on Close and the job will validate the things in the background, i.e. if the mailbox from the CSV mapping file exists in your tenant.
- Next, you can start the PST import to your mailbox by clicking on Import to Office 365.
Figure 9: Import to Office 365
- In the next screen, you can optionally filter your data based on age and exclude messages from specific users.
Figure 10: Filtering options
The most likely error you will run into is a PST that either hasn’t been uploaded or can’t be found by the import job because the path you specified in the CSV file is not correct. The PST file names and paths are also case sensitive so you should take care to use the correct case in the CSV file when you create it.
Issues with the Office 365 PST Import Service
The Office 365 Import Service is a relatively straightforward tool that handles the basics of securely importing PST file data into Office 365 mailboxes. However, as with the Exchange Server Mailbox Import Request method, the Office 365 Import Service has several weaknesses that you should be aware of.
- No transfer of PSTs to a staging area – collection of PSTs to a central location ready for upload to Azure needs to be performed separately.
- Identifying ownership – neither AzCopy nor the Import Service provides assistance with identifying the owners of PSTs, so this activity needs to be performed manually.
- No duplicate handling – when multiple PSTs exist for the same mailbox user duplication of items may occur.
- Logistics and performance – uploading with AzCopy is a slow process, while the alternative of packaging data onto encrypted hard drives and transporting them to Microsoft datacenters is logistically complex and time-consuming. In addition, Microsoft charges $2/GB for data ingested into Office 365 via the drive shipping method.
- Elimination of PSTs – disconnection of PSTs from Outlook profiles and prevention of new PST file creation needs to be handled separately.
- Passwords must be manually remediated for each file, by either breaking them with a tool or getting them from the user and removing them.
Quadrotech’s PST Migration Service
If you find that you have a more complex problem or are running into the issues discussed here, a comprehensive migration tool may be in order.
Quadrotech has focused on its PST Flight Deck solution, and their team helps handle everything from the software to implementation. Their focus has been on providing an upgrade experience, which extends to:
- Users keeping access to files while they are being migrated
- Bandwidth control options
- Consistent user communications to ensure happy administrators
- Speed! Powered by Quadrotech’s Advanced Ingestion Protocol, the fastest PST import to Office 365 is guaranteed
- Automated filtering, password removal, corruption handling, and more
For further information on Quadrotech’s ‘fixed-cost, fixed-outcome’ PST file discovery, elimination and migration services, please click here.