See why Archive Shuttle should be your #1 choice for migrating email archives to Office 365
Migrating Archives At Scale
We are often asked during evaluations whether our products are truly scalable, or whether we are suitable for smaller migrations. Well, the answer to both questions is an unequivocal YES.
Archive Shuttle has been used to migrate archive emails to Office 365 in excess of 250 TB so our products are just as usable for a 5 TB migration. Indeed, because we allow cloud-controlled migration, where a migration server is not required on-site, several of our customers are using us to migrate 2 TB archives quickly and at a realistic cost.
In cooperation with Delegate, a premium European Microsoft and Quadrotech partner based in Denmark, and a customer with three different EV sites in three different continents, we tested our Advanced Ingestion Protocol into Office 365 and compared it to the traditional Exchange Web Service Approach (EWS).
Today we received the first “real-life” results:
The data was migrated during the POC three different times using two different approaches. In the first approach, pure EWS (Exchange Web Services) was used, resulting in a peak performance of 11 items/second in EMEA, 11 items/second in USA and 9 items/second in APAC. The average performance was around 24 items/second.
In the second attempt, a fully functional AIP Office365 Module in Archive Shuttle was used to assess the real-world performance increase when ingesting the items into Office365.
The performance increase was significant, resulting in a peak performance of 55 items/s in EMEA, 48 items/s in USA and 44 items/s in APAC, with each region using just 8 ingestion threads. The average performance showed a performance of around 125 items/second across all three regions.
Based on the performance numbers above, the customers migration duration has dropped significantly. The effect is shown in the estimates below:
Archived Items Count in EV: 520,817,280
Approach 1: (traditional EWS)
520,817,280 / 24 (items/s) / 60 (sec) / 60 (min) / 20 (h/day) = 336 days (net) migration time
Approach 2: (Advanced Ingestion Protocol)
520,817,280 / 125 (items/s) / 60 (sec) / 60 (min) / 20 (h/day) = 58 days (net) days migration time
The Quadrotech Research Team are always working on improving project efficiency and migration speeds to help customers complete migrations as fast as possible.
The Advanced Ingestion Protocol will be released with Archive Shuttle 6.6. As our research team optimizes our technology further, you can expect more exciting Office 365 innovation announcements very soon!
In many instances, it will be desirable to limit the amount of data that is migrated to the new environment.
From a business perspective it is essential to follow email retention policies, and these are likely to include deletion of emails that are older than a certain time, typically 7 years.
From a technical point of view filtering out message classes (for example Notes, Tasks, Drafts etc.), shortcuts and other unwanted data will be required.
Archive Shuttle includes a powerful filtering engine that caters for powerful filter stacks on multiple email attributes including:
– Item date
– Archived Date
– Retention Category
– Folder in Archive
– Shortcut status
– Message Class
As a result the new target environment can be guaranteed to only include data that is current and relevant.
Coordinated Peer-to-Peer Architecture
Archive Shuttle employs a coordinated peer-to-peer architecture that is designed for maximum scalability and resiliency.
The coordination is carried out by a CORE server which passes instructions to peer-level agents that are responsible for extraction, ingestion, synchronization, switching etc. This approach ensures central coordination, control and reporting while providing a totally scalable solution with multiple agents carrying out the heavy lifting.
2) The source agent will export data to a staging area as instructed
3) Upon successful completion the source agent will send a notification to the CORE
4) The CORE will identify an available target agent and send an import command detailing the import criteria
5) The target agent will import the data directly from the staging area
6) Upon successful completion the target agent will notify the core
At this point the data has been successfully synchronized, the CORE will now issue the necessary commands to source and target agents to initiate the switch, so that the “Sync’n’Switch” is complete.
This architecture is well suited to on-premise and hybrid cloud deployment where the CORE can be located remotely. It also lends itself well to very fast performance using a multi-threaded approach. Real world migration speeds for a migration of Enterprise Vault from Centera to Netapp are shown below. In this case three export and three import agents were used under the control of a single CORE.
Multiple Sources and Targets
For maximum flexibility Archive Shuttle supports migration between a multitude of archive sources and archive or mailbox targets. The include:
- Symantec Enterprise Vault
- The HP archive family including Zantaz and Autonomy
- EMC Source One
- HP Autonomy / Zantaz / EAS
- EMC EmailXtender
- Symantec Enterprise Vault
- Microsoft Exchange
- Microsoft Office 365
- Global Relay
In addition Archive Shuttle can import from PST Files, .eml and .msg files allowing even more flexibility for migration.
The modular nature of Archive Shuttle means that each source and target is optimized according to their own requirements.
Each target system is unique and treated differently for the best possible experience. As an example, Quadrotech has implemented functionality to take advantage of the latest Office 365 “capabilities” and is working closely with Microsoft to implement advanced migration technology into Office 365 and the Quadrotech Products.
Another example is the Exchange 0n-premise ingestion module where we utilize two completely different protocols (MAPI & EWS) to make sure that the customer experiences a smooth migration.
A third example is the Enterprise Vault ingestion module, where it is the fastest in the market. Years of optimization and close work with the Symantec Engineering teams on the APIs allow Archive Shuttle to adopt a massively multi-threaded approach with stunning migration rates.