Fast and efficient ingestion for Office 365 and Exchange

Advanced Ingestion Protocol (AIP) is Quadrotech’s default method for migrations to Exchange 2010 Service Pack 1 and above. It dramatically speeds up ingestion into Office 365 and Exchange environments and, critically, maintains the integrity of the source data.

While Exchange Web Services (EWS) or MAPI are traditionally the primary methods of archive ingestion to Office365 or Exchange, they offer low performance, have high bandwidth requirements, and are potentially harmful to future forensic analysis due to the way the email items are reformed upon ingestion.

With AIP, items transferred remain exactly the same as when they went in. The process is completed quickly, securely and with integrity as a priority.

AIP compared with EWS and other protocols

AIP provides a massive increase in speed compared with traditional EWS. Tests have indicated that an average performance of 125 items/second is achievable. Compare that with average performance of EWS, which has been measured at just 24 items/second.

When bandwidth and throttling (by Office 365, for example) are taken into account it becomes apparent how much difference that speed can make. Real-life migrations demonstrate that a large Enterprise Vault (EV) archive containing 521,000,000 items takes 336 days (net) to migrate using EWS. AIP achieves the same result in only 58 days. AIP’s speeds are enhanced through the intelligent management of multi-threaded ingestion processes, intelligent back-off handling from Office 365, and smart stream compression.

While some competitive products may appear to provide adequate performance when migrating to Office 365, they are also resource-heavy and require a vast number of virtual machines pushing mail via EWS. In contrast, even for the largest migrations AIP ensures that only a maximum of two machines are required to push data volumes far in excess of 12TB/day, given the right circumstances.

How AIP aids Chain of Custody and compliance

Quadrotech’s ArchiveShuttle (AS) product, which incorporates AIP, was created with compliant Chain of Custody (CoC) in mind. AS itself keeps a digital fingerprint of every item, at every stage of the process. From extraction to ingestion you can interrogate the metadata in the database to ensure there’s been no interference during migration, and files in the target destination are totally reconciled against the source.

AIP streams the migrated archive into Exchange and Office 365 fully preserved. This makes CoC much easier to validate than using EWS for ingestion, which breaks the data into smaller pieces (senders, recipients, dates and other properties). Unlike AS and AIP, EWS cannot even preserve some information – like the date the item was created – meaning there’s much more of an opportunity for information to be lost or corrupted.