Pages

Sunday, March 14, 2010

We loaded 1TB in just 30 Minutes

Overview:
In February 2008, Microsoft announced a record-breaking data load using Microsoft® SQL Server® Integration Services (SSIS): 1 TB of data in less than 30 minutes. That data load, using SQL Server Integration Services, was 30% faster than the previous best time using a commercial ETL tool. This paper outlines what it took: the software, hardware, and configuration used. We will describe what we did to achieve that result, and offer suggestions for how to relate these techniques to typical scenarios. Even for customers who don't have needs quite like this benchmark, such efforts can teach a lot about getting optimal performance.

Details:
This system is very easy to understand. There are 56 streams of data generated as a flat file that means there will be 56 instances of the SSIS package executing in parrallel and they all will write to a single Database.

Physical Topology:



For more details please click here