14 DAY TRIAL // February 2009 was synonymous with Microsoft setting the ETL World Record, a move that the company regarded as both very important and unimportant at the same time. Extract, Transform, Load is a process associated with data warehousing, with a label that is self explanatory, and with various tools associated to it including the Microsoft SQL Server Integration Services (SSIS). In fact, it was via Microsoft SSIS that the company set the new ETL record, namely 1 TB of data in less than 30 minutes. After managing to load 1 TB in under half hour, the software giant published a technical article that makes it possible for all customers to perform similar operations.
“Microsoft and Unisys arranged to load over 1 terabyte (TB) of data that was read from flat files on four source servers into a SQL Server database on a single destination server. In this test, the data was read, converted from text fields to database data types, transferred over the network, and inserted into the destination database in less than 30 minutes. To be precise, 1.18 TB of flat file data was loaded in 1,794 seconds. This is equivalent to 1.00 TB in 25 minutes 20 seconds or 2.36 TB per hour,” revealed Microsoft's Len Wyatt, Tim Shea, David Powell.
According to the Redmond company the process achieved a data load no less than 30% faster than any other commercially available ETL tool. In this regard, the software giant has made public the software and hardware products used, as well as the configuration deployed, in order to ensure that customers are able to enjoy similar performance.
“Setting the ETL World Record is simultaneously very important and unimportant. It is important because it clearly shows that SSIS is among the world leaders in ETL performance. In doing this exercise, we have illustrated techniques that can be applied to achieve outstanding performance. These techniques can be applied by customers today. We have also learned things that can be used to build a better product. Some of these things have been implemented; some will come in following releases,” the company added.