Merge pull request #10 from EnableSoftware/streaming-exporter

Add support for a streaming parser and streaming exporter

CHANGELOG.md

@@ -5,34 +5,43 @@ This project adheres to [Semantic Versioning](http://semver.org/).
 
 ## [Unreleased][unreleased]
 
+### Added
+
+- Add support for a streaming reading of delimited data files.
+- Add support for a streaming export of delimited data files from a `DbDataReader`.
+
 ## [3.3.2] — 2016-08-03
 
 ### Fixed
 
-Remove check from Exporter preventing TabSeparator being used when IncludeEscapeCharacters disabled.
+- Remove check from `Exporter` preventing `TabSeparator` being used when
+  `IncludeEscapeCharacters` disabled.
 
 ## [3.3.1] — 2016-05-04
 
 ### Added
-Add support for a blacklist of field values. Fields that start with a value from the blacklist should have a single quote prepended to them.
+
+- Add support for a blacklist of field values. Fields that start with a value
+  from the blacklist should have a single quote prepended to them.
 
 ### Fixed
 
-Fix a bug with the `IncludeEscapeCharacters` setting. Previously this setting
-could only be disabled if using a tab character as a field separator.
+- Fix a bug with the `IncludeEscapeCharacters` setting. Previously this setting
+  could only be disabled if using a tab character as a field separator.
 
 ## [3.2.0] — 2016-04-29
 
 ### Added
 
 - Add XML documentation to NuGet package.
-- Support for clearing column names specified via ExtendedProperties on a DataColumn.
+- Support for clearing column names specified via `ExtendedProperties` on a
+  `DataColumn`.
 
 ## [3.1.0] — 2016-04-28
 
 ### Added
 
-- Support for using ExtendedProperties on a DataColumn.
+- Support for using `ExtendedProperties` on a `DataColumn`.
 
 ### Changed
 

README.md

@@ -15,6 +15,23 @@ using (DataTable myData = parser.Parse(myTextReader))
 	// Make use of `myData` here…
 }
 ```
+
+For processing a large amount of delimited data, the `Parser.ParseReader()` method takes a `TextReader` and returns `System.Data.Common.DbDataReader`, which provides a fast, forward-only stream of rows. This allows for processing each row in turn, rather than reading a whole file into memory.
+
+```c#
+var parser = new Parser();
+
+var reader =  parser.ParseReader(myTextReader))
+
+while (reader.Read())
+{
+	// Each field for the current row can be retrieved using the column index:
+	var field1 = reader[0];
+	var field2 = reader[1];
+	// etc…
+}
+```
+
 ### Configuration properties
 
 * `FieldSeparator` - the character used as field delimiter in the text file. Default: `,` (i.e., CSV).
@@ -30,9 +47,20 @@ using (var writer = new StringWriter())
 {
 	var exporter = new Exporter();
     exporter.Export(myData, writer);
-	myCsv = writer.ToString()
+    myCsv = writer.ToString();
+}
+```
+
+For exporting a large amount of delimited data, the `Exporter.ExportReader()` method takes a `System.Data.Common.DbDataReader` instance and writes to a specified `TextWriter`, which provides a fast, streamed-based generation of row data. This allows for processing each row in turn, rather than needing to retain the whole data source in memory.
+
+```c#
+using (var fileWriter = File.CreateText("my.csv"))
+{
+    var exporter = new Exporter();
+    exporter.ExportReader(myDataReader, fileWriter);
 }
 ```
+
 ### Configuration properties
 
 * `FieldSeparator` - the character used as field delimiter in the text file. Default: `,` (i.e., CSV).

src/DelimitedDataParser/DelimitedDataParser.csproj

@@ -50,11 +50,11 @@
     <Reference Include="System.Xml" />
   </ItemGroup>
   <ItemGroup>
+    <Compile Include="DelimitedDataReader.cs" />
     <Compile Include="Exporter.cs" />
     <Compile Include="Parser.cs" />
     <Compile Include="GlobalSuppressions.cs" />
     <Compile Include="Properties\AssemblyInfo.cs" />
-    <Compile Include="Table.cs" />
   </ItemGroup>
   <ItemGroup>
     <None Include="packages.config" />