Package com.norconex.committer.core.fs.impl

Class CsvFileCommitter

java.lang.Object
com.norconex.committer.core.AbstractCommitter<CsvFileCommitterConfig>
com.norconex.committer.core.fs.AbstractFsCommitter<org.apache.commons.csv.CSVPrinter,CsvFileCommitterConfig>
com.norconex.committer.core.fs.impl.CsvFileCommitter
All Implemented Interfaces:
Committer, Configurable<CsvFileCommitterConfig>, AutoCloseable
public class CsvFileCommitter extends AbstractFsCommitter<org.apache.commons.csv.CSVPrinter,CsvFileCommitterConfig>

Commits documents to CSV files (Comma Separated Value). There are two kinds of document representations: upserts and deletions.

If you request to split upserts and deletions into separate files, the generated files will start with "upsert-" (for additions/modifications) and "delete-" (for deletions). A request "type" field is always added when both upserts and deletes are added to the same file. Default header name for it is type, but you can supply your own name with CsvFileCommitterConfig.setTypeHeader(String).

The generated files are never updated. Sending a modified document with the same reference will create a new entry and won't modify any existing ones. You can think of the generated files as a set of commit instructions.

The generated CSV file names are made of a timestamp and a sequence number.

You have the option to give a prefix or suffix to files that will be created (default does not add any).

Content handling

The document content is represented by creating a column with a blank or null field name. When requested, the "content" column will always be present for both upserts and deletes, even if deletes do not have content, for consistency.

Truncate long values

By default, values longer than 5096 are truncated. You can specify a different maximum length globally, or for each column. Use -1 for unlimited lenght, or 0 to use the the global value, or 5096 if the global value is also zero.

CSV Format

Applications consuming CSV files often have different expectations. Subtle format differences that can make opening or parsing a generated CSV file difficult. To help with this, there are preset CSV formats you can chose from:

  • DEFAULT
  • EXCEL
  • INFORMIX_UNLOAD1.3
  • INFORMIX_UNLOAD_CSV1.3
  • MONGO_CSV1.7
  • MONGO_TSV1.7
  • MYSQL
  • ORACLE1.6
  • POSTGRESSQL_CSV1.5
  • POSTGRESSQL_TEXT1.5
  • RFC-4180
  • TDF

More information on those can be obtained on Apache Commons CSV website. Other formatting options you explicitely configure will overwrite the corresponding setting for the chosen format.