Package com.norconex.committer.core.fs.impl

Class CsvFileCommitterConfig

java.lang.Object
com.norconex.committer.core.BaseCommitterConfig
com.norconex.committer.core.fs.BaseFsCommitterConfig
com.norconex.committer.core.fs.impl.CsvFileCommitterConfig
public class CsvFileCommitterConfig extends BaseFsCommitterConfig

Commits documents to CSV files (Comma Separated Value). There are two kinds of document representations: upserts and deletions.

If you request to split upserts and deletions into separate files, the generated files will start with "upsert-" (for additions/modifications) and "delete-" (for deletions). A request "type" field is always added when both upserts and deletes are added to the same file. Default header name for it is type, but you can supply your own name with setTypeHeader(String).

The generated files are never updated. Sending a modified document with the same reference will create a new entry and won't modify any existing ones. You can think of the generated files as a set of commit instructions.

The generated CSV file names are made of a timestamp and a sequence number.

You have the option to give a prefix or suffix to files that will be created (default does not add any).

Content handling

The document content is represented by creating a column with a blank or null field name. When requested, the "content" column will always be present for both upserts and deletes, even if deletes do not have content, for consistency.

Truncate long values

By default, values longer than 5096 are truncated. You can specify a different maximum length globally, or for each column. Use -1 for unlimited lenght, or 0 to use the the global value, or 5096 if the global value is also zero.

CSV Format

Applications consuming CSV files often have different expectations. Subtle format differences that can make opening or parsing a generated CSV file difficult. To help with this, there are preset CSV formats you can chose from:

  • DEFAULT
  • EXCEL
  • INFORMIX_UNLOAD1.3
  • INFORMIX_UNLOAD_CSV1.3
  • MONGO_CSV1.7
  • MONGO_TSV1.7
  • MYSQL
  • ORACLE1.6
  • POSTGRESSQL_CSV1.5
  • POSTGRESSQL_TEXT1.5
  • RFC-4180
  • TDF

More information on those can be obtained on Apache Commons CSV website. Other formatting options you explicitly configure will overwrite the corresponding setting for the chosen format.

}

  • Field Details

  • Constructor Details

    • CsvFileCommitterConfig

      public CsvFileCommitterConfig()

  • Method Details

    • getColumns

      public List<CsvColumn> getColumns()

    • setColumns

      public CsvFileCommitterConfig setColumns(List<CsvColumn> columns)

    • getFormat

      public CsvFileCommitterConfig.CsvFormat getFormat()
      Optional base CSV formatter to use. Settings explicitly set will overwrite these defaults ("quote", "escape", etc.).

    • getDelimiter

      public Character getDelimiter()
      Delimiter character to use.

    • getQuote

      public Character getQuote()
      Quote character to use.

    • isShowHeaders

      public boolean isShowHeaders()
      Whether to write column names on the first line.

    • getEscape

      public Character getEscape()
      Escape character.

    • getTruncateAt

      public int getTruncateAt()
      Number of characters from which long text will be truncated. Default is 5096

    • getMultiValueJoinDelimiter

      public String getMultiValueJoinDelimiter()
      String used as delimiter when joining multi-value fields.

    • getTypeHeader

      public String getTypeHeader()
      When generating CSV files containing both upserts and deletes requests, what column name to use to display the request type.

    • setFormat

      public CsvFileCommitterConfig setFormat(CsvFileCommitterConfig.CsvFormat format)
      Optional base CSV formatter to use. Settings explicitly set will overwrite these defaults ("quote", "escape", etc.).
      Returns:
      this.

    • setDelimiter

      public CsvFileCommitterConfig setDelimiter(Character delimiter)
      Delimiter character to use.
      Returns:
      this.

    • setQuote

      public CsvFileCommitterConfig setQuote(Character quote)
      Quote character to use.
      Returns:
      this.

    • setShowHeaders

      public CsvFileCommitterConfig setShowHeaders(boolean showHeaders)
      Whether to write column names on the first line.
      Returns:
      this.

    • setEscape

      public CsvFileCommitterConfig setEscape(Character escape)
      Escape character.
      Returns:
      this.

    • setTruncateAt

      public CsvFileCommitterConfig setTruncateAt(int truncateAt)
      Number of characters from which long text will be truncated. Default is 5096
      Returns:
      this.

    • setMultiValueJoinDelimiter

      public CsvFileCommitterConfig setMultiValueJoinDelimiter(String multiValueJoinDelimiter)
      String used as delimiter when joining multi-value fields.
      Returns:
      this.

    • setTypeHeader

      public CsvFileCommitterConfig setTypeHeader(String typeHeader)
      When generating CSV files containing both upserts and deletes requests, what column name to use to display the request type.
      Returns:
      this.

    • equals

      public boolean equals(Object o)
      Overrides:
      equals in class BaseFsCommitterConfig

    • canEqual

      protected boolean canEqual(Object other)
      Overrides:
      canEqual in class BaseFsCommitterConfig

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class BaseFsCommitterConfig

    • toString

      public String toString()
      Overrides:
      toString in class BaseFsCommitterConfig