Package com.norconex.importer.util.chunk

Class ChunkedTextReader

java.lang.Object

com.norconex.importer.util.chunk.ChunkedTextReader

public class ChunkedTextReader
extends Object

Supplies chunks of text to the provided consumer. If a field matcher is supplied, the text will be coming from matching document fields. Otherwise, the document body is used. In either case, if the text is greater than the maxChunkSize, it is split and read in chunks. This class is a convenient way to apply similar logic whether the text source is a document content, field, huge, or small.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	ChunkedTextReader.ChunkedTextReaderBuilder

Method Summary

Modifier and Type	Method	Description
static ChunkedTextReader.ChunkedTextReaderBuilder	builder()
static ChunkedTextReader	from(ChunkedTextSupport chunkedTextSupport)
boolean	read(@NonNull DocHandlerContext docCtx, @NonNull FailableFunction<TextChunk,Boolean,IOException> textConsumer)	Handles the processing of document text, invoking the consumer as many times as necessary.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

from

public static ChunkedTextReader from(ChunkedTextSupport chunkedTextSupport)

read

public boolean read(@NonNull
 @NonNull DocHandlerContext docCtx,
 @NonNull
 @NonNull FailableFunction<TextChunk,Boolean,IOException> textConsumer)
             throws IOException

Handles the processing of document text, invoking the consumer as many times as necessary. If the field matcher is set, the streams to handle are coming from fields, otherwise it is the document content.

Parameters:

docCtx - document context

textConsumer - text consumer

Returns:

true if all chunks were read. false if the chunk consumer ever returned false.

Throws:

IOException - problem reading

builder

public static ChunkedTextReader.ChunkedTextReaderBuilder builder()