Package com.cognite.client.stream

Class RawPublisher

  • public abstract class RawPublisher
extends Object
    This class produces a continuous data stream of rows from a raw table. The raw table is monitored for changes and all new or changed rows are streamed. The publisher polls Raw for updates at the pollingInterval (default every 5 sec.) and push the resulting batch of RawRow to the registered Consumer.

    • Field Detail

      • LOG

        protected static final org.slf4j.Logger LOG

    • Constructor Detail

      • RawPublisher

        public RawPublisher()

    • Method Detail

      • of

        public static RawPublisher of​(RawRows rawRows,
                              String rawDbName,
                              String rawTableName)
        For internal use. Configures a publisher to stream rows from the specified raw table.
        Parameters:
        rawRows - The read raw rows api to use for querying Raw.
        rawDbName - The raw database to read from.
        rawTableName - The raw table to read from.
        Returns:
        The configured RawPublisher

      • withConsumer

        public RawPublisher withConsumer​(Consumer<List<com.cognite.client.dto.RawRow>> consumer)
        Add the consumer of the data stream. The consumer will be called for each batch of RawRow. This is potentially a blocking operation, so you should take care to process the batch efficiently (or spin off processing to a separate thread).
        Parameters:
        consumer - The function to call for each batch of RawRow.
        Returns:
        The RawPublisher with the consumer configured.

      • withStartTime

        public RawPublisher withStartTime​(Instant startTime)
        Sets the start time (i.e. the earliest possible created/changed time of the CDF Raw Row) of the data stream. The default start time is at Unix epoch. I.e. the publisher will read all existing rows (if any) in the raw table.
        Parameters:
        startTime - The start time instant
        Returns:
        The RawPublisher with the consumer configured.

      • withEndTime

        public RawPublisher withEndTime​(Instant endTime)
        Sets the end time (i.e. the latest possible created/changed time of the CDF Raw Row) of the data stream. The default end time is Instant.MAX. I.e. the publisher will stream data indefinitely, or until aborted.
        Parameters:
        endTime - The end time instant
        Returns:
        The RawPublisher with the consumer configured.

      • withPollingInterval

        public RawPublisher withPollingInterval​(Duration interval)
        Sets the polling interval to check for updates to the source raw table. The default polling interval is every 5 seconds. You can configure a more or less frequent interval, down to every 0.5 seconds.
        Parameters:
        interval - The interval to check the source raw table for updates.
        Returns:
        The RawPublisher with the consumer configured.

      • withPollingOffset

        public RawPublisher withPollingOffset​(Duration interval)
        Sets the polling offset. The offset is a time window "buffer" subtracted from the current time when polling for data from CDF Raw. It is intended as a safeguard for clock differences between the client (running this publisher) and the CDF service. For example, if the polling offset is 2 seconds, then this publisher will look for data updates up to (and including) T-2 seconds. That is, data will be streamed with a 2 second fixed latency/delay.
        Parameters:
        interval - The interval to check the source raw table for updates.
        Returns:
        The RawPublisher with the consumer configured.

      • start

        public Future<Boolean> start()
        Starts the streaming job. The job is executed on a separate thread and this method will immediately return to the caller. It returns a Future that you can use to block the execution of your own code if you want to explicitly wait for completion of the streaming job.
        Returns:
        A Future hosting the end state of the streaming job. The future returns true when the polling loop completes (at its specified end time). false if the job is aborted before the specified end time.

      • abort

        public void abort()
        Aborts the current stream operation. It may take a few seconds for this operation to complete.