Typically, a stream processing worker may retrieve data records of a partition for processing. The stream processing worker may update partition checkpoint metadata for the partition to indicate the intention to process the retrieved records. Once processing and emitting of the retrieved records is successfully completed, then the stream processing worker may update the partition checkpoint metadata to indicate that processing of the records is completed. As discussed above in FIG. 1, idempotent processing of a data stream may rely upon the partition checkpoint metadata to determine whether or not data records retrieved by a stream processing worker have been emitted to a destination. In this way, another stream processing worker that assumes responsibility for processing the data records may not reprocess and emit data records that have already been processed and emitted. FIG. 5 is a sequence diagram illustrating interactions of a stream processing worker to perform idempotent processing of data records of a data stream, according to at least some embodiments.
