Skip to main content
This document describes what bookmarks are, and how they are used in client-server protocol. A stream is a sequence of entries, where these entries belong to some specific operation, and are identified by an entry number. In the generic sense, an entry can be any piece of data relevant to an application’s context. They can be either events or bookmarks. Events are defined by the application. Stream clients provide relevant information about an event that must be streamed. In the case of the Polygon zkEVM, entries are actually L2 blocks, while operations are thought of as batches. So then, when the stream source triggers the StartAtomicOp()\texttt{StartAtomicOp()} function and the corresponding message is sent to the stream server, that message in the Polygon zkEVM context, amounts to an instruction to “begin a batch, and prepare to receive its related entries (i.e., blocks) that are about to follow.” Again, in the Polygon zkEVM context, the message sent when the CommitAtomicOp()\texttt{CommitAtomicOp()} function is called, is tantamout to saying: “close the batch with the last entry received.” Note that without these two messages, the stream server has no means of knowing where a batch starts and ends. Similarly, in the client-server protocol, there is a need for a stream client to indicate to the stream server what entry the streaming should begin with. A bookmark is used for this purpose by the stream client.

​
What is a bookmark?

A bookmark is an entry in a stream, and therefore has an entry number, denoted by entryNumber\texttt{entryNumber}. A bookmark is essentially a string of bytes. It links an entryNumber\texttt{entryNumber} to a string of bytes in a way that is meaningful to an application. Stream clients can request stream servers to start the stream from a particular bookmark using the StartBookmark()\texttt{StartBookmark()} method. A bookmark is a type of a stream entry, used as an identifier. It points to a specific position in the stream from which the stream server must begin the streaming. Figure

​
StartBookmark command

In addition to the start()\texttt{start()} and stop()\texttt{stop()} commands of the client-server protocol, a command called StartBookmark()\texttt{StartBookmark()} is added. In the similar way the start()\texttt{start()} and stop()\texttt{stop()} commands are identified by 1\texttt{1} and 2\texttt{2}, respectively, the StartBookmark()\texttt{StartBookmark()} command is identified with the number 44. The StartBookmark()\texttt{StartBookmark()}command is as follows: 1.u64 command = 42.u64 streamType// e.g. 1: zkEVM Sequencer3.u32 bookmarkLength// Length of fromBookmark4.u8[ ] fromBookmark\begin{aligned} 1.\qquad &\mathtt{u64\ command\ =\ 4} \\ 2.\qquad &\mathtt{u64\ streamType} \qquad //\ \texttt{e.g. 1:}\ \mathtt{zkEVM\ Sequencer} \\ 3.\qquad &\mathtt{u32\ bookmarkLength} \qquad //\ \texttt{Length of fromBookmark}\\ 4.\qquad &\mathtt{u8[\ ]\ fromBookmark} \end{aligned} As previously seen with other commands, streamType\texttt{streamType} refers to the network associated with the stream source. When calling the StartBookmark()\texttt{StartBookmark()} function, and the corresponding message is sent to the stream server, the stream client must provide a bookmark. That is, a string of bytes together with the length of the string, denoted by bookmarkLength\texttt{bookmarkLength}. Example: Consider the Polygon zkEVM case, where entries are L2 blocks. As seen in the above diagram, a bookmark precedes events. In this context, the data of each block is preceded by a bookmark entry, and this bookmark entry contains among other things the block number. It is more meaningful to use the blockNumber\texttt{blockNumber} in the Polygon zkEVM context. That is, the blockNumber\texttt{blockNumber} of the block with which the streaming must begin as requested by the stream client.

​
Bookmarks DB

Although stream entries are recorded in the stream file, there’s a database called Bookmarks DB, which is specifically for storing all bookmarks. The Bookmarks DB is a key-value database that maps a bookmark (used as a key) with its entry number (used as the value). The technology behind Bookmarks DB is LevelDB, which is an open-source, fast key-value storage library written by Google’s developers. So, when a stream client requests the stream server to start streaming from a specific bookmark, it provides the bookmark as a string of bytes. The stream server performs a binary search in order to locate the bookmark in the Bookmarks DB, and fetches the corresponding entryNumber\texttt{entryNumber} from the Bookmarks DB. It then begins to stream entries, starting from the entryNumber\texttt{entryNumber} onwards. Figure All commands made by stream clients return a response, called a Result\texttt{Result} entry. The format of such a Result\texttt{Result} entry is as follows: 1.u8 packetType // 0xff:Result2.u32 length // Total length of the entry3.u32 errorNum // Error code (e.g. 0:OK)4.u8[] errorStr\begin{aligned} 1.\qquad &\texttt{u8 packetType // 0xff:Result} \\ 2.\qquad &\texttt{u32 length // Total length of the entry} \\ 3.\qquad &\texttt{u32 errorNum // Error code (e.g. 0:OK)} \\ 4.\qquad &\texttt{u8[] errorStr} \end{aligned} Whenever a stream client calls the start()\texttt{start()} function, the stream server responds with the Result\texttt{Result} entry first, followed by all the entries. In the client-server protocol, each message from the stream server has its identifier or packetType. As seen in the code above, the Result\texttt{Result} entry has the identifier, 0xff\texttt{0xff}, at the protocol level. The Result\texttt{Result} identifier is followed by the total length of the entry. Next is a code of an error message, errorStr\texttt{errorStr}. If everything goes well then a 0\texttt{0} is sent, meaning OK\texttt{OK}. A string is also sent, which is in a more human readable text. An error is sent in the form of a code and a string, where the string is just an array of bytes, and each byte is an ASCII character.