In some embodiments, the requester specifies a confidence level threshold. In the ML model output 206, prediction information associated with objects that meets the confidence level threshold is kept and the rest is discarded.
It is assumed that the ML models are trained on relatively small sample sets and are less accurate than human annotators; therefore, the initial object predictions generated by the ML model are verified and/or adjusted by the human annotators to achieve greater accuracy. Compared with not having any initial predictions, having the initial ML-generated object predictions as a starting point allows the annotators to go through images at a much faster rate. As will be discussed in greater detail below, the initial set of annotations coupled with appropriate user interface tools can improve annotation throughput significantly while maintaining human-level accuracy.
The annotator interacts with an annotation engine 208 via a client application on client device 212. In this example, the client application and annotation engine 204 cooperate to provide a user interface that displays the image and optionally at least a portion of the initial object prediction information to the human annotator.
The client application (e.g., a browser-based application or a standalone application) provides a user interface configured to display the image and associated object prediction information to the annotator user. As will be explained in greater detail below, in some situations not all of the bounding boxes are displayed in order to avoid a cluttered image that may cause user fatigue and reduce accuracy.