Module x_robots_tag

Structs

CustomRule

A non-std rule that we are not aware of. An unknown rule.

DirectiveDateTime

RobotsTag

A single element of X-Robots-Tag corresponding to the valid values for one bot name

XRobotsTag

Enums

MaxImagePreviewSetting

The maximum size of an image preview for this page in a search results. If omitted, search engines may show an image preview of the default size. If you don’t want search engines to use larger thumbnail images, specify a max-image-preview value of standard or none.

Functions

robots_tag_parse_iter

Create an iterator to try to parse a byte slice as one or multiple RobotsTags.