Skip to main content

tokenize

Function tokenize 

pub fn tokenize<S>(input: &[u8], sink: &mut S) -> Result<(), TokenizerError>
where S: TokenSink,
Available on crate features html and http and std only.
Expand description

Tokenizes input in one pass with the default (non-strict) tokenizer, dispatching events to sink.

§Errors

The default tokenizer is non-strict, so this does not abort on ambiguous contexts; build a strict Tokenizer (via with_strict) to get a ParsingAmbiguityError. A construct over DEFAULT_MAX_BUFFERED_BYTES fails with a MemoryLimitExceededError.