Java tokenizer, C++ tokenizer and strtok

I Want to know what is the algorithmic complexity of following 1. Java String tokenizer 2. C++ STL based tokenizer 3. strtok开发者_如何学运维.

Is there any faster algorithm then rudimentary strtok to tokenize a string based on custom delimeter.

Regarding Java, there are 3 main techniques for tokenizing (String.split(), StringTokenizer and StreamTokenizer). if you refer to the java.util.StringTokenizer class (which tokenizes by breaking the input string S on every occurrence of a character from a given string D), then the complexity is O(|S|*|D|). I.e., if you have only one delimiter char, this will be linear.

Note that the other tokenizers are more powerful in their abilities. String.split() for example can split around any pattern matching a given regex.