Scala tokenizer

Today I found out that scala compiler allows to print stats about tokens used in code with command line tokenizer. To use it from eclipse:

1. create debug configuration and point Main class to scala.tools.cmd.program.Tokens
2. on program arguments set HelloWorld.scala --verbose –freq
3. add standard scala classpath for scala debuging you can find it: here

On the output you will see tokens stats like below:

Tokenizing: HelloWorld.scala
2 '}'
2 '{'
1 '['
1 '='
1 args
1 main
1 ':'
1 'def'
1 'object'
1 '('
1 ']'
1 Hello
1 String
1 Array
1 ')'

Rather simple but sometimes useful.