Please note that as of October 24, 2014, the Nokia Developer Wiki will no longer be accepting user contributions, including new entries, edits and comments, as we begin transitioning to our new home, in the Windows Phone Development Wiki. We plan to move over the majority of the existing entries over the next few weeks. Thanks for all your past and future contributions.
How to define a custom separator in TLex
TLex does not provide support for user defined separators, hence a possible workaround could be using the MarkedToken(), Mark(), Get(), and Peek() methods for achieving the desired results.
// As an example taking "," as a separator.
_LIT8(KSomeConstString, "first, second, third, fourth,");
while((ch = lex.Get()) != 0 )
while ((ch = lex.Peek()) != ',')
/* Now we have the string as the token,
* do something.. */