Tokenizing Data Using c:forTokens in JSTL

While the <c:forEach> action is powerful for iterating, sometimes data is presented to us in the form of strings. A common reason for getting string data (no matter how much we dislike it) is because we are integrating or interfacing with a legacy system. Many times, the data will come in a form where there is some delimiter (or delimiters) being used, and it’s up to our application to get the data into a form in which we can use it. This is just one example of when string data needs to be processed. There might be a speci?c reason in your application why you are passing string data around on purpose.

This is when the <c:forTokens> comes into play. <c:forTokens> has the same basic attributes as the <c:forEach>, but with two important differences. First, the items attribute is a String of tokens to iterate over. When iterating over tokenized Strings, it is necessary to de?ne a delimiter to determine the characters that separate the tokens in the String. Therefore, the <c:forTokens> also has a delims attribute. This is a String that contains the set of delimiters to use to iterate through the tokens. If the delims is null, then the items are treated as one contiguous String. If <c:forTokens> reminds you a lot ofjava.util.StringTokenizer, that’s a good thing. The tokens of the string are retrieved using an instance of StringTokenizerwith the attributes items and delims as arguments.

Let’s clarify the terms token and delimiter. A token is a sequence of consecutive characters that are not delimiters. The delimiter characters are used to separate tokens. Ascacquickcsample, givencacstringc“a-b-c” and a delimiter character of “-” the following tokens would be returned : a b c.

When using the <c:forTokens>, you specify a string in the items attribute. Using the delims attribute, you specify what the delimiter characters are. This can be one character, like a comma, or any number of characters. Each one will be used to determine if a new token is created. If the delims attribute is null, then the items are treated as one token. Let’s look at a couple of examples.

Our ?rst sample in Example shows a null delims de?ned. We are using a <c:set> for all of our samples to set a string variable called tokens for our page. The string consists of a number of letters, and various characters. We’ll use the tokens variable to see how changing the delims attribute will affect the outcome of the tokenize operation.

 <c:forTokens> with a null delims Attribute:

<%-- set some tokens to work with --%> 
<c:set var="tokens" value="a,b,c:d:e,f#g,h" /> 
<%-- Tokenize using null delim --%> 
<b>Tokenize the string "<c:out value="${tokens}" />" with a null delimiter</b> <br> 
<c:forTokens var="token" items="${tokens}" delims="" > 
  Found token: <c:out value="${token}" /><br> 

By de?ning a value for delims in following example, we see in how the tokens are now returned. A token is produced only when the character de?ned in delims is found in the string.

 <c:forTokens> with delims Value:

<%-- Tokenize using 1 delim --%> 
<b>Tokenize the string "<c:out value="${tokens}" />" with delims = ","</b> <br> 
<c:forTokens var="token" items="${tokens}" delims="," > 
  Found token: <c:out value="${token}" /><br> 

Next, we add the whole ball of wax. We de?ne multiple delims so that our string token is completely tokenized. This is shown in the code in following Example

<c:forEach> with multiple delims:

<%-- Tokenize using multiple delims --%> 
<b>Tokenize the string "<c:out value="${tokens}" />" with delims = ",:#"</b> <br> 
<c:forTokens var="token" items="${tokens}" delims=",:#" > 
  Found token: <c:out value="${token}" /><br> 

Obviously, from our samples, it is important to know what the delims are that you are interested in, otherwise your tokens might not be what you are expecting. It’s possible to use the EL to access a variable for the delims and then set the data dynamically, perhaps with a request parameter that might have been de?ned on an HTML form. This is shown in following Example

Using Dynamic Values for the delims Attribute:

<c:forTokens var="token" items="${tokens}" delims="${param.delims}" > 
  Found token: <c:out value="${token}" /><br> 

The <c:forTokens> action has the same attributes available as the <c:forEach> that we have already gone into in depth. These include the varStatus for accessing status information about the current iteration loop, the begin, end, and step attributes. The step will be used to indicate that the iteration should process only those tokens de?ned by the step value. All of these attributes work exactly the same in the <c:forTokens> as they do in the <c:forEach>, so I will refrain from going over them again.