How to extract HTTP header fields in a REST API

To be able to extract HTTP headers fields from incoming requests accessing a REST API is particularly useful when trying to log the transactions that are coming through the REST based service.

In this example, I use Java based JBoss RESTEasy together with @Context java annotation to extract the header information.

One of the REST API operation is /version that returns the current version of the API in a JSON format.The interface operation is defined as follow:

     import javax.ws.rs.core.HttpHeaders;

     /**
     * Get the current version of the REST API.
     */
     @GET
     @Path("/version")
     @Produces("application/json")
     @GZIP
     public Response getVersion(@Context HttpHeaders headers);

The @Context annotation allows you to map request HTTP headers to the method invocation.

One way to safely extract all available header parameters is to loop through the headers.The method getRequestHeaders from javax.ws.rs.core.HttpHeaders returns the values of HTTP request headers. The returned map is case-insensitive and is read-only.

   if (headers != null) {
       for (String header : headers.getRequestHeaders().keySet()) {
          System.out.println("Header:"+header+
                             "Value:"+headers.getRequestHeader(header));
       }
   }

When querying the REST API via a browser:

  http://www.acme.com/api/version

you obtain a minimum set of headers fields:

   Header:cache-control Value:[max-age=0]
   Header:connection Value:[keep-alive]
   Header:accept-language Value:[en-us,en;q=0.5]
   Header:host Value:[www.acme.com]
   Header:accept Value:[text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8]
   Header:user-agent Value:[Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0)    
   Gecko/20100101 Firefox/8.0]
   Header:accept-encoding Value:[gzip, deflate]
   Header:session-id Value:[d636369c]
   Header:accept-charset Value:[ISO-8859-1,utf-8;q=0.7,*;q=0.7]

To be able to extract a specific header field (e.g. Host), you can use the getRequestHeader function:

 public Response getVersion(@Context HttpHeaders headers) {
     if (headers != null) {
        List<String> hostHeader = headers.getRequestHeader("host");
        if (hostHeader != null) {
           for (String host : hostHeader) {
              LOG.debug("Host:"+host);
           }
        }
     } 
  
     // Get the version
     final Version current_version = new Version();
     final ObjectMapper mapper = new ObjectMapper();
     final JsonNode rootNode = mapper.createObjectNode();
     ((ObjectNode) rootNode).putPOJO(Version.XML_ROOT_ELEMENT, current_version);
     
     final ResponseBuilder builder = Response.ok(rootNode);
     return builder.build();
   }

If you are using a test harness tools like cURL, you can also easily add additional HTTP header fields
(e.g. the email address of the user making the request or the referer) and test the logging functionality of your REST API:

curl -H "From: user@example.com" http://www.acme.com/api/version

curl -H "Referer: http://consumer.service.acme.com" http://www.acme.com/api/version

The headers can also be obtained through the HttpServletRequest object.
This can be done using the @Context HttpServletRequest annotation and can provide additional information about the incoming request:

public Response getVersion(@Context HttpServletRequest request) {
   LOG.debug("Host:"+request.getHeader("host"));
   LOG.debug("Request-URL:"+request.getRequestURL());
   ...
} 

Since HttpServletRequest extends ServletRequest you also getting useful methods providing information on the Internet Protocol (IP) address, host and port of the client or last proxy that initiated the request.

public Response getVersion(@Context HttpServletRequest request) {
   LOG.debug("Remote-IP:"+request.getRemoteAddr());
   LOG.debug("Remote-Host:"+request.getRemoteHost());
   LOG.debug("Remote-Port:"+request.getRemotePort());
   ...
} 

cURL tests harness and TLS

In a previous post, I have explained how to use cURL to test harness REST based Web services. One thing I did not described was how to add Transport Layer Security (TLS) in your tests. In other words, how to successfully test REST Web services over HTTPS?

The cURL manual describes a certain number of options that can be used. One of the most convenient option is -k or --insecure. It allows curl to  perform  "insecure" SSL connections and transfers. All SSL connections are attempted to be made secure by using the CA certificate  bundle  installed by  default. So if you SSL connection does not require client side authentication it is a very quick way to test your web service over SSL:

  curl -k https://www.acme.com/api/version


Another useful option can be used if in conjunction to SSL, you need to compress your payload via GZIP for example to optimize the transfer of large messages. In this case, you will use the option -H or --header that will help you specify custom headers to your request:

  curl -H "Accept-Encoding: gzip,deflate" -k https://www.acme.com/api/users/list


Notice in that case that the -k or --insecure option is always placed just in front of URL.

Of course, other headers such as the one described previously can be combined:

  curl -c ./cookies.txt --data-binary @login_password.json -H "Content-Type: application/json" -H "Accept-Encoding: gzip,deflate" -k https://www.acme.com/api/users/token

How to test harness REST web services with cURL

cURL is a very convenient command line tool to send and retrieve data using the URL syntax. It supports a large number of protocols: HTTP, HTTPS, FTP, FTPS, SCP, SFTP, TFTP, LDAP, LDAPS, DICT, TELNET, FILE . IMAP, POP3, SMTP and RTSP.

I have been using cURL for quickly and conveniently testing RESTFul APIs.

Since I am doing most of my development on Windows platforms, I am using cURL in conjunction with Console, a Windows console window enhancement tool.

 First you need to download and install cURL for your platform (windows, Linux ect). Then you can do a quick test by accessible the home page of your favorite web site:

This should display the HTML content of the home page. At this point I would suggest to look at the documentation to see what you can do with cURL including the FAQ.

I usually use simple test harness such as 'curl http://www.acme.com/api/versions' if you REST API exposes available versions.

You can then start to do more sophisticated tests such as authentication credentials from files (in my case I use a JSON data structure to automatically authenticate to my web service), saving cookies on files (using the the -c option):

curl -c ./cookies.txt --data-binary @login_password.json -H "Content-Type: application/json"  http://www.acme.com/api/users/token

With my JSON file as follow: {"login":{"username":"user1","password":"StrongPassword1"}} 

The cookie in my case contains a session token that I can reuse between each cURL calls. In the next call I read the cookie (via the -b option):

curl -b ./cookies.txt http://www.acme.com/api/users/userid1234567890/orders

The output of the command can be saved in a file using the option -o or redirecting our output:

curl -o google.html www.google.com

or

curl www.google.com > google.html 

Enjoy!

In my next post on this topic, I will explain how to use SSL/TLS and specify GZIP headers to your cURL requests.

Response objects and the use of GenericEntity class with RESTEasy

Recently during the implementation of a REST API, I wanted to return a complex response containing a list of objects (Patients). The issue was that the RESTEasy build-in JAXB MessageBodyWriter could not directly handle lists of JAXB objects (Java has trouble obtaining generic type information at runtime).

I was recently in a situation where I had to create a complex response to a HTTP POST for my REST API. I am using JAXB /JSON support from RESTEasy.

I found some element of answer in the book "RESTFul Java with JAX-RS" from Bill Burke (pp 102). However the code snippet had a couple of errors:

  • the GenericEntity object cannot be passed to the Response.ok() method directly (a ResponseBuilder is required).

  • references to GenericEntity needs to be parameterized.

My use case is a little more complex than in the book. I am receiving a user-name and password from a POST (e.g. a form submit). I then perform the authentication and returns a list of Patient objects in a JSON/GZIP compressed format (instead a list of Customer objects) together with an authentication token.





The resulting code looks like this:

   @POST
   @Path("/token")
   @Consumes("application/x-www-form-urlencoded")
   @Produces("application/json")
   @GZIP
   public Response getPatientsWithToken(@FormParam("username") String username, @FormParam("password") String password) {
  
        Login login = new Login(username, password);
        // ... perform authentication here ....
    
        // Build the returning patient list
        List<Patient> returnList = new ArrayList<Patient>();
        returnList.addAll(patients.values());
        Collections.sort(returnList);
      
        GenericEntity<List<Patient>> entity = new GenericEntity<List<Patient>>(returnList){};
      
        // Create the response
        ResponseBuilder builder = Response.ok(entity);
        return builder.build();
   }

Of course you will have to import the following classes as well:

import javax.ws.rs.core.GenericEntity;
import javax.ws.rs.core.Response;
import javax.ws.rs.core.Response.ResponseBuilder;