Wednesday, June 13, 2007

Putting the Pieces Together

You can imagine how frustrated I was when the day after my last post, Google decided to release version 1.4 of the GWT, which may have fixed the difficulties with cross-domain scripting in the GWT. However, I am not even going to bother testing whether or not the changes they made are adequate until I've finished developing the g3p with the implementation (mac-1.3.3) I've got.

Anyway, after I added the saving of preferences/favorites, it was time to work on the actual search mechanism. In order to do this I ended up wrapping the Entrez utilities with a Java API. This went surprisingly smoothly, and even though it still had bugs in it the first try, the error reporting mechanism was correct enough to identify where they were. Here is a diagram overview of the way the search works:



I completely isolated the Entrez search system into a subpackage of the g3p project, and I am considering releasing it as a separate project for AJAX apps that need to access the Entrez databases. Basically it allows you to create pipelines using the Entrez utilities. Each utility can format its results however it wants to, since each receives XML of a different DTD. Certain utilities provide input for other utilities and so you can chain these together in different ways to achieve different searching goals. I am not yet sure how many distinct pipelines will be needed for the g3p, right now I am only using ESearch->EFetch for searching articles in the Pubmed database.

The XmlParser and XmlRequestor interfaces that the Entrez system talks to have to be implemented differently depending on how the search system is going to be embedded. For the g3p I wrote an IGXmlRequestor class that uses the gadget IG_FetchContent javascript method to retrieve its results. It would seem more natural to use the IG_FetchXmlContent, but this returns a document parsed by the browser DOM instead of text. There may be advantages to doing it this way, but for now parsing using the GWT XML packages seems to provide a much cleaner implementation.

When exception or errors occur during the search process, they are all bubbled back to the ResultsBox also, which can then display what went wrong. The overhead of wrapping the entire Entrez system in this way is worth the time it saves in debugging and adding features.

The other major change I made was to make the overflow scroll properly. Everything in the user interface is more complicated than it seems like it should be, because I want the g3p to work not just as a gadget, but as a standalone web application. This means heights and widths are almost never fixed, and so setting the proper max-height for the scroll bars to start showing up meant writing a WindowResizeListener to figure out the height of the window and set max-height accordingly.

In short, the project is coming along wonderfully, but there are still a number of issues that need to be dealt with. Some of these are minor things like connecting the display results checkbox or changing the number of results displayed. Other things however, will require feedback from the people who will be using it. These are things like what the results should look like, what types of searches are most useful, whether or not to have advanced search options and when to link to external webpages. Depending on the answers to these questions, the UI may still need to be completely redesigned. All in all though, I'm very happy, because its already time to start asking and answering these questions.