XSL for the Archivits’ ToolKit

August 1, 2008

I just finished the XSL work I’ve been doing for the Archivists’ ToolKit. The stylesheets will both be included in the next release which I believe is scheduled for September. I’m fairly pleased with how they came out, the layouts had to be simple, no images and neutral color schemes in order to be generic enough for any institution to use, but at the same time I wanted them to look attractive. Here is a sample of the HTML output, and here is a sample of the PDF (this is an EAD from UVM’s collection). In fact I think I like the design quite a bit more than the one I did for UVM, it is nice and simple.

The work itself was enjoyable though a bit more of a challenge then I had anticipated, partially due to my daughter not napping as much as I thought she would, and partially due to the limitations of XSL 1.0. I prefer 2.0, it has a lot of nice functions that are lacking in 1.0, but in the end it was not a problem, just took me a little longer to do some things. I ended up starting from scratch rather than using the existing XSL (which was derived from one of the EAD Cookbook examples). I think this made things easier, I didn’t have to wade through anyone else’s code, although it may have taken a bit longer.

I’m still waiting for UVM to come through with my consulting contract, but am also starting to look around for other possible consulting opportunities. So if you need any XSL, XQuery or XForms work done, drop me a line.

Gone Fishing…

April 14, 2008

I’ve been a little distracted lately:

My daughter Hannah was born last Friday, and as a result, I’m taking a little break from work and blogging. I’ll be back in oh… six weeks or so.


March 27, 2008

I pretty much dread every Firefox upgrade, because of the havoc it could wreck on my XForms. The past few upgrades have been problem free, but yesterday’s automatic update has put a halt to metadata production once again. I’m having trouble explaining myself to the disgruntled catalogers who had just gotten back to work.

I’ve e-mailed the list and an updated version of the extension should be out in a few days, in the meantime I’ve been poking about unsuccessfully looking for a nightly build that works with FF I’ve also finally installed Orbeon on my development server and ported over the MODS XForm. It was surprisingly painless to get it working. However the working form is just the first step, since I have a whole metadata management system that has to be integrated. Most of the system is built using xquery, but could possibly be done entierly in XForms, it will just require some thinking through.

Unfortunately I’m on a bit of a tight schedule, I need to have the forms working reliably as soon as possible because I’m going on maternity leave some time in the next 3 weeks. I’ve got a lot of wrapping up and documenting to do before that time, so I’m not sure how far I’ll get with Orbeon.

**This version (for Windows) of the XForms extension does work with FF, just be sure to uninstall your earlier version of the extension before you install the new one. I didn’t do an uninstall, and was having all sorts of problems with submitting data. Everything seems to work just fine now.

Session Solutions

March 12, 2008

So I came up with two possible solutions to my sessions problem, I’m sure there are others as well.

1) I could store all the xqueries that build my metadata admin interface in eXist with restricted read permissions so that uses would be presented with a login from the server when they arrive at the page, this would initiate the REST session, which would presumably perpetuate as long as the window was active (and thus hopefully by-pass the random logout issue.)

I haven’t actually tried this one, I just assume it would work. I don’t particularly like the solution because I don’t store most of my xqueries in eXist. However, I haven’t ruled it out.

2) Solution 2 is the one I worked on all day yesterday. First I’ve gone back to submitting the data via a POST which sends the data to an xquery, this query then uses the xmldb:store() function to save the data to eXist. This xquery also checks to see if there is a current session using session:exists(), if not it will send back “Not logged in!” if there is a current session the xquery will attempt to save the record to the database, if successful the xmldb:store() function returns the path to the resource, if it fails it returns an empty string.

My XForm now uses an event observer to check if the submit action was successful, if not it will warn the user that they are either not logged in, or have insufficient privileges to edit the resource and toggles them to a login form located at the top of their current page. Users can then login, a separate event is fired that changes or creates a new session and the user can save the data in the form, all without refreshing or leaving the form.

After much tweaking this seems to work. The nice thing is that having the login on the form saves the user from losing the data that they have been working on.

Here are what the additions the form look like:

Two new instances, the first to check what sort of error the xquery sends back if it is not successful:

<xforms:instance id="submitError">
  <dummy xmlns="">

The second to hold the login information:

<xforms:instance id="login">
  <dummy xmlns="">

Additions to the submission action:

<xforms:submission id="submit" method="post" ref="instance('metadata')"
 replace="text" instance="submitError" action="xqueryGoesHere.xql">
   <xforms:action ev:event="xforms-submit-done" ev:observer="submit">
     <xforms:message level="modal">Item Saved!</xforms:message>
   <xforms:action ev:event="xforms-submit-error" ev:observer="submit">
     <xforms:message level="modal" 
       ref="instance('submitError')/child::*[. = '']">You do not have permission 
       to edit this resource! Please log in.</xforms:message>
     <xforms:message level="modal"  
       ref="instance('submitError')/child::*[. = 'Not logged in!']">You are no longer 
       logged in! Please log in.</xforms:message>
     <xforms:toggle case="case-login"/>

I’ve then added a xforms:switch to the top of the MODS form that toggles between a login form and summary information about the resource. As you can see in the submission, if there is an error, the top part of the form will toggle to display a login form for the users, case-login. Once the user logs in with the new name and password, they click login which fires this submission action:

<xforms:submission id="do-login" method="get" replace="instance" 
 instance="login" separator="&" ref="instance('login')" action="login.xql">
  <xforms:action ev:event="xforms-submit-done" ev:observer="do-login">
    <xforms:message level="modal">You are now logged in. Please re-save your data.
    <xforms:toggle case="ready"/>
  <xforms:action ev:event="xforms-submit-error" ev:observer="do-login">
    <xforms:message level="modal">Incorrect login information</xforms:message>
    <xforms:toggle case="case-login"/>

The login.xql looks very similar to this example from the eXist website. If the login is successful the form toggles from the login info to the MODS form info. The user is then prompted to re-save their data as the new user.

I’m still a little nervous putting the catalogers back to work with these forms because I haven’t been able to replicate some of the problems they had reported, but  I’m hoping even if they do get logged out this new form will allow them to log back in without losing any data. I’ve also finally made it possible to page from one record to the next in the queue, maybe that will buy me a little extra goodwill.

Struggling with sessions

March 10, 2008

Right before I left for Code4lib 2008 we had a serious db crash, this was an eXist issue, which I’m hoping was partially solved by upgrading to 1.2 as well as some re-writing of xqueries to eliminate most of our temporary fragments creation. We haven’t had any problems with crashing since the upgrade, but a by product of the crash was some odd behavior that was reported to me by one of the catalogers. She was getting logged out in the middle of editing a record. Because of the way my forms had been set up she wasn’t getting any reliable messages about the status of the record (if it had been successfully saved or not), so she assumed a record had been saved, but it wasn’t, several records were lost, and worse, cataloger good will is now a bit more shaky than it was.

I had been submitting the data from our XForms to an xquery via post, the xquery then did some post processing and submitted the data to eXist using xupdate. Thinking it would be more straight forward I switched to using the RESTservlet, which allowed me to do a simple PUT from my XForm.  However,  I can’t seem to persist my session through the PUT request. My metadata administrative interface is a series of xqueries, the user must log in on the fist page using the eXist xmldb:login function, they then can browse through records in process, pull up completed records to edit and create new collections. However now when the user presses the save button on a record they are asked to login again. Once they have logged in from an XForm, they can save as many records as they like, so essentially it creates another session.

After some back and forth on the eXist mailing list it appears that sessions persist across xqueries, but that the RESTserver does not reuse this session information. The suggested solution was to post from my XForm to an xquery… which is what I had been doing. The real problem I’m having is how to get reliable information back to the user on whether or not the save was successful, and if not (due to an expired session) allow them to log back in without navigating away from the form they are on.

I’m wondering if I can use an XForms alert that is tied to an xforms-submit-error response. I haven’t found any examples of this so far, but I’m still looking.  Also, I seem to be getting an error message back from my form even when the data has submitted successfully.

I’m racking my brains for a workaround solution, because I’d like to get metadata work back underway, so I have a few weeks to monitor it. Any suggestions, or examples you know of would be greatly appreciated. Otherwise I’ll just carry on testing various solutions until I run across one that works.

MODS for everyone

February 18, 2008

When I started working on the CDI project almost 2 years ago there were no easy solutions for integrating the creation and editing of complex metadata into a project workflow. Most DAMS (Digital Asset Management Systems) came with the ability to create Dublin Core, or some form of modified Dublin Core, but the creation of MODS was not supported.

Today there are a plethora of people working on MODS editor solutions. Here are just a few:

  1. The University of Tennessee Libraries – The University of Tennessee has released their MODS workbook as open source, it is a web based form that I believe is javascript based.
  2. Peter Binkley at the University of Alberta – Peter has just announced his MODS editor is complete, or very nearly so and is built using the Cocoon Forms Framework and and you can try it out here.
  3. Michael Park at Brown University – Mike has been working on a MODS editor using Orbeon Forms.
    **edited to add links from our recent presentation:
    Code and documentation: http://dl.lib.brown.edu/its/software/metadata/
    Example: http://riker.services.brown.edu:8080/repo/mods/demo.html
  4. Parmit Chilana formerly at Princeton University – Wrote a MODS editor using Orbeon, I’m not sure who has taken over now that she has moved on.
  5. Clay Redding at the Library of Congress
  6. Me – I just got permission to release our metadata editor as open source from the UVM powers-that-be. The forms need some cleaning up, but I’m posting my demo forms here, and will be slowly adding the code for the entire metadata editing interface, including our forms for posting data to Solr. I’ve also had some time to refine the MODS editor we are using in production, there are two versions, the simple version – used by the copy catalogers – and the full version for editing the full record.
    **links from my recent presentation:
    Code: http://code.google.com/p/xforms4lib/

Its great to see so much development happening in this area, it always seemed crazy to me that we had so many library metadata standards but but no way for our users to create these records. I still prefer the XForms solution to most of the other solutions I’ve seen, just because XForms seems to me the most logical and simple (in spite of all the trouble I’ve had with it) method for creating and editing XML data.

Now, if only someone would build a decent METS editor.

Fixing temprorary fragments

February 12, 2008

Surprise! Or at least to me… My XForms are not the source of the majority of my temporary fragments, at least as far as I can tell. This is good news because the XForms constitute the most complicated part of my web app, and would take the longest amount of time to troubleshoot and fix.

I’ve follwed the trail of temprorary fragments on my development machine by tracking the exist logs. After every query/page request I check the logs to see if a temprary fragment has been created. In addition  I’ve set up a little xquery that pulls the results out of /db/system/temp so I can see excatly what the temprary fragment is, allowing me to pin point the problems in my xqueries.

The results supprised me, although after a little more reading on eXist and temporary fragments, they make sense. In particular this thread on the eXist mailing list was enlightening. I use the doc() function to return the xml response from Solr and then transform it with XSL stored in eXist. For some reason I was calling the results like this:  doc(‘solrResults’)/child::*  As noted in the above thread, applying an xpath to the returned fragment causes it to be stored as a temporary fragment. The fix has been easy enough, I simply removed the child::* operator and adjusted my XSL.

I have a few other queries that were also creating temporary fragments, and for the most part the issues are very similar, the use of xpath on a returned fragment, rather than using xsl, or even creating variables in the xquery to get the values. For the most part it has been pretty simple to rewrite these queries, and has also given me a chance to clean up some of my code.  In addition I’ve allocated more memory to Tomcat. Hopefully these two adjustments will alleviate the issues we have been having the past few weeks.