For now however, I’m going to see what I can come up with that is relatively generic. One thing I suspect about METS is that many institutions (including UVM)  are generating large portions of the records automagically and that a form would actually slow this process down. The place I see XForms being the most useful for METS is the structMap section where you build a representation of your digital object by organizing the files in the fileSec.

How do you create your METS records? What would you like to see in a METS editor?

I’ll be posting code when I get something half-way presentable, but it would be nice to have a discussion about the variouse ways METS is being used and created and how best to support that.

The work itself was enjoyable though a bit more of a challenge then I had anticipated, partially due to my daughter not napping as much as I thought she would, and partially due to the limitations of XSL 1.0. I prefer 2.0, it has a lot of nice functions that are lacking in 1.0, but in the end it was not a problem, just took me a little longer to do some things. I ended up starting from scratch rather than using the existing XSL (which was derived from one of the EAD Cookbook examples). I think this made things easier, I didn’t have to wade through anyone else’s code, although it may have taken a bit longer.

I’m still waiting for UVM to come through with my consulting contract, but am also starting to look around for other possible consulting opportunities. So if you need any XSL, XQuery or XForms work done, drop me a line.

My work status remains a little up in the air. I took a year leave of absence from UVM, with the understanding that I would be doing about 20 hrs worth of consulting work for them while on leave. I haven’t started doing work for UVM yet but I have started doing some consulting. Currently I’m working on revamping the XSL stylesheets for HTML and PDF displays for the Archivists’ ToolKit. This has been a great project to ease back into working, because while EAD is a bit of a headache, I really enjoy working with XSL. It feels so uncomplicated after XQuery and XForms. Both stylesheets should be done by the begining of August for the next AT release.

My daughter Hannah was born last Friday, and as a result, I’m taking a little break from work and blogging. I’ll be back in oh… six weeks or so.

I’ve e-mailed the list and an updated version of the extension should be out in a few days, in the meantime I’ve been poking about unsuccessfully looking for a nightly build that works with FF 2.0.0.13. I’ve also finally installed Orbeon on my development server and ported over the MODS XForm. It was surprisingly painless to get it working. However the working form is just the first step, since I have a whole metadata management system that has to be integrated. Most of the system is built using xquery, but could possibly be done entierly in XForms, it will just require some thinking through.

Unfortunately I’m on a bit of a tight schedule, I need to have the forms working reliably as soon as possible because I’m going on maternity leave some time in the next 3 weeks. I’ve got a lot of wrapping up and documenting to do before that time, so I’m not sure how far I’ll get with Orbeon.

**This version (for Windows) of the XForms extension does work with FF 2.0.0.13, just be sure to uninstall your earlier version of the extension before you install the new one. I didn’t do an uninstall, and was having all sorts of problems with submitting data. Everything seems to work just fine now.

1) I could store all the xqueries that build my metadata admin interface in eXist with restricted read permissions so that uses would be presented with a login from the server when they arrive at the page, this would initiate the REST session, which would presumably perpetuate as long as the window was active (and thus hopefully by-pass the random logout issue.)

I haven’t actually tried this one, I just assume it would work. I don’t particularly like the solution because I don’t store most of my xqueries in eXist. However, I haven’t ruled it out.

2) Solution 2 is the one I worked on all day yesterday. First I’ve gone back to submitting the data via a POST which sends the data to an xquery, this query then uses the xmldb:store() function to save the data to eXist. This xquery also checks to see if there is a current session using session:exists(), if not it will send back “Not logged in!” if there is a current session the xquery will attempt to save the record to the database, if successful the xmldb:store() function returns the path to the resource, if it fails it returns an empty string.

My XForm now uses an event observer to check if the submit action was successful, if not it will warn the user that they are either not logged in, or have insufficient privileges to edit the resource and toggles them to a login form located at the top of their current page. Users can then login, a separate event is fired that changes or creates a new session and the user can save the data in the form, all without refreshing or leaving the form.

After much tweaking this seems to work. The nice thing is that having the login on the form saves the user from losing the data that they have been working on.

Here are what the additions the form look like:

Two new instances, the first to check what sort of error the xquery sends back if it is not successful:

<xforms:instance id="submitError">
  <dummy xmlns="">
    <submit/>
  </dummy>
</xforms:instance>

The second to hold the login information:

<xforms:instance id="login">
  <dummy xmlns="">
    <user/>
    <pass/>
  </dummy>
</xforms:instance>

Additions to the submission action:

<xforms:submission id="submit" method="post" ref="instance('metadata')"
 replace="text" instance="submitError" action="xqueryGoesHere.xql">
   <xforms:action ev:event="xforms-submit-done" ev:observer="submit">
     <xforms:message level="modal">Item Saved!</xforms:message>
     <xforms:refresh/>
   </xforms:action>
   <xforms:action ev:event="xforms-submit-error" ev:observer="submit">
     <xforms:message level="modal"
       ref="instance('submitError')/child::*[. = '']">You do not have permission
       to edit this resource! Please log in.</xforms:message>
     <xforms:message level="modal"
       ref="instance('submitError')/child::*[. = 'Not logged in!']">You are no longer
       logged in! Please log in.</xforms:message>
     <xforms:toggle case="case-login"/>
     <xforms:refresh/>
   </xforms:action>
</xforms:submission>

I’ve then added a xforms:switch to the top of the MODS form that toggles between a login form and summary information about the resource. As you can see in the submission, if there is an error, the top part of the form will toggle to display a login form for the users, case-login. Once the user logs in with the new name and password, they click login which fires this submission action:

<xforms:submission id="do-login" method="get" replace="instance"
 instance="login" separator="&" ref="instance('login')" action="login.xql">
  <xforms:action ev:event="xforms-submit-done" ev:observer="do-login">
    <xforms:message level="modal">You are now logged in. Please re-save your data.
    </xforms:message>
    <xforms:toggle case="ready"/>
    <xforms:refresh/>
  </xforms:action>
  <xforms:action ev:event="xforms-submit-error" ev:observer="do-login">
    <xforms:message level="modal">Incorrect login information</xforms:message>
    <xforms:toggle case="case-login"/>
    <xforms:refresh/>
  </xforms:action>
</xforms:submission>

The login.xql looks very similar to this example from the eXist website. If the login is successful the form toggles from the login info to the MODS form info. The user is then prompted to re-save their data as the new user.

I’m still a little nervous putting the catalogers back to work with these forms because I haven’t been able to replicate some of the problems they had reported, but  I’m hoping even if they do get logged out this new form will allow them to log back in without losing any data. I’ve also finally made it possible to page from one record to the next in the queue, maybe that will buy me a little extra goodwill.

I had been submitting the data from our XForms to an xquery via post, the xquery then did some post processing and submitted the data to eXist using xupdate. Thinking it would be more straight forward I switched to using the RESTservlet, which allowed me to do a simple PUT from my XForm.  However,  I can’t seem to persist my session through the PUT request. My metadata administrative interface is a series of xqueries, the user must log in on the fist page using the eXist xmldb:login function, they then can browse through records in process, pull up completed records to edit and create new collections. However now when the user presses the save button on a record they are asked to login again. Once they have logged in from an XForm, they can save as many records as they like, so essentially it creates another session.

After some back and forth on the eXist mailing list it appears that sessions persist across xqueries, but that the RESTserver does not reuse this session information. The suggested solution was to post from my XForm to an xquery… which is what I had been doing. The real problem I’m having is how to get reliable information back to the user on whether or not the save was successful, and if not (due to an expired session) allow them to log back in without navigating away from the form they are on.

I’m wondering if I can use an XForms alert that is tied to an xforms-submit-error response. I haven’t found any examples of this so far, but I’m still looking.  Also, I seem to be getting an error message back from my form even when the data has submitted successfully.

I’m racking my brains for a workaround solution, because I’d like to get metadata work back underway, so I have a few weeks to monitor it. Any suggestions, or examples you know of would be greatly appreciated. Otherwise I’ll just carry on testing various solutions until I run across one that works.

Here are the slides, video will be coming eventually, although I need to redo the audio portion of my half of the talk due to a technical snafu.

Doing a joint 20 minute presentation was quite a challenge, I think both Mike and I could have easily used the entire 20 minutes, but I think it was a good thing for people to see the two different editors, and be able to talk to us both after the presentation about which ever approach might be most appropriate for their institution.

Both Mike and I have released the code for our editors as opens source (Mike’s is quite well documented, I’m still working on documentation). You can see examples of the forms and get the code at the locations below.

UVM

• Code: http://code.google.com/p/xforms4lib/
• Examples:
  • http://cdi.uvm.edu/exist/xforms/mods
  • http://cdi.uvm.edu/exist/xforms/modsSimple

Brown

• Code and documentation: http://dl.lib.brown.edu/its/software/metadata/
• Example: http://riker.services.brown.edu:8080/repo/mods/demo.html

Today there are a plethora of people working on MODS editor solutions. Here are just a few:

1. The University of Tennessee Libraries – The University of Tennessee has released their MODS workbook as open source, it is a web based form that I believe is javascript based.
2. Peter Binkley at the University of Alberta – Peter has just announced his MODS editor is complete, or very nearly so and is built using the Cocoon Forms Framework and and you can try it out here.
3. Michael Park at Brown University – Mike has been working on a MODS editor using Orbeon Forms.
   **edited to add links from our recent presentation:
   Code and documentation: http://dl.lib.brown.edu/its/software/metadata/
   Example: http://riker.services.brown.edu:8080/repo/mods/demo.html
4. Parmit Chilana formerly at Princeton University – Wrote a MODS editor using Orbeon, I’m not sure who has taken over now that she has moved on.
5. Clay Redding at the Library of Congress
6. Me – I just got permission to release our metadata editor as open source from the UVM powers-that-be. The forms need some cleaning up, but I’m posting my demo forms here, and will be slowly adding the code for the entire metadata editing interface, including our forms for posting data to Solr. I’ve also had some time to refine the MODS editor we are using in production, there are two versions, the simple version – used by the copy catalogers – and the full version for editing the full record.
   **links from my recent presentation:
   Code: http://code.google.com/p/xforms4lib/
   Examples:
   • http://cdi.uvm.edu/exist/xforms/mods
   • http://cdi.uvm.edu/exist/xforms/modsSimple

Its great to see so much development happening in this area, it always seemed crazy to me that we had so many library metadata standards but but no way for our users to create these records. I still prefer the XForms solution to most of the other solutions I’ve seen, just because XForms seems to me the most logical and simple (in spite of all the trouble I’ve had with it) method for creating and editing XML data.

Now, if only someone would build a decent METS editor.

I’ve follwed the trail of temprorary fragments on my development machine by tracking the exist logs. After every query/page request I check the logs to see if a temprary fragment has been created. In addition  I’ve set up a little xquery that pulls the results out of /db/system/temp so I can see excatly what the temprary fragment is, allowing me to pin point the problems in my xqueries.

The results supprised me, although after a little more reading on eXist and temporary fragments, they make sense. In particular this thread on the eXist mailing list was enlightening. I use the doc() function to return the xml response from Solr and then transform it with XSL stored in eXist. For some reason I was calling the results like this:  doc(‘solrResults’)/child::*  As noted in the above thread, applying an xpath to the returned fragment causes it to be stored as a temporary fragment. The fix has been easy enough, I simply removed the child::* operator and adjusted my XSL.

I have a few other queries that were also creating temporary fragments, and for the most part the issues are very similar, the use of xpath on a returned fragment, rather than using xsl, or even creating variables in the xquery to get the values. For the most part it has been pretty simple to rewrite these queries, and has also given me a chance to clean up some of my code.  In addition I’ve allocated more memory to Tomcat. Hopefully these two adjustments will alleviate the issues we have been having the past few weeks.