Author: Jane Stevenson

Do we need index terms?

/ Jane Stevenson

image of road signArchival descriptions need to include associated subjects, names and places as index terms. Is that self-evident? Well, certainly we need to do what we can to provide ways into an archive, and you might say the more ways to access it the better. But do archival descriptions need index terms? Do they add anything that keyword searches don’t have?

The Archives Hub encourage our contributors to add access points, which is EAD speak for index terms for subjects, names and places that reflect the content of the description, and therefore the archive. But if those terms are already included in the description, with the technology at our disposal, maybe we can dispense with them as access points and simply query the main body of the description? What are the arguments in favour of keeping index terms?

1. It’s about what is significant. One of the great challenges with archives is drawing out what is important within the archive; enabling researchers to know whether the archive is relevant to them. But this is always going to be a very imperfect exercise. I remember cataloguing an architect’s diaries (Robert Mylne, architect of Blackfriars Bridge) and ending up taking months because I couldn’t bear to leave out any people, or place names or buildings, or building techniques, etc. What if someone really wanted to know about stanchions? If I didn’t mention them, then a search would not bring back the Mylne diaries, and I would have failed to connect researcher to research material. The reality is that with the time and resources at our disposal, what we need to try do is reflect what is ‘most significant’ and include ‘key concepts’, accepting that this is a somewhat subjective judgement and hoping that this is enough to lead the researcher in the right direction. For the Hub we usually recommend adding somewhere between 3 and 10 index terms to a description. It means that the archivist can (arguably) draw out the most pertinent subjects and list the most significant people.

2. It allows for drawing out entities. So, in a sentence like “The collection comprises of material relating to the British National Antarctic Expedition, 1901-1904 (leader Robert Falcon Scott), the British Antarctic Expedition, 1907-1909, led by Shackleton, correspondence with his family, miscellaneous papers and biographical information”, you can separate out the entities. Corporate bodies such as British National Antarctic Expedition, 1901-1904, and personal names such as Robert Falcon Scott.  This is very useful for machine processing of content, as machines do not know that Robert Falcon Scott is a personal name (although we are increaingly developing sophisticated text mining techniques to address this).

It can be particularly useful where the entities are not obvious from the text, such as “[A]s well as material relating to his broadcast and published works, the archive also includes many scripts…”. Notice a lack of definite subject terms such as ‘playwright’, or ‘writer’.  A human user may infer this, but a general search on ‘playwright’ will not bring back any results becauase a machine has to know it too, in order to serve the human user.

3. You can then apply consistency to the entities, in terms of using a pre-defined controlled vocabulary.  Bu in a world where folksonomies are becoming increasingly popular, with increasing use of user tagging, does it make sense to insist on controlled vocabularies?

Take the example above, which is about Arthur Hopcraft. The index terms do include ‘playwrights’ and ‘writers’ so that the user can do a keyword search on these terms, or a specific subject search, and find the description. However, there is an obvious flaw here: the archivist has chosen these terms. Whilst they do both come from the Unesco thesaurus, she could easily have chosen different terms. The index terms do not include ‘scriptwriter’ for example. They do not include ‘television’ or ‘journalism’, both of which could have reasonably been used for this description. We end up with some descriptions that use ‘playwrights’ as a controlled vocabulary term, but others that don’t, and some that maybe use ‘scriptwriters’ when they are essentially about the same subject, or ‘authors’ which is the Unesco preferred term for scriptwriters.

But you cannot cover everything, so you have to make a choice about which subject terms to use. The question is: is it better to have some subject terms rather than none, even if they do not necessarily cover ‘all’ subjects, and so the researcher may carry out a subject search and not find the archive? One important point is that with our without subject terms, you have the same problem; it is just that a specific subject search does actually narrow what the researcher is searching on – the search may not include other fields, such as the scope & content or biographical history. Therefore whilst a subject search helps the researcher to find the most significant collections, it may exclude some collections that might be very pertinent for their research (collections that they may find through a keyword search).

4. Index terms allow for clarification of which entity you are talking about. This can be particularly helpful with identifying people and corporations. The scope and content may refer to Linsday Anderson, but the index entry will provide the dates and maybe an epithet to clarify that this is Lindsay Gordon Anderson, 1923-1994, film director. You could add this information to the scope and content, but it would tend to make it much more dense and arguably more difficult to read if you did this with all names. It would also imply that all names are of equal significance, and it would not be very helpful for machine processing unless you marked it up so that a machine could identify it as a personal name.

5. Index terms allow for connecting the same entity throughout the system. A very useful and powerful reason to have index terms. The main issue here is that contributors do not always enter the same thing, even with rules and sources to draw upon. Personal and corporate names are usually consistent, but inevitably the addition of the epithet, which is much more of an archival practice than a library practice, means that one person often has a number of different entries. If you took the epithet away, at least for the purposes of identifying the same entity, then things would work reasonably well. For subjects it’s more a case of just the amount of subjects that can be used to describe an archive. If you look for all the descriptions with the subject of ‘first world war’, then you won’t find all the descritions that are significantly about this subject because some of them are indexed with ‘world war one’, and other may use ‘war’ and ‘conflict’.

The way around this for the Hub is our ‘Subject Finder’. This is different from a straightforward subject search. It actually looks for similar terms and brings them together. So, a search for ‘first world war’ will bring back ‘world war one’. Similarly, a search for ‘railways’ will bring back the Library of Congress heading of ‘railroads’.

The Subject Finder helps, but does not comletely address this problem of the differing choice of terms. It cannot by-pass the fact that sometimes descriptions do not include any subject terms, so then they will not show up in a subject search. Recently I was looking for archives in the Hub on ‘exploration’, and was surprised to find that many of the Antarctic expeditions collections were not listed in the results. This was because some repositories did not use this subject term; a perfectly legitimate choice not to use it, but many other similar archives do use it.

I still feel that it is worth adding the significant entities as index terms, even with the problems of selecting what is ‘significant’ and with the inconsistencies that we have. Cataloguing as a whole is a subjective exercise, and it will never be perfect. For those who say that index terms are out-dated, I can only say that they are proving pretty useful for our current Linked Data project, and that is certainly pretty up to the minute in terms of Web technologies.

One final point in favour: the Archives Hub index terms exist within the descriptions as clickable links. This allows researchers to carry out ‘lateral’ searches, and it is a popular means to traverse descriptions, exploring from one subject to another, from one person to another.

Whether we should also consider enabling researchers to tag descriptions themselves is a whole other issue for another blog post…

This is not a complete case for and against by any means, but I think I’d better leave it there. I’d love to hear your views.

First class citizens of the Web

/ Jane Stevenson / 5 Comments

Linked Data enthusiasts like to talk about making concepts within data into first-class citizens. This should appeal to archivists. The idea that the concepts within our data are equal sounds very democratic, and is very appealing for rich data such as archival descriptions. But, where does that leave the notion of the all important top-level archival collection description? Archivists do tend to treat the collection description as superior; the series, sub-series, file, item, etc., are important, but subservient to the collection. You may argue that actually they are not less important, but they must be seen in the context of the collection. But I would still propose that (certainly within the UK) the collection-level description generally tends to be the focus and is considered to be the ‘right’ way into the collection, or at least, because of the way we catalogue, it beomes the main way into the collection.

Linked Data uses as its basis the data graph. This is different from the relational model and the tree structure model. In a graph, entities are all linked together in such a way that none has special status. All concepts are linked, the links are specified – that is to say, the relationships are clarified. In a tree structure, everything filters down, so it is inevitable that the top of the tree does seem like the most important part of the data. A data graph can be thought of as a tree structure where links go both ways, and nothing is top or bottom. You could still talk about the collection description being the ‘parent’ of the series description, but the series description is represented equally in RDF. But, maybe more fundamentally than this, Linked Data really moves away from the idea of the record as being at the heart of things and  replaces this with the idea of concepts being paramount. The record simply becomes one other piece of data, one other concept.

This type of modelling accords with the idea that users want to access the data from all sorts of starting points, and that they are usually interested in finding out about something real (a subject, a person) rather than an archive per se. When you model your data into RDF what you are trying to think about is exactly that – how will people want to access this data. In Australia, the record series is the preferred descriptive entry, and a huge amount has been written about the merits of this approach. It seems to me, with RDF, we don’t need to start with the collection or start with the series. We don’t need to start with anything.

Linked Data graph

This diagram, courtesy of Talis, shows part of a data graph for modelling information about spacecraft. You can see how the subjects (which are always represented by URLs) have values that may be literal (in rectangular boxes) or may point to other resources (URLs). Some of this data may come from other datasets (use of the same URL for a spacecraft enables you to link to a different resource and use the values within that resource).

The emphasis here is on the data – the concepts – not on the carrier of the data – the ‘record’.

In our LOCAH project we will need to look at the issue of hierarchy of multi-level descriptions. In truth, I am not yet familiar enough with Linked Data to really understand how this is going to work, and we have not yet really started to tackle this work. I think I’m still struggling to move away from thinking of the record as the basis of things, because, to coin a rather tiresome phrase, RDF modelling is a paradigm shift.  RDF is all about relationships between concepts and I will be interested to see where this leaves relationships between hierarchical parts of an archive description. But I am heartened by Rob Styles’ (of Talis) assertion that RDF allows anyone to say anything about anything.

Who is the creator?

/ Jane Stevenson / 9 Comments

I am currentphoto of quill pensly working on an exciting new Linked Data project, looking at exposing the Archives Hub metadata in a different way, that could provide great potential for new uses of the data. More on that in future posts. But it has got me thinking about the thorny issue of ‘Name of creator(s)’, as ISAD(G) says. The ‘creator’ of the archive. In RDF modelling (required for Linked Data output) we need to think about how data elements relate to eachother and be explicit about the data elements and the relationships between concepts.

Dublin Core has a widely used ‘createdBy’ element – it would be nice and easy to use that to define the relationship between the person and the archive. The ‘Sir Ernest Shakleton Collection’ createdBy Sir Ernest Shakleton. There is our statement. For RDF we’ll want to identify the names of things with URIs, but leaving that for now, what I’m interested in here is the predicate – the collection was created by Sir Ernest Shakleton, an Arctic explorer whose papers are represented on the Hub.

The only trouble with this is that the collection was not created by him. Well, it was and it wasn’t. The ‘collection’ as a group of things was created by him. That particular group of things would not exist otherwise. But people will usually take ‘created by’ to mean ‘authored by’. It is quite possible that none of the items in the collection were authored by Sir Ernest Shakleton. ISAD(G) refers to the ‘creation, accumulation and maintenance’ and uses ‘creator’ as shorthand for these three different activities. EAD uses ‘origination’ for the ‘individual or organisation responsible for the creation, accumulation or assembly of the described materials’. Maybe that definition is more accurate because it says ‘or assembly’. The idea of an originator appears to get nimbly around the fact that the person or organisation we attribute the archive to is not necessarily the author – they did not necessary create any of the records. But the OED defines the originator as the person who originates something, the creator.

It all seems to hang upon whether the creator can reasonably mean the creator of this archive collection – they are responsible for this collection of materials coming together. The trouble is, even if we go with that, it might work within an archival context – we all agree that this is what we mean – but it doesn’t work so well in a general context. If our Linked Data statement is that the Sir Ernest Shakleton collection ‘was created by’ Sir Ernest Shakleton then this is going to be seen, semantically, as the bog-standard meaning of creator, especially if we use a vocabulary that usually defines creator as author. Dublin Core has dc:creator. Dublin Core does not really have the concept of an archival originator, and I suspect that there are no other vocabularies that have addressed this need.

I would like to end this post with an insightful solution…but none such is coming to me at present. I suppose the most accurate one word description of the role of this person or organisation is ‘accumulator’ or ‘gatherer’. But something doesn’t sound quite right when you start talking about the accumulator. Sounds a bit like a Hollywood movie. Maybe gives it a certain air of mystery, but for representing data in RDF we need clarity and consistency in the use of terms.

Ditchling: A Craft Community

/ Jane Stevenson

Ethel Mairet's Ditchling workshop Photo: Ethel Mairet’s workgirls and apprentices at her ‘Gospels’ workshop, Ditchling, in the 1930s; copyright © the Crafts Study Centre, and courtesy of VADS.

In 1921, the letter-cutter, sculptor, artist and writer Eric Gill founded an arts and crafts colony in Ditchling, East Sussex. Known as The Guild of St Joseph and St Dominic, it was a unique experiment in communal life in the early twentieth century, and survived until 1989.

This month we highlight descriptions for the Ditchling collections held by The Crafts Study Centre, which are especially rich in the work of the calligrapher Edward Johnston (1872-1944) and the weaver and dyer Ethel Mairet (1872-1952).

All for one and one for all: the management of cultural collections

/ Jane Stevenson
Collections Trust Collections Management guide
Collections Management: A Practical Guide, by the Collections Trust

PAS 197 has recently been published as a new standard, sponsored by the Collections Trust and developed by the BSI. At this week’s meeting of the Archives and Records Association’ Data Standards Group we heard more about the involvement of archivists in the development of the standard.  This came about originally following a talk at the DSG by Nick Poole of the Collections Trust; there was a feeling that it was important to ensure the standard met the requirements of archives as well as museums and libraries.  Susan Snell (Archivist at the Library of Freemasonry) and Teresa Doherty (Collections Manager at The Women’s Library) put themselves forward to work on a standard for collections management that would be truly cross-domain, and look to fulfill the MLA ideal of one standard for all three sectors.

Essentially, what we needed was a standard that said what we do with materials in our care.  The Collections Trust approached the British Standards Institute (BSI) to develop a publicly available specification (PAS) that would provide a broad overview of how heritage collections should be managed. Susan and Teresa only became involved after a number of meetings, but from the presentation that they gave, we were in no doubt that they made their voices heard very successfully. Dr Norman James ( TNA) and Christopher Marsend (V&A) was also there representing our profession.

PAS 197 sets out clearly how you manage an archive and how and why archives are different from other collections. It gives a broad overview of how the collections should be managed and lists the key standards. Susan and Teresa explained to us the way that the different contexts within which we work can create barriers. For example, the museum fraternity talk about ‘objects’, which is not appropriate for archives (‘item’ was used instead in PAS 197, as the best compromise). There was a danger that the standard would not lead to cross-sectoral compatibility if these issues were not addressed, but it needed to be a consensus document.

Teresa emphasised that PAS 197 is not aimed at the average archivist or curator or librarian; it is aimed at senior managers, or at helping us to deal more effectively with senior managers, funders and other similar stakeholders. It codifies our best practice for our boards and financial advisers. It sets out what needs to be resourced.

Teresa pointed out a diagram within the standard that summarises the four main areas of work:

i) Developing collections – acquisitions and disposals

ii) Information procedures – catalogues, indeces, survey lists, accessions registers, etc.

iii) Access

iv) Care and conservation

This may seem straightforward to us, but it needs to be spelt out. We should have policies, procedures and documentation for these areas, and resources directed to each area.  Teresa and Susan pointed out that the scale should be suitable for the repository. A very small repository may just need one document with four paragraphs addressing each area of collections management. Also, it is a generic model, so it can apply to cross-domain collections. The four areas map to what is required for accredication for musuems and TNA self-assessment. The area that is not addressed in PAS is governance, which is also part of accreditation. This is addressed by other BSI standards.

Susan took us through a diagram that sets out processing collections and clarifies terminology – pre-accession, accession, appraisal, cataloguing, deaccessioning/disposal. So, for example, accession in archives is the same as acquisition in the museum and library world, and appraisal only happens in the archival world.

Susan and Teresa felt that working with the BSI was very productive. They were very professional and gave a neutral perspective, looking to ensure a balanced approach so that all voices were heard.  They also told us that we should be pleased with ourselves as a profession, as we lead the way in terms of the development of useful standards to help us do our work more effectively – see the appendix of the standard for proof of this!

We were informed that there is a move for archives to get accreditaion by 2012, to take over the TNA self-assessment scheme.  There may be issues around scalability here, but hopefully, if the accredication procedure is guided by PAS 197, it will be achievable for very small collections. Cross-domain accreditation may encourage institutions that are primarily museums or libraries to ensure that their archives are well cared for, catalogued to the appropriate standards and accessible for use.

The Collections Trust have now produced Collections Management: a practical guide (by Susanna Hillhouse, priced at £29.99).

If you are interested in getting a copy of PAS 197, being a BSI standard, it is a little expensive, at about £56. But, it sounds like it may be well worth having. Thumbs up to Susan and Teresa for helping to ensure that this key standard is relevant for archives as well as museums and libraries.

Democracy 2.0 in the US

/ Jane Stevenson

Democracy 2.0: A Case Study in Open Government from across the pond.

I have just listened to a presentation by David Ferriero – 10th Archivist of the US at the National Archives and Records Administration (www.archives.gov). He was talking about democracy, about being open and participatory. He contrasted the very early days of American independence, where there was a high level of secrecy in Government, to the current climate, where those who make decisions are not isolated from the citizens, and citizens’ voices can be heard. He referred to this as ‘Democracy 2.0.’ Barack Obama set out his open government directive right from the off, promoting the principles of more transparecy, participation and collaboration. Ferriero talked about seeking to inform, educate and maybe even entertain citizens.

The backbone of open government must be good record keeping. Records document individual rights and entitlements, record actions of government and who is responsible and accountable. They give us the history of the national experience. Only 2-3 percent of records created in conducting the public’s business are considered to be of permanent value and therefore kept in the US archives (still, obviously, a mind-bogglingly huge amount of stuff).

Ferriero emphasised the need to ensure that Federal records of historical value are in good order. But there are still too many records are at risk of damange or loss. A recent review of record keeping in Federal Agencies showed that 4 out of 5 agencies are at high or moderate risk of improper destruction of records. Cost effective IT solutions are required to address this, and NARA is looking to lead in this area. An electronic records archive (ERA) is being build in partnership with the private sector to hold all the Federal Government’s electronic records, and Ferriero sees this as the priority and the most important challenge for the National Archives. He felt that new kinds of records create new challenges, that is, records created as result of social media, and an ERA needs to be able to take care of these types of records.

Change in processes and change in culture is required to meet the new online landscape. The whole commerce of information has changed permanently and we need to be good stewards of the new dynamic. There needs to be better engagement with employees and with the public. NARA are looking to improve their online capabilities to improve the delivery of records. They are developing their catalogue into a social catalogue that allows users to contribute and using Web 2.0 tools to allow greater communication between staff. They are also going beyond their own website to reach users where they are, using YouTube, Twitter, blogs, etc. They intend to develop comprehensive social media strategy (which will be well worth reading if it does emerge).

The US Government are publishing high value datasets on data.gov and Ferriero said that they are eager to see the response to this, in terms of the innovative use of data. They are searching for ways to step of digitisation – looking at what to prioritise and how to accomplish the most with least cost. They want to provide open government leadership to Federal Agencies, for example, mediating in disputes relating to FoI. There are around 2,000 different security classification guides in the government, which makes record processing very comlex. There is a big backlog of documents waiting to be declassified, some pertaining to World War Two, the Koeran War and the Vietnam War, so they will be of great interest to researchers.

Ferriero also talked about the challenge of making the distiction between business records and personal records. He felt that the personal has to be there, within the archive, to help future researchers recreate the full picture of events.

There is still a problem with Government Agencies all doing their own thing. The Chief Information officers of all agencies have a Council (the CIO Council). The records managers have the Records Management Council. But it is a case of never the twain shall meet at the moment. Even within Agencies the two often have nothing to do with eachother….there are now plans to address this!

This was a presentation that ticked many of the boxes of concern – the importance of addressing electronic records, new media, bringing people together to create efficiencies and engaging the citizens. But then, of course,  it’s easy to do that in words….

Archives Hub contributors’ survey

/ Jane Stevenson

We would be very grateful if contributors to the Archives Hub could fill in this short survey for us. It is invaluable in helping us to understand your needs and priorities, and to plan for future development and enhancements.

Contributors’ survey: http://www.archiveshub.ac.uk/blog/?page_id=2317

This survey is specifically for contributors of archival descriptions rather than researchers. However, we are always keen to hear any views that you have on the strengths and weaknesses of the Hub, so do please email us with any feedback you have.

Many thanks,
from the Archives Hub team.

Opening up UK archives data (II)

/ Jane Stevenson / 1 Comment

This is the second post relating to the recent UKAD meeting, concentrating on the brainstorming that took place around digital and digitised archives.

The driving forces that were identified:

  • Crowd-sourcing – metadata generation
  • Attracts funding
  • Promotes access
  • Open up wealth of possibility
  • Remain relevant
  • Meet user expectations
  • Centres of excellence in digitisation – common approach
  • Collections already digitised are hidden – in silos – return on investment
  • Potential to capture richer information about users
  • Potential to draw people in
  • Increasing ‘digitisation on demand’ – needs to be harnessed effectively
  • Increasing amount of born-digital media need to be made accessible online – drive to discoverability of digital materials
  • Changing profession – becoming more confident in this area as a result of above
  • Web makes it much easier

The group felt that it all added up to a resouding “we have to do this!”.

The resistors included:

  • Systems don’t talk to each other
  • Insufficient metadata of legacy digitised material – retroconversion – cost*
  • Copyright/IPR – complex, lots of local specificity
  • Work needed to marry user generated content and standard metadata
  • Community resistance to UGC
  • Vast amounts of content – prioritisation is intellectually challenging
  • Bulk digitisation is happening commercially – restricted rights
  • Clashes with business models – or perception that it does (e.g. models based on commercial digitisation assume increasing return on investment; the opposite may occur if the most commercially enticing material digitised first)
  • Fears – grounded in truth – could affect funding: diminish user/visitor numbers on site, diminishes value of on-site expertise
  • Challenges in bringing catalogue data and digital object systems together
  • Query: not ultimately cost effective
  • Cost
  • Web makes it easier – but it’s hard to keep up…

The group looked at actions that are required:

1. Accrue evidence of user demand and current behaviour

  • Identify user communities (family, academic, student researchers)
  • Secondary research of existing analysis
  • Market research
  • Produce cost-benefit analysis – impact on site visits?

2. Systems talking to each other

  • People talking to each other about systems!
  • Develop definitive list of systems in use – a picture of UK situation > crosswalks/maps between (see Library world)
  • Needs to cover both catalogue and digital object management systems
  • Discmap?

3. Copyright/IPR

  • Produce decision tree to help archivists make decisions – risk assessment but beware risk aversion
  • Encourage sharing of experience/lessons learned
  • Gathering what has already been done

4. Impact of digitised resources

  • Gather existing articles/research
  • Share practice in assessing impact in differing contexts

5. Metadata and costs

  • Establish costs of differing levels of metadata generation
  • Identify how much data needs to be converted into digital metadata (how much is not online?)

6. Identify quick wins!

  • Working together to create user cases and examples, sharing experience, getting onvolved in Resource Discovery Task Force and linking projects to this

Of course, the gathering of such evidence can help us to see where we are and where we need to go, and also how to get there. But implementation is quite another thing. The UKAD Network is hoping to build upon this work to encourage collaborative initiatives and the sharing of expertise and experiences. We are considering events and training opportunities that might help. We do feel that it will be useful to create a stronger presence for UKAD, as a means to provide a focus for this work, and we are looking at low-cost options to do this.