Emulation makes WORM media a reasonable digital... - digitalcontinuity
I’m posting this on my personal blog as it’s not ready for anywhere more formal. There are few citations, and it’s not fully formed, but hopefully of interest. 

 

 In a paper (P225-labeled/P226-actual) I recently presented with Daniel Chadash from Twist BioScience at iPRES 2022 Dan and I discussed how Write Once Read Many (WORM) media is becoming a more viable and attractive option for ensuring the long term preservation of digital content. In the paper we go into detail about why WORM media is now more attractive and discuss how DNA data storage is becoming a particularly practical WORM storage option (and soon part of a write-many solution also). As part of the explanation we briefly discuss the role of emulation in making WORM media more viable. It wasn’t the focus of the paper (which was primarily about DNA data storage) and therefore didn’t get a great deal of exposition. So for this year’s World Digital Preservation Day (#WDPD2022) I thought I’d spend a little time expanding on it.  

 Most organizations doing digital preservation, and most digital preservation software systems assume and are designed around regular content migration (i.e. some information from interactive software-dependent experiences stored in software and “data” files is going to be selected and reimplemented in new, more modern, software and data files - a process often called “format migration”). Content migration based approaches don’t seem to be compatible with WORM media as they seem to require stored data to be regularly changed or deleted. For example: 

 
 There seems to be an expectation of regular file deletion: If content migration is ever going to happen (it’s hard to find many large scale examples of it “in the wild” so I’m going to be particularly cagey with my language) the process should result in new files being added to the organization’s storage. And the old files that make up the original content are usually proposed to be deleted to save resources. Alternative, (at least in theory) sometimes the original and most recent sets of files after multiple content migrations are kept and interim file sets deleted.  So, content migration as an approach likely requires files to regularly be deleted from storage. 

 There may be a requirement for changing the stored file structure: There is sometimes an expectation in the digital preservation community that the files that make up preserved digital objects, along with their metadata, should be stored in a file structure that maintains them together as a findable intellectual unit. Whether or not any of us agree with that (in my opinion there are good arguments both ways in different contexts), if we assume this is a requirement, a content migration approach likely requires regularly adding new files to those file structures in order to ensure each digital object remains together on the storage media.  In other words, content migration requires changing the file structure of the stored content.  


 This expectation of having to delete or change stored objects if following a content migration based digital preservation approach has led to an assumption that WORM media is not a great solution for long term digital preservation as it doesn’t allow for either action. 

 Emulation changes this. A digital preservation approach that instead assumes that we will preserve and use emulation to maintain access to the original software files that make up the digital experiences we’re preserving, doesn’t have this same problem. With an emulation based approach, outside of unavoidable changes resulting from the right to be forgotten, or take-down requests, files never need to be changed once stored on WORM media. Provided the original interaction software files are stored with the “data” files, and emulators stored along with them, then the content should still be accessible at any point in the future. 

 A concern could be raised that at some point, our current emulators will not work on modern computers. We have at least two options for managing this. 

 
 We can rewrite the emulators 

 We can create new emulators that can run the operating systems required to run the emulators we already have, then nest the old emulators inside the new ones 


 Both of these have pros and cons and both require writing new software which could be considered expensive. In our iPRES 2022 paper on the Emulation as a Service Infrastructure (EaaSI) Program of work we briefly discuss the economics involved which also seem to come out in favour of emulation: “Re-writing one emulator could ensure access to many emulated computers, which could ensure access to many legacy software applications which could ensure access to virtually unlimited digital objects.” 

 e.g. 

 But most importantly, once either solution has been selected, the new emulators can be added to the same type of storage and don’t need to be stored in the same folder structure. Given the economy of scale of emulation (indicated in the quote and diagram above), even if we re-write the emulators occasionally and store those along with the old “obsolete” ones, that should only add a relatively insignificant amount of additional storage usage to our requirements. In other words, an emulation based approach can treat storage as something we only ever add to. By only being additive, and not destructive or requiring changes, an emulation based approach makes WORM media much more feasible. 

 To finish, we do seem to be living in interesting times. If there were a major event after which society one day rebuilt itself such that it could recover data from long life WORM media, having usable software available to decode all the other digital files out there seems like an additional valuable benefit of this approach. In alignment with this, the team I manage recently deposited a collection of both software installation media, and usable emulated environments with software installed on them (along with the EaaSI source code and binaries), to the Arctic World Archive stored on Piql film (a WORM media). And as the paper that motivated this post (P225) describes, we’re also testing DNA storage as a long-life WORM storage option. With large scale use of emulation now practical through endeavours like our EaaSI program of work I hope that we’re helping to ensure some digital artifacts persist regardless of how “interesting” our times become.