In this article, which summarizes some of the key findings of Jean-Christophe Kummer (NOA CEO) and Silvester Stöger (NOA Project- & Account Manager) in their academic deep-dive into best practice archive structure, setup and operation - we examine some of the key issues which faced SBA, and the technologies and expertise NOA implemented to overcome them. The original academic paper - published in the Henry Stewart Journal of Digital Media Management - is linked at the end of the article.
It was Winston Churchill who said ‘the longer you can look back, the further you can look forward’. The world of broadcast occupies a very special role in the sense that day-by-day, it catches small artefacts of life; our culture, society, values and practices. It doesn’t aim to do this (or at least, not always), but it is inherent in the nature of broadcast, TV and entertainment that it does. TV represents a constant window on our lives.
Sharjah Broadcasting Authority (SBA) recognized the cultural importance of their broadcast history, and were keen to preserve it. As the main broadcaster for Sharjah - one of the seven emirates of the UAE, SBA have always played a central role in securing and promoting the values of education, arts and culture which sit at the heart of Sharjah. They wanted not just to preserve their media history passively - as a ‘museum’ of content, but also in a way that would allow them to dynamically and actively make use of the footage on an on-going basis in future productions. Furthermore, the database would need to be ‘living’, with newly produced assets added to it every day.
SBA had a hard decision ahead of them: should they outsource the digitization process, or develop the capabilities to digitize in-house? Whilst it would be daunting to develop the in-house technical and project management knowledge needed to match the abilities of contracted specialists, there was recognition that if this could be achieved, then creating an in-house digitisation factory would stabilize costs and deliver a much lower OPEX (OPerational EXpense), reduce the risk of overrun costs from miscalculated project scope, bypass a number of logistical issues arising from strict IP laws, and represent an achievement that would significantly enhance the prestige and reputation of the broadcaster.
However, to pull off this ambitious undertaking, it would be vital that the company SBA contracted to supply and install the relevant systems was far more than just a technology provider. Instead, SBA needed a solutions provider who would actively consult in the creation of a custom system that took account of SBA’s specific needs rather than implementing a one-size-fits all option, who would create systems where knowledge and best practices were actively embedded in the technologies and user-interfaces, and most importantly, they needed a company who would provide guidance and support on an ongoing basis; from pre-consultation to post-installation.
It is here that NOA entered the scene.
Establishing SBA’s needs
SBA’s central aim was simple in its objectives, but more complex in its execution: SBA wanted to digitize all existing assets and archive these within an intuitive, easy-to-use system that made use of comprehensive metadata in order to make the archive easily searchable. Further, they wanted to be able to add newly produced assets to the system on an ongoing basis, make use of their assets in daily productions, and finally, wanted to ensure that a second physical backup was integrated at another broadcast site in order to secure disaster prevention.
NOA delivered a system which was custom in its setup, but which made use of a number of Commercial-Off-the-Shelf (COTS) elements to drive efficiency, reduce cost, and embed automation and usability features which would allow the new in-house digitisation team to get up to speed with the process quickly. There were, in essence, three separate elements to the solution; the digitisation process, the archival database construction, and the provision of overall disaster-proofing.
The key aim for the capture workflow as a whole was to make the process as comprehensive and detail-oriented as possible, without being overly complex. Automating, streamlining and employing auto-error detection elements within the individual processes was key to allowing a workforce with no specialist knowledge to quickly train on the workflow and process artefacts in bulk, with high levels of accuracy and efficiency Indeed, most processes took little more than the press of a button, and were all embodied in a highly usable, intuitive user-interface.
When it came to archive and database construction, NOA’s main mission was to create a system that was intuitive and usable to a workforce who - at least initially - would not be intimately familiar with how an effective archive is structured. Designing an effective metadata structure lay at the heart of this; a scheme which both incorporated concepts of best practice archiving and data handling, but was also highly contextual and specific to the nature of the data and the needs of SBA. This called for a highly collaborative effort, combining NOA’s knowledge of archive structures with careful consultation of SBA to establish completely their expectations and desires for the system’s use.
Establishing how the system as a whole would be accessed and used - and what it would be used for was also key. SBA had expressed the idea that they needed a carefully preserved record of the culture embedded in their broadcasting legacy, but they also needed something more active and dynamic - a database that could be searched and used to pull forward assets and use them in ongoing productions.
This meant that the system needed to be implemented both as an Archive Asset Manager (AAM), but also a Production Asset Manager (PAM). NOA delivered on this; they combined an AAM (in the form of the NOA mediARC), PAM (through the AVID Interplay system) and Hierarchical Storage Management (through Oracle’s DIVA software). The core features of the AAM were absolute stability, integrity and longevity of the assets (preserving the assets ‘for eternity plus a day’). In relation to production though, the PAM allowed for the assets to be stored in a format and manner which made them easily searchable, retrievable and usable - with speed, efficiency and flexibility being key considerations.
These two systems - the AAM and the PAM - needed to exist independently, but interact seamlessly, so their integration was undertaken on a dedicated layer using the Marquis Medway. - an approach which carried a multitude of technical advantages. But the assets also needed to be divided (or more accurately, duplicated) physically across two geographic locations to provide for security of the assets against physical disaster. As a result, a fully redundant database cluster, an archive database backup, and an archive media storage backup were all created at both the main and secondary site - preserving the enduring safety of the assets against disaster or single-point-of-failure.
The outcomes
42,000 physical archive carriers (approx. 50,000 hours of footage) have been ingested by SBA over the last three years, and every day the organization continues to add productions to the system, whilst also accessing historical elements to integrate into their new creations. SBA has moved from an organization with no archiving experience to now maintaining a fully-trained team well versed in the intricacies of the process, and have achieved all of their organizational goals, maintaining a secure yet highly flexible database of media assets that they are able to maximize the value of on a continuing basis.
At the heart of this has been NOA’s approach to the creation of end-to-end digitisation and archiving systems; ones which put the client front-and-center by being responsive to the specific needs and context of the system, and implementing systems which make use of COTS solutions, but which are applied in a customized workflow which is adapted specifically to the needs and abilities of the organization.
Further, NOA were able to deliver such successful outcomes for SBA by virtue of the three key ‘philosophies’ which underpin their design and implementation process; prioritizing usability and building best-practice knowledge directly into the way the systems are designed and operated, maximizing automation and error detection wherever possible, and developing close customer relationships which offer ongoing support before, during and beyond implementation.
