Digital Library Division, Kansai-kan of the National Diet Library

This is based on an article of the same title in the series
<“1998-2008” Topics during the last decade and future prospects>
in NDL Monthly Bulletin No. 569/570 (August/September 2008).

“Digital Library from the Meiji Era” used in Leiden University (the Netherlands)
Digital library enhances availability overseas

In 1998, 10 years ago, the Internet population in Japan passed the 10-million mark. In the same year, W3C (World Wide Web Consortium), an organization which sets the Internet standards, made a recommendation of XML1.0, and Google was established in the United States. In the previous year, 1997, Aozora Bunko in Japan, a volunteer-run digital library service on the Internet, launched its website, and Elsevier, one of major academic publishing companies, started Science Direct, an electronic journal service. Meanwhile, 2channel, the biggest Internet forum in Japan, was launched in the next year, 1999.

Big changes were also happening in government policies. Going back a little further, development of information infrastructure and utilization of information and communication technology in academic research and government administration became policy issues in each country, such as the “National Information Infrastructure” (1993) of the United States. In these “information policies,” informatization of libraries was made an important issue. The situation was the same in Japan. The National Diet Library (NDL) launched a “Pilot Electronic Library Project” with the Ministry of International Trade and Industry (formerly the Ministry of Economy, Trade and Industry) in 1994, and conducted experimental operation of digital library systems including the digitization of rare books and books printed in the Meiji era. In 1998, the NDL formulated the “National Diet Library Electronic Library Concept” (“Concept”) and showed the vision of the digital library for which the NDL was heading¹. Here we look back the NDL’s efforts for the digital library in the last decade beginning with this “Concept.”

up

The stacks of the Tokyo Main Library: digital library services are based on the NDL collections

The “Concept” mentions digitization of existing materials in paper form as a way of “collection” development of a digital library. Especially, it emphasizes the digitization of information about legislation and other activities of the National Diet. In this respect, significant progress was made, for example, the launch of the full-text database of the Minutes of the Diet (cf. “Services for the National Diet - providing legislative support” National Diet Library Newsletter No. 161, June 2008).

In addition, since the International Library of Children's Literature (ILCL) opened, services such as the “Digital Library of Children's Literature” and the “Picture Book Gallery” have been developed. In particular, the latter is a realization of the “digital museum function” of the ILCL mentioned in the “Concept.”

up

Among digital library services, we should remember the “Rare Books Image Database” and “Digital Library from the Meiji Era” (both in Japanese) as services which provide traditional library materials in digital form via the Internet. The “Rare Books Image Database” was released in March 2000 to provide color images of rare books held by the NDL. Starting with about 23,000 images from 459 titles of materials, it has enhanced its contents to about 49,000 images from 949 titles as of July 2008.

The “Digital Library from the Meiji Era” is a service which provides the NDL collection of books printed in the Meiji and Taisho eras in digital form. To provide materials as exhaustively as possible, the NDL has been conducting large-scale copyright clearance without parallel in Japan. We are still involved in the work of identifying creators of each illustration, preface, etc. and confirming the status of copyright of all of them. The “Digital Library from the Meiji Era” started with roughly 33,000 volumes in 20,000 titles in 2002, timed with the opening of the Kansai-kan (cf. “Kansai-kan of the National Diet Library opened on October 7” National Diet Library Newsletter No. 127, October 2002). As of August 2008, it has grown to what deserves to be called a library “collection,” 148,000 volumes in 101,000 titles, as a result of the steady work of copyright clearance.

Increase in the number of volumes provided in the “Digital Library from the Meiji Era”

In addition, modeled on the “American Memory” project in which the Library of Congress digitizes a wide range of American historical and cultural materials, it is stated in the “Concept” that “the NDL will plan a similar project to electronically edit, compile, and supply Japanese materials in these fields.” It was realized by the “Online Gallery,” which consists of exhibitions providing via the Internet digital images of the NDL collections by theme in combination with comments and columns relating to the collections. Starting with the “Rare Books of the National Diet Library” released in 1998, we currently provide 13 themes and aim for further expansion.

up

The “Concept” indicates the importance of collecting and preserving digital information including Internet information.

It took form with the inclusion of offline electronic publications in the legal deposit system in 2000, and efforts were made toward the collection and preservation of online electronic publications including Internet information (cf. “Tracking the acquisition of materials - during the last decade -” NDL Newsletter, No. 162, August 2008).

With the Web ARchiving Project (WARP) (in Japanese) that started as an experimental project in 2002 (upgraded to a full-scale project in 2006), the NDL collects websites of national government agencies, etc. and free online journals and information on websites by agreements. The NDL collected about 600 websites (total 32 gigabytes) in the beginning of the project (as of March 2003), and at present, the collection has increased to nearly 4,000 websites (10 terabytes² ) an increase of 300 times by volume.

The WARP contains the former NDL websites, which have a totally different design from the present version (the website in 2002 above, and in 2004 below). However, to see a much earlier version, it is necessary to rely on the Wayback Machine³ provided by the Internet Archive (IA), an NPO in the U.S. Although websites are regarded as “faces” of organizations nowadays, we had not recorded our former “faces” ourselves until the WARP started. To avoid such situations occurring in the future, the NDL is making efforts to make its collection on a legal basis, rather than with license agreements, starting with websites of national government agencies and institutions.

On the other hand, it is technically difficult to collect and preserve databases on the Internet. The NDL launched the Database Navigation Service (Dnavi) (in Japanese) in parallel with the WARP to introduce such databases, and to guide users to database retrieval screens. The number of databases accessible through Dnavi was approximately 6,000 at the beginning of the service (as of the end of March 2003), and currently, it increased to about 12,000.

Collection and preservation of Internet information has become an international issue because of its cross-border nature. To tackle this issue, the NDL became a member of the International Internet Preservation Consortium (IIPC)⁴ in 2008. The IIPC is an organization established by national libraries in the western countries and the IA in 2003, and aims to work together to tackle various challenges in the collection and preservation of Internet information. The NDL will continue its efforts to hand down Internet information, the new cultural property, to future generations in cooperation with the IIPC and other countries which share the same issues.

up

The NDL’s future plan concerning the digital library has not yet been completed in the “Concept.”

In February 2004, the NDL formulated the National Diet Library Digital Library Medium Term Plan for 2004 (Plan 2004) intended to “set forth in detail the direction in which electronic library services will be oriented at the NDL over the next five years” and “present the frameworks required for realizing such services.”

In 2004, the Internet population in Japan exceeded 60 million, and websites searchable with Google exceeded 6 billion. In the same year, the Japanese language version of Wikipedia, which was launched in 2001, exceeded 1,000 pages. Since the major company Amazon.com had launched business in Japan in 2000, online bookstores became common at that time, while paid music distribution services such as the Chaku-Uta Full(R) service started to spread. In 1998, diffusion of the Internet had only begun to take off, but in 2004, it had already become part of the indispensable social infrastructure for many people. In response to the change, national policies had been considerably developed. The Basic Law on the Formation of an Advanced Information and Telecommunications Network Society came into effect in 2001, and the e-Japan Strategy II, which was formulated by the IT Strategic Headquarters of the government, mentioned the creation, distribution, digitization and archiving of contents.

Construction of digital archive is a main pillar of Plan 2004. “Digital archive” is a broadly used word to mean the accumulation of digitized information of cultural properties in some form.

In Plan 2004, we proclaimed “the NDL serves as a vital base for digital archiving in Japan.” The term “digital archiving” here includes digitization of materials and collection of Internet information, which were mentioned above. Besides, Plan 2004 newly set some services and projects, which were not defined in the “Concept.”

The following are the projects newly set in the Plan 2004.

up

In Plan 2004, we divide the collection and preservation of Internet information into two parts. One is collecting and preserving the whole of a website, called web archiving, of which WARP bears the function. The other is digital deposit (described as “online deposit” in Plan 2004), which maintains diversified online information in smaller units, that is, on work basis, with assignment of metadata such as name of title and author.

Regrettably, this digital deposit has not yet been taken form as a project. We will promote the development of the necessary systems aiming at opening the digital deposit by the beginning of 2010.

up

In Plan 2004, it is stated that “in order to meet the demands of diverse users both inside and outside Japan, a comprehensive portal site is being constructed by the NDL, which will navigate them to digital information throughout Japan.” However, it is not an easy task to guide “the entire digital information.” One of the measures is to collect, edit, organize and provide information useful for information retrieving. It includes “Subject Information Services” such as Guide for Search by Theme (Japanese only) and “Knowledge Services (developed version of the former)”⁵ , and we are putting efforts for further enhancement of this.

Another measure is the construction of a digital archive portal which enables us to make integrative and crossover search of multiple digital archives provided by various institutions and organizations. As a result of a prototype system, we started the “National Diet Library Digital Archive Portal (PORTA)” in October 2007.

As of November 2008, digital archives of only 13 institutions and organizations excluding the NDL can be searched integrally through PORTA. We take pride in presenting a frame of cooperation with a wide range of institutions providing digitized resources, such as the National Archives of Japan, the Japan P.E.N. Club and Aozora Bunko, rather than confining PORTA to within libraries. PORTA is extending gradually, aiming at becoming in deed as well as in name a navigator to “all of the digital information in Japan” in future.

up

A great amount of creations and fruits of intellectual activities is being digitally born on a daily basis around the world. On the other hand, contents in analog media such as printed materials are being digitized as digital archive.

The fact that digital information is not damaged by copying tends to lead to a misunderstanding that long-term preservation of such information is possible. In practice, however, it is not easy to preserve and keep it available for the long period because digital media including CDs and DVDs, and the equipment and software needed to play them are all short-lived. While it is not difficult to preserve a book printed on high-quality paper for 100 years, it is an extremely hard task to hand down digital information for the same period.

In Europe and the US, measures to tackle this issue have been promoted since no later than the early years of the 1990s, and currently a lot of national projects are being progressed. In Japan, by contrast, people are only just starting to recognize the issue. In Plan 2004, we introduce the Charter on the Preservation of the Digital Heritage⁶ adopted at UNESCO's 32nd General Conference (2003) and raise a question.

The NDL started research on this issue in 2002 and at present a 5-year research plan from 2006 including a study on floppy discs from the NDL collections is under way. This research has been phased and a guideline for the long-term preservation of digital information will be drawn up on its completion⁷ .

up

These have been our efforts at providing digital library services for the past decade. Based on these efforts, the NDL is now involved in the development of the NDL Digital Archiving System which enables the integration of the system infrastructure of the Digital Library from the Meiji Era, WARP and others. This new system is to be launched in the beginning of 2010.

In future, we will carry out the following actions:

1. To discuss with copyright holders and publishers about conditions of digitization in order to digitize a broader range of materials;
2. To push forward the formulation of legislation required to exhaustively collect and preserve websites of mainly national institutions;
3. By transcending the framework of libraries, to have further cooperative relationships with institutions which create and provide “digital archives” such as archives and museums, and also support the construction of “digital archives” and the coordination among the various institution in each region in Japan;
4. To facilitate the use of standard search interface (API⁸ ) in order to make PORTA a portal of “digital archives” in Japan, and make metadata and API public in order to promote the use of the NDL’s various services;
5. To develop policies (guidelines, systems, etc.) for the long-term preservation of digital information, including Internet information, through international cooperation.

Support and cooperation by many people will be vital for these efforts to bear fruit. We will be continuously and actively engaged in the digital library for the next decade.

up

¹ Taya, Hiroyuki. “The National Diet Library Electronic Library Concept" NDL Monthly Bulletin No. 450 (September 1998) 3-10

² 1 terabyte=1,000 gigabytes

³ Wayback Machine. Internet Archive. http://www.archive.org/web/web.php (last access 2008.12.22)

⁴ For information about the IIPC, see http://current.ndl.go.jp/ca1664 (Japanese only).

⁵ See series of the NDL Monthly Bulletin (Japanese only) from No.553 (Apr. 2007) to No.559 (Oct. 2007), “Give shape to knowledge – NDL headed to “Subject Information Services”” (Japanese only)

⁶ Charter on the Preservation of the Digital Heritage. UNESCO. ORG. (last access 2008.12.22)

⁷ Ensuring long-term preservation and usability of digital information, NDL website

⁸ Application Programming Interface, a system making services interoperable between different systems

up