FAQ on web archiving
The National Library’s statutory mandate is to collect and archive information about Switzerland and make that knowledge accessible over the long term. Websites are part of that: today, a lot of material is no longer printed but is instead published on the Internet.
Web archiving is carried out by the National Library in cooperation with other memory institutions in Switzerland. Partners include the cantonal libraries and other specialised archives and libraries. The full list of partners can be found in the background document (in German) on Web Archive Switzerland.
The National Library does not carry out domain crawls, and consequently does not automatically save every website with the .ch suffix. There is no legal basis for doing so: Switzerland does not have a national copyright deposit act for digital online media. The National Library therefore adopts a selective approach.
The Internet Archive is a non-profit organisation in San Francisco which has copied selected websites from around the world since 1996 and makes them available free of charge via the Wayback Machine. Web Archive Switzerland focuses on documenting web content with a strong connection to Switzerland, irrespective of whether the website concerned has a .ch suffix.
The challenge of a selective approach lies in making a sensible and representative choice from the large amount of material on offer. The websites are selected by Web Archive Switzerland’s partners, i.e. the Swiss cantonal libraries, other special libraries and archives, and the National Library itself. Each institution contributes its specialist know-how to the collection.
The websites must be freely available on the Internet and have a strong connection to Switzerland. The selection criteria are described in detail in the “Collecting” information sheet (in German).
You are welcome to nominate your website by sending an e-mail to the Coordination Office for Web Archive Switzerland. Your suggestion will be examined and a decision on whether to include the website will be taken on the basis of the collection guidelines.
The National Library uses a crawler – the open-source software Heritrix – to copy the websites. The open-source software PhantomJS also assists the crawler in identifying all the relevant links. The Heritrix crawler follows the links within a website, collects all the files it finds, and copies them onto the National Library’s server. The aim is to archive a version of the website that is as comprehensive as possible and is displayed correctly. Pages that are password-protected and links to other sites are not collected.
Normally, a website is harvested every two years. Other options include harvesting it only once or every four years. The interval is very much dependent on the content concerned and can be set individually for each website.
The National Library sends an e-mail or letter to all website owners whose sites have been selected for Web Archive Switzerland. It includes information on the purpose, the copying process and accessing the archived websites. You do not have to reply, unless you do not wish to allow your website to be archived.
There is no charge for archiving.
No, website owners are still responsible for hosting their websites. Regular copying only provides snapshots of a website that are archived individually and then used for research purposes.
You do not need to prepare anything or update your website before copying. A website that has been chosen is harvested regularly (every 2 years). This allows the changes that the website has undergone over the years and decades to be documented.
The crawler operated by the National Library is configured to keep the server load to a minimum. If technical problems do occur due to copying, please contact the Coordination Office for Web Archive Switzerland. If desired, the crawl speed can be reduced.
Protected areas, such as intranets and data with access protection, are not copied by the crawler. Web Archive Switzerland only contains content that is published and made accessible freely on the Internet.
If robots.txt and robots meta tags were taken into account when copying, there would be a risk that the website obtained would not be complete and that its layout would not be reproduced correctly. To avoid this, robots.txt and robots meta tags are ignored.
It is possible that a website cannot be archived for technical reasons, even though it meets the Web Archive Switzerland collection criteria.
Large quantities of data, missing content or menu functions, Flash animations, dynamic script-based functions and crawler traps such as calendars or maps can make archiving more difficult. Some websites are archived despite quality issues, so that they are at least documented. If a website is incorrectly displayed, this may also be due to shortcomings in visualisation tools, e.g. of the Wayback Machine.
One key feature of a crawler-friendly website is links in HTML or XHTML format that are not embedded in Flash or JavaScript. Alternative navigation options via a text-based version or sitemap are also helpful. If you are interested in receiving further information, please contact the Coordination Office for Web Archive Switzerland.
If you are a website owner and you have been notified by the National Library that your website is going to be harvested and archived, you can contact the Coordination Office for Web Archive Switzerland and advise it of your concerns.
Archived websites can be full-text searched and displayed in e-Helvetica, the access portal for the digital collections. For copyright reasons, the Web Archive can only be accessed on the premises of the National Library and those of its partner institutions. Reproduction in any form – saving to media, printing out, etc. – is not permitted.
Web Archive Switzerland is for historical purposes only, and no commercial use is made of the archived websites. Web Archive Switzerland is available to users on the premises of the National Library and those of partner institutions free of charge.
The Coordination Office for Web Archive Switzerland will be happy to provide further information.