Electronic imaging is currently the low-cost way of storing business documents. But this has not always been the case. For many years microfilm was the storage media of choice, so many companies have a large amount of microfilm (roll and fiche) that is part of their records storage. Consideration of the use or conversion of microfilm requires familiarity with the film technology. If you need a review of the types of microfilm and it's use for business documents, see the film tutorial available on this web site at www.plesums.com/image/filmintro.html
Every company with older documents on microfilm and newer documents on a digital document image system faces some choices:
Many companies did not follow good practices or standards when making the original film years ago, so quality is often bad. Users often hate using film, because retrieval is time consuming and the poor quality makes the documents difficult to read. Searching for a particular document on microfiche may require scanning 50 or more pages. Retrieval of digital images has always been faster and more economical, so (as the storage costs for electronic images have become cheaper) many companies are moving to the digital image technology.
Conversion costs in this paper are based on multiple studies in the 2003-2004 timeframe, with vendors throughout the country. There have not been dramatic changes in technology that would lead to drastically different prices today. These are not exceptionally high or low prices, but estimates of the competitive prices available from national vendors. Specific local vendor prices were often much higher.
Most companies find that it costs several dollars to find and mount a roll of microfilm, locate a document on that roll, rewind and re-file the roll, but only a few cents to capture each image desired from that roll. Further, since roll film almost always has an index (or we wouldn't be able to find a particular document), the primary cost of converting roll film is mounting and scanning. Therefore it is often cheaper to convert all the images on a roll of film, while it is mounted, rather than mounting the roll hundreds of times to convert one document at a time.
Further, since all the images on the roll were taken and processed together, the images are relatively consistent throughout the roll (whether good or bad). Since any adjustments would apply to thousands of images, during a conversion it pays to optimize the scanning for each roll.
If we are going to do a bulk conversion of all the images on a roll, then we could also use a service company to do all the rolls, since these companies have high-speed machines and staff specialized in optimizing the images quality and speed. There are exceptions, of course.
The next couple steps involve the actual data on the images. Some service companies will perform these steps (for an additional fee), while others just return all the images, and leave these steps to the company that owns the data. Since they are critical steps as well as a good way of providing quality control of the scanning operation, there is a compelling argument for keeping these steps in-house even though it is part of the conversion.
Until recently the programs to match the index to the scanned documents were not widely available - some service bureaus provided the function, but most organizations had to build custom systems. Iimage Retrieval Inc. in Dallas has published their "Iiridium" program that will take most index files, match against a set of scanned images, and "ripple" the index along as pages are added and deleted to synchronize the set of images with the index.
It is unlikely that thousands of rolls of film will be sent to a service bureau in a single shipment. Therefore a logging system needs to be established, to track each roll of film, and to be sure all the output makes it through the process once and only once.
Some service bureaus have all new equipment, and are more expensive. Others have a variety of new and old equipment. Sample runs are performed on the newest equipment, but high volume work (or rush work) may be performed on the older equipment with less experienced operators. Therefore it is important to monitor the quality of the work delivered - not just assume that all work will be as good as the sample.
A large service bureau may be able to scan 200 rolls of film per day or more, representing 500,000 image pages per day (40-50 CDs per day). Realistically few operations can perform a quality check on the service bureau at that rate, nor can most image system store images that quickly without special facilities or additional capacity. How fast can this type of conversion proceed? There is no absolute answer, but for planning, assume 2 CDs or about 25 rolls of film per day can be handled routinely without operational impact. One modern scanner (if the work is done locally) can theoretically handle 3-4 rolls per hour, but in practice only count on about 2 rolls per hour, so one on-site roll film scanner may be able to handle as much film as the people and image system can handle.
If there is a very small volume of roll film, or if only a few documents are required from a roll, or if a roll has been damaged (without backup) and requires special handling, it may pay to do the conversion manually.
Many of today's reader printers use digital technology internally, at least in the printing process. Many reader-printers have a low cost feature to allow that digital image to be saved to an attached PC (even multiple times as the image is viewed and the quality is adjusted). A few reader-printers have a standard digital interface, and still others fit into a complex (expensive) system for collecting digital image output.
A document is located on the reader printer, just as it would be for viewing and/or printing. In a process similar to printing, each page is captured as a digital image. The image can be cropped and the quality checked on the computer screen, just as it would have been checked on the reader printer. As each page is complete, the unit manually or automatically moves to the next frame. The rated speed to print pages may be as high as 10 to 20 pages per minute, but in actual use (since each page is manually handled) don't expect to average more than about 3 pages per minute.
As the several pages of a document are completed, that document must be added to the digital image system. This normally requires a custom program, to get the image into the format required by the image system, to store the image, and to index the document.
As individual documents are converted, it is wise to update the CAR system to note that the documents are done, so that multiple copies will not be converted and added to the image system.
A reader printer process, similar to this, may be a component of the quality checking of a bulk conversion, to determine what is actually on the film and to immediately scan missing pages or to rescan unreadable pages and add them to the document.
Most microfiche consolidate all the documents for a customer or contract in a single "jacket". Large files may consist of several jackets kept together, just as a large paper file might be in several adjacent file folders. Indexing is based on the right documents being placed in the jacket, and finding that jacket.
When a document on microfiche is requested in most companies today, the fiche is pulled from the film cabinets, and is copied or delivered to the users. A copy of the entire fiche can be made in seconds at a cost well under a dollar (probably less than 25 cents). Each user works with the copy using a microfiche viewer at their workstation. There is probably also a film printer in the area, in case a hard copy is required.
Microfiche are indexed for each contract/policy/customer, but do not have any identification of individual documents (other than, perhaps, the first document is always the original application, or other conventions). If the entire "file" must be reviewed, the fiche is easy, but if a particular document is required, the users may have to look through all 50 or so pages to find the one needed.
When doing a conversion, it is tempting to treat the entire fiche as a single 50-page document, but that approach will put a heavy load on the image system and network, as well as the user, whenever the documents are used in the future. It can become a productivity disaster for years to come. Therefore special effort is required to identify individual documents. (See the white paper on conversion of paper files at www.plesums.com/image/PaperConv.html for a discussion of indexing documents in a conversion. That article concludes that each document, or small groups of documents, should be stored separately, even it if is not completely indexed initially.)
If any part of the fiche is required, the entire fiche should be converted. It would be very hard to identify which documents had been converted and which remain only on a fiche, since there is no formal index of the individual documents in the fiche.
When converting microfiche, there is a significant labor cost to identify the documents, and perhaps to adjust the quality of the scan (since each document in the fiche was captured at a different time). Typical service bureau costs for converting jacket microfiche are 7-10 cents or more per frame for scanning, plus another 3-5 cents per frame if manual cropping is required. Thus microfiche is far more expensive to convert, both in terms of scanning and indexing cost, than roll film. Therefore it is generally better to only convert the fiche as needed, rather than trying to convert all of the fiche. (There is no economy of scale with fiche, as there was by doing an entire roll of microfilm at a time.)
In an on-demand conversion, the turn-around time required is normally short (the user is waiting), so it is not as convenient to use an off-site service company. The best solution generally involves on-site microfiche scanner(s), and custom programs to help identify and index the individual documents. Assuming that there are thousands of fiche to be converted, the scanner should be a production grade unit (50-200 pages per minute), not a reader printer with an image capture adapter (3-6 pages per minute).
One of the big challenges with microfiche conversion is how the quality (darkness) can vary drastically from page to page, since they were filmed at different times, with different cameras and processing. Some of the newest scanners save the gray scale image, at least temporarily, with great detail. These gray images are far larger than practical for permanent storage, but they allow the virtual rescan of a "bad" page, in a quality checking or indexing step, without returning to the scanner. Vendors are only starting to build systems that take advantage of this technology.
Early Sunrise brand scanners were the standard of quality for many years, but were complex to set up and operate. A replacement line of Sunrise 2000 models were remarkably unreliable - between the quality and company problems, many distributors dropped the Sunrise line. Presumably the problems have been fixed with new management and new products, but they have not completely rebuilt their reputation or distribution and support network, and their software quality is suspect.
Mekel is another popular brand of microfiche scanner. The older models had very poor image quality but the newer model M565 is greatly improved. They have an automatic feeder, so theoretically no operator intervention is required between fiche. However, in practice, manual intervention is needed, so the automatic feed is disabled. Contrast adjustment and recognition of each frame is faster/easier than the early Sunrise, but the actual scan is slower, so the net result is comparable performance. Mekel was bought by another company, who said they did not change the system, but one installation I was involved in before the purchase worked well, and a different installation after the purchase, using similar procedures and interface software, was a major headache.
Wickes and Wilson are the market leaders in aperture cards, good on roll film, and reportedly slow and steady on microfiche scanners, but I have not worked with their equipment, nor worked with anyone who has direct experience.
nextScan, recognized this dismal selection of units a few years ago, and formed a new company to enter the film conversion marketplace. Their roll film scanner is impressive, and after numerous delays, their microfiche scanner looks good. The company appears to have good technology but is getting a bad reputation for support and customer service.
I am starting to get rumors that a new product may be coming in the microfiche conversion marketplace... stay tuned.
Microfiche conversions are fun. Once the equipment is installed and the users are trained, fiche can be converted as needed by the users. If 100 files (fiche) are required per day, all 100 users will have the images of the fiche within minutes or hours. In fact, many scanning units can handle around 200 fiche per day with skilled operators, so half the capacity of the equipment might be used to immediately satisfy user needs, and the other half of the capacity might be used to "convert the rest" of the fiche - to gradually eliminate the separate film records operation.
Roll microfilm conversions are less fun. If high speed equipment is installed and the operators trained, 20-50 rolls can typically be converted per day. But if 100 files are required each day, and each file has an average of 10 documents, as many as 1000 rolls would have to be converted to meet the requirement. Only a few users, not all 100 users, will be satisfied when the conversion is begun. And most of the requests still will need to be handled following the "old" procedures that we were trying to eliminate.
After the first few months, thousands of rolls will have been converted. Hopefully the most active rolls will be converted first, so many of the 1000 documents that might be required to satisfy 100 requests per day will already be on the image system. A few requests may already be complete, and not require a request at all. Some requests will only need a few documents, which might be satisfied by moving the required rolls to the "head of the list." The "old" conversion procedures will probably still be required, but the number of documents that have to be sent through the old (expensive) process will be dropping.
Eventually the roll film scanner will be able to handle all the requests received each day, and the old conversion process can be discontinued. How long before this happens depends on the size of the roll-film library, and the pattern of requests. The conversion should continue "full speed" as time allows, until all the film is converted.
The use of a service bureau will probably cost more than an in-house roll-film scanning operation, but will allow the conversion to proceed more rapidly. Thus the expensive old procedures can be eliminated far sooner than with an in-house operation, perhaps making the total cost of the conversion less than with in-house scanning. This is certainly an option that should be considered.
Back to the home page at www.plesums.com
Back to the Document Imaging index at www.plesums.com
Send e-mail comments to Charlie@Plesums.com
©2004, 2007 by Charles A. Plesums, Austin, Texas USA. ALL RIGHTS RESERVED. You may license additional copies of this document through a nominal royalty payment as specified on www.plesums.com.