Do all documents need to be transferred to a new system? Not necessarily. This article explains why migrating documentation requires far more work than archiving, and how a professionally informed approach to categorisation can improve quality, save resources and create more breathing space in the run-up to go-live. 

We can refer to a case we encountered at Strator; as part of the replacement of a document management system, we had to handle several million documents in a highly regulated environment. Around 10 per cent of the documentation was to be transferred for active use in the new system. The remainder was to be preserved but not used in day-to-day work, and was therefore placed in an archive. 

The distribution of the workload was striking. Migrating the approximately 10 per cent required around 90 per cent of the workload. Archiving the remaining 90 per cent – approximately three million documents – required the remaining 10 per cent, even though the documents had to be tagged with selected metadata and retention periods and organised into virtual archive boxes as part of the archiving process. 

This did not mean that archiving was taken lightly. The difference stemmed from the fact that documentation intended for active use in a new system must be adapted to a new context. The documents must fit into the system’s structure, metadata must be mapped and, where necessary, enriched, and the result must be checked. Information that is to be retained but is no longer part of day-to-day use can be handled far more simply, as it does not need to be adapted to all the structures and workflows that apply to active documentation. 

This example highlights a key decision in a major migration: not all information necessarily needs to be transferred to the new system. Instead, the organisation should determine what the future of the information should be. Some information should be migrated for continued use, some should be archived, and some may be discarded following a controlled assessment. 

The costly part is making the documentation usable in a new context 

The approximately 10 per cent of the documentation that was to be incorporated into the new system was first selected, and then it had to be prepared for a new way of being used. 

The aim of introducing the new system was to support the automation of parts of a process that had not previously been automated. This required more and more detailed data on the products covered by the documents. Key product information was already available in the organisation’s ERP system. The task was therefore to establish a logical link between the documents and the relevant information and to enrich them during the migration. 

Some of the documentation could be automatically matched with the information from the ERP system. However, the data quality and recording practices in the old document management system meant that the link could not be established automatically for all documents. Some documents therefore required manual preparation before the enrichment could be carried out in a controlled manner during the migration itself. 

In other words, it was not merely a matter of moving or adapting existing metadata from one field to another. The documentation had to be linked to information from another business system and brought up to a standard where it could be incorporated into new, automated workflows. 

The 10 per cent did not constitute a single, uniform set of documents. They consisted of several groups of documents, each with its own characteristics, peculiarities and enrichment requirements. The rules for selection, matching and adaptation therefore had to be defined separately for the different groups. 

This explains why a relatively small proportion of the total volume of documents could account for the bulk of the workload. The documents did not simply have to be transferred. They had to function correctly in the new context and support the purpose for which the new system had been established. 

This is one specific example, but the combination is a familiar one from our work on migrations: the new system must support a new or changed practice; data in the existing material is not necessarily sufficient for the new purpose; and only part of the documentation will continue to be actively used. 

Expert insight with a high ‘hourly rate’ 

In this case, it is worth paying a little attention to the client; Anja, an employee at our client’s organisation, sat down with us and used her in-depth professional insight to help us distinguish between the documentation that should continue to be used actively and that which should simply be retained. 

Just think how much her time saved the organisation in consultancy fees to us. The organisation avoided spending resources on adapting, enriching, migrating and checking documents that were not to be used in the new system. Her actual ‘hourly rate’ on the project was therefore quite high. 

A technical analysis can show which documents exist, how large they are, and what metadata they contain. We external specialists can act on the basis of the insight and understanding we can provide. However, a deep business understanding of what the documents are used for, what documentary value they hold, and whether they should continue to form part of active processes must come from skilled people with professional insight into the material. 

In this case, that insight had a very tangible financial value. At the same time, it made it possible to focus the bulk of our efforts on precisely the documentation that needed to work well in the new system. 

Everything must be handled – but not in the same way 

In this specific case, there were two destinations for the documentation. The part that was to continue to be used actively was migrated to the new system. The rest was archived. In this particular example, there were no parts of the material that needed to be disposed of. 

In other contexts, controlled disposal may be a third option. This does not alter the fundamental principle: the entire volume of information must be managed, but it does not need to be managed in the same way. 

Documentation that is to continue to be used actively must function within the new system’s structure and work processes. It must be locatable, comprehensible and usable, and it must contain the data required by the new processes. Therefore, migration may involve selection, cleansing, transformation and enrichment. 

Documentation that is to be retained but not used in day-to-day operations serves a different purpose. It must still be retrievable and comprehensible, and it must be stored in accordance with the rules that apply to it. However, it does not need to be adapted to all the functions and workflows that apply in the active system, so this is usually less labour-intensive. 

Information that has neither continued practical value nor needs to be retained as documentation may be disposed of following a controlled assessment. This still requires a decision and a documented process. However, it does not require the information to be prepared for a new system first. 

The classification is therefore not a choice between taking the information with you or letting it go its own way. It is a way of ensuring that each part of the information set receives the treatment appropriate to its future function. 

A smaller volume can be processed more effectively 

If all those several million documents had been processed with the same level of rigour as the active documentation, the scale of the task would have been entirely different. Within a fixed budget and timeframe, the alternative might have been to reduce the quality of the processing in order to get through the entire volume. 

By first distinguishing between the documentation intended for active use and that intended solely for preservation, resources could be concentrated on the active part. Here, thorough work could be carried out on selection, data quality, enrichment and verification, so that the documentation could actually support the new workflows. 

The separation based on future function therefore did not merely reduce the scope of the migration. It made it realistic to process the most important documentation to a high standard and thus derive greater value from the new system. 

The separation can also create some space before go-live 

In fact, the separation also provided a little more flexibility in the project’s timetable. 

Projects can be under considerable time pressure in the run-up to go-live, and this was certainly the case here. Because the volume of information had been separated, efforts could be concentrated on the documentation that needed to be ready for active use in the new system from day one. 

Archiving and, where necessary, the controlled disposal of the remaining information can, in many cases – as here – be carried out at a later stage as part of the planned phasing out of the old system. This allows the remaining volume of information to be dealt with at a leisurely pace, without it putting a strain on the most critical part of the project’s schedule. 

However, ‘subsequently’ must not come to mean ‘indefinitely’ or ‘never’. The phasing out must be planned, and there must be accountability, resources and a clear end to the work. Otherwise, the old system may end up as a ghost that continues to be operated and incurs costs, even after the new system has been put into use. 

The right outcome is not necessarily to migrate as much as possible 

A successful migration does not, therefore, necessarily mean that as much of the old system’s content as possible has been transferred to the new one. 

A successful outcome is when the information that is to be used going forward works effectively in its new context. Historical documentation that needs to be retained is archived securely. And information for which there is no longer a basis for retention is dealt with through a controlled decision-making process. 

This requires the organisation to review the information prior to migration and decide on its future. This professional assessment takes time, but the case also demonstrates just how much time it can save – and how significant an impact it can have on the quality of the new system. 

So does everything really need to be transferred to the new system? No. But everything must have a controlled future.