In the era of the paper office, there were always some documents that companies stored securely in their safe. But since we have moved to the electronic age, the habit of selective storage has given way to a 'store everything' mindset. Many companies now treat all their data in the same way: they store all of it.

The consequence of this is that storage needs have grown to swallow an average two-thirds of the IT budget. And an email containing a funny clip gets the same generic encryption and preservation period as an electronically signed SLA with the company's chief customer.

Some companies have refined this store-all strategy by using hierarchical storage management (HSM) to move files to a secondary - and less expensive - storage media after 30 days of non-usage. But this is still a more or less non-selective technology that fails to take into account the importance of specific documents.

"It will be difficult to get away from the 'store everything' approach," warned Nick Charles, vice president and managing director of storage specialist Overland. "The whole business with data is not just capacity, it is also its management. Today, this is usually done through human decision, even though there is legal pressure to preserve data for an extended period with maximum security. But the network manager doesn't always know where to put things."

Charles was surprised at Bill Gates's admission last month to Network News that storage was merely a hardware problem that did not concern Microsoft. According to Charles, there is a need for intelligent data software, using a data object-linked environment.

"Microsoft knows the need for storage through its own court case," said Charles. "You need certain data, even when a user deletes an email. HSM needs to be taken to the next step. A high percentage of data is rubbish, but 30 days is too simple as a measurement for validity. Data is the lifeblood of a company."

Geoffrey Cole, manager at the network storage advisory group at IBM, agreed there was a great need for managing stored data. He believes users tend to be very inefficient in their use of storage. Storage virtualisation techniques could handle the cost of storage hardware management effectively, he said, but did nothing about the issue of identifying important data.

New technologies are emerging to tackle the growing data problem. IBM last year announced a new software development, called StorageTank, which aims to create a consolidated storage hardware environment. This creates a "pool of capacity" across platforms and applications in a multi-supplier environment. The next phase of the project is to look at data management, and create an administrative tool that identifies the way specific data should be treated.

"We see the beginnings of software developments that target the data issue," said Cole. "Once it is fully developed, the StorageTank architecture will allow users to set up administrative policies to indicate which level of performance, backup, life and security the data should receive. These policies will be created at the birth of data. But today, this is still impossible."