Technology is developing too fast for organisations, both public and private, to form lasting data handling principles, Minister for Justice Michael Wills told a House of Lords Constitution Committee gathering on Wednesday.

The Lords met to question Wills and Tony McNulty, the Minister of State for Security, Counter-Terrorism, Crime and Policing, on issues surrounding the Surveillance Society report, a Lords public inquiry into the principles and technology by which the public sector administers public data. The inquiry followed numerous data losses by public and private organisations and growing public anxiety surrounding the government's data collection and sharing policies.

“It is clear we need a radical change in government in how we handle data,” said Wills. “Over the years the government has become scrupulous about how it handles money and has put lots of checks and balances in place, but the case for data is less clear,” he added.

Wills acknowledged the government needs to give the public more confidence that it does not intend to become a data hound, collecting as much individual data as possible. “I don't think anyone wants to see huge databases where anyone can go and search,” he added. “The security implications of that are horrendous.”

McNulty denied claims made by the Information Commissioner that Britain is “sleepwalking” towards a “big brother state”, but he acknowledged a government “struggle” between striking a balance between the duty of the state to protect the public and the individual's right to privacy, as well as “how to deal with the positive benefits of new technology”.

Also debated was the possibility of a privacy impact assessment for new government data-sharing legislation, which would require the government to clarify exactly what individual data would be sought and shared when putting forward new legislation. McNulty said it was an idea that was “worth exploring”.