Hidden data

File deletion

In most filesystems deleting a file means that the disk space containing the file is marked "free". It means that the operating system can now overwrite this space when it needs to. However, exactly where and how a file is written on a disk depends on the filesystem and operating system in use, so an ordinary user has no way of knowing when a specific disk region is overwritten with new data.

As long as the deleted file is not overwritten with new data, it is still accessible (with special software). Hence, to securely delete a file, it must be overwritten at once (in case of hard disk drives).

Securely erasing whole disk

It is advisable to securely erase the whole disk if the disk (or the whole computer with the disk) is sold or given to somebody else. The methods of performing secure erase depend on the type of disk.

For traditional hard disk drives (HDD), the data can be securely deleted by overwriting it. In most cases secure deletion software overwrites the deleted data many times with zeros and/or random data. For example, one can use Darik's Boot And Nuke to securely delete the whole disk, but there are also many alternatives.

Screenshot of DBAN (image source)

For solid state drives (SSD) overwriting the whole disk stresses the disk too much and does not work as intended. SSD-s have a built-in controller that controls where and how data is written so that all disk areas are worn out evenly. This means that the operating system does not know where exactly the file resides on the disk.

Solid state drive (SSD) on the right and traditional 3.5" hard disk drive (HDD) on the left.

To securely erase the whole SSD, a special ATA Secure Erase command has to be used if it is supported by the disk. This command temporarily raises the voltage used by the drive and thus destroys the electrons in the flash memory.

Securely deleting individual files

Securely deleting individual files on HDD-s follows the same logic as erasing the whole disk: the files have to be overwritten several times with zeros and/or random data. In Windows, there are several alternatives to choose from, for example Eraser or File Shredder. From command-line, one can use SDelete by SysInternals.

Lab exercise. Start the program Eraser in your virtual machine. Delete a file securely by overwriting it. If you don't have a file to delete then download a file or create the file yourself.

In Mac OS X prior to El Capitan (10.11), the possibility to securely delete files was built into the operating system. First, the files had to be moved to Trash as usual and then choose Finder -> Secure Empty Trash... from Finder's menu. Unfortunately, this functionality is removed starting from Mac OS X El Capitan due to a security issue. Hence, a third part tool has to be used, although encrypting the whole disk may also solve the issue in some cases. The functionality to securely erase a whole disk or partition is still built into Disk Utility.

In most Linux distributions either shred or srm command-line programs are available.

To securely delete files on SSD, it must support the TRIM functionality. This allows the operating system to tell disk controller which data blocks are no longer in use. Unfortunately, many older SSD-s do not support TRIM functionality. Moreover, its support is determined by the combination of the disk, operating system, file system and the way disk is connected to the computer. USB flash drives also do not support TRIM functionality.

For example, TRIM is not supported if:

SSD is part of a RAID array
SSD is connected by FireWire
SSD is used as an external disk with USB
SSD is part of Network Attached Storage (NAS)
the operating system is too old
if a file system other than NTFS is used in Windows

More information about SSDs can be found from:

Temporary files

In every day usage, the operating system creates a lot of temporary files in order to accelerate the work flow or perform some task (e.g. printing). Web browsers save browsing history, cookies, temporary and downloaded files. Windows stores log files, list of recently opened documents, etc. Most of these files can be deleted individually, but it makes sense to use special software to delete most or all of them at once.

Windows has a built-in application Disk Cleanup that it automatically invokes when the system drive is running out of space. However, it can also be run manually.

Disk Cleanup in Windows

By default, Disk Cleanup prompts to delete current user's temporary files but it is advisable to click "Clean up system files" if the user has administrator permissions. Among other, this allows to remove old system updates that are outdated and may take a lot of disk space.

Disk Cleanup is a quick and simple solution, but there are also alternatives with more features, e.g. CCleaner. While CCleaner has been a convenient tool it has had several issues since 2017. The most significant issue involved hackers having compromised the tool for over a month before it was detected. The latest change in 2018 is that the CCleaner installer bundles Avast Antivirus. Thus, use the tool at your own risk.

CCleaner searches temporary files from more locations (e.g. from application-specific locations) and offers more granular control of their removal. However, one must be careful as CCleaner may offer some files which are sometimes necessary for the system.

Data recovery

If secure file deletion is not used, deleted files can still be accessed until they are overwritten. This means that it is possible to recover some files that are accidentally deleted. Moreover, this method is used by digital forensics when investigating confiscated computers. Even if the file cannot be restored in full, it may be possible to obtain some meta information (time of creation, author) that can be used as evidence.

There are a lot of file recovery tools for all operating systems. Here is a short list of free software for Windows:

Recuva Free - http://www.piriform.com/recuva
Undelete Plus - http://undeleteplus.com/
Undelete 360 - http://www.undelete360.com/
TestDisk - http://www.cgsecurity.org/wiki/TestDisk
- TestDisk can repair corrupted partitions and file systems and recover deleted files. Also works in Mac OS X and Linux.
PhotoRec - http://www.cgsecurity.org/wiki/PhotoRec
- PhotoRec ignores the file system and reads file identifiers from block device directly. Hence it is possible to use it even with corrupted file systems. Also works in Mac OS X and Linux.

Lab exercise: Start Recuva Free and understand how it works.

Task 1: Download some documents and pictures and delete them in the usual way. Try to find and restore them with Recuva Free. Now delete the files securely and run recovery software again.
Task 2: try to find information from the partition that is named "Virtual USB". This task is a part of the first homework. The upload form for the restored files can be found under the first homework task.

Infoturve 2018/19 sügis