How to install and use the alpha version of Disk & Tape to find and delete duplicates on multiple devices

WARNING: Our MVP is not yet fully functional, contain known bugs and deficiencies, and is still in active development. We strongly advise to try it only on data that you can afford to lose, like a secondary copy or a filesystem snapshot.

  1. System requirements

    2. We have tested our software on Windows 10, Ubuntu Linux 22.04, and macOS Ventura 13.3.

    The early version of our application has not been fully optimized for memory consumption. As such, the more RAM — the better.

  2. Request an invite code

    3. We have a limited number of invite codes. We send them via e-mail (make sure to check your Junk folder).

    Request an invite  

  3. Create a user account

    4. Please sign up on our website  

  4. Register hosts

    5. Files generally reside on disk drives, which are connected to computers (permanently or temporarily). In order to organize and manage files stored on a particular computer, it must be registered with your account as a "host". Every host must have an agent we call "dtrm" (Disk & Tape Root Manager) installed. Install "dtrm"  

    How to register hosts on Linux  

    How to register hosts on Windows  

    * NOTE: Registering a host does not scan or upload any information about your files to the cloud.

  5. Register roots

    6. Roots are locations (directories or entire drives) containing files that you want to organize and manage. You need at least one to begin with (you will be able to add more roots later). It's up to you how to split your files into roots. Some people use one root for the entire host, some people use one root per drive, some people use separate roots for music, movies, etc.

    How to register a root on Linux  

    How to register a root on Windows  

    * NOTE: Registering a root does not scan or upload any information about your files to the cloud.

  6. Scan and upload file metadata

    7. All files under all roots must be enumerated and scanned to identify duplicates. Scanning a file computes the hash of its entire content and attempts to determine its actual type (JPEG, MP3, etc.). Note that large, multi-terabyte roots might take significant time (up to several days). After scanning files under the root, the encrypted list of file names and metadata (referred to as "manifest") will be uploaded to Disk & Tape Cloud. The encryption keys are never uploaded to the cloud and always remain in your sole possession. Nobody but you will be able to decrypt those lists.

    For each registered root, run either of the following commands:

    
    
    
        Command Prompt
    
    
    
    
    # Example
johndoe@client-server:~$ dtrm sync <path to root or any file under it>


    

    
    
    
        Command Prompt
    
    
    
    
    # Example
johndoe@client-server:~$ dtrm -n <root name>


    

    
    
    
        Command Prompt
    
    
    
    
# Example
johndoe@client-server:~$ dtrm -i <root id>

    * NOTE: Files not located under any registered root are not enumerated or scanned. Depending on permissions, you may need to use "sudo" or run the command as administrator.

    ** NOTE: We understand it's a lot of manual work, and we will make it easier in the future. For now, we want to do it on the per-root basis to give better visibility to possible errors.

  7. Install the control application on your client machine

    8. The client application, or "dtcli", is used for downloading manifests from the cloud, decrypting them, and merging them into a unified list. Also, it is used to analyze and operate on this list, creating file management tasks, and dispatching them to hosts for execution.

    You can install "dtcli" on multiple machines but you can be logged in from only one location at a time.

    Install "dtcli"  

    Once installed, run the following command:

    
    
    
        Command Prompt
    
    
    
    
    # Example
johndoe@client-server:~$ dtcli login

    * NOTE: Logins expire periodically for your privacy. You may be asked to log in again.

  8. Fetch manifests from the cloud

    9. Use the "dtcli fetch" command to download the most recent root manifests from the cloud.

    
    
    
        Command Prompt
    
    
    
    
    # Example
johndoe@client-server:~$ dtcli fetch

    Because manifests are encrypted by the hosts and the encryption keys are not uploaded to the cloud, you will need to enter the encryption keys manually. This needs to be done once per registered host (5 hosts = 5 passwords).

    Use the "dtcli add_host_key" command to decrypt the file lists encrypted by each host. You will need to run the following command for each host:

    
    
    
        Command Prompt
    
    
    
    
    # Example
johndoe@client-server:~$ dtcli add_host_key

    How to add a host encryption key to dtcli  

  9. Find duplicates

    10. Text  

  10. Mark duplicates for deletion.

    11. Bulat  

  11. Send decrees to host.

    12. Decree is the list of commands that applies to the host.  

  12. Run the command from section 6 on each host you have.

    13. Decree is the list of commands that applies to the host.