Home | Appendix

Quick links: |  | Missing and Empty Files Orphaned Files Pending Files File Integrity Verify & Repair Rebuild Your Database Restore Internal Backups Recover or Manual Rebuild Recover Files Contact Support

|  |  | Repairing a defective database

DEVONthink databases are packages containing your files and the indices containing metadata, thumbnails, etc. These package files are normally robust and trouble-free but problems sometimes can arise. If your database starts behaving strangely, or if DEVONthink finds inconsistencies during start-up, it is time for some housekeeping. In this section we look at the most common issues and their usual causes, followed up by explanations of the maintenance commands in the File menu. Missing and Empty Files All documents in a DEVONthink database have a path the file system where their content file is located. This path is stored for every document, imported or indexed. If DEVONthink can't find a file at a path stored in its index, it will be reported as missing. The most common causes of missing files are: indexed files being moved in the Finder, the indexed parent folder being renamed, or the indexed parent being moved. In the first case, the file isn't in the location it was indexed from. Moving the file back into place will resolve this issue. In the second case, the path no longer exists as the names in the path have changed. Changing the name back will resolve this. For the third case you may have had to relocate the folder for space or other organizational reasons. Select the indexed parent group in your database and open the Generic Info inspector. Click the dropdown arrow next to the Path field and choose Select. In the Open dialog, locate the indexed folder in its new location and press the Choose button. DEVONthink will update the paths for the contents of the folder in its new location. Regarding indexed items, it's also possible for out-of-date items to be reported as missing, especially in cloud-synced locations. Remember that items indexed in cloud-synced locations may require a manual update via File > Update Indexed Items since a filesystem event may not be triggered by the cloud software. Also DEVONthink tries to make sure it doesn't cause any conflicts with those applications. The second most common cause of missing files is people modifying the internal contents of a database. Some Finder replacement applications, like PathFinder, allow you to access package files like normal folders. The internals can also be accessed by other methods, like the command line or the Finder's context menu. If you delete or reorganize anything, you can easily cause missing files. Empty Files: Empty files, also known as zero-byte files, are just that: files with no content and listed as having zero bytes for the file size. DEVONthink is aware of certain file types that can legitimately have empty files, but will report ones that can't, e.g., images. These aren't very common but are sometimes caused by a crash, incomplete file transfers, or temporary files created by some applications. They are handled in the same way as missing files. Dealing with missing or empty files: Missing files are reported in DEVONthink's Window > Log. To deal with these files, Control-click the item in the Log window and choose one of two options: Reveal: Reveals the item in the expected location in the database. If the view/edit pane is visible, the expected file path is shown beneath a document thumbnail. This path is especially useful when dealing with missing indexed files as it shows the path in the Finder from which the file was indexed. If the path shows the file inside a Files.noindex directory, you may use the following option if the file is of no concern or contact our support team. Move to Trash: If the missing file is of no value to you, this command moves the file to the database's Trash. Then choose the DEVONthink > Empty Trash command to completely remove the file from the database. Orphaned Files Just as with any database application, controlling the data flow in and out is a critical process. DEVONthink should be the gatekeeper of the files in its databases. If an external agent, whether an application or a user, adds files outside DEVONthink, they will cause an internal inconsistency and lead to orphaned files when following the steps below. The most common cause of orphaned files is a simple and common one: attempting to create revisions. Say you open a file, a Word document, in an external application. As you are reading it, you decide to make some changes. You type your edits, and select File > Save As (or File > Duplicate the file first). The application most commonly will open the Save dialog in the location of the original file, in this case, inside the internals of the database. You have just created an orphaned file DEVONthink knows nothing about. So the question becomes, how do I create a revision without causing this issue? One simple way is… Duplicate: Duplicate the item in DEVONthink first. Rename: Rename the duplicated item. This can be quickly done by immediately pressing ↩ Return, typing the change, then pressing ↩ again. Open and Edit: Now you can open the new file and make your edits. The other common cause is the same as with missing files: getting into the database package and adding files manually.
Note: Modifying the internal contents of a database outside DEVONthink is unsupported behavior and can lead to broken or inconsistent databases. You should only get into a database's internals when instructed to by our support team.
Pending Files You may see a message in the Log window about "n items to be uploaded" or you may see a icon to the right of an item's name. These are pending, meaning the metadata for the item is in the database but the content is not. As this is something that can propagate to other devices, it shouldn't be ignored. A common cause is indexing files and syncing while the files aren't available. For example, indexing files on a connected hard drive then ejecting the drive or indexing cloud synced files that have been evicted from the local machine by the cloud service. Syncing while the content is unavailable pushes only the metadata from the database. This means another device, Mac or mobile, won't have contents available to download, leaving them with pending files. This also should underscore why it's important to read and understand the In & Out > Importing & Indexing section before committing to it. To address this, start with a toolbar search for `item:pending` in your database to help isolate the files. If none, are found, try emptying the trash in DEVONthink. If any pending files found and you don't need them, move them to the databases' Trash and empty it. Also, make sure the indexed files are available, whether in your cloud-synced folder or by connecting the volume from which you were indexing. You may need to select the indexed parent and choose File > Update Indexed Items to reconnect them with their contents. If needed, check the database on other devices to see if the items are intact. If they are, you could put them on an external drive and replace them in the affected database. In the worst case scenario, you would need to restore from your backups. File Integrity Another report you may see in the Log window is regarding file integrity. DEVONthink scans and stores a checksum for each file in a database. This value is updated as the files change. However, if you use the utility command File > Check File Integrity, it may report an integrity error. This means the stored checksum doesn't match the newly scanned checksum. This may indicate a problem with the file itself. However, this error will also be reported for indexed items that are not up to date. Using Files > Update Indexed Items may resolve the issue with indexed files. XLS Files: Though much less commonly used but still prevalent in legacy files, Microsoft `.xls` files will report a checksum error after you've opened and closed them, even without modifying the document. Converting these the newer `.xlsx` format is recommended. We will now look at the maintenance options available, from the simplest health check to trying to recreate a damaged database. Verify & Repair One of the first options to explore is File > Verify & Repair Database. This checks a database's consistency to ensure it's healthy. If prompted there are problems, press Repair and DEVONthink will try to correct it. It will report missing files and import orphaned ones. But be aware this will not fix missing or empty files. That is handled as described above. If Verify & Repair Database was able to repair your database, use File > Optimize to optimize your database and have DEVONthink create an internal copy of the index.
Note: Verify & Repair isn't just for emergencies. Running this command on an occasional basis is not a bad idea. In fact, if you're a fairly heavy user of DEVONthink, running it weekly or bi-weekly does no harm. It's just a routine checkup.
Rebuild Your Database A deeper maintenance routine, a database rebuild is typically used with more serious database errors, e.g., with reports of file errors. It can also be used when searches, classification, etc. are clearly and consistently behaving erratically. To rebuild the database from scratch, use File > Rebuild Database. During the rebuild process, your files are exported and reimported, recreating the internal index afresh. You should not interrupt this process or let the machine power down while it's working. And for safety, a new restore point is created at the beginning of the rebuild process. A progress report or errors can be viewed in the Log window or popover. The Rebuild command won't be available if a database is actively syncing. Also, bear in mind you'll need to have sufficient hard drive space to do a database rebuild. While DEVONthink uses clever mechanisms to reduce overhead and disk usage, you should still be mindful of the available disk space. This includes when you're dealing with an encrypted or audit-proof database. If the Rebuild command isn't enabled, you may have insufficient disk space to run it. Try emptying the database's Trash and recheck if the command is enabled.. Restore Internal Backups DEVONthink doesn't do file backups but does an internal metadata backup for support purposes. Weekly, it creates a copy of a database's metadata, saving it as a restore point. Only two restore points are kept, with the oldest one being purged. A less frequently used mechanism, it can be useful after a crash, especially if the crash occurred during a drag and drop of a large number of items between databases. When the database is opened and appears to be empty, restoring an internal backup could resolve this with a few mouse-clicks. To restore the metadata files from a previously saved internal backup, hold the ⌥ Option key and select File > Restore Backup. DEVONthink presents you with a list of all available backups. Choose the latest one or the one with a size that appears to be correct and click Open. This swaps the current set of metadata files with the chosen backup metadata (the current set becomes the backup, the backup becomes the current set) so that data is never overwritten. Run File > Verify & Repair to ensure everything is healthy. If everything looks good, you can manually use File > Optimize to optimize your database create a new restore point. Recover or Manual Rebuild If your database is constantly crashing, it's possible it is – or is becoming – damaged. In this case, there are two options to consider: Restore From Backups: Restoring a database from your external backups is usually a quick and effective way to get back up and running. As noted in the linked section, if your data is important to you then diligent backups should be high on your priorities. Manual Rebuild: If you find yourself in the unfortunate situation where you have no backups, it may be possible to rebuild a damaged database manually, following these steps: Locate the database (`.dtBase2` file) in the Finder. Rename the file by adding, e.g., an `x` to the beginning of the name. Control-click the database and choose Show Package Contents in the context menu. Copy the most recent `backup` folder where your database is located. Copy the `Files.noindex` folder into the copied `Backup` folder. Rename the copied `Backup` folder to the previously used name and add the file extension `.dtBase2`, okaying the extension when prompted. Double-click the new database file to open it in DEVONthink. Use File > Verify & Repair to ensure that the database is operating properly. Recover Files In a worst case scenario, if you do not have a current backup of your documents and a manual rebuild didn't work, you can rescue your files manually, but you will lose any group structure. To manually copy your files from the defective database, show the database package in the Finder, Control-click it, and choose Show package contents from the context menu. Copy the `Files.noindex` subfolder to a safe location, e.g., the desktop. If you look inside the folder, you will see a series of subfolders — one per file extension, e.g., `.rtf` — with subfolders in each. Your imported documents are stored within those. It is possible to create a new database and import this folder but you will have to rebuild the database's structures and apply any metadata again. Contact Support If none of the above steps helped solving the problems you have with your database, please contact us with a precise-as-possible description of your problem.