Adventures in Digital Research – Devonthink: Escaping the Silo

Silo Explosion, Dan Browne, Flickr.
Silo Explosion, Dan Browne, Flickr.
In my last post, I explained why I have been moving away from Evernote as the go-to destination for my research material. While I have got tremendous use out of Evernote over the last few years – and will continue to use it heavily in the future – it simply cannot cope with the volume of material I have been throwing at it. Or at least, it cannot arrange and process that information in a way that is useful to my workflow.

In this post, I will introduce why I have been using another app – Devonthink Pro Office (DTPO) – to organise my research projects. Devonthink is an Apple-only app, with both MacOS and iOS variants, so apologies in advance for those of you who only use Windows. However, over the last couple of months Devonthink has become an indispensable part of my workflow, so, on the basis that I know many academics who now use Macs, I would like to share how I use Devonthink to conduct my research in a way that overcomes Evernote’s main drawbacks.

Introduction to Devonthink

Despite its weird name, Devonthink has got nothing to do with the English county, or custard. The name, apparently, refers to the Devonian period (419–359 million years ago)1, though what this has to do with document management, I have no idea. Devonthink has actually been around for several years now, predating Evernote by some way, and its Idaho-based developers have done a lot to harness the Mac’s power in managing your information. Devonthink is essentially a database – or a collection of separate databases – that can store or index whatever you like, and while Evernote’s main selling point is search, Devonthink’s is artificial intelligence, which can automatically group or classify your documents, as if by magic. DTPO is the highest tier of Devonthink available, enabling scanner support and baked-in OCR (optical character recognition) of all your documents. It’s pretty expensive, but unlike Evernote it is not a subscription product and I picked it up at a pretty hefty educational discount. It’s worth every penny.

Separate databases – search, sync and sleep soundly

In my Evernote post, I explained how one of my biggest gripes is that it is a single database. If you use it like I did, your personal stuff is mixed in with your work stuff, your private with the professional, so without diligent organisation (whether through separate notebooks or the liberal application of tags), you can end up inadvertently showing your colleagues all your shakshuka recipes, or clips of Tim Farron you’ve grabbed from the web.2 Devonthink gets around this by allowing you to set up multiple databases. When you first start Devonthink, you are asked to set up a database (an Inbox database comes as standard). This database can live locally on your hard drive, but recent versions of Devonthink also include a generous set of sync options, including iCloud Drive, Dropbox, WebDav, and local wifi-sync (the latter of which is secure, and crazy-fast). These options allow you to access your databases on all your Apple devices, including on iOS with Devonthink to Go. Databases can be password protected, and if you choose to sync, are encrypted both in situ and transit. When it comes to security, therefore, Devonthink has all the bases covered.3 This cloud-optional solution to information management means I am far more happy to store my confidential files such as bank statements in Devonthink’s ginormous brain (specifically, in a dedicated financial database). With OCR and scanner support, I can continue to pursue a paper-light existence without fear of my documents being lost in the cloud, or hacked.4 Above all, however, it means I can have separate databases for my personal life and separate research projects. This means that when I look through my database, I only see relevant material. Because my separate Devonthink databases are lightweight compared to my former (enormous) Evernote database, search is blisteringly fast – much faster than Evernote – and I can sleep soundly in the knowledge that nobody has seen my numerous shakshuka recipes, unless I wanted them to.

No siloing here – unless you want it

Again, another problem I find with Evernote is that the more and more data you put in it, the wider the gulf between your magic Evernote documents, and your regular-joe documents on the file system. It’s like your Evernote docs have been invited to some crazy, cocaine-fuelled party, where anything can happen, and your regular-joe docs are stuck at home sipping orange juice.5 Do you want to organise and search documents on your file system, such as project files or files that are too large to import (e.g. movies)? Tough, they’re not invited.

With Devonthink, you can index regular files on your hard drive, as well as import files into your database. The difference is that while imported files exist within the database – à la Evernote – and are only stored there, indexed files continue to sit on your file system; they’re just referenced within Devonthink. You can open them within Devonthink, search them, edit them and organise them as you see fit, but the actual files sit elsewhere.6 The benefit of this is obvious to any academic who has done a significant amount of research and amassed a pile of digital documents. You can organise and search your project files alongside relevant reference material. You can organise and search through material that is too big to fit in your database, ad hoc files you don’t need to store for long, or files which for whatever reason should not be stored in the database. Above all, it allows your files to be accessed by multiple external apps at the same, avoiding the siloing problem whereby you need to put several copies of the same files in different databases to secure the most functionality (thus taking up unnecessary hard drive space). So, my reference manager can ‘see’ the pdfs of my historical sources, and use them to make citations, but Devonthink can ‘see’ the same pdfs and use them for organisation, search and annotation.

Better organisation and visualisation

Screenshot of Devonthink Pro Office
Screenshot of Devonthink Pro Office
As I explained previously, Evernote’s organisational features are flexible, but limited. Tagging gets round many of the problems associated with a two-tier storage hierarchy (notebooks and stacks). However, tagging is extremely difficult to maintain. Evernote search is amazing, and using its advanced search syntax, you can sift through your database in a more focused way. However, when it comes to the more sophisticated data organisation and visualisation required for advanced qualitative research, Evernote is out of its depth.

I find Devonthink’s search to be as powerful as Evernote’s – perhaps even more. Not only does it search the contents of your files and documents, as per Evernote Premium (including OCR’d text), but you can run far more granular searches on your files, using Boolean operators. Further, Devonthink has better support for saved searches, with ‘smart groups’ that automatically retrieve relevant files according to specific criteria. This functionality is ideal for academics, especially when it comes to visualising and analysing your myriad of documents. I use smart groups to hunt for particular keywords and phrases, so I can always be sure to find relevant material when I index or import new documents.

With Devonthink, you can create as many nested folders as you want alongside tags and smart groups to create a highly personalised filing system. Devonthink also allows you to mark your documents with flags and coloured labels so you can see at a glance if a particular document is important or needs action. The upshot of this is that Devonthink is extremely scalable, easily handling vast databases of several thousand documents without loss of functionality. It also permits far better visualisation of your data, allowing you to tease out connections and think about it in new and creative ways (hence, the ‘think’ in Devonthink). Devonthink is thus ideal for academics, lawyers and other information workers who handle a lot of data and need finer control over how it is organised and presented.

Artificial intelligence – not quite Skynet, but still scarily good

Devonthink’s main selling point, as I mentioned above, is the use of ‘artificial intelligence’ to automatically file your documents – a functionality that has not yet been fully replicated in Evernote.7 We are not quite talking about Skynet levels of artificial intelligence just yet, but Devonthink does a fairly good job at analysing your documents based on their contents and metadata, and automatically grouping or classifying them. So, you can put a particular historical source in your database and Devonthink will work out which group it belongs in. Awesome.

Accessibility, import and export

One of Devonthink’s greatest strengths is that the documents you put in your databases are kept in their original format (e.g. docx, rtf, pdf) and are accessible to the file system as a whole. Indexed documents are kept in their original locations on your file system, but even imported documents within Devonthink’s database package can be located and searched using Spotlight.

Devonthink’s advanced document handling also means that import and export are really easy. You can import individual files or entire folders, and when the time comes to export, it’s simply the case of dragging them out of Devonthink onto your file system. And since the documents are preserved in their original format, you don’t need to worry about converting them.

Does not care to share

By now the advantages of Devonthink for digital research should be obvious – but all apps have their downsides, and Devonthink has its fair share. Firstly, the interface is a bit old-fashioned and clunky – something from 2006, rather than 2016. It is no way as polished as Evernote. While this does not affect its functionality, if you care about aesthetics and plan to spend a lot of time in your databases, it’s something that’s worth thinking about.

Secondly, Devonthink’s sharing and collaboration features are also pretty basic compared to Evernote. While Evernote has support for shared notebooks, Devonthink uses its own ad hoc web server to share your database with friends and colleagues over a local network. It’s not ideal, and I hope that in time Devonthink will include more advanced sharing features, in particular those useful to users in a corporate/enterprise environment.

Conclusion

This post has been a whirlwind tour around Devonthink’s main features, which are too numerous to mention in a single post. There are many more advanced features (like replicants and duplicates, RSS feed support and web bookmarks) that make Devonthink truly ideal for doing digital research, rather than merely collecting research material. Using Devonthink (and in particular, its advanced strain, Devonthink Pro Office), many of the practical shortcomings of Evernote, such as its limited organisation and siloing of your important documents, can be overcome.

In my next post, I will explain how I use the two apps alongside each other to get the best of both worlds. While Evernote and Devonthink share many of the same core features (such as search, indexing, and OCR), differences in their advanced feature sets and implementation mean that they are suited to slightly different purposes. By putting them both to work as systematically as possible, the benefits of both apps can be maximised and their weaknesses cancelled out. Devonthink might be the obvious home for my research material, but for what it’s worth, I continue to put my shakshuka recipes in Evernote.

  1. Thanks Wikipedia!
  2. @MillieQED
  3. Importantly, if you choose to sync using a popular cloud service like Dropbox, your database package itself is not stored in the cloud – which can lead to data corruption. Rather, it sets up a sync store that acts as a kind of holding pen for your data, allowing you to work on multiple devices at once without fear of data loss.
  4. Naturally, I back up ALOT.
  5. Don’t do drugs, kids.
  6. What actually happens behind the scenes, cleverly, is that Devonthink extracts all the relevant information from your indexed document to make search and organisation happen, but when you go to open the document you open the file on your file system.
  7. Evernote’s web clipper has a quasi-AI functionality, that does a good job at figuring out which notebook you want to put a clip in. But this is not at the level of Devonthink.
  • Apologies: I initially posted this comment on your why-I’m-leaving-Evernote post. Trying again here.

    Thanks for this extremely helpful post. I am a PC user considering switching to Mac purely in order to use DevonTHINK. I wonder if you could answer two questions, based on your experience with DTPO.

    1. I’m a social scientist who relies partially on archival research. I have converted photos of archival documents from JGPs to PDFs and combined these into multipage documents to replicate, say, a government report or a pamphlet. Will DTPO let me assign tags to specific locations in a multipage PDF document, or only to the document as a whole?

    2. Your description of indexing vs importing is very helpful. Many of the more affordable Macs have very limited storage space, so I am curious whether it is possible to index files that are located in the DropBox cloud, for example, but have not been downloaded to the local harddrive. Is this possible? Does it slow down DTPO so badly that it’s not worth considering this option?

    I’ve been looking for the answers to these questions in user fora but haven’t found anything definitive on either one. Your help is greatly appreciated!

    • Magda – replying again here because it’s probably the more relevant post!

      Thanks Magda for your comments – I’m glad you found my post useful. In answer to your questions, alas, Devonthink does not go that far, although it is the most fully featured document manager I’ve found on the Mac to date. As far as I know, Devonthink does not allow you to tag particular locations in pdf documents, but that would be a really useful feature, and I hope that they implement it in future. Something like that would approach the ‘coding’ functionality of NVivo, but that is a very specialist bit of software for qualitative research. If you have a text document, e.g. in rich text, it is possible to create wiki-style links between documents, or manually insert hyperlinks, but this is not quite the same thing as tags. While I have not tried this out, another option would perhaps be to use manual hashtags like on Twitter, and run a content search for words containing that hashtag. For example, in a text document you could insert the hashtag “#risk” where the subject matter turns to risk, and then retrieve all documents containing that hashtag. (Advanced search is brought up by Cmd + F, then limit the search to content only).

      Indexing, while allowing you to search documents outside your database package, is unfortunately limited to files on your local hard disk – it would be amazing if Devonthink would allow you to index files stored in the cloud, but as far as I know there is no database software out there that can do this, since the files still need to be ‘read’ by the app for indexing to work. One way around the limited storage on newer Macs would be to store the database package on an external hard drive. While this is not quite as convenient as storing internally, you can still use the cloud (Dropbox, iCloud etc) or your wifi to synch your database(s) between computers.

      Hope this answers your questions, and good luck finding a research solution that works for you.

Leave a Reply

Your email address will not be published. Required fields are marked *