Data modeling and management
This list is about the basic need of many researchers to organise, manage, publish and eventually archive information collected in the course of their research. Simple tools dealing with one type of information are rarely adequate because of the rich connections within Humanities data.
A number of open-source, web-based tools now available allow you to model, collect, manage and publish data characterised by heterogeneous entities which are linked together through relationships. These can include documents, images, events, people, places, organisations, structures, portable objects, and many other entities which form rich networks of relationships.
While any sort of complex data modelling and data management/publishing can be achieved - laboriously and at huge expense - by programming in conventional relational (and other) database tools, newer web-based approaches can provide nearly ready-made open-source solutions to a wide range of basic research data management needs. However many researchers are unaware of what is available. This list aims to highlight suitable tools for consideration.
<p>HEURIST (http://HeuristNetwork.org) is an extremely flexible, end-user oriented, web-based data management system designed specifically for Humanities data. Developed since 2005, it has been in active use across many projects since 2009. It is available both as a free web service for researchers (hosted at the University of Sydney Data Centre) or for installation on a physical or virtual server (Open Source on gitHub). </p><p></p><p></p><p></p><p>Researchers can design, create, manage, analyse, visualise and publish their own richly-structured database(s) through a simple web interface, without the need for programmer(s). Quite complex databases can be built in a few hours by borrowing structures and vocabularies published by other users. Databases can be designed and built incrementally, as existing data are not affected by changes in structure. Databases created by Heurist are stored in MySQL with a repeatable structure facilitating sustainability and independent access by other software. </p><p></p><p></p><p></p><p></p><p></p><p>Advanced features include record linking, graph structure, drill-down facet searches, rule-based queries, custom reports, linked map-timelines, network visualisation, normalised spreadsheet import, crosstabulation, XML feeds, XSLT transforms. The team provides initial email and skype assistance for project setup at no cost, and special customisations at modest cost.</p><p></p><p></p><p></p><p></p><p></p><p>Heurist is applicable in many domains including History, Archaeology, Anthropology, Cultural Research, Linguistics, Media Studies etc. It handles everything from text, tags and categories, through networks of rich relationships and offset annotations, to images, video, spatial data and dating (including non-western systems and dating uncertainty).</p><p></p><p></p><p></p><p></p><p></p><p>Sophisticated searches (including faceted and rule-based) can be saved, organised and rendered as layered interactive maps, timelines, network visualisations, crosstabulations, lists, custom reports, XML and XSLT transforms, all of which can easily be printed or embedded (live) in a website - Heurist provides the html code required to embed results in a web page or CMS.</p><p></p><p></p><p></p><p></p><p></p><p>Databases can easily be migrated later to a personal or institutional server. Since the data are stored in a standard MySQL database with a fixed and well-documented structure, the data can be accessed directly via SQL from most programming languages independently of Heurist.</p><p></p><p></p><p></p><p></p><p></p><p>Version 4.2 (Jan 2016 onwards) is available as a service via the Heurist website. Version 4 delivers a user-customisable interface of widgets, along with better mapping and visualisation. Existing version 3 databases are bidirectionally compatible with version 4.</p>
<p>KORA is an open-source, database-driven, online digital repository application for complex multimedia objects (text, images, audio, video) created by MATRIX. The application ingests, manages, and delivers digital objects with corresponding metadata that enhances the research and educational value of the objects. KORA's flexible architecture can accommodate any metadata scheme allowing for individualized digital collections with differing contents and metadata structures.</p>
<p>Pliny was a tool for working with annotations when conducting personal scholarly research, and can be used with both digital and print materials. It also had data manangement and organizational features, and was designed to facilitate reading, reacting to that reading and forming an interpretation. Pliny stored all data on the user's computer rather than remotely on the web.</p>
nodegoat creates visualizations and provides a graphical user interface for the maintenance of data. It combines a variety of data management functions into one program.
Other list by Ian Johnson