TAPoR 3 Discover research tools for studying texts.
General Inquirer

General Inquirer

The General Inquirer is a historically important program for content analysis of textual data originally developed in the 1960s by Philip Stone and his colleagues at the Harvard Laboratory of Social Relations. Though the original release used punched cards, a version the program is available by request, and can be obtained by e-mailing Roger Hurwitz from an academic address. A minimalist web version capable of processing small files is also available. The General Inquirer's homepage is maintained in Philip Stone's memory.

Documentation
Creator(s): Philip Stone (c/o Roger Hurwitz)
Creator's Email: rhhu@csail.mit.edu
Repository:
Attributes

List this tool appear on

Comments

Amy Dyrbye
Amy Dyrbye

More information on the General Inquirer and its use is available in the following:

Dengler, Ralph. \"A General Inquirer Analysis of Sixteenth Century and Contemporary Catechisms.\" Computers and the Humanities 8.1 (1974): 5-19. Web.

Kadushin, Charles, Joseph Lovett and James D. Merriman. \"Literary Analysis with the Aid of the Computer.\" Computers and the Humanities 2.4 (1968): 177-202. Web.

Psathas, George. \"The General Inquirer: Useful or Not?\" Computers and the Humanities 3.3 (1969): 163-174. Web.

Rice, Sandra. \"Software Review: INQUIRER III (Edinburgh Edition) A Computer Based System for Content Analysis.\" Computers and the Humanities 10.6 (1976): 332. Web.

Stone, Philip J., Dexter C. Dunphy, Marshall S. Smith and Daniel M. Ogilvie. The General Inquirer: A Computer Approach to Content Analysis. Cambridge: MIT Press, 1966.

Amy Dyrbye
Amy Dyrbye

Program Overview

The General Inquirer was a program for processing natural language text with a focus on content analysis from Philip Stone and his colleagues at Harvard University (Rice 332). It was developed in the 1960s at the Harvard Laboratory of Social Relations, and as late as 1984, Robert Oakman described it as a \"pioneering computer application\" for its capacity to automate the process of applying preset categories to the words of a text from a dictionary such as the Harvard Psychosocial Dictionary (Oakman 150).

The General Inquirer lemmatized each word, assigned content tags, performed graphical and statistical analysis, and produced a report containing sentences with significant content words (Oakman 151). Oakman, reflecting back, noted that it was most useful to the social sciences as \"with literary texts, its usage has been minimal and the results often disappointing\" (151).

In the manual released in 1966, The General Inquirer: A Computer Approach to Content Analysis, the developers described the system as

a set of computer programs to (a) identify systematically, within text, instances of words and phrases that belong to categories specified by the investigator; (b) count occurrences and specified co-occurrences of these categories; (c) print and graph tabulations; (d) perform statistical tests; (e) sort and regroup sentences according to whether they contain instances of a particular category of combination of categories. (Stone et al 68).

The General Inquirer's applications and value was a subject of debate amongst researchers seeking computer automation within the Humanities. The journal Computers and the Humanities published a review symposium focused on the General Inquirer as an aid to literary analysis in 1968 (Kadushin, Lovett and Merriman 177). All three reviewers criticized the system for its reliance on dictionaries for categorizing words. Charles Kadushin found that, due to the dictionaries, \"[t]he Inquirer is indeed limited to small-unit coding, in either broad or narrow contexts. That it does its coding by means of single words, even in contexts, is its major drawback as well as its chief virtue\" (Kadushin, Lovett and Merriman 180). He concluded that, with respect to categorizing words,  \"making do with general dictionaries or a slight modification of them leads, in many cases, to almost unbearable intellectual costs\" (Kadushin, Lovett and Merriman 182).

Joseph Lovett criticized the General Inquirer, and its manual, sharply. In his view, \"[t]he pitfall of these chapters [on content analysis] lies in the biases of the authors, for these biases of the General Inquirer approach influence its implementation in any specific research problem\" (Kadushin, Lovett and Merriman 184). He disliked the emphasis on procedural, rules-based content analysis, and the system's inability to handle homographs (184). With respect to homographs, he regarded the challenges with them as a shortfall of the developers as much as the program itself:

That homographs would be confused more by the computer than by a human coder is admitted by the authors. They view the problem as one of category consistency, however, not of coder reliability. It seems unclear, if 'river bank' and 'savings bank' are assigned by the computer to the same category, why the researcher is to blame. (184)

Like Kadushin, Lovett identified the dictionary as the root of the General Inquirer's shortfalls, finding that \"[t]he burden of proof of the General Inquirer...is on the method and logic by which the descriptors are conceived to bring about the many-to-few mapping, i.e., the dictionary of the program\" (Kadushin, Lovett and Merriman 185).

James Merriman provided the final assessment of the roundtable. In his view, computers had proved their value with automatic data processing, and that the General Inquirer was a welcome attempt to extend the usefulness of computers by leveraging them to tease meaning from literature (Kadushin, Lovett and Merriman 187). For him, the General Inquirer's use of dictionaries for categorizations was the \"real novelty of the approach, and it is precisely the difficulties of 'dictionary' construction that raise major doubts about the present and perhaps even future utility of the General Inquirer to literary studies\" (Kadushin, Lovett and Merriman 189). He criticized the challenges of selecting and constructing categories through dictionaries (Kadushin, Lovett and Merriman 191) and the Harvard Third Psychosocial Dictionary in particular as problematic when used for Humanities applications (Kadushin, Lovett and Merriman 192). However, Merriman judged the suggestion that users construct dictionaries tailored to texts unrealistic and representative of \"abandoning the General Inquirer\" (Kadushin, Lovett and Merriman 200). Rather, he suggested that \"[t]he hope for fully automatic preparation of texts...will almost certainly have to be abandoned in favor of extensive manual pre-editing\" (200). Whether a universal dictionary was a realistic outcome, Merriman called for researchers to work on creating better dictionaries, and a better solution for creating categories and category systems (Kadushin, Lovett and Merriman 201).

George Psanthas disagreed with Kadushin, Lovett and Merriman's criticisms, and contributed a rebuttal to Computers and the Humanities in 1969 for the purpose of dispelling misconceptions about the General Inquirer (GI) and its value to Humanities research applications. In his view,

Many criticisms of the GI, based on the description of the system and its uses by Stone et al., can be shown to be inappropriate, since they are based on a misunderstanding of the system and the failure to make certain basic distinctions among hardware, software, dictionary construction, data preparation and input, data analysis, and inferences from data analysis...[m]uch work needs to be done to improve the GI but it does offer possibilities for use in literary and content analysis, as well as other application. (Psanthas 163)

He stressed that the dictionaries were distinct from the General Inquirer as a system, and that any dictionary may be used, not just the Harvard Third Psychosocial Dictionary (Psanthas 164). Psanthas described the General Inquirer as \"a system that can be used for making classifications of words (idioms), or sentences and need not be used for content analysis in the traditional sense of the word\" (164). He noted that it offered a classification/tagging system for words and sentences, with 'word' being better defined as 'string', and 'sentence' as \"any string of words marked by a beginning and end marker\" such as a space or punctuation mark (164).

With respect to criticism of the General Inquirer's capacity to classify words, Psanthas found that if a scholar's classifications were not programmable into the General Inquirer, they were butting up against the limitations of the software and/or hardware, and their own understanding of what computers can't do yet (Psanthas 168). It was his view that, \"in all honesty, it must be stated that all of our desires for a completely automated system of classification still remain dreams\" (168). Like Merriman, Psanthas called for scholars to focus on creating better dictionaries as the solution to categorization problems (Psanthas 169), but added that

[i]f there is an inherent set of limitations built into the dictionary approach to the classification of natural language, it is simply that at this time we do not have enough knowledge to enable us to use the computer to automatically and correctly classify words, idioms, or even whole sentences into a clearly defined category...[t]he search for better methods for analyzing language is of more proper concern. (169)

Psanthas criticized the General Inquirer for its reliance on statistics, and suggested that new models of peering into a text were required (Psanthas 172). He also cited the labour of keypunching a text, and the limited symbols available, as a drawback (172), in addition to the challenges of pre-editing texts by hand (Psanthas 173).

Despite its shortcomings, the General Inquirer remained in use into the 1980s, though even in 1984 Robert Oakman noted that the system was only as good as the content categories of the dictionary it drew from (Oakman 151). It saw continued development, including adaptation, as with the INQUIRER III (Edinburgh Version) reviewed by Sandra Rice in 1976 (Rice 332). The Edinburgh Version retained a reliance on punched card input and a preset dictionary (332). Rice found the supplementary job control cards \"difficult to prepare and comprehend for the user not conversant with IBM OS system [sic],\" while the program itself was poorly commented and required precise input to receive comprehensible results (332).

Rice and Oakman's interest in the General Inquirer well after its initial release demonstrates the long term impact of its approach to automating the analysis of natural language text. Notably, its shortcomings prompted other researchers to develop their own programs to accomplish similar tasks, as Howard P. Iker cited the General Inquirer's reliance on dictionaries as the spur for the WORDS system, which used the text at hand to isolate words instead (Oakman 152).

Amongst Humanities researchers interested in applying computers within their scholarly practices, conversations surrounding the General Inquirer were characterized by ambivalence due to the tension between its potential and individual scholars' experiences of its shortcomings. Psanthas, though an advocate for the system and its manual, nevertheless concluded that

the book was unduly optimistic in proposing that the system could be applied fairly easily and quickly to a variety of problems....Unless the investigator is willing to make a long-term commitment to research with computers, the use of the General Inquirer system is not to be recommended at this time....For those who wish to focus their research on computer program development for natural language processing, the General Inquirer already provides a reality; for those who wish substantive results by applying a \"ready-made\" system, caveat emptor. (Psanthas 174)

 

Sources

Kadushin, Charles, Joseph Lovett and James D. Merriman. \"Literary Analysis with the Aid of the Computer.\" Computers and the Humanities 2.4 (1968): 177-202. Web.

Oakman, Robert L. Computer Methods for Literary Research. Athens: University of Georgia Press, 1984.

Psathas, George. \"The General Inquirer: Useful or Not?\" Computers and the Humanities 3.3 (1969): 163-174. Web.

Rice, Sandra. \"Software Review: INQUIRER III (Edinburgh Edition) A Computer Based System for Content Analysis.\" Computers and the Humanities 10.6 (1976): 332. Web.

Stone, Philip J., Dexter C. Dunphy, Marshall S. Smith and Daniel M. Ogilvie. The General Inquirer: A Computer Approach to Content Analysis. Cambridge: MIT Press, 1966.

Rating
Suggested Tools