Evidence and Scarcity

As dig­i­tal tech­nol­ogy expands, our whole rela­tion­ship to pre­sent­ing evi­dence will have to change.

Other dis­ci­plines roll their eyes at his­to­ri­ans for hav­ing  too many exam­ples. A lit scholar might read a his­tor­i­cal arti­cle and think “he had me at five: the forty five exam­ples that fol­lowed  didn’t accom­plish a damn thing.” And even his­tory grad stu­dents learn to skim the long train of exam­ples, unless they’re directly related to their research. It’s a legacy of his­tory as a sci­ence: enough exam­ples proves repeata­bil­ity and the­o­ret­i­cally estab­lishes truth.

Set that aside for a moment, though, and think about the other rea­son why we needed to present the exam­ples: scarcity. Just as the struc­ture of aca­d­e­mic paper pub­lish­ing grew from the scarcity of print, so too the struc­ture of evi­dence pre­sen­ta­tion grew from the scarcity of time and access.

Print­ing a jour­nal or book was/is expen­sive. Even today paper is expen­sive, print­ing is expen­sive, bind­ing and ship­ping are expen­sive; they were even more so in 1870, when the pro­fes­sion got started. Peer review helped con­serve this scarce media, by mak­ing sure that resources–physical resources,  but also resources of time and attention–only went to vet­ted work. Print­ing was a way to dis­ci­pline scarcity.

The historian’s march of exam­ples also rep­re­sented a rela­tion­ship to scarcity–scarcity of time to do the research, scarcity of mate­r­ial, and scarcity of access. Good evi­dence was/is often hard to find, some­times located only in an archive, and some­times buried deep within other sources. The his­to­rian needed pro­fes­sional cre­den­tials to get into the archive, and he or she needed the time required to do the work. Archives had lim­ited work­ing hours. Both were rel­a­tively scarce, and the parade of exam­ples demon­strated the historian’s tri­umph over scarcity.

That scarcity is less and less of a prob­lem. When I wrote my dis­ser­ta­tion, on the inven­tion of stan­dard time, I spent about 9 months look­ing at every Amer­i­can news­pa­per extant for the time period Sep­tem­ber through Decem­ber 1883. Sit­ting in front of a micro­film reader for long hours every day, turn­ing the han­dle, feel­ing slightly sick, stop­ping to make a note­card when I found some­thing use­ful. The New York Times index helped, but oth­er­wise, I was hunt­ing for ref­er­ences to “time,” “stan­dard time,” “rail­roads,” etc. by skim­ming the newspapers.

Today, most of those news­pa­pers are dig­i­tized and word search­able. A search that took months takes ten min­utes. The scarcity is gone. Over a decade ago, at the begin­ning of my research for Face Value, I looked for texts about money, and searched them for metaphors that used racial or genetic terms. Today, most of those same texts are word searcheable: I can find the same results almost instantly across a broad range of texts.

Google has dig­i­tized I believe more than ten mil­lion books. In my field, 19th cen­tury US, that means that nearly every pub­lished book is avail­able instantly on  line, fully searcheable 24/7. Scarcity is gone.

So one of the things the pro­fes­sion rec­og­nized and rewarded–the sheer effort required to plod through scarce resources–is now out­dated. And while of course not all archives are online, and some may never be, who can doubt that arc points toward ever increased digitization?

If the profession’s byways and prac­tices emerged in an era of tex­tual and evi­den­tiary scarcity, how will we remake it to fit an era of evi­den­tiary abun­dance? Should we keep act­ing as if  access to old books is lim­ited to peo­ple with Uni­ver­sity library cards? Do we really need to keep parad­ing the fruit of hours in the library, when any casual user can find the same results in a few seconds?

Clearly, an aca­d­e­mic his­to­rian still has things that can’t be casu­ally acquired–context, depth of knowl­edge, a com­par­a­tive frame­work. These things are still essen­tial. And just as clearly, some kinds of research, some kinds of ques­tions, fore­ground the end of scarcity more than others.

I’m not sure exactly what this change in the axis of scarcity means. It’s par­tic­u­larly acute in the kind of his­tory I do, cul­tural his­tory inflected by ‘post­mod­ernism.” Per­haps in  the future, rather than trot­ting out 40 exam­ples, we will sim­ply describe the “infor­ma­tion archi­tec­ture” of our searches. Thought­ful prac­ti­tion­ers of his­tory quickly real­ize how the kinds of ques­tions they ask deter­mine the range of pos­si­ble out­comes. All his­tor­i­cal inves­ti­ga­tions depend on a scaf­fold­ing of assump­tions and pre0questions, “met­ta­nar­ra­tives” that we assume in order to make our inves­ti­ga­tions cohere.

Per­haps future his­tor­i­cal inquiries will spend more time describ­ing the search terms used, and less fur­nish­ing results. Describ­ing the terms one searches for: “gold” and “negro” and “amal­ga­ma­tion,” within 50 words of each other; or “time,” “rail­road” and “stan­dard­ized” on the same page, not only pro­duces cer­tain results: it reveals the struc­ture of historian’s inquiry, and invites oth­ers to use it or mod­ify it.

The Cen­ter for His­tory and New Media at GMU is work­ing on new ways of “min­ing” text. We are all more and more used to “dumb” word searches that pro­duce huge results. How might we refine and rethink the  dig­i­tal search process, to make it more use­ful for historians?

Imag­ine search­ing a range of documents–the col­lected writ­ings of Char­lotte Perkins Gilman, say–and being able to get a list of all the adjec­tives she used, bro­ken out by fre­quency. Or search­ing for a term like “equal­ity” and see­ing the results ranked by the words most fre­quently pared with “equal­ity?” Or hav­ing the results returned as a tag cloud, with syn­onyms and antonyms dis­played in dif­fer­ent col­ors and sizes, by fre­quency. The limit here is our imag­i­na­tion, not tech­no­log­i­cal possibility.

So it seems to me that it’s now essen­tial to rethink how his­tory will look. The things that drove the forms and modes of schol­arly work–the scarcity of time, mate­ri­als and access–are less and less present. Elec­tric­ity was once a scarce resource. When it became cheap and abun­dant, it changed every­thing about the way we lived. Now it’s threat­en­ing to become more dear, and so we are hav­ing to rethink what it does for us and how we use it. It’s time to rethink how we use evi­dence in an age of evi­den­tiary abundance.


My col­league Sean Takats made an excel­lent response to this post, point­ing out that the abun­dance of evi­dence might only cause his­to­ri­ans to “dou­ble down” on quan­tity. I fear he might be right. It increases my sense that we have an oblig­a­tion to think more about the “infor­ma­tion archi­tec­ture” of our ques­tions. No amount of evi­dence can con­ceal the banal­ity of a weak premise.

Comments are disabled for this post