Unique identifiers: the key to data interoperability

Unique identifiers solve problems. In the course of building Ultimate Consequneces, an interactive database of lives lost in Bolivian political conflict, I’ve had to coordinate the data I’m generating from a number of other sources: Census tables, data on Indigenous identification, shapefiles that show the size, location, and outlines of municipalities, geographic datasets mapping every rural community in the country.

Essentially each of these forms of data are big data tables whose rows are places. And I need to make them work with my own data table of deaths, each row of which contains a set of place information, laid out across six variables like this:

Now if I want to create a municipal map like this one, I have to count the number of deaths in each municipality and also figure out which item on the map shares that name. But nowhere in the world is there a consistent unambiguous spelling for all places. And in Bolivia in particular names have shifted dramatically, as the country’s municipal map was radically decentralized, adding three hundred new municipalities between 1994 and 2010. Some municipalities have multiple names, but each data source either only lists one, or occasionally both or all three.

Now I can just use the lookup table to add an INE code to any dataset, with a command add_id_for_municipality() that I just throw into my workflow:

locations$municipality_centroid <- lookup$municipality %>%
  add_id_for_municipality() %>%
  left_join(select(gadm_muni_centroids, cod.mun, lon, lat),
            by = c("id_muni" = "cod.mun"))

Wikidata: A global index of unique identifiers

Put most simply, Wikidata as an open knowledge graph that anyone can edit and approve. Where Wikipedia primarily hosts article, Wikidata is filled with structured data, something familiar to most people only through the infoboxes at the upper right of Wikipedia pages and the similar tables of properties served alongside Google searches (which have become much less common there recently, supplanted by paragraph-long AI summaries). A popular single-domain knowledge graph is the Internet Movie Database

Arguably what Wikidata is best use for is an index. Started as a way of aligning the various Wikipedia article with one another across all of the online encyclopedia editions, Wikidata has become a massive respository of entities and their properties, stored in a structured data format whose basic elements are items and their properties.

Each item is a single entity, typically a noun and prototypically one element in a large conceptual set, whether an administrative or a species or human being or a work of art. And such categories are merely the root values of long conceptual chains of categories. For example, New York City (assigned the sequential code Q60) is an instance of a city (Q515), which is a subclass of a human settlement (Q486972), through several intermediate categories (city or town, urban settlement). Embedded in this chain of connections is a complex conceptual answer about what New York City is. And this is only one of three chains extending upwards from New York City’s Q60 page, which also lead through “municipality in the United States” (Q3327870) to “administrative territorial entity of a single country” (Q12076836) ultimately to “type of region” (Q137022846).

These are kinds of descriptions that librarians, archivists, encyclopedia designers, and computer programmers might all love. Given a well-organized system, you can do a lot with such indices. And if nothing else, Wikidata has become a global repository for this kind of index. Once the meeting place just for Wikipedia entries, it now archives hundreds (perhaps thousands of such indices).

One might know a lot about New York City (my former home of seven years) and only encounter a few such indices for the city in the course of a year: its six area codes and a couple dozen ZIP codes, its three major airport AITA codes (none of which directly link to Q60 since they are all describe smaller entities within it). You might come across its top-level Internet domain .nyc, which is on Q60. But there you’ll also find scores of other index ID’s: codes used to identify the city in library catalogs, online encyclopedias, even as a sub-Reddit.

Where this really matters is in research applications, like genomics or data science, where knowing which precise entity is being named and measured across different sources is absolutely crucial. In any one source, we use unique identifier codes to tell things apart. Wikidata makes it possible to link those codes together and thereby to combine different forms of data without going line-by-line through the datasets to find matches. (Possible, but not easy, it must be said.)

Map of unmatched facilities

Map of unmatched facilities in FY25

map_data <- peak_fy25 |>
  filter(peak_pop > 0, !is.na(latitude), !is.na(longitude))

map_sf <- st_as_sf(map_data, coords = c("longitude", "latitude"), crs = 4326)

type_colors <- c(
  "Dedicated"     = "#e41a1c",
  "Non-Dedicated" = "#377eb8",
  "Federal"       = "#4daf4a",
  "Hold/Staging"  = "#ff7f00",
  "Family/Youth"  = "#984ea3",
  "Medical"       = "#a65628",
  "Other/Unknown" = "#999999",
  "Unclassified"  = "#333333"
)

map_sf$type_grouped <- factor(map_sf$type_grouped, levels = names(type_colors))
present_levels <- levels(droplevels(map_sf$type_grouped))
pal <- colorFactor(palette = type_colors[present_levels], domain = present_levels)

# sqrt scale for area-proportional circle sizing
radii <- scales::rescale(sqrt(map_sf$peak_pop), to = c(3, 20))

map_sf$peak_class <- factor(map_sf$peak_class, levels = c("Peak < 2", "Peak ≥ 2"))

leaflet(map_sf) |>
  fitBounds(lng1 = -125, lat1 = 17.5, lng2 = -65, lat2 = 49.5) |>
  addProviderTiles("CartoDB.Positron") |>

  # Layer 1: Peak < 2  (small solid points, no border)
  addCircleMarkers(
    data        = subset(map_sf, peak_class == "Peak < 2"),
    radius      = 3,                # small fixed dot
    stroke      = FALSE,            # <- no outer border
    fillColor   = ~pal(type_grouped),
    fillOpacity = 1,                # solid
    label       = ~paste0(detention_facility, " (", state, ": peak ", peak_pop, ")"),
    popup       = ~paste0(
      "<b>", detention_facility, "</b><br>",
      state, " | ", type_grouped, "<br>",
      "Peak: ", peak_pop, " (", peak_date, ")<br>",
      "Mean ADP: ", mean_pop
    ),
    group = "Peak < 2"
  ) |>

  # Layer 2: Peak ≥ 2
  addCircleMarkers(
    data        = subset(map_sf, peak_class == "Peak ≥ 2"),
    radius      = radii[map_sf$peak_class == "Peak ≥ 2"],
    color       = "black",
    weight      = 1,
    fillColor   = ~pal(type_grouped),
    fillOpacity = 0.7,
    label       = ~paste0(detention_facility, " (", state, ": peak ", peak_pop, ")"),
    popup       = ~paste0(
      "<b>", detention_facility, "</b><br>",
      state, " | ", type_grouped, "<br>",
      "Peak: ", peak_pop, " (", peak_date, ")<br>",
      "Mean ADP: ", mean_pop
    ),
    group = "Peak ≥ 2"
  ) |>

  # Keep your facility-type legend
  addLegend(
    pal = pal, values = ~type_grouped,
    title = "Facility type", position = "bottomright"
  ) |>

  # Add the layer toggle
  addLayersControl(
    overlayGroups = c("Peak < 2", "Peak ≥ 2"),
    options = layersControlOptions(collapsed = FALSE)
  ) |>
  addSearchFeatures(
    targetGroups = c("Peak < 2", "Peak ≥ 2"),
    options = searchFeaturesOptions(
      zoom = 9,
      openPopup = TRUE,
      position = "topleft",
      hideMarkerOnCollapse = TRUE
    )) |>
  addControl(
    position = "bottomright",
    html = htmltools::tags$style(htmltools::HTML("
      /* Force legend text left in Quarto/Bootstrap contexts */
      .leaflet-container .leaflet-control {
        text-align: left !important;
      }
      .leaflet-container .leaflet-control .legend {
        text-align: left !important;
      }

      /* Make the color swatch and label behave like a left-aligned row */
      .leaflet-container .leaflet-control .legend i {
        display: inline-block !important;
        width: 14px !important;
        height: 14px !important;
        margin: 0 8px 0 0 !important;
        vertical-align: middle !important;
      }
      .leaflet-container .leaflet-control .legend span {
        display: inline !important;
        vertical-align: middle !important;
      }
    ")))

Verifying Wikipedia in 2025

When I took my first critical look at the text, I saw a lot of very detailed, but weakly sourced text, sometimes with out any sources and sometimes with a single parenthetical citation. These references to articles and books published decades ago were once difficult for Wikipedians to verify,but not anymore.

Much more of the academic workflow has become digital since Wikipedia started. With the Internet Archive and my university library access, I could read almost all of the sources online and match up citations with sources side-by-side. Sometimes I did this to fill in where exactly the author said what is claimed on Wikipedia.

The good news is that these tools make verifying poorly Wikipedia text much easier. One can find each of the criticisms from Derek Freeman described in this paragraph in his book criticizing Mead’s work. And then cite them individually, producing a verifiable paragraph (for reasons of length and balance, I put this level of detail in the Coming of Age in Samoa article.)

While most of this work was just attaching clear sourcing to good text, it’s also a chance to improve the article, and sometimes (as I did here) to find the dramatic quote that brings the whole argument into focus. I plan to incorporate this kind of hard verification into a new assignment for my courses that edit Wikipedia, getting students familiar with checking up on and improving sources.

And sometimes, more often than I expected, the works cited on Wikipedia failed to verify the facts claimed. For example, there’s this text on Wikipedia:

Deborah Gewertz (1981) studied the Chambri (called Tchambuli by Mead) in 1974-1975, and found no evidence of such gender roles. Gewertz states that as far back in history as there is evidence (1850’s) Chambri men dominated over the women, controlled their produce and made all important political decisions.

Gewertz said Chambri women weren’t dominant in the 1970s, but she certainly didn’t reject Mead’s ethnography on this issue, writing instead (in Gewertz 1981): “I believe that the relative ‘dominance’ of Chambri women during 1933-or perhaps more accurately, the reduction of symmetrical competition between Chambri men-reflects a temporary shift in the balance.” Elsewhere she wrote, “We argue in Errington and Gewertz, 1987a, that … the Chambri never developed a male-oriented military organization comparable to that of the Iatmul; relations between Chambri men and women were, therefore, much more egalitarian than between Iatmul men and women.”

Similarly, Wikipedia claimed:

Mead’s Sex and Temperament in Three Primitive Societies became influential within the feminist movement since it claimed that females are dominant in the Tchambuli (now spelled Chambri) Lake region of the Sepik basin of Papua New Guinea (in the western Pacific) without causing any special problems.

This was the first sentence introducing Mead’s book. While not the worst summary of one of Mead’s particular claims, it totally misunderstands the relevance of the book to feminism. It wasn’t the reported female dominance among the Tchambuli that led to Sex and Temperament’s role in the women’s movement, but rather the book’s explanation of the sex-gender distinction. But it’s hard to counter a statement like that one without some detailed knowledge of both Mead and her social impact. This is something we fortunately have for Mead in high-quality sources like Nancy Lutkehaus’s Margaret Mead: The Making of an American Icon.

Deep verification of the article uncovered multiple problems. Text attributed to Mead wasn’t hers, facts cited to Gewertz and to Mead weren’t in those texts. And the summaries of Friedan and Bamberger confused facts or didn’t convey their full meaning. Perhaps just as important, finding a narrow fact in context sometimes revealed that isolating the fact was a bad idea.

How did we get here?

Here’s a list of some plain-text segments from the Margaret Mead article (in October 2025, before I started tinkering with it). Some are problematic in themselves, others index sections that I was interested in changing or balancing out. By implementing Wikiblame in R, I could draw up this table of who exactly added this text in the first place.

By discovering when the first three sentences were put onto Wikipedia, I unearthed an early moment in the encyclopedia page when it functioned as basically an online debate over Margaret Mead’s legacy.

First an anonymous user (known only by their IP address) made this addition in September 2004:

Another extremely influential book by Mead was “Sex and Temperament in Three Primitive Societies.” This became a major cornerstone of the women’s liberation movement, since it claimed that females are dominant in the Tchambuli tribes of the South Sea islands, without causing any special problems. However, looking up Tchambuli (sometimes spelled Chambri or Chimbu) in the Encyclopedia Britannica and other such modern references, the reader finds that later anthropological studies do not confirm this, and that males are dominant in Melanesia, except for the belief that some female witches have special taboo powers. (See also Gender Relations in Melanesian Culture, in the Encyc. Britannica.) Also, Mead claimed that the Arapesh people were pacifists, but later studies did not confirm this.

Then Arnold Perey clapped back in a two-part contribution in April 2005:

Neither contribution would meet Wikipedia’s ultimate standards, some of which were still under development when these sections were added to the encyclopedia. The anonymous IP contribution is argumentative (contra Wikipedia’s Neutral Point of View policy) and it begins with a flat-out falsehood: the women’s movement had not in fact taken a New Guinea matriarchy as its cornerstone. Arnold Perey’s response is also argumentative, but it embeds a long attempt at neutral description, initially properly cited to his Livejournal blog, Anthropological Anecdotes, which is impressively still online. He wrote the posts in question on the same day he contributed to Wikipedia.

Perey’s lecture notes from Mead-turned-blog post-turned-Wikipedia contributions weren’t a bad summary. In fact, their opening paragraph reads pretty close to what Mead herself says in her 1950 preface. But they end up standing in the way of readers finding out what Mead actually said herself. A current Wikipedia contributor would be strongly cautioned not to use such a blog post (though blogs by published subject matter experts are not strictly against policy). Current Wikipedia practice would strongly caution Peney against self-citation and encourage him to use higher-quality reliable sources instead.

But by and large, these two positions stayed up through twenty years of edits and were still the core of Wikipedia’s coverage of Mead’s Sex and Temperament in 2025. (See the colored sequence of these edits here.) Instead of a description of Mead’s book, the section became a gradually widening argument between critics and defenders of Mead, exploring issues like the prevalence of male dominance in Melanesia. When editors wanted to shift this debate, they typically added more text rather than going back to verify what was already there for problems.

Why? Perhaps because this kind of back and forth is the familiar practice of online debates. Perhaps because deleting material feels more likely to spawn conflicts and eit wars. Perhaps because Wikipedians assumed good faith by prior writers and so didn’t fact check what was already there.

An Error on Wikipedia Becomes Global

One consequence, unique to Wikipedia, is that Perey’s blog quote gradually got transformed into an uncited quote, and then finally into a misquote from Sex and Temperament itself. This is a mis-quote that has now gone round the world through translations of the English Wikipedia article, as in this segment from Japanese Wikipedia.

Perey’s text also appears, properly cited, in Spanish and Portuguese Wikipedia. Pieces of his description of Mead’s argument have been scattered across the Internet.

Example: Generating Redirects

Redirects allow multiple names to link to the same encyclopedic content on Wikipedia.

We create two functions to create redirects on Wikipedia:

• create_redirect(from_title, to_title): Generates a link to create a redirect from from_title to to_title if the from_title page does not already exist.

• create_multiple_redirects_to_one(redirects_from, redirect_to): Uses create_redirect to generate multiple redirects from a list of titles to a single target title.

create_redirect <- function(from_title, to_title) {
  
  # 1. Manually define the Wikipedia redirect syntax with Hex Encoding
  # # = %23 | [ = %5B | ] = %5D | Space = %20
  prefix  <- "%23REDIRECT%20%5B%5B"
  suffix  <- "%5D%5D"
  
  # 2. Encode the destination title for use inside the URL
  # We use URLencode with reserved = TRUE to catch spaces and special chars
  encoded_destination <- utils::URLencode(to_title, reserved = TRUE)
  
  # 3. Assemble the preload text and the edit summary
  preload_content <- paste0(prefix, encoded_destination, suffix)
  edit_summary    <- paste0("Redirecting%20to%20%5B%5B", encoded_destination, "%5D%5D")
  
  # 4. Check if the page exists using your existing logic
  wikipedia_pages <- page_info("en", "wikipedia", page = from_title)$query$pages
  
  if (names(wikipedia_pages) == "-1") {
    # Construct the raw URL targeting index.php
    # We use from_title for the 'title' param so we are creating the new page
    base_url <- "https://en.wikipedia.org/w/index.php"
    auto_fill_url <- paste0(
      base_url, 
      "?title=", utils::URLencode(from_title, reserved = TRUE),
      "&action=edit",
      "&preloadtext=", preload_content,
      "&summary=", edit_summary
    )

    # 5. Return as a raw HTML link to protect the %23 from Quarto's renderer
    link_html <- paste0(
      '<a href="', auto_fill_url, '" target="_blank" ',
      'style="color: #0056b3; text-decoration: underline; font-weight: bold;">',
      '🚀 Create redirect: ', from_title, ' → ', to_title, '</a> ',
      '<br>Add this text:'
    )
    
    # We also keep the copy-paste box as a fallback
    redirect_text_plain <- paste0("#REDIRECT [[", to_title, "]]")
    copy_box <- paste0(
      '<div style="display: flex; align-items: center; width: 80%; border: 1px solid #dee2e6; border-radius: 4px; padding: 5px; background-color: #f8f9fa; margin: 10px 0;">',
      '<code style="flex-grow: 1; overflow: hidden; text-overflow: ellipsis; white-space: nowrap;">', redirect_text_plain, '</code>',
      '<button onclick="navigator.clipboard.writeText(\'', redirect_text_plain, '\'); this.innerText=\'✅\'; setTimeout(()=>this.innerText=\'📋\', 1000)" ',
      'style="border:none; background:none; cursor:pointer; padding-left: 10px;">📋</button>',
      '</div>'
    )

    return(paste0(link_html, "<br>", copy_box))
    
  } else {
    wikipedia_url <- wikipedia_pages[[1]]$fullurl
    return(paste0("Page [", from_title, "](", wikipedia_url, ") already exists."))
  }
}

create_multiple_redirects_to_one <- function(redirects_from, redirect_to) {
  if (length(redirect_to) != 1) {
    stop("This function needs one redirect destination.")
  }
  if (assertthat::is.string(redirects_from)) {
    return(create_redirect(redirects_from, redirect_to))
  } else {
    redirects_md <- sapply(redirects_from, function(from) {
      create_redirect(from, redirect_to)})
  }
  
  return(paste(glue("## Redirects to '{redirect_to}'\n\n"), 
               paste(redirects_md, collapse = "\n\n---\n\n")))
}

Here are three sample uses of the create_multiple_redirects_to_one function. The last two are practical uses, coming from the corresponding pages: Hickory Ground and Occaneechi people. The latter ultimately comes from the name lists (or synonymy) generated by James Mooney as shown in the margin.

create_multiple_redirects_to_one(
  c("Alpha", "Beta", "Gamma", "Delta"),
  "Greek Letters")

# A Native American settlement known by multiple names
create_multiple_redirects_to_one( c("Otciapofa", "Odshiapofa", "Ocheopofau",
                                    "Ocheubofau", "Oce Vpofa"), "Hickory Ground")

# A Native American people referenced by different names in the historical record
create_multiple_redirects_to_one( c("Achonechy", "Aconechos", "Akenatzy",
                "Hockinechy", "Occaneches", "Occaanechy", "Occhonechee",
                "Occonacheans", "Occoneechee", "Ockanechees", "Ockanigee",
                "Okenechee", "Acconeechy", "Occaneeches", "Ochineeches", "Ockinagee"),
                "Occaneechi")

Where is the Wikipedia community on AI?

Heather Ford formally researched Wikipedia community concerns about generative AI and gave this talk presenting this paper in late 2025. Here’s her summary of community concerns:

Does the potential use of generative Al threaten knowledge integrity on Wikipedia? If so, how?

• Reliability, trustworthiness, accuracy: proliferation of Al-generated misinformation, hallucinations and biased content because of the difficulty in fact-checking and verifying Al outputs.
• Equality: uneven distribution of risk because genAl tools are not available for all languages and because low editor numbers in smaller versions could tip the balance away from human control over curation, especially among non-English speakers.
• Exploitation: the commercial exploitation of open content and editor labour by genAl companies. - Sustainability: the long-term sustainability of Wikipedia with the potential for people to turn increase in Al for their information needs.
• Verifiability: when the link between a claim and its source is broken, readers are no longer able the veracity of a claim or participate in the process of knowledge curation leading information pollution on Wikipedia and the degradation of the broader information environment, or even model collapse.

In the in-person Wikimedia community, I think that opinions continue to vary widely, with both a heavy interest in experimentation and a desire to protect the encyclopedia. (I’m basing this on WikiConference North America.) WMF continues to demo or profile its AI-powered improvement experiments, such as “semantic search” that would allow visitors to ask questions in the Wikipedia search bar. And a lightning talk promoted a “release the bots” perspective that describes generative tools as powerful tools.

On English Wikipedia, discussion of LLM edits had deadlocked a couple of years ago: This proposed policy was never approved, but kept as an essay. Still, the community seems to be gradually moving towards a consensus of specific limitations on adding unfiltered LLM content to the encyclopedia. These online discussions end up being lengthy and multi-sided, with one axis of contention pitting “defend the rules, whether AI or not” versus “limit (or even ban) mass LLM content.” This is complicated by the cross-cutting issue of smaller use cases for AI, where AI-inclusionists tend to use the prospect of forbidding AI grammar checks as a straw man, while AI exclusionists struggle to find a minimum set of off-limits use cases that can reach consensus. A series of hard limits in particular areas has, however, reached consensus: WP:LLM forbids the creation of new article with generative AI; new articles with clear AI artifacts may be speedily deleted; WP:AITALK limits the use of LLMs in formal talk page debates and allows for new topics to be closed if they weren’t opened by a human willing to articulate their ideas for themselves (but reopened once an initial user does so). A very long RFC is underway on a somewhat stronger LLM edit guideline, which seems likely to be modified once more before it’s approved.

WikiProject AI Clean-Up and the essay at WP:LLM have become a central repositories for demonstrations of valuable LLM-assisted editing, the policy discussions on Wikipedia. One impressive recent demonstration appeared on Wikipedia Signpost: user Tilman Bayer prompted GPT-5 Thinking to find errors in every daily featured article and it suggested 56 errors over a month, 38 of which Bayer confirmed.

Persistent critical voices:

• Emily Bender (Wikipedia | Google Scholar), a linguist who entered the Generative AI discussion as an expert on large-language models. Co-author of “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜” research paper with Timnit Gebru. Bender’s take is that hype is driving the AI discussion, and that there are substantive ethical problems with current models. More than other researchers, Bender inclines towards detailed collaborative critiques on questions like: Can AI’s that pass human benchmark tests be said to have capacities they test for? What are the dangers of using AI for search? Bender has a new book out in 2025, The AI Con,⁴ making a practical case that LLMs are being oversold and an ethical case they aren’t building the future we want.
• Gary Marcus (Wikipedia Substack blog) — researcher and proponent of a structured approach to building artificial intelligence, but who thinks that hallucinations and failure to understand edge cases are invariable traits of neural network architectures. Marcus has emerged as a sharp critic of the current generative AI wave as unlikely to every generate sophisticated intelligence and a potential diversion of hundreds of billions of dollars and societal distrust in an unproductive direction. Marcus published his views in Taming Silicon Valley (2024).⁵
• Ed Zitron — Technology critic and writer. Focused on declining tech user experience and risks of a financial bubble around generative AI. His general take is that generative AI has yet to show a vital use case and could dramatically underperform market expectations.
• Jaron Lanier, whose You Are Not a Gadget and Who Owns the Future? raise important questions about power and economy of technology, while accepting that technology will solve whatever problems it creates. Who Owns the Future also provides a smart typology of perspectives on technology.
• Cory Doctorow, futurist sci-fi writer with a hard economic critique of Big Tech in Chokepoint Capitalism.
• Shoshana Zuboff. It’s worth taking a moment and imagine AI chatbots and on-the-horizon agents, not as potentially intelligent machines, but rather as data collectors to the corporations that she thoroughly laid out in The Age of Surveillance Capitalism.

Visions of AI entrepreneurs and business leaders

• Mustafa Suleyman, CEO of Microsoft AI, His talk, “What Is an AI Anyway?,” characterizes AI as “a new digital species”

• Dario Amodei, CEO of Anthropic. February interview on Hard Fork. November appearance on NYT DealBook (video). In essence, Amodei who has prophesied the near-term arrival of AGI continues to believe that scaling up LLMs is the way to get there.

• Sam Altman, CEO of OpenAI: Instead of listening to Sam Altman, let me suggest reading one of the two up-close books on Altman and OpenAI’s rise published this year: Karen Hao’s Empire of AI⁶ which describes the AI industry as not just about its product, but about the acquisition of creative labor (through training data) and coordination of human labor and energy (through its overseas workforce and data centers) in ways that echo colonial enterprises. Keach Hagey’s The Optimist: Sam Altman, OpenAI, and the Race to Invent the Future⁷ is more, well, optimistic. As a comparative review put it, Hagey “stands Altman as the secular prophet preaching human progress and boundless optimism.”

  This interview quote from Karen Hao on Altman is informative, suggesting that his ability to shift narratives makes him something of an unreliable narrator on even his own views:

  “He’s a once-in-a-generation fundraising talent. That is his particular skill. And he’s also a once-in-a-generation storytelling talent, which is effectively why he’s so good at fundraising. He is able to paint these extremely persuasive visions of the future. He was already prominent within Silicon Valley, and Silicon Valley very much runs on stories and telling stories about the future. And one of the things that I sort of concluded through the reporting of my book is that when he says something to someone, what he’s saying is more tightly correlated with what he thinks they need to hear than what he actually believes or the ground truth of the thing. He’s able to say the things that really provoke people to kind of rally towards a general, broad, sweeping mission that he paints.⁸

A suprising, one could also say disturbing, share of AI visions are wrapped up in some peculiar ideas about the future that Timnit Gebru and Émile P. Torres have christened the TESCREAL bundle.⁹ (Gebru is a Google engineer fired for her critical views. Torres is a philosopher who was once a transhumanist.)

The bundle consists of: “Transhumanism, Extropianism, Singularitarianism, (modern) Cosmism, Rationalists (the internet community), Effective Altruism, and Longtermism” (as summarized and linked in Wikipedia). In essence, all of these ideologies envision a future “beyond humanity” itself, though with different variations. Many argue that cultivating a distant sci-fi future into existence is a supreme ethical goal, outweighing say the ecological survival of planet Earth or the near-term wellbeing of most humans.

If you want a readable summary of these ideas and their influence, More Everything Forever¹⁰ is your guided tour. Torres and comedian Kate Willett’s podcast Dystopia Now (Apple | Spotify) is a regular look at this world.

Theoretical Influences

Rubin draws on a truly remarkable number of theoretical referents, making “The Traffic in Women” into something of a theoretical crossroads. On the one hand, we have been tracing our way along many of these pathways, making us especially well-informed readers of this piece. On the other hand, there will be other threads that are unfamiliar. Don’t worry about just trusting Rubin’s account of these sources.

Nevertheless, if you want to follow some of these pathways yourself, here are some resources.

On Boundaries: Purity and Danger

Mary Douglas’s Purity and Danger: An Analysis of Concepts of Pollution and Taboo⁹ stands as a historical landmark in the anthropology of religion.

It is only by exaggerating the difference between within and without,
above and below,
male and female,
with and against,
that a semblance of order is created.

Her chapter on magic and religion confronts and challenges the line of anthropological scholarship on “primitive religion” going back to James Frazer’s The Golden Bough.

Structures of Kinship

In Elementary Structures of Kinship,¹¹ Claude Lévi-Strauss proposes a structuralist explanation for the circulation of spouses mentioned in Mauss’s The Gift. The core insight of the book is to unite the near-universal taboo on incest (in some form) with the complex kinship structures that anthropologists had been mapping since at least Lewis Henry Morgan. (Like Douglas, he rejects the functionalist explanation that incest taboos prevent interbreeding.)

Lévi-Strauss argues that by separating marriageable from un-marriageable individuals, kinship systems generate the necessary boundaries for lineages that are the collective social actors which exchange spouses—glossed in his account, as in Mauss’s brief phrase, exclusively as women.

Gayle Rubin will build upon Elementary Structures of Kinship in her pivotal essay “The Traffic in Women,”¹² which theorizes kinship and gender systems as integrated whole, subject to historical change, and as systems of power.

And if men have been sexual subjects—exchangers—and women sexual semi-objects—gifts—for much of human history, then a great many customs, clichés, and personality traits seem to make a great deal of sense (among others, the curious custom by which a father gives away a bride).

This essay proves foundational to Gender Studies. Both Rubin and Judith Butler will bring vast inquiries into gender, the body, and sexuality into dialogue with Lévi-Strauss’ account.

Malinowski’s Argonauts of the Western Pacific

Bronisław Malinowski (), a native of Krakow, received his doctorate there, only turning his attention to anthropology at that time, inspired by James Frazer’s The Golden Bough. He worked at the London School of Ecoonomics from 1910 and late in his life at Yale University.

It’s difficult to underestimate either the influence of Malinowski on the traditions of anthropological fieldwork or the broad impact of his students on mid-century social anthropology, which parallels that of Boas’ students in the United States. Among his students were E. E. Evans-Pritchard, Raymond Firth, Meyer Fortes, and Edmund Leach, each noted for their contributions to mid-century anthropological theory. Jomo Kenyatta studied social anthropology with Malinowski and crafted his essays on his Kikuyu people into the book Facing Mount Kenya (1938),¹ which also stands as an indictment of British colonial misunderstandings of Kenyan society. Kenyatta returned to his native land in 1947, led the Kenyan African Union, and became the first president of Kenya until his death in 1978.

It’s remarkable how Malinowski’s argument for theoretically informed fieldwork in Argonauts is mirrored by Vine Deloria’s polemic² against anthropologists in Native American communities who are concerned with theoretical questions rather than the practical problems of their research subjects.

In Argonauts, Malinowski also outlines a three-part vision for anthropological writing and data collection: (1) systematic data on a tribe’s organization and cultural forms; (2) elegant writing on “the imponderabilia of actual life” and behavior as recorded in ethnographic diaries; and (3) a corpus of native-language statements, myths, magical formula, and narratives. These lines of evidence converge in Malinowski’s widely cited statement of purpose for ethnography:

These three lines of approach lead to the final goal, of which an Ethnographer should never lose sight. This goal is, briefly, to grasp the native’s point of view, his relation to life, to realise his vision of his world. We have to study man, and we must study what concerns him most intimately, that is, the hold which life has on him. In each culture, the values are slightly different; people aspire after substance of their happiness—is, in my opinion, to miss the greatest reward different aims, follow different impulses, yearn after a different form of happiness. In each culture, we find different institutions in which man pursues his life-interest, different customs by which he satisfies his aspirations, different codes of law and morality which reward his virtues or punish his defections. To study the institutions, customs, and codes or to study the behaviour and mentality without the subjective desire of feeling by what these people live, of realising the which we can hope to obtain from the study of man.

Gluckman’s “Analysis of a Social Situation in Modern Zululand”

Max Gluckman () was born in Johannesburg, South Africa in 1911. Gluckman studied anthropology at the University of the Witwatersrand in Johannesburg under Winifred Hoernlé and I. Schapera. He would later describe this training as part of the “sociological stream” of anthropology, which he traced upstream to those scholar’s teacher, Alfred “A.R.” Radcliffe-Brown, and through him to Emile Durkheim.³ His doctoral work in anthropology at Oxford brought him into direct contact with Radcliffe-Brown as well as E. E. Evans-Pritchard and solidified his attachment to social anthropology as his intellectual home.

Gluckman traces his own intellectual biography in detail in an introductory essay to Order and Rebellion in Tribal Africa. After completing his doctorate, Gluckman returned to South Africa to conduct fieldwork among the Zulu people of Natal. His article written in the field, “The Kingdom of the Zulu in South Africa” considers 120 years of Zulu history from the nation’s founding by Shaka Zulu in 1818-24. The focus is within the Zulu Nation, but is on turbulent processes of political change through rebellions that nonetheless maintain the nation and its monarchy. Gluckman considers the Zulu people’s relationships to the British colonial power as part of this history.⁴

However, his article “Analysis of a Social Situation in Modern Zululand” (1940),⁵ commonly referred to as “The Bridge,” casts the largest shadow on broader anthropology, precisely because it re-conceptualizes the colonial society in South Africa as a single biracial society and because it introduced an approach to situational analysis that tacks betweeen specific events and the broader social structure. In effect, Gluckman imports the concept of segmentary society, previously applied to complex African polities to the colonial situation of racially stratified power in South Africa.

Gluckman’s approach shifts the framing of anthropology from “the Native” society to the social world that includes the anthropologist him or herself, the colonial government, missionaries, employers, and the entire complex power structure. At root is consideration of what we now call “his own subject position” and a kind of narration later termed “reflexivity.”

Some amazing context for Max Gluckman’s “The Bridge” article is offered in Robert Gordon’s “On Burning One’s Bridge: The Context of Gluckman’s Zulu Fieldwork,”⁷ to which is appended Gluckman’s unpublished note that is quoted above. Here is its suggestive abstract:

Abstract: Gluckman’s 1940 essay “An Analysis of a Social Situation in Modern Zululand” (aka as “The Bridge”) is widely acknowledged as one of the most influ- ential in social anthropology for signalling a paradigm shift. This paper publishes a chapter of a previously unpublished, undated manuscript by Gluckman – probably from 1946 – describing how he did fieldwork in Zululand. A contextualizing essay discusses why Gluckman went to Zululand, what his preparations were, how he famously got “banned,” the role of the regent and the native commissioners in orchestrating this, and how local anti-semitism also played its part. It also suggests that his “banning,” coupled to his marriage in 1939 to Mary Brignoli, a member of the communist party, were crucial factors leading to this paradigm change. Finally, the afterlife of “The Bridge” is sketched showing how dismal was its original reception among peers and colleagues.

Fieldwork by Margaret Mead

List consolidated from Library of Congress. Manuscript Division¹¹

Biography

Margaret Mead’s life is the subject of books including:

• Charles King’s Gods of the Upper Air on multiple members of the Boas circle.
• Mary Catherine Bateson’s With a Daughter’s Eye: A Memoir of Margaret Mead and Gregory Bateson on her parents.
• Paul Shankman’s The Trashing of Margaret Mead on the posthumous Freeman controversy over her Samoa fieldwork.
• Elesha Coffman’s Margaret Mead: A Twentieth-Century Faith, which makes senes of the Christianity of a woman who once wrote, “Shorn of all the things in which I can’t believe—and don’t want to—an omnipotent God, immortality, and original sin—Christianity is still the most beautiful thing I know, and the fact that Jesus lived the most satisfactory justification of life.” See also, Coffman’s summaries of her favorite Mead books.
• Benjamin Breen’s Tripping on Utopia: Margaret Mead, the Cold War, and the Troubled Birth of Psychedelic Science¹² on Mead and Bateson’s intersections with both early experiments in psychedelic use and covert experimentation organized by the US government.
• Lily King’s novel Euphoria, a very thinly pseudonymized retelling of Margaret Mead, Reo Fortune, and Gregory Bateson in Papua New Guinea, Euphoria fleshes out these young Euro-Americans rethinking gender and community in the 1930s. Substantial attention is paid to Mead and Bateson’s ill-fated theory of the Squares among cultural personalities and individuals, and its entanglement with the love triangle of these three.
• Deborah Blum’s Coming of Age: The Sexual Awakening of Margaret Mead, covering Mead’s education at Barnard/Columbia, fieldwork in Samoa, and summer in Europe, and relationships with Cressman, Benedict, Sapir, and Fortune from 1921 to 1926.
• Jean Walton’s Fair Sex, Savage Dreams: Race, Psychoanalysis, Sexual Difference devotes its last two chapters to Mead, with a lengthy interpretation of her conversation with James Baldwin in A Rap on Race.
• Nancy Lutkehaus’s Margaret Mead: The Making of an American Icon, which intertwines her look at Mead with “the study of fame … as a particular social practice” and of “the meanings associated with her that transcend or go beyond Mead.”

Historical context of the Ghost Dance

The colonization of the Great Plains through the Indian Wars is the context for the spread of the Ghost Dance religion. The Fort Laramie Treaties had recognized native sovereignty over the Great Plains in 1851, but volunteer and Federal Army troops began a new round of Indian Wars in the 1860s. Genocidal massacres, military internment, and the undermining of the Indian food supply followed in the succeeding decades. Killing buffalo as a tactic of counterinsurgency was pursued with abandon, as praised by General Philip Sheridan in 1875:

These measures constituted successive attacks on the physical and cultural mechanisms of survival of the Indians. And consciously so, as can be seen from the opinion of General William Tecumseh Sherman as early as 1867, “We must act with vindictive earnestness against the [Lakota], even to their extermination, men, women and children.”.² The impact was devastating, as Dee Brown describes for the summer of 1874,

Those who chose the path of a negotiated surrender to reservations, as did those led by Sitting Bull in 1881, saw agreements for rations and safety undermined. Just two years after that agreement, Sitting Bull—noting “our rations have been reduced to almost nothing” and “many of our people have starved to death”—made a plea for material equality as a corollary of negotiation: