Bolivia CPV 2024: Population QuickStatements for Wikidata
NoteAI-generated text, click for process details
This page was first generated using the Posit AI plug-in running Claude Sonnet 4.6, operating in my repository wiki-graph, which already contained the functions referenced here. The prompt I used was:
For each of the following tasks, build functions and embed the process in a qmd that explains the needed steps.
Before we can deploy the quickstatements below, we need to clear out the small number of P1082 values. They only appear in these ID’s: “Q628665” “Q1516972” “Q1814603” “Q1814613” “Q1814898” “Q1814929” “Q1952982”
We’re going to build a new table building off of pop_muni, with qid values added, using muni_id_lookup_table to connect them through id_muni.
Each row will drive a series of columns with quickstatements produced by add_quick_statement_column, one for each of the three total population columns. These quickstatements will add a value for P1082 (population) for the corresponding year of the census. All will be sourced to Q139668018.
The page reflects extensive manual and automated revisions to the code following a debugging process. Several sections below were added by hand.
Overview
This document adds P1082 (population) statements to Wikidata items for Bolivia’s 339 municipalities and for Bolivia’s 9 departments and 112 provinces using census counts from three surveys: 2001, 2012, and 2024 (CPV 2024).
Each statement is sourced to Q139668018 (the Wikidata item for the CPV 2024 publication) and carries a P585 (point in time) qualifier for the relevant census year.
Before adding new statements, a small number of pre-existing P1082 claims on seven municipality items must be removed. Those items were tagged during a prior data-cleaning pass; their existing values are undated or unreferenced and would otherwise create duplicates.
Setup
library(tidyverse)
── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ dplyr 1.2.0 ✔ readr 2.1.5
✔ forcats 1.0.1 ✔ stringr 1.6.0
✔ ggplot2 4.0.2 ✔ tibble 3.3.1
✔ lubridate 1.9.5 ✔ tidyr 1.3.2
✔ purrr 1.2.1
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag() masks stats::lag()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
library(readr)library(httr)library(jsonlite)
Attaching package: 'jsonlite'
The following object is masked from 'package:purrr':
flatten
# CPV 2024 publication item used as "stated in" (P248)SOURCE_QID <-"Q139668018"p585_for_year <-function(year) sprintf("+%d-01-01T00:00:00Z/9", year)# Wikidata instance-of classesBOLIVIA_QID <-"Q750"DEPARTMENT_CLASS_QID <-"Q250050"# department of BoliviaPROVINCE_CLASS_QID <-"Q1062593"# province of BoliviaMUNICIPALITY_CLASS_QID <-"Q1062710"# municipality of Bolivia
Step 1: Fetch current Wikidata municipality data
A single get_wikidata_instances() call retrieves every Bolivian municipality item, collecting both the INE code (P14142) used for QID mapping and any existing P1082 (population) claims needed to generate the removal statements in the next step.
Seven items carry one or more P1082 statements that were added before this structured import. They must be deleted first, or QuickStatements will create duplicate claims.
make_p1082_removals() extracts the current claim values from the already-fetched municipalities_wd tibble (no second API call needed) and emits one removal line per claim in QuickStatements batch syntax (-QID | P1082 | value).
#' Generate QuickStatements removal lines for all P1082 claims on a set of items.#'#' @param municipalities_wd Tibble returned by get_wikidata_instances(), which must#' include population_n, population_1 … population_N columns (numeric list#' property columns produced when numeric_list_properties = "P1082").#' @param target_qids Character vector of QIDs whose P1082 claims should be removed.#'#' @return A tibble with columns `qid`, `label_es`, `claim_slot`, `value`, and#' `removal_statement` (the QuickStatements deletion line).make_p1082_removals <-function(municipalities_wd, target_qids) { df <- municipalities_wd |>filter(qid %in% target_qids)if (nrow(df) ==0) stop("None of target_qids found in municipalities_wd.")# Identify the per-claim value columns: population_1, population_2, ...# (exclude population, population_n, population_*_year, population_*_ref) value_cols <-names(df) |>str_subset("^population_\\d+$") df |>select(qid, label_es, all_of(value_cols)) |>pivot_longer(cols =all_of(value_cols),names_to ="claim_slot",values_to ="value" ) |>filter(!is.na(value)) |>mutate(value_str =as.character(as.integer(value)),removal_statement =str_c("-", qid, " | P1082 | ", value_str) ) |>select(qid, label_es, claim_slot, value, removal_statement)}
pop_muni → muni_id_lookup_table on municipio == muni_cpv2024 and departamento == department. The department key is required because many municipality names (e.g. “San Pedro”, “Santa Rosa”) are shared across departments; omitting it inflates the row count from 340 to 369.
muni_id_lookup_table → Wikidata on id_muni == ine_code using the municipalities_wd tibble already in hand.
TIOC rows in pop_muni (e.g. “TIOC-Raqaypampa”, “TIOC-Territorio Indígena Multiétnico”) have no matching muni_cpv2024 in the lookup table and will produce NA after the join; they are dropped and reported to the console.
#' Join pop_muni to Wikidata QIDs through muni_id_lookup_table.#'#' @param pop_muni Tibble of municipality-level population rows from CPV 2024.#' @param lookup muni_id_lookup_table with columns id_muni, muni_cpv2024,#' and department.#' @param ine_qid_map Tibble with columns id_muni (INE 6-digit code) and qid.#' Must have NA rows filtered out before calling (to avoid spurious TIOC matches).#'#' @return A tibble of matched rows, one per municipality, with all pop_muni#' columns plus id_muni and qid. Unmatched rows are dropped and messaged.build_population_qs_table <-function(pop_muni, lookup, ine_qid_map) { ine_qid_map <- ine_qid_map |>filter(!is.na(id_muni)) joined <- pop_muni |>left_join( lookup |>select(id_muni, muni_cpv2024, department),by =c("municipio"="muni_cpv2024", "departamento"="department") ) |>left_join(ine_qid_map, by ="id_muni") unmatched <- joined |>filter(is.na(qid))if (nrow(unmatched) >0) {message(nrow(unmatched), " pop_muni row(s) had no QID match and will be dropped:")walk(unmatched$municipio, ~message(" - ", .x)) } joined |>filter(!is.na(qid))}
cat("Provinces with INE code: ", sum(!is.na(provinces_wd$ine_code)), "\n")
Provinces with INE code: 112
Wikidata already has some values for population. Let’s examine them. All nine departments have two values for population. Most provinces have no population data.
departments_wd <- departments_wd |>filter(!is.na(ine_code)) # filter out Litoral# One Bolivian department has not been controlled by Bolivia since the 19th century# It has no INE code.departments_wd |>count(population_n)
# A tibble: 1 × 2
population_n n
<int> <int>
1 2 9
provinces_wd |>count(population_n)
# A tibble: 3 × 2
population_n n
<int> <int>
1 0 107
2 1 4
3 2 1
# A tibble: 5 × 7
qid label_en ine_code population_1 population_1_year population_2
<chr> <chr> <chr> <dbl> <int> <dbl>
1 Q389311 Nor Yungas Prov… 0214 36983 NA NA
2 Q638697 Belisario Boeto… 0108 12277 NA 11161
3 Q1355903 Franz Tamayo Pr… 0207 26997 NA NA
4 Q1366004 Capinota Provin… 0307 26725 NA NA
5 Q1420274 Poopó Province 0406 14984 NA NA
# ℹ 1 more variable: population_2_year <int>
Looking at the results we see our task: We will have to double-check departmental population values against our new sources and add infromation for 2001. We can clear the five undated data points for province populations.
Step 5: Build province/department population tables via INE codes
The CPV 2024 population extracts already include department/province names. We add INE codes as follows:
Departments: take the first two digits of row_num-independent codes by grouping on departamento and joining to Wikidata via department INE code. (In practice pop_dept already has one row per department.)
Provinces: take the first four digits of the 6-digit municipality code. Because pop_prov does not contain a province code, we compute it from the department/province order using the INE code on Wikidata: we join on (departamento, provincia) after extracting a clean province label from WD.
Note: Wikidata province labels are not perfectly standardized (e.g. multiple “Cercado” provinces). We therefore match primarily by INE code (P14142). Where that is missing, we fall back to a conservative name+department join and report unmatched rows.
# pop_dept has one row per department; create dept INE code = 2 digits.# We derive it from the province/municipality-style codes when available, but# for CPV 2024 tables we only need the mapping through Wikidata.# Here we join by department Spanish label as a fallback if needed.# Prefer matching by INE code if pop_dept already has it (it currently doesn't),# otherwise match by department name against WD Spanish label without prefix.wd_dept_name <- departments_wd |>transmute( qid, ine_code,dept_name = label_es |>str_remove("^Departamento de(l )?\\s*") |>str_replace("Potosí", "Potosi") )# pop_dept_qs <- pop_dept |># left_join(wd_dept_name, join_by(id_dep==ine_code)) |> # relocate(id_dep, departamento, qid)pop_dept_qs <- pop_dept |>left_join(departments_wd, join_by(id_dep==ine_code)) |>relocate(id_dep, departamento, qid)unmatched <- pop_dept_qs |>filter(is.na(qid))if (nrow(unmatched) >0) {message(nrow(unmatched), " department row(s) had no QID match:")walk(unmatched$departamento, ~message(" - ", .x))}pop_dept_qs <- pop_dept_qs |>filter(!is.na(qid))
# Province mapping strategy:# 1) If Wikidata has INE code (P14142) for a province item, use it.# 2) Compute province code by joining pop_prov to a lookup built from WD labels.wd_prov_lookup <- provinces_wd |>transmute( qid, ine_code,department = label_es |>str_extract("\\(([^)]+)\\)$") |>str_remove_all("[()]") |>str_replace("Potosí", "Potosi"),province = label_es |>str_remove("\\s*\\([^)]+\\)$") |>str_remove("^Provincia de\\s+") |>str_remove("^Cercado$") )# Some province items may not have "(Department)" disambiguation.# Fall back to located-in labels when present would be better, but this QMD# keeps it simple: we primarily rely on INE codes being present.pop_prov_qs <- pop_prov |>mutate(department = departamento |>str_trim() |>str_replace("Potosí", "Potosi"),province = provincia |>str_trim() ) |># join on (province, department) to recover province QIDleft_join(wd_prov_lookup |>select(qid, province, department, ine_code),by =c("province"="province", "department"="department"))pop_prov_qs <- pop_prov |>left_join(provinces_wd, join_by(ine_code)) |>relocate(ine_code, provincia, qid)unmatched_prov <- pop_prov_qs |>filter(is.na(qid))if (nrow(unmatched_prov) >0) {message(nrow(unmatched_prov), " province row(s) had no QID match and will be dropped (inspect manually):")walk(head(unmatched_prov$provincia, 50), ~message(" - ", .x))}pop_prov_qs <- pop_prov_qs |>filter(!is.na(qid))
Step 6: Compare existing and new population data in departments
Looking at the existing population data on Wikidata for departments, we see the most recent year is 2024 (the census year) for all nine. The one older year is 2012 for seven departments. These are the values that might match up.
pop_dept_qs |>count(population_1_year)
# A tibble: 1 × 2
population_1_year n
<int> <int>
1 2024 9
pop_dept_qs |>count(population_2_year)
# A tibble: 3 × 2
population_2_year n
<int> <int>
1 2012 7
2 2016 1
3 2020 1
The 2024 data are close, but unequal in all values. These values were initial releases by the Census and we will delete them to update them.
The 2012 data match in two cases, but none of theme are cited. We’ll delete all of these as well and replace them all with new, cited information.
Finally, we have two data points, without source, for 2016 and 2020. Since we are adding authoritative numbers for 2012 and 2024, we will delete these as well to avoid confusion.
Copy the lines below into QuickStatements batch mode and run them before proceeding to Step 7.