Introduction

Building a collection of URLs from Google or DuckDuckGo has become an increasingly brittle endeavor. Search-result markup and access rules change, automated requests get blocked, and endpoints disappear. Commercial search APIs are more stable, but they introduce keys, usage fees, and dependence on changing terms of service.

That problem is especially acute for local news, which is scattered across individual outlet sites and difficult to search as a collection. RSS provides a practical alternative: choose the publishers, collect what they publish, and search the resulting material locally.

textpress ships with rss_local_rags, a national catalog of validated local news feeds with outlet and geographic metadata. The catalog is a pinned snapshot from the local-rags-rss project, which builds on the outlet registry assembled by 3DLNews and the Local Memory Project. Here we use that built-in catalog to collect a recent national news snapshot, search RSS descriptions with a BM25 index, narrow results by place, and retrieve full articles for a small RAG example.

A National Directory of Local Feeds

library(dplyr)

We begin with the validated feed directory bundled with textpress.

local_rags <- textpress::rss_local_rags

Each row pairs a local outlet and validated feed with county and Census geography. The feed URL tells us where to collect recent items; the geographic fields let us map coverage and filter search results by region, state, county, or metro status.

Coverage by Census division

The directory reaches every state and Washington, DC, but it is not evenly distributed. The first map counts validated feeds in the nine Census divisions.

options(tigris_use_cache = TRUE)
conus <- c(state.abb, "DC") |> setdiff(c("AK", "HI"))

states_sf <- tigris::states(
  cb = TRUE,
  resolution = "20m",
  year = 2020
) |>
  filter(STUSPS %in% conus) |>
  sf::st_transform(5070) |>
  select(state_abbr = STUSPS, geometry)

division_counts <- local_rags |>
  filter(state_abbr %in% conus) |>
  count(census_division, name = "feeds")

division_sf <- states_sf |>
  left_join(
    local_rags |> distinct(state_abbr, census_division),
    by = "state_abbr"
  ) |>
  group_by(census_division) |>
  summarise(.groups = "drop") |>
  left_join(division_counts, by = "census_division")

division_colors <- c(
  "New England" = "#9467bd",
  "Middle Atlantic" = "#5ba3a0",
  "South Atlantic" = "#DD8E58",
  "East North Central" = "#708A81",
  "East South Central" = "#C2956E",
  "West North Central" = "#B08968",
  "West South Central" = "#A8757B",
  "Mountain" = "#C48A5A",
  "Pacific" = "#7A8B99"
)

ggplot2::ggplot(division_sf) +
  ggplot2::geom_sf(ggplot2::aes(fill = census_division), color = "white", linewidth = 0.25) +
  ggplot2::geom_sf_text(
    ggplot2::aes(label = paste0(census_division, "\n(", feeds, ")")),
    color = "white",
    size = 2.7,
    fontface = "bold"
  ) +
  ggplot2::scale_fill_manual(values = division_colors, guide = "none") +
  ggplot2::labs(title = "Validated RSS Feeds by Census Division") +
  ggplot2::theme_void() +
  ggplot2::theme(plot.title = ggplot2::element_text(size = 16))

County-level gaps

Division totals make the collection look denser than it is. County geography shows both the reach of the directory and the large parts of the country for which no validated feed is present.

conus_state_fips <- local_rags |>
  filter(state_abbr %in% conus, !is.na(fips)) |>
  transmute(STATEFP = stringr::str_sub(fips, 1, 2)) |>
  distinct() |>
  pull(STATEFP)

counties_sf <- tigris::counties(
  cb = TRUE,
  resolution = "20m",
  year = 2020
) |>
  filter(STATEFP %in% conus_state_fips) |>
  sf::st_transform(5070) |>
  select(fips = GEOID, geometry)

county_counts <- local_rags |>
  filter(!is.na(fips)) |>
  count(fips, name = "feeds")

counties_sf |>
  left_join(county_counts, by = "fips") |>
  mutate(has_feed = !is.na(feeds)) |>
  ggplot2::ggplot() +
  ggplot2::geom_sf(ggplot2::aes(fill = has_feed), color = "white", linewidth = 0.03) +
  ggplot2::geom_sf(data = states_sf, fill = NA, color = "white", linewidth = 0.3) +
  ggplot2::scale_fill_manual(
    values = c(`TRUE` = "#708A81", `FALSE` = "grey85"),
    labels = c(`TRUE` = "Has a feed", `FALSE` = "No feed"),
    name = NULL
  ) +
  ggplot2::labs(title = "County Coverage in the RSS Directory") +
  ggplot2::theme_void() +
  ggplot2::theme(plot.title = ggplot2::element_text(size = 16), legend.position = "bottom")

Metro and nonmetro outlets

The directory includes the 2023 Rural-Urban Continuum Code for each outlet’s county. We use its broad metro/nonmetro distinction rather than all nine codes.

local_rags |>
  summarise(
    feeds = n(),
    counties = n_distinct(fips),
    .by = metro_status
  ) |>
  DT::datatable(rownames = FALSE, options = list(dom = "t"))

Metro status describes an outlet’s home county, not the full reach of its reporting. A paper may cover neighboring counties, and one feed may serve several publications. Even so, the distinction supports useful comparisons between metro and nonmetro results.

Build the National RSS Snapshot

textpress::fetch_rss() retrieves recent entries from every feed in the local directory. The descriptions arrive with the feeds, so this does not scrape the linked articles or download their full text.

The collection step is shown below. The analysis uses a saved snapshot so the same set of stories is searched each time the post is rendered.

rss_items <- textpress::fetch_rss(
  local_rags$url,
  cores = 4
) |>
  left_join(
    local_rags |>
      select(
        url, state_abbr, census_division,
        county, fips, metro_status
      ),
    by = c("feed_url" = "url")
  )

The saved snapshot contains 31,385 feed items collected on August 27, 2026.

rss_items <- rss_items |>
  mutate(
    published_at = lubridate::ymd_hms(published_at, quiet = TRUE),
    description = description |>
      stringr::str_remove_all("<[^>]+>") |>
      stringr::str_remove(stringr::regex("The post .* appeared first on .*", ignore_case = TRUE)) |>
      stringr::str_replace_all("&#[0-9]+;|&[A-Za-z]+;", " ") |>
      stringr::str_squish()
  )

A few cleaned feed items look like this.

Dates, Feed Depth, and Syndication

RSS feeds expose different amounts of history. One publisher may return ten items from this week, another thirty from the past month, and a quiet feed may return much older material. We use a common 30-day window anchored to the newest item in the snapshot.

snapshot_date <- max(as.Date(rss_items$published_at), na.rm = TRUE)

recent_items <- rss_items |>
  filter(
    as.Date(published_at) >= snapshot_date - 30,
    !is.na(description),
    description != ""
  )

Syndication presents a different problem. The same press release or wire story can appear at several outlets. We collapse exact matches after normalizing titles and descriptions, while retaining the number of outlets that carried each item. Near-duplicates remain in the corpus.

search_items <- recent_items |>
  mutate(
    title_key = title |> stringr::str_to_lower() |> stringr::str_remove_all("[^[:alnum:]]"),
    description_key = description |>
      stringr::str_to_lower() |>
      stringr::str_remove_all("[^[:alnum:]]")
  ) |>
  group_by(title_key, description_key) |>
  mutate(syndication_count = n_distinct(source)) |>
  slice_max(published_at, n = 1, with_ties = FALSE) |>
  ungroup() |>
  mutate(search_id = paste0("item-", row_number()))

A Local BM25 Index

textpress also supports search_regex(), search_dict(), and search_vector(). Here we use search_index() and BM25: a good middle ground for exploratory keyword searches when the wording is not known exactly but the query terms should still appear in the result. BM25 ranks documents using the query terms they contain, how often those terms occur, how unusual the terms are across the collection, and document length. Here the searchable document is only the publisher-provided RSS description.

tokens <- search_items |>
  transmute(search_id, text = description) |>
  textpress::nlp_tokenize_text(
    by = "search_id",
    id_col = "search_id",
    include_spans = FALSE
  )

index <- textpress::nlp_index_tokens(
  tokens,
  stem = TRUE
)

Each query calls textpress::search_index() directly. We join all matching descriptions to the feed metadata before filtering by geography, then keep the highest-ranked local results.

show_hits <- function(hits, caption) {
  hits |>
    mutate(
      date = as.Date(published_at),
      story = paste0(
        '<a href="', url,
        '" target="_blank" rel="noopener noreferrer">',
        title,
        "</a>"
      )
    ) |>
    select(date, source, state_abbr, county, metro_status, story) |>
    DT::datatable(
      rownames = FALSE,
      escape = FALSE,
      caption = caption,
      options = list(pageLength = 5, scrollX = TRUE, dom = "tip")
    )
}

What Is Local News Talking About?

A national search can surface subjects that are locally important but diffuse across thousands of publishers.

county_fair_hits <- textpress::search_index(
  index, "county fair livestock",
  n = nrow(search_items), stem = TRUE
) |>
  left_join(search_items, by = "search_id") |>
  slice_head(n = 15)

county_fair_hits |>
  show_hits("National results: county fair livestock")

The geographic metadata makes more pointed comparisons possible. The same index can ask about wildfire in western states or water across the Mountain division.

wildfire_hits <- textpress::search_index(
  index, "wildfire evacuation",
  n = nrow(search_items), stem = TRUE
) |>
  left_join(search_items, by = "search_id") |>
  filter(state_abbr %in% c("CA", "OR", "WA", "ID", "MT", "NM", "AZ", "CO")) |>
  slice_head(n = 15)

wildfire_hits |>
  show_hits("Western states: wildfire evacuation")
water_hits <- textpress::search_index(
  index, "water drought irrigation",
  n = nrow(search_items), stem = TRUE
) |>
  left_join(search_items, by = "search_id") |>
  filter(census_division == "Mountain") |>
  slice_head(n = 15)

water_hits |>
  show_hits("Mountain division: water drought irrigation")

Metro status adds another useful comparison. The spread of data centers into rural counties connects a national industry to local questions about infrastructure, land, water, taxes, and public services.

data_center_hits <- textpress::search_index(
  index, "data center",
  n = nrow(search_items), stem = TRUE
) |>
  left_join(search_items, by = "search_id") |>
  filter(
    metro_status == "Nonmetro",
    stringr::str_detect(
      paste(title, description),
      stringr::regex("\\bdata centers?\\b", ignore_case = TRUE)
    )
  ) |>
  slice_head(n = 15)

data_center_hits |>
  show_hits("Nonmetro counties: data centers")

From descriptions to full articles

Descriptions are enough to find the stories. Once the rural data-center search has narrowed the collection, textpress::read_urls() adds the full article text to the result set while carrying its RSS and geographic metadata forward.

data_center_articles <- data_center_hits |>
  textpress::read_urls(cores = 4)

Full text was available for ten of the 15 selected stories. The rendered post uses that saved article text.

For the RAG step, we keep each article intact and attach its title, outlet, county, state, publication date, author, article URL, and feed URL. The model receives the complete text of each selected article.

rag_documents <- data_center_text |>
  arrange(doc_id, node_id) |>
  summarise(article_text = paste(text, collapse = "\n"), .by = doc_id) |>
  inner_join(
    data_center_hits |>
      select(
        doc_id, title, source, state_abbr, county, published_at,
        author, url, feed_url
      ),
    by = "doc_id"
  )

A small RAG synthesis

The final step uses ellmer and OpenAI’s Responses API to organize the full articles by theme. Counties and states appear in the prose, while parenthetical citations name the publishing outlets.

rag_context <- rag_documents |>
  transmute(
    source = paste0(
      "OUTLET: ", source, "\n",
      "TITLE: ", title, "\n",
      "LOCATION: ", county, ", ", state_abbr, "\n",
      "PUBLISHED: ", published_at, "\n",
      "AUTHOR: ", coalesce(author, "Not listed"), "\n",
      "ARTICLE URL: ", url, "\n",
      "FEED URL: ", feed_url, "\n",
      "FULL ARTICLE:\n", article_text
    )
  ) |>
  pull(source) |>
  paste(collapse = "\n\n")

chat <- ellmer::chat_openai(
  system_prompt = "Answer using the supplied local-news articles.",
  model = "gpt-5-mini-2025-08-07",
  params = ellmer::params(reasoning_effort = "low"),
  echo = "none"
)

rag_answer <- chat$chat(paste(
  "Write cohesive prose for publication. Start directly with the topic and",
  "its central tension. Organize the main patterns under short Markdown",
  "subheadings formatted as `#### Heading`, followed by developed paragraphs.",
  "Write",
  "location names naturally into sentences and use",
  "publishing outlet names for parenthetical citations.\n\n",
  rag_context
))

LLM-generated synthesis across local reporting

The boom in AI-driven data centers has opened a sharp tradeoff: communities from Jefferson County, Arkansas to Hayden, Colorado and Pahrump, Nevada are weighing the promise of construction dollars, tax revenue and new infrastructure against worries about water and power use, sprawl, and a loss of local control.

Economic promises and local limits

Developers and some local leaders point to large fiscal benefits and few service demands from data centers, arguing these facilities can dramatically boost property tax bases and fund schools and roads. Oasis Digital Properties, for example, touts the ability of a Russell County, Virginia site to transform underused land and generate “extraordinarily high” tax revenue (thelebanonnews.com). Advocates in McKenzie County, North Dakota press a similar case: data centers as engines of construction jobs and long-term revenue that can lower residential taxes (watfordcitynd.com). But careful, county-level analysis tempers that optimism. University of Arkansas researcher Frank Seo simulated one year of activity and found that while data centers can compare favorably with manufacturing in statewide output, much of the indirect and induced spending leaks outside sparsely populated counties where local supply chains and consumer markets are thin (www.pbcommercial.com). In short: headline tax numbers can obscure limited local circulation of benefits in many rural places.

Zoning, codes and the drive to get ahead of speculation

Local governments are responding by writing data-center specifics into land-use rules or tightening permitting. Hayden, Colorado amended its municipal code to define data centers as a heavy industrial use with strict caps on size and electrical demand, mandated energy and water plans, and required conditional-use review to close loopholes that previously allowed “all heavy industrial uses” by right (craigdailypress.com). Pennsylvania’s executive order from Gov. Josh Shapiro took a statewide tack, making local approvals and tougher GRID standards central to any fast-tracked permitting pathway while discouraging speculative proposals by raising the bar for state permits (explorevenango.com). These actions reflect a common impulse: communities want the leverage to shape projects early, not be faced with near-complete proposals that force rushed local approvals.

Water, power and environmental tradeoffs

Concerns about resource use are driving much of the opposition. In Georgia and elsewhere, public and environmental groups point to large water withdrawals and the indirect water costs of energy generation; researchers and trade analyses place data centers among the top water-consuming commercial sectors when scaled for AI growth (statesboroherald.com). Developers counter that modern closed-loop cooling and on-site generation can reduce withdrawals or shift impacts, but closed systems often increase energy demand, and energy production itself can consume or thermally affect water resources. Policymakers are responding: some states and towns now require energy-supply plans, water-demand analyses, noise studies and thermal-mitigation measures as part of conditional approvals (craigdailypress.com; explorevenango.com). The tug-of-war is between technological mitigation claims and the practical realities of local grids and watersheds.

Industry messaging and political contests

Industry and pro-business groups are mounting messaging campaigns to reshape public opinion and influence policymakers. The Georgia Chamber formed a Digital Infrastructure Alliance to highlight jobs and revenue and to rebut what it says are inflated fears about water and electricity (statesboroherald.com). At the same time, opposition has hardened: national polling shows rising resistance to local data-center construction, and politicians from both parties have taken strong positions — from calls for pauses to demands for tighter oversight — as the issue becomes a visible campaign topic (statesboroherald.com). That politicization complicates negotiations and makes transparent, locally relevant information more urgent.

Community mobilization and demands for transparency

Where projects are proposed, grassroots pushback is frequent and intense. Nye County residents organized town halls, presentations, and petitions that helped secure a local ban on a proposed campus after sustained public comment and scrutiny (pvtimes.com). In Russell County, neighbors revived the organizing tactics learned during a past landfill fight, mobilizing to question water capacity, energy impacts and tax deals tied to a possible Moss 3 data center (thelebanonnews.com). A recurring complaint is lack of clear, locally specific data from developers — a gap that both advocates and skeptics want filled through third-party studies, surety bonds to cover abandonment, and binding conditions in permits (statesboroherald.com; thelebanonnews.com).

Across these patterns, the core tension persists: data centers can bring concentrated fiscal gains and national strategic value, but their localized impacts — and who captures or bears them — are uneven. The emerging response is pragmatic and procedural: require rigorous local studies, harden permitting and zoning rules, and insist on transparent mitigation plans so rural and small-town governments can decide whether, how and on what terms to host the next wave of AI infrastructure (www.pbcommercial.com; craigdailypress.com; explorevenango.com).

Summary

Together, these tools build a searchable national collection from known local publishers. local-rags-rss supplies the publishers and geography, RSS supplies a recent stream of stories, and textpress indexes their descriptions on a local machine. Full articles enter only after a geographic BM25 search has narrowed the collection to a manageable set.

The larger value is visibility. At a time when local reporting is thinning, this collection makes work scattered across hundreds of small outlets easier to find and compare without stripping away its source or geography. It keeps reporting about local institutions, conflicts, and everyday civic life in view, while making patterns across communities easier to see.