Expecting surprise

My (long) journey in graduate school has finally come to an end. Aristotle wrote in the Nicomachean Ethics that contemplation is the highest form of happiness¹, and despite the difficult tradeoffs of pursuing an advanced degree, it has been a truly happy time waking up each morning, putting the kettle on for coffee, and getting out of my mind's way for long periods of quiet contemplation about water, how we observe it, and how we use observations to make decisions about its use under uncertainty. I'm grateful to my supervisor Dr. Steven Weijs for his enthusiasm, patience, and support.

The last course I enrolled in as a graduate student was in the philosophy of science on the Evolution of Cognition. I was completely out of my depth, and I remain grateful and apologetic to my classmates for enduring my crimes of enthusiasm. One assigned reading in particular has remained with me from that class: "Signal, Decision, Action" (Godfrey-Smith 1991) reminded me of how entropy was introduced through Maxwell's demon in undergraduate thermodynamics. In Godfrey-Smith's paper, he explores rationality as an adaptive evolutionary trait using a similar idealized animal that featured a sensor to detect its environment, cognitive faculties to develop an internal model of its world, and physical capacity to act on the model beliefs.

Figure: Signal, Decision, Action (Godfrey-Smith, 1991). S is the signal in a noisy environment, N. X is the sensor variable. C is the internal cognitive model. M is the physical capacity for action.

The human capacity for visual information processing is enormous compared to Godfrey-Smith's idealized animal, yet simple, aggregate views of complex environmental systems are what we generally use to support the design of infrastructure and to guide the operation of systems we take for granted today. We apply increasingly complex models in an effort to squeeze more information from data that requires generations to accumulate.

If we take roughly 1000 streamflow monitoring stations in the Pacific Northwest (PNW) that have operated for some time since 1950², assign each location a pixel in a (roughly) geographically proportioned grid, and shade each pixel in 8-bit greyscale³ corresponding to streamflow volume, the result (below) resembles a coarse, incomplete glimpse of the state of surface water over this roughly 1 million square kilometer region.

Figure: Streamflow monitoring in the PNW, with British Columbia outlined in red. Each pixel represents a flow value recorded at a streamflow monitoring station, where the arrangement of pixels roughly preserves the spatial arrangement of the monitoring network.

If we then play a sequence of 365 images representing ~27,000 daily records since 1950, the result (below) contains much of the collective historical record of surface water in the region. This 74-year history of streamflow fits in a file smaller than one of the short videos we scroll past on social media in idle or uncomfortable moments of daily life.

Figure: Estimated daily streamflow across the monitoring network, 1950-2024. Each pixel represents a station, and brightness indicates flow (dark blue-grey pixels are gaps with no data). The network forms a crude collective eye through which we observe rivers of the Pacific Northwest.

My research was tasked with quantifying the marginal value (cost) of each pixel (station) for the broadest set of questions, a compelling and practical but complex problem. Streamflow observations have long guided decisions about how water infrastructure is planned, designed, and operated. These data are used to inform design for floods, to better understand predictors of drought, to estimate water availability, and to decide how available water should be allocated among industry, agriculture, and ecosystems. The processes governing the rainfall-runoff response are complex, and their drivers and mediators are changing, not least by the loss of perennial ice and snow across BC (below).

Figure: For non-overlapping catchments between 400 and 500 square kilometers, the decade between 2010 and 2020 saw substantial loss of perennial snow and ice cover. Reproduced from Kovacek and Weijs (2025).

In the last thirty years, our coarse perception of surface water has deteriorated in response to budget decisions that have reduced the density of monitoring in British Columbia, and more generally across Canada. The effect of reduced investment in streamflow monitoring has progressed like cataracts impeding the senses of analysts and decision makers. But unlike cataract surgery, our collective vision of surface water can't be restored by a simple intervention, since many hydrological applications rely on long-term continuous records to support robust analysis.

Government delivering on strong open data policies has been an enormous benefit to hydrology research and practice. But to take advantage of these opportunities, programming skills were essential. Massive open online courses (MOOCs) helped renew my own interest in computer programming nearly 20 years ago, and they helped me develop skills indispensable to graduate research and to the consulting engineering practice I maintained through graduate school to get by.

Even with some degree of education and practical experience, data and computational tools are evolving at a rate that is both fascinating and overwhelming. The most difficult concepts I wrestled with in my studies were the interrogation of models and the interpretation of probabilistic metrics. The tools we use to process environmental data can be seen as a medium through which we interact with information, and the nature of the interaction governs interpretation. Without careful interrogation, metrics can quietly decide what counts as insight or success, regardless of needs or intentions. The famous quote from Marshall McLuhan, "the medium is the message"⁴, occupies a deep, recurrent corner of my brain.

In witnessing and experimenting through the emergence of large language models (LLMs), I see their extraordinary potential for overcoming productivity bottlenecks. But the promise of coding at the speed of thought lies in conflict with how I feel about the hard-won crumbs of knowledge I scavenged throught the process of building datasets customized for my own research, including the physical and climatic characterization of 1.2M ungauged catchments in the PNW, and the processing of daily meterological forcings for 800+ monitored catchments in the same region. With much credit to the Stack Overflow community (which was the mainstay of programming help prior to LLMs), writing code to build and refine these procedures revealed a sometimes overwhelming depth of complexity and subjective judgment underlying seemingly straightforward problems.

The acceleration of analytical capacity enabled by LLMs has shifted the bottleneck to interpretation. One consequence of this shift is that these tools risk becoming the means by which we interpret results. LLMs have pushed us into a new regime where the technical distance between an idea and the means to test it can be reduced, but it will take time to harness these powerful inference engines for building intuition and understanding⁵.

The datasets I built for my own research took years to process and publish, and the persistence to complete the work required embracing their imminent obsolescence. What feels more lasting are general skills in identifying how subtle, subjective procedural choices can produce materially different outcomes⁶, and how to make the most of modern technology to explore and explain these effects in new ways. The computer scientist Donald Knuth beautifully conveyed the idea that the choice of technology (or programming language) is less important than how it is explained⁷. This idea seems like a useful compass to orient my path going forward.

The final product is a thesis ambitious in scope, hopefully clear and modest in its claims, and available through UBC Open Collections: Expecting surprise: towards information-based streamflow monitoring network evaluation through flow duration curve estimation in ungauged basins.

References:

• ECCC. (2016). Hydrometric Manual, Stage-Discharge Model Development and Maintenance. Environment and Climate Change Canada, Ottawa, ON. Report En37-464/2016E-PDF. https://publications.gc.ca/collections/collection_2021/eccc/en37/En37-464-2016-eng.pdf
• Godfrey-Smith, P. (1991). Signal, decision, action. The Journal of philosophy, 88(12), 709-722.
• Klowden, T., & Tao, T. (2026). Mathematical methods and human thought in the age of AI. arXiv. Preprint posted March, 27. https://arxiv.org/abs/2603.26524
• Knuth, D. E. (1984). Literate programming. The computer journal, 27(2), 97-111.
• Kovacek, D., & Weijs, S. (2025). BCUB-A large sample ungauged basin attribute dataset for British Columbia, Canada. Earth System Science Data Discussions, 2024, 1-19.
• Kratzert, F., Nearing, G., Addor, N., Erickson, T., Gauch, M., Gilon, O., ... & Matias, Y. (2023). Caravan-A global community dataset for large-sample hydrology. Scientific Data, 10(1), 61.
• McLuhan, M. (2019). The medium is the message (1964). In Crime and media (pp. 20-31). Routledge.
• Thornton, Peter E., Rupesh Shrestha, Michele Thornton, Shih-Chieh Kao, Yaxing Wei, and Bruce E. Wilson. (2021). Gridded Daily Weather Data for North America with Comprehensive Uncertainty Quantification. Scientific Data 8 (1). https://doi.org/10.1038/s41597-021-00973-0.