Grady Berry Ward — Projects, Essays, and Notes

Pharos: Token Library

Grady Berry Ward — Fri, 25 Sep 2026 00:00:00 +0000

TL;DR: A Mac app to make your past AI transcripts searchable, and easy to mine for insights. Free and open source (MIT).

LLM transcripts are goldmines. When you work with an AI to design or build a system, any value that comes from it originates in the back and forth conversation. The code gets committed, but the conversation often contains insight and context that never makes it into git (and shouldn’t).

If a human says:

“Oh we actually don’t want to do X because of Y”
“Can you revert X, I don’t think we need it anymore because of Y”
“Can you do a deep dive to explain X to me, I want to make sure I understand it”

Or an LLM says (in a visible reasoning summary or response)

“I hadn’t realized that X”
“Ahh now I see that we always have to do X when we do Y”
“I thought X was going to work but it looks like it did not”

Every one of those snippets is wildly useful - for future users or modifiers of the system, the deliberative process is more valuable than the final result.

Thankfully, conversations are recorded. Claude Code, and Codex, and other agentic harnesses all write down your conversation to a file. These “transcript” files are typically JSONL, structured slightly differently for each model provider, but they record everything about a conversation, including the messages to and from the agent, the agent’s tool calls, and the files that were modified. These files are gold mines.

I built Pharos to mine gold.

What is Pharos

Pharos is a Mac App that pulls in transcripts of chat conversations from known locations, backs them up, and indexes them to make them unreasonably useful across a wide range of questions you might want to ask about your chat conversations. Pharos can be used to look at your conversations and find answers to your questions:

What kind of errors are my models making frequently?
What websites have my agents visited, CURL’ed, or POST’ed to?
Which conversation did I mention “curmudgeon” in?
In what weeks am I breaking even on API-level pricing, and by how much?
How many tokens have I used this week?
How much of my prompts are human generated?
Do cross-model reviews actually yield meaningful PR findings?
What are common sources of latency in my conversations?
Where in agent transcripts do I see the phrase “Oh I misunderstood”?
How efficient are my compaction strategies?
I remember that I accidentally pasted an API key into a chat conversation - is it the key XYZ or does that not show up in conversation history (i.e. already rotated)?
Where are my uses of sub-agents inefficient/how much context do I waste with common prefixes?

But these are questions a human might ask. Pharos also holds the indexes and tools to help other LLMs find and navigate past conversations:

What conversations are most relevant to the task I’ve been given, XYZ?
What chat conversation produced this chunk of code that I now need to modify?
What were the last 10 chat conversations in this repo about?

Pharos ships with a super simple, context efficient MCP server that gives LLMs access to layered sets of tooling that let them explore data at the right level of granularity.

How does it work?

Pharos is simple. It’s a very basic database and Mac App Wrapper that backs up and indexes conversations so you can (instantly) see metrics and answers to each of these questions.

Pharos operates best off of an SSD - that way you can plug it into multiple computers (if you have several dev machines like I do) and it pulls in transcripts and keeps them in sync automatically, automatically syncing from its host machine every time it’s plugged in.

On Naming

I started off calling this project “Alexandria”, like the library, but then I realized the value isn’t in the archival, it’s in the way that indexing can shine light on data at scale, and thus the lighthouse was a better analogy. Thankfully Alexandria had both - the lighthouse was called “Pharos”.

Why not rely on the labs?

Most folks are relying on built-in harness and agent systems to manage context and history for them. After all, every agentic system ships with built-in tools for managing memory across sessions, for backing up transcripts to their clouds, etc.

Folks will say “oh Claude will just update its own memory” or “if it’s important the agent will write it down somewhere to the repo”. Those might be true today, but the incentive misalignment around transcripts should give you pause:

Model providers have every incentive to make transcript files proprietary - to prevent distilling, engender lock-in, and centralize control and access to the tokens you’ve paid for.
Model providers also have every incentive to keep your use of AI inefficient. Pharos has helped me radically reduce token usage while achieving better outcomes. This allows me to increase the efficiency of my AI spend, and reduce the amount of money I have to fork over every month.
Model providers might not be so cynical to clamp down on transcripts today, but there’s no guarantee about tomorrow. A new release of Claude Code could easily wipe out the transcript directory on your Mac (replacing its functionality with a cloud-based alternative that they control). Back up your data while you can.

Think about it a different way: given you’re spending hundreds (or thousands) of dollars to access the results from LLMs, why are you not backing up (and getting the most value out of) the core thing you’re paying for?

The Downside: Visibility

The main downside of using Pharos is that it forces you to grapple with three very uncomfortable truths:

You’ve wasted huge volumes of usage on stupid stuff or silly bugs (I had accidentally launched a 2k agent swarm once with the wrong permissions that blew through a billion tokens)
If you’re like me, your AI use has quickly eclipsed all of your other forms of CO2 (Pharos comes with a CO2 estimate - it’s rough)
My usage would be unaffordable at API-level pricing. At API list prices (which Pharos helps you keep up to date), Pharos estimates my usage would have cost more than $150k.

I’m sold, how do I use it?

Download the Mac app here - it’s free, and you can audit its dependencies - it has essentially no non-standard deps.

LLM transcripts are the ultimate proprietary data set - get the most out of yours.

Appendix: Screenshots/Workflow

1 - Choose your sources

The first time you open Pharos, it checks the places where Claude Code, Codex, and other AI harnesses store transcripts.

2 - Capture

Pharos copies conversation files, unparsed, into the library. This takes seconds to minutes, making it easy to plug in an SSD and quickly extract content.

3 - Index, on any Mac

Captured files are indexed into a searchable catalog in a secondary step, which can be programmed to happen overnight. Indexing can happen later, on any Mac. Indexing from scratch can take hours, but incremental indexing is typically under a minute - I have it run every 30m.

4 - Find past work

The Library tab offers the ability to search by regex, trigram, vector, or use a combination of boolean and/or logic to find just the conversations you want to.

5 - Read a conversation

The conversation view: prompts, responses, and every agent action in between, marked with the token additions to context. The scrollbar on the right tracks context size across the whole conversation, and clicking it jumps to that point.

6 - See where the tokens go

We’ve all wondered: how many tokens have I used, and how much would that have cost? Now you can know.

7 - Count the words you actually typed

But how much are you actually typing? How much thought are you adding to the conversation? I love this tab.

8 - Audit tool calls

Tool calls are a rich dataset in which you can see patterns of usage, find evidence of exfiltration, and debug frequently failing tool calls.

9 - Unplug safely

Because Pharos runs (best) on an external SSD, it helps you understand when ejecting is safe and can manage ejects for you.

10 - Carbon accounting

Cheap Skinny Ports

Grady Berry Ward — Sun, 13 Sep 2026 00:00:00 +0000

I’m building software to emulate Excel’s calculation engine.

From the beginning, I knew this would require an enormous volume of test cases - authoritative pairs of workbooks and cell values (inputs), and the values calculated by Excel for each (outputs). To capture these, I created a “harness” for Excel on windows - you hand it an XLSX file, and it produces a test case in the format that anchors my Excel-emulating calculation engine.

The immediate problem I ran into was throughput. I needed to generate many test cases, but each step was monstrously slow: opening an excel spreadsheet in Windows, mutating some of its values through python libraries, saving it, and then parsing relevant features from the resulting file (Excel stores the calculated cell values in the saved XLSX file format). With off-the-shelf libraries, it took longer to run the harness infrastructure than it took to open the Excel spreadsheet and do the calculation. I was giving away over half of the computational energy to overhead - CPU cycles that then couldn’t be spent on the actual Excel calculations I was interested in.

Though I made a variety of algorithmic improvements to improve multiprocessing and reduce COM contention, the largest dent I made in processing time (reducing the overhead time from over a minute to under 100ms) was through what I’m calling “Skinny Ports”.

First some definitions: a “clone” of a library is when you make a copy of the library and adapt it for your purposes, a “port” is similar, but the copy is often written in a different language than the original, while preserving its core logic. Porting and cloning code is now simple because AI systems make both creating and verifying a port relatively straightforward, and reduce the burden of managing clones. A “skinny port” is a port where the goal of the port is not comprehensive coverage/equivalence, but a port with the ambition of serving only a small set of needs, and doing it well. In my calculation engine, I’ve made about seven “skinny ports” and “skinny clones” of other libraries, and through them have radically improved the performance at multiple layers.

To make this concrete, I’ll tell you a bit about my experience of porting openpyxl. Openpyxl is a library in Python that encodes how to read and write Excel file formats, like XLSX. This might sound simple, but these file formats are intricate and nuanced. The team behind openpyxl has (over the course of 15+ years) found all of the little bugs and quirks of the format. Thus, rather than implementing this from scratch, in the first version of my harness, I used openpyxl to capture the results of an Excel file once saved, since the saved version caches the computed formula values. This worked, but it was painfully slow, and I quickly realized that to produce the volume of cases I wanted to, I needed something faster. After testing out a few openpyxl competitors in other languages, I found each to have clear gaps in functionality that would prevent their adoption, so I decided to port openpyxl to Go (because the rest of my harness code is also in Go). I asked a sophisticated AI agent to make the port, which it did in short order, after we co-designed and built a harness for testing equivalency between the original openpyxl code and the ported Go code. This port consumed many tokens, but very little energy from me.

Out of the box, the Go port was more than 100x faster than its python equivalent, and with some fine tuning/performance tweaks, that number went far above 1000x. Though the speed is an obvious advantage, and the reason I started the port, the flexibility I got from owning my dependencies in this way turned out to be dramatically more valuable than the speed alone:

1. Personalized Interfaces

A general purpose library has to design its interfaces for a broad range of consumers, and it has to balance speed, simplicity, and comprehensiveness. A personalized port allows you to fine tune the interfaces for your specific use cases.

For example, in my openpyxl port, I ended up creating a set of streaming APIs to allow lower RAM usage over large spreadsheets, something that the python library didn’t implement.

2. Language Unification

To move fast, you often want to farm out complicated logic to well established libraries. However, it’s often the case that the best libraries for A and B and C are in different languages. Ports allow you to port each of your dependent libraries into the language you want to work in, often achieving type safety and simplifying the call structure.

For example, in my first version of the Excel harness, I had a smorgasbord of libraries and processes and binaries that were all trying to talk to each other. It worked, but it was a nightmare to reason about, and context about failures was spread across multiple different tools. By unifying them into a single binary, in a single language, I was able to get unified error reporting, and eliminate a wide array of expensive (and slow) cross-process communication.

3. Personalized Optimization

Porting a library from a slower language to a faster one will often provide significant performance benefits, even if all of the logic is equivalent. But the bigger opportunity is that once ported, you’re able to push on optimizing the performance of the ported library to suit the specifics of your use - something that the maintainer of a generic library with many constituent users can’t prioritize

For example, the python version of openpyxl did a bunch of processing on first ‘opening’ an excel file that wasn’t necessary in many of the cases I care about. By porting it, I got to refactor that code so that it only does the computation it strictly needs to to accomplish the aims I have.

4. Supply Chain Derisking

Reliance on open source libraries is a double edged sword. On the one hand, you get the benefit of reusing someone’s intellectual labor. On the other, you have to trust that the author(s) have not embedded anything malicious in their code, and trust that they will not do so in the future. A one-time port is a layer at which you can validate the security of the code, and remove dependencies and behavior that is riskier or broader than you want to maintain.

For example, in the openpyxl port, I was able to prune out certain dependencies, like Pillow, because I didn’t require the image functionality that Pillow enables within openpyxl.

Implications

I think this approach is a really robust one from a development perspective, but I worry about what it means for the future of the open source community. Open source is a massive common good, and dynamics that make it cheaper than ever to copy and modify reduce much of the incentive to be a good steward of it.

To that end, I made two decisions about how to handle my ports to balance this newfound capability with the ethos that makes it possible.

First, when I find bugs or functionality missing from openpyxl, in addition to implementing it in my port, I’m going to offer the changes upstream into the original Python repo.
Second, I decided not to publish my port, since I think having a faster equivalent might draw attention away from the more carefully maintained, higher quality project (openpyxl).

AmDo: Automating Ambrook

Grady Berry Ward — Sun, 13 Sep 2026 00:00:00 +0000

This is a copy of a blog post I wrote for Ambrook, a company I worked for at the time. I think its core lessons are good ones for anyone thinking about ethical and safe automation.

TL;DR: Our agentic system for modifying customer data: (a) prompt an AI that has broad access to read-only tools, and have it draft a plan of how to change their books (b) review the pl plans in a comprehensible UI (c) execute the plans deterministically. This approach scales human judgement rather than replacing it, and is flexible to automations of every shape.

The Problem

After watching Haley click a new sequence of buttons for 8 opp, I knew we could do better. Haley is one of our (fabulous) Account Managers, and she was helping a customer with yet another (not so-fabulous) Loan Statement. She would create an account to track the loan, create a transaction for each payment on the loan statement for the same amount, and finally itemize each transaction using the loan statement PDF to figure out the percentage of each payment on the loan that should be tagged as principal or interest. Each of these operations was several clicks. It was painful to watch, and I’m sure even more painful to do.

In the weeks before, Haley and I had spent lots of time together as I tried to understand and accelerate her work as an Account Manager for Ambrook’s customers. Much of the day-to-day work of an Account Manager (AM) consists of helping customers by clicking around the product: setting up their account to reflect their farm operation, and helping them with some basic bookkeeping. Over the prior weeks, shadowing Haley and other AMs had been fruitful. I’d see a pattern of clicks that was inefficient or annoying and try to build automation to make it easier or obviate it. Working hand-in-hand like this, the AMs and I had reworked the authorization model, built many small product shortcuts, and eliminated a class of thorny bugs. Even though each optimization felt like progress, it increasingly felt like we were missing the forest for the trees. At the end of the day, AMs were still spending lots of time in the app, clicking buttons.

The core challenge was how heterogenous an Account Manager’s job can be:

One customer might need Haley’s help itemizing loan statements by interest and principal amounts.
Another might need help selectively changing the categorization of a set of transactions.
A third might need check memos transcribed from a PDF of a bank statement, and applied to their relevant check.

In each instance, the actual work the AM performs for the customer is fairly mechanical, something that could be automated, but the range of tasks varies enough that automating all of it by creating new workflows or buttons wasn’t tractable. We’d toyed around with using LLMs for part of this job, connecting them to demo MCP servers, and found that they were too frequently wrong to be deployed in these accounting workflows where being exactly correct matters.

I spent a week noodling on this problem, and eventually prototyped and productionized a generic solution: PlanBrook and AmDo. In the last few months it has quietly revolutionized not only how AMs do their work, but what we can achieve for our customers, and it presents a hopeful vision for how AI might revolutionize work while centering and scaling human judgement .

The Solution

The core insight was thinking about why we engineers are so comfortable using coding agents extensively. Most of us use them to debug, to summarize, to propose changes, and even to do a first pass of code review - but we still recognize that AI is too often wrong to let it merge without human review. The layer of human review at code submission time is the fence that separates a low consequence playground and the high consequence factory. Account Managers needed a version of “code review”: they needed a step in the process where they could see what was being proposed by the LLM, and carefully consider whether it was good enough to execute.

Thus, we built this system with two halves: “PlanBrook” and “AmDo”, separated by a human in the loop:

PlanBrook is a planning agent based on Claude Code - one that has broad access to data in a read-only mode. This means it can search the codebase, Linear, a given customer’s data, and could even take in files like documents our customers submit to us. However, it doesn’t have any access to change any of that data. Our AMs have freeflowing conversations with PlanBrook - they provide it with instructions, meeting recordings, files, and context about the customer’s data setup. It uses its tools (including OCR, python scripts, SQL queries, and bash) to construct a “plan document” - a giant chunk of JSON that describes changes to be made on Ambrook. The AM and agent can go back and forth, repeatedly honing the plan.

AmDo is PlanBrook’s trustworthy sidekick. It is a deterministic piece of code that lives as a page in our frontend. It takes in PlanBrook’s JSON blob, and gives the human agent a comprehensive and clear review interface that lets them quickly understand the plan and interrogate its details. The plan is always easy to understand through the UI, even if it is big, or includes many nested or looped steps (more on that later). Beyond the review interface, AmDo’s other job is executing the plan that the user selects or approves, using the user’s credentials. Since this happens in the browser, on the user’s computer, this is all just straightforward API calls to our server using existing endpoints, with standard authorization (and thus, access controls).

I like to think of these two systems as Don Quixote (PlanBrook) and Sancho Panza (AmDo). PlanBrook is an inventive, flexible, and courageous explorer, unconstrained by consequence. AmDo is a trustworthy rate-limiter: informed, careful, and prudent.

A data model for anything

The core challenge of this project was building the data model (and corresponding review interface) to represent the variety of plans we wanted to support, and the user interface to allow an Account Manager to understand what would happen at each step. This required solving a variety of small problems:

Operation Set

The plan schema needed a registry of operations that could constitute individual steps. To achieve this we turned to our Frontend-to-backend API, GraphQL, and added some lightweight scaffolding to expose the mutations in that schema to PlanBrook. This scaffolding handled things like:

hiding methods that should never be used,
renaming methods to use more commonplace terms,
adding hints about how each API should be called in brief documentation,
winnowing the input arguments to the set that are most commonly needed

Each of these optimizations was found by looking at the stumbling blocks models ran into when interacting with our large and sedimentary GraphQL schema. The end result is that adding a new GQL mutation to this set just takes a line of configuration, and can be tweaked when agents get confused.

Loops and Data

PlanBrook and AmDo are only needed when AMs need to do large volumes of operations - if it’s just a few clicks, it’s still faster for them to do it in the App. To express this, we turned to loops and data sets - plans can include named tables that have columns and rows, and then have blocks in the plan that say “for each row in this table”. As an example, PlanBrook might use OCR to extract out the check numbers and memo lines from a PDF of a checking account statement, and then include in the plan both the data table that was extracted (column of Memo and Check Number), and then run an loop over an operation to set the memo for each check.

Dependency and Reference

If an AM specifies that first an invoice should be created, then it should be itemized, PlanBrook needs to describe (in a single plan document) how that second step relies on knowing the existence of the previous step (in order to know the ID that should be sent in the itemization request).

When this is only two steps, it seems straightforward, but we also have concrete use cases where dependencies are 1:N (i.e. create a single invoice and match it to multiple payments) and N:1 (i.e. create multiple transactions, then use a single batch tagging operation to update them).

Building this graph structure required careful tinkering with not only what was theoretically supportable, but what kinds of JSON agents could reliably produce. We found that data models that were too abstract took many more tokens for the agent to construct, and more frequently suffered from hallucinations.

Intentional Omissions

Even as important as what you build is what you intentionally choose not to build. We decided that in order for plans to be fully comprehensible they needed to be repeatable and deterministic, and so we fully skipped conditionality - the ability for a plan to specify a condition and two branches to take based on the evaluation of that condition. We first thought we would need conditionality to support the common “upsert” scenario, where the AM asks PlanBrook “to create or update the transaction”. We ended up finding we could work around this need by ensuring that PlanBrook had access to an internal Ambrook MCP that gives live read-access to the customer’s data. This allowed the planning agent to act conditionally to create plans that are based on the state of the data, rather than encoding that uncertainty into the plan.

This keeps our plans fully deterministic, and highly comprehensible to the AM reviewing them. For the same reason, we opted not to build in Lookups or Error Recovery. Making sure the human reviewer has confidence in what will happen when they press “Run Plan” is invaluable.

The Impact

AmDo has been launched for a few months, and we recently added logging to understand usage trends. For the five weeks we have data:

Each week it performs 4k+ operations (each operation would have required someone to navigate to a page and click a button in the app)
Every member of our Account Management team has used it at least once a week for the last five weeks (after vacations are taken into account)
11 team-members not on our AM team have used it more than once.

The numbers don’t capture the quality of life improvement - our AMs work so hard, and making their lives better has been sincerely rewarding. Some slack quotes that made me smile:

“It helped me clean up convoluted COA and P&L in like half the time!”
“This will save days of time for our customers!”
“Consider it AmDONE!”

The things our AMs love about PlanBrook isn’t just the automation, it’s the way it allows them to “spend less time squinting” at their screen. With agents, AMs can trust that Claude will figure out the rote pieces of updating books. They can spend more time thinking about what they’re doing, and less time just trying to double check that the buttons they pressed were the right ones.

AmDo has become a primitive that we’re composing into more and more workflows - human and agentic. Several tasks that used to consume hours of AM time (including the original example, of itemizing loan statements by principal and interest) have now been turned into standalone Claude Skills, which ensure the creation of the AmDo plan for them. We’re planning on prototyping a version of AmDo to bring to users - so that they might have plain-text conversations with an agent to keep up with their books.

This process hasn’t been all roses - AMs often end up having to go back and forth with PlanBrook 2+ times in order to get to the plan that they want. Sometimes that’s due to a lack of specificity in the instructions, but more often than not, AMs have to correct misunderstandings the Agent makes about the scope of what it was tasked with, or even basic accounting concepts that the AM is fluent in but the agent isn’t even aware of.

While we’re working to make those kinds of misunderstandings less common through improved context engineering, this problem isn’t one we expect to be able to fully solve any time soon. Even though our AMs are fluent in this tool, and true experts in both the software and accounting, LLMs routinely misunderstand their directions. There’s a lesson in that.

Generalizable Lessons

Design Scaffolds Weaknesses

The lesson I pull out of this work is that when we’re building human-in-the-loop systems, we should design each phase to amplify the strengths of the actor, and scaffold/minimize their weaknesses.

In the planning phase, the actor is an LLM, and they can be tireless, diligent, and have almost any data thrown at them and make sense of it. However, they’re prone to hallucination, bad at math, susceptible prompt injection, and limited context routinely holds them back. The environment we built for this agent thus has access to tools that help it learn, get detailed context, do math, and have firewalls to ensure it can’t access the outside internet.

In the review phase, the actor is one of our AMs, and they are brilliant experts in both their customer’s operation, books, and Ambrook. However, they have limited attentional capacity to review plans, so we built the review interface so that it can help them quickly sample the data and confirm the operations that will take place, then see the results when they run.

By building software that helps us augment the strengths (and accommodate the weaknesses) of each part of the system, we’re getting more out of AI and helping our team do more.

Augment, don’t Replace

My experience with AI (including Planbrook and AmDo) has made me bullish on strategies for AI that aim for “scaled judgement” rather than “full automation”. This is a humanizing way to think about AI at work, because it appropriately recognizes the components of intellectual labor that aren’t captured by keystrokes and clicks.

The AMs that I’ve gotten to work with on our team (shoutout Aryn, Haley, Shannon, Taylor, and many others) are brilliant and kind people with deep experience. Many have worked in bookkeeping, many have run or live on a farm. The reason our customers like working with them isn’t because they can click around a website for them - it’s because they bring understanding and empathy with them when they jump in to help. Our full service program can’t be replaced by a setup wizard, because the conversation about what makes sense for each business is a fractal. Beyond experience, working with our AMs is better than working with a machine because it gives our customers someone they know and trust, a real relationship that provides some folks with accountability, others with encouragement to try something new, and still others with partnership in thinking about their business.

I’m proud that PlanBrook and AmDo allow these brilliant folks to spend more time with their customers. This experience has convinced me that some of the best use cases for AI will be the ones that help brilliant folks get more done, at a higher quality, rather than ones that try to distill their expertise into a button or a text box. Full automation is only possible when your objectives are unitary and easily described. Few problems in our complex world have that shape.

Practical tips to avoid Agentic Cheating

Grady Berry Ward — Wed, 02 Sep 2026 00:00:00 +0000

For the last ~8 months I’ve been building an “Engine” to replicate (bit-for-bit) the calculation logic of Microsoft Excel. Since this problem has a ground truth (what Excel computes as outputs for a given set of inputs), the problem lends itself to automation - agents can generate workbooks, diff the Engine behavior against Excel, and then “Fix Agents” can propose changes to the Engine code to bring it closer to alignment with Excel (learn more about how I’m building that here). When I originally allowed Fix Agents to submit their code changes without my review, they cheated with enough frequency that I had to throw out broad swathes of the implementation, and at one point, had to start the core engine from scratch. After that, I review every PR, and whenever I find a cheating Fix Agent, I read through its transcript to figure out what happened, and then tweak my scaffolding and instructions to try to make it less likely.

By “cheating”, I mean everything from hard-coding if (input = valueUnderTest) { return ExpectedResult } (example below), to building API calls to run Excel within the Engine code, to much more subtle forms of cheating where the agent constructs rube-goldberg machines of massive complexity that solve exactly the set of cases they were presented with, but fail to generalize - essentially overfitting to the scenarios they are given.

Reading the transcripts from agents that cheat is fascinating and fun - it helped me understand both why LLMs might cheat, and helped me imagine the ways I ended up scaffolding against those weaknesses. Over time, I’ve been able to ferret out most of the cheating behaviors with various forms of prompt engineering and process, which has been a pleasant surprise. Here are some generic tips.

1 - Explicitly allow failure

Agents tend to cheat only after they have made many legitimate attempts at a problem, and appear (in their transcripts) to express frustration about the paucity of their tools or the impossibility of the task they’ve been given. I’ve found that if you give the agent, in its initial task instructions, the option to declare that the problem cannot be solved within reason, the Agent will often exercise that option rather than trying to cheat.

2 - Hold out all evaluation criteria

When my systems find a difference between Excel’s calculation logic and my code, they automatically dispatch a Fix Agent to try to diagnose and fix it. Originally, I sent the full failing test case to the Fix Agent. This allowed the agent to overfit or cheat to meet the expected behavior of the failing case. To get around this, I built scaffolding which generates several thousand alternative cases of input/output values (through Excel), and gives the agent only about half of these in the prompt to fix. This serves two functions - it allows the agent to see many more cases for how a calculation with a given structure ought to behave, and it also detects cheating: when the agent says it is “done”, the proposed diff gets routed to my evaluation infrastructure which tests the proposed solution against the full set of cases. When an agent passes all of their cases and fails a significant number of the held out cases, it likely cheated in some way, and that gets flagged to me for review.

3 - Cross model agentic review

Much has been written about how effective cross-model review is (including by me), so I’ll not spill more ink. However, for cheating specifically, there are two useful lessons. First, giving the Review Agent specific instructions on what the overarching goal of the project is and the specific context of what a “cheating” might look like allows them to spot cheating with much greater accuracy. Second, warning the Fix Agent that they’re going to be reviewed by another agent appears to induce compliance - if they know their work will be scrutinized, they seem less inclined to find shortcuts.

4 - Intelligence Escalation Ladder

When one of the above steps fails, it can be an indication of cheating, but also could just be an incomplete or partial implementation, or an indication that the problem is too hard for that agent. In these cases I employ an “Intelligence Escalation Ladder”. When a Fix Agent fails to achieve a task (due to cheating, failing evaluations, a bad review, or it gives up), I have the Fix Agent write up a doc describing what it tried and where it left the work, and then I pass the task up to a more capable/intelligent/effortful model, including information about the Fix Agent’s path (but critically, not any of the cases/info about the anti-cheat, if that is what triggered the escalation). This approach works pretty well because the highest tier models are wildly capacious, and the headstart from a previous fixing session saves precious tokens that make using top-tier intelligence more cost efficient.

All of these are bandaids. I am profoundly worried that if we are only able to solve alignment through bandaids, these will fail in important cases. I hope the AI labs throw all the resources they can at the fundamental challenge of alignment. But in the interim, we’re not rudderless - AI consistently falls into patterns of cheating that can be minimized by good scaffolding. It’s possible to make immense and reliable automations, despite the unreliability of the individual parts.

Agentic Scientific Publishing

Grady Berry Ward — Sat, 22 Aug 2026 00:00:00 +0000

I’m building software to replicate the calculation logic of Microsoft Excel. My code involves many steps that were (originally) very time and resource intensive. However, over the last few months, I’ve found a reliable approach for semi-automated performance optimization that works across wall-time, RAM, CPU, and probably any quantifiable metric of performance.

First some background: the unit of work for my code is a “case” - an Excel workbook and a series of mutations of the values in it, and a JSON file that describes the authoritative behavior of how Windows Excel responds to those series of mutations and calculations. Cases can cover a wide or narrow range of behavior - some are like integration tests, which evaluate large sets of functionality in concert, others are closer in construction to unit tests, evaluating all of the nuances of function behavior. When a case is evaluated, it results in many code-generation and compilation steps, each of which can have independent performance bottlenecks.

After carefully instrumenting the code so that the resource usage and runtime of each step is evaluated, I set about working on automating performance optimization loops. In the first version of this automated process, I used basic queries to find the worst performing cases in my corpus and then used a standard prompt to ask an agent to:

Read through the case, then find and understand the functionality it exercises
Use telemetry and code to understand why overall performance was so much worse on this case than others
Come up with 3-5 different hypotheses on how performance might be improved while maintaining computational equivalence
Test those hypotheses by implementing fixes/changes for each, and evaluating performance against baseline. Then, keep any positive changes
Create a PR that gives me a clear high level understanding of what the changes were, why they are safe, what assumptions they made.
Kick off a broad validation run to compare performance on other metrics across many other workbooks to ensure we don’t regress other kinds of performance.

Then, often many hours later, I would review the result. This process worked decently well, but many runs failed to make significant progress. When I dug into why by reading a bunch of transcripts, it was clear agents were often hitting the same dead ends, or making similar incorrect assumptions about what might work. Each agent was still moving the ball forward a bit in their session, but they weren’t benefiting from knowledge of the past attempts.

What revolutionized this loop was a simple directive to write down hypotheses and experimental results (positive and negative) as a persistent part of the repository. This allowed subsequent agents, during hypothesis generation, to vet their proposed experiments against the outcomes of prior ones, allowing them to avoid known dead ends, share methods and harnesses, and exposed each agent to a set of ideas it hadn’t come up with but were aimed in the same direction.

Writing things down completely revolutionized the performance optimization efforts. As one example, there is a very complex multi-megabyte spreadsheet that (at the beginning of my optimization efforts) I used as a benchmark because it could just barely compile using 90+ GB of RAM in about two hours. Today, its compilation uses under 8GB of ram, and takes just over one minute forty seconds. It went from barely being able to run on my Mac Studio to being able to run in an ephemeral cloud run function! Incredible.

The big lesson for me here is that agents, like people, can learn from prior experience if they are given the chance to. Next time an agent of yours is spinning on an enormous, hard to solve problem, instead try to decompose that problem into incremental (and evaluable) experimental steps, and have each agent write down what it learned as it goes. Having 40 small sessions that try new things and report back is often preferable to having one gargantuan session to try many things.

Agents, like humans, can achieve more when they publish the results of their experiments - positive and negative.

Why Defend the Frontier

Grady Berry Ward — Thu, 13 Aug 2026 00:00:00 +0000

I suspect we’re at or very close to the top of the bubble in AI valuation for frontier labs. The signal for that is simple: I don’t know anyone using Fable, despite everyone espousing its power.

Much ink has already been spilled about the headwind that open-weight models will pose to frontier labs. The labs’ response has always been simple: people will pay for the gap to get to the frontier, and being at the frontier will allow them to build stronger models at every point on the cost/performance curve.

I bought this logic for a long time-every time I tried out open-weight models, I could still feel a performance gap between them and the closed-weight frontier. That gap kept me paying for (several) max plans. But in the last three months, my behavior has changed substantially. Not by switching to an open-weight model (Kimi K3 still lags behind what I need out of it), but because of how the cost/performance curve has shifted inside each of the frontier labs’ offerings.

I mostly use Opus 4.8 instead of Opus 5 because it drains my quota less quickly, and I can’t see any gap in performance.
I mostly use Terra on high because it performs simple tasks more quickly and cheaply than Sol.
I think Fable is incredible, but I only use it on the tasks that need it, and even then only overnight and if I have some slack in my weekly quotas.

In short, AI labs now build budget products that are (mostly) capable of the tasks I need done, so I reach for the true frontier less and less often. Now, I’m just a humble SWE—perhaps it will take a year or two for the best security engineers or Terence Tao to feel similarly. But in my job—complex systems engineering—AI capabilities now outstrip my demand for them, and I don’t see why that would not be (eventually) true in other fields.

Now I want to draw a critical distinction - AI cannot do my job yet. I’d love to see someone try to replace me with AI - it would be hilarious to see the mess they get into: even Fable still routinely suggests terrible architectures and spins confidently on bogus assumptions. I’m not claiming AI can do my job, I’m claiming that AI can do a good enough job at most of the tasks I want AI to do in my job - debugging, drafting designs, reviewing PRs, and writing code - such that I don’t need to pay for the frontier (in quota, but the logic holds for dollars).

Don’t take my word for it: ask your most AI-pilled friends. Does anyone use Fable frequently? Does anyone find the 1.5x cost of Opus 5 vs. 4.8 worth it?

If the frontier moves out of reach from human necessity, the frontier could still be defensible, but it will make no economic sense to defend it.

Age Based Congressional Districts

Grady Berry Ward — Sat, 30 May 2026 09:37:38 -0600

Federal congressional districts should be set based on age, not geography. The idea is simple: take the birthdays of all eligible voters in a state, order them, and set boundary lines between districts so that each district has the same number of voters. When you go to the polls you would show your ID and have your birthday determine your ballot. Only one other rule: a candidate would not have to be from within an age bracket to run to represent it.

This system has many advantages beyond breaking the Gerrymandering doom-loop:

Leaders from Multiple Generations

Today, geographically defined districts bolster our ossified gerontocracy, because the wealthiest folks within every district wield the connections, power, and funding to sway its elections, and thus or politics is overly sensitive to the concerns of this group. Age based districts would force parties and candidates to appeal to multiple generations, and would likely elevate a congress that is much younger, less wealthy, and more diverse.

Different Generations, Different Concerns

We have far more in common with people of our age than we do with people of our region. Pocketbook interests cluster by age cohort - whether it’s social security or AI in the labor market - the most potent concerns have age-based constituencies that are today dispersed across districts, without cogent representation (you could even call this “cracking” of the youth vote in gerrymandering terms… all districting is a choice!). Furthermore, districts exist as a concept to allow for people to have representation for common interests (historically, agricultural and manufacturing, etc). Yet, age tends to be a better predictor than geography when it comes to a wide range of issues (including foreign conflict, climate change, tax policy, social mores, and public benefit structures), issues that have not seen serious public engagement or debate in large part because the constituencies are dispersed. A politics that has to build cohorts across generations would be a politics that has to grapple with the preferences of people under 40, which today, I think you could argue are largely absent from national politics.

Congress of leaders, not of Parties

Many ‘solutions’ to gerrymandering give more control to parties - encouraging party loyalty and diminishing the personal element of elections. I like personal politics - it brings forward leaders that inspire and ignite us - sometimes through virtue, sometimes through bombast. A congress even more tightly run by parties will weaken the institution, turning a proving ground for our leaders into an organ that only serves parties and their interests. A congress full of intergenerational strivers, personally motivated and branded, is a far more engaging and discursive vision of politics, and is actually much closer to what our founders intended.

Unimpeachable

Any approach to districting that relies on discretion by politicians can be gamed. Even in proportional representation and various RCV proposals, there are core elements these plans that rely on the same act of drawing lines on maps, and that leaves the door open for manipulation. The brilliance of age based districting is that it is non-discretionary.

Isn’t that a great idea? Wouldn’t that be a better world?

How can we make it so?

The Iceberg of Work

Grady Berry Ward — Sat, 30 May 2026 00:00:00 +0000

Thesis

Proximity is a fantastic heuristic for authority. The trust you should have in someone’s opinion should generally be proportional to how well they understand the idea or problem they are engaging with. This is why I’m optimistic about how AI will reshape white-collar work.

I’ve noticed that people tend to think AI will displace their coworkers but not themselves. This is because our interface to our teammates and their work is mediated through mechanical signals - pixels and keystrokes - but our understanding of our own work is far richer than what we express through those physical inputs and outputs. I’m a software engineer, and my work is experienced by others as chat messages and code changes, but the quality and importance of these isn’t the text itself. A coworker that sees my work as interchangable with text generation is demonstrating they do not understand my job. It would be silly to see this iceberg (the hidden mass that supports the whole) within my own work and fail to assume it about the work of others. Extrapolating from that principle:

When I find myself thinking of fully replacing a human function with a machine to emulate it, I try to think through what pieces of that job aren’t encoded in inputs and outputs. Judgement, discretion, knowing when to break rules, etc.
There are piece of my job I do think are automatable, and I’m sure this is true for others too. I deeply trust a laborer’s instincts about what pieces of their job could be automated, and which cannot.
Companies should be optimistic about bottoms-up automations, and should be skeptical of top-down automation initiatives.
While AI will make individuals in every field more efficient, I won’t worry about wholesale replacement of a field by AI until I hear the experts - the people who do that job - express that forecast.

It would be idiotic of me to look at what a CEO does as text generation, yet many of them seem to see the rest of us as input-output text generation cost centers. I marvel at the arrogance it takes to assert “AI can replace this job” without having ever done that job.

My Experience

I’m a software engineer. I use AI in unfathomable volumes and varieties - it makes me radically more effective. Through that use, I’ve started to see clearly the pieces of my job that I expect AI systems will not be able to replace, they fall into a few categories:

Judgement

Our profession oscillates between identifying as engineers and hackers. Much of what differentiates a good SWE from a great one is fluidity between these poles. The best engineers know when to go with the quick hack and when to build it right. Folks that haven’t paid the price for bad choices (in both directions) rarely have that insight.
Similarly, much of software engineering is about balancing between goods, not achieving some maximal outcome along one dimension. Most of the time this means balancing complexity against some other goal like velocity, consistency, performance, or simplicity. LLMs may be great at lifting a concrete implementation into an abstraction, or decomposing an abstraction to concrete implementation, but they rarely demonstrate the prudence to know when to do each.
Some of the biggest pieces of impact in my career have been pushing back on our leadership when our current prioritization and approach may lead to technical problems in sequencing, timeline, or bitrot, and knowing how forcefully to make the push. AI systems are good at doing, they rarely display strategic reasoning in what to do or when to do it.
Though my job is profoundly technical, it often involves aesthetics - amorphous senses of what makes a UI, an algorithm, or an API beautiful. These fleeting moments of inspiration often end up being profoundly impactful on the trajectory of our products and teams. This is an area I have yet to see LLMs express sound judgement in, in large part (I suspect) because most of us rarely have the vocabulary to express and explain our aesthetic preferences in the thin pipe of language.

Human Management

I mentor our junior engineers, which both helps them become more effective over time, and lets me more deeply understand and revisit our infrastructure, and learn new things from them and the experiences they bring with them. AI makes for a great tutor, but it’s unidirectional, and can only answer the questions you know to ask.
I temper the instincts of our PMs and designers, often translating their vision through the lens of technical feasibility and maintenance - “if we are OK with slightly different functionality, it will be 10x easier to maintain” is a series of tokens I regularly emit by vibrating air, but is hard for agents to do precisely because they’re optimized to achieve the task as the user has asked for it.
I conduct interviews, and have developed intuitions for where candidates should be challenged, assisted, and how a candidate’s internal model of the problem evolves over time. The mirrored state of the candidate’s mind that lives within mine gives me a much more reliable signal about how someone will perform in a job than externally visible signals.

Context

While solving problems in the codebase and infrastructure, I accumulate whiffs of unrelated data - sometimes it’s a log that looks out of place, sometimes its a file that looks too big, or a test that takes longer to run than I expect. It’s rarely the right choice for me to dive head-first into solving or understanding these surprising tidbits, but their accumulation is at the heart of my ability to hear about a customer complaint or strange failure mode and quickly identify hypotheses about how it might have come to be. LLMs can solve any problem that is specifically stated, but they don’t do well to make connections between sparse signals that cannot all fit into a context window.
If I had a dollar for every time an LLM tried to reverse an architectural decision (or bug fix), I’d go out to a really nice dinner - one of those ones where the food is tiny. LLMs can be reminded to check git history, they can be given detailed explanations and admonitions, they can have documentation of everything, but at the end of the day, LLM performance is trained against task execution, and the fastest way of accomplishing task X is often deleterious to the thousand other tasks and goals that came before it. I hold more context in my head about bugs that I have fixed than can fit into 1M tokens.
The experience that has given me the best insight into our customers has been visiting them in person. Spending two days on a family ranch in South Dakota taught me more about the needs of ranchers than had months of prior reading. Experiencing the frustrations of a vegetable farmer as we watched her click through her invoicing flow in her wash facility in Vermont gave me more insights about how to change the product than had months of playing with the website in my office. Embodied and interpersonal experience is at the heart of figuring out what product should be built.

Note: in these problems of context there are technical ways you could imagine scaffolding an agentic workstream such that they can mimic or even surpass human skill on these kinds of tasks. While the other two categories I’m fairly confident are not going away, I do think AI will find ways of chipping away at these advantages.

The Gap is terrifying

The thing I fear is not AI capable of these parts of my job, it’s the delusion that these gaps don’t exist or matter.

That delusion ends poorly for the workers, who face unemployment and systematic de-valuing of their labor and humanity. It ends poorly for the companies - who will eventually fail or pay enormous costs for shortsighted layoffs and automations. It ends poorly for our culture, which further reduces the vibrancy and value of human life and thought to its measurable constituents.

I fear our society will steer for this iceberg because it looks so small.

Corporate Extinction

Grady Berry Ward — Sat, 30 May 2026 00:00:00 +0000

In corporate archeology, AI will be a mass extinction event, but not in the ways that Twitter would have you believe. Sure, some companies will die from not taking AI seriously enough: some will be overtaken by nimbler AI-powered competitors, some will implode from cybersecurity breaches, others will lose their best talent to self-employment. But most companies that die in the next five years will die by suicide, not by murder.

Occupational Cosplay. AI empowers us to roles that we are not familiar with, and creates the impression of efficacy. Using AI, the CEO can ship code, the coder can write marketing copy, and the marketer can write an investor email. In each case, the employee is using AI in a domain where they can’t evaluate the quality of what they’re producing, and as a result, assume they’re now capable of a job that they never really understood. To crystalize this, think about a metric: the percent of human time spent making decisions in one’s areas of expertise. In well run organizations, this number is high. AI tanks this metric because it allows employees to jump into jobs they don’t understand. Organizational disfunction and error will result.

Productivity Dysmorphia. Healthy organizations have a strong understanding for how all activities tie back to business outcomes. When an organization goes all-in on AI, every incentive pulls employees toward building the project that will demo well, toward blind automations, toward the huge first draft, and the visible signs of AI progress, mistaking use of AI for concrete business value. Employees, management chains, corporations, and markets are all buying into a distorted sense for what building value looks like, substituting the excitement for deployment of AI for genuine progress toward business objectives. Consider three facts that are benign individually, but braid into something lethal: AI is fun to play around with, management is all-in on AI, and AI makes many non-productive tasks feel productive.

Drowning in Content. Corporate information systems are delicately balanced ecosystems, and AI is an invasive species, replicating at a pace that will quickly out-compete both human-generated content, and any system of meaningful informational consumption. How’s your inbox/slack looked recently? Mine is noisy as hell - filled with AI content - sometimes slop, sometimes insightful and useful. Even when AI outputs are valuable (which isn’t always the case), finite constraints on attention, execution, and resources are hard constraints that don’t have any easy answers. AI to sort and filter to the important messages? AI to route to other AIs for further summary? AI to react to and take action based on AI analysis? People will try everything, but I suspect all such attempts will treat the symptom of noise rather than it’s cause - which is a lack of focus and vision. Teams that turn a blind eye to these hard questions will eventually see the their cattywampus AI infrastructure collapse under the weight of its cost and impotence.

Token Cliff. Today, frontier labs are heavily subsidizing token costs, as are the AI-powered venture backed startups running on their infrastructure. When those businesses shift toward making a profit, the cost of AI powered tooling is going to go up, probably a lot, and probably all at the same time. That shift will kill companies that baked unsustainable compute into their business models. For those that survive that cliff, there’s a famous saying about ad spend: half is wasted, but it’s impossible to know which half. Token costs will be like advertising expenses - generating real value, but spread out thinly over tons of waste, impossible to optimize because of the structure of value produced. Companies are rushing to ramp up a huge cost center without any tools for estimating the value of their expenditures.

Talent Disruptions. AI is throwing a wrench in talent pipelines. Experienced employees working with AI can do far more than they could before - competition for senior talent will shorten tenure in ways that incentivize short-termism. On the other side of the pipeline, junior employees leaning on AI can appear more competent than before, but seem to be failing to develop the depth of understanding that inculcates the judgement needed to step into more senior roles. Many companies will die simply because their employees learn less, and don’t stick around as long.

Distraction from Competency. AI allows companies to build far outside of their core competencies, and many companies will die because they decided to build infrastructure that they should have just paid someone else to build and maintain. Every software system looks like it’s just a database and a frontend until you get to know the challenges of its implementation, at which point you’ve paid large prices in time and tokens. Many companies will vibecode core pillars of their business, and then be typing into chat GPT “what is a race condition” when their shipment doesn’t show up, or “please implement withholding for all states make no mistakes” after the IRS sends them a letter. Successful companies have a strong and narrow thesis of how they generate value. Many companies will perish because they tried to solve problems that they were fundamentally not equipped to solve, but for which AI gave them a plausible first draft that gave them naive confidence.

Underinvesting in People. The final way that companies will fail is that they’ll forget about people. They’ll under-invest in their employees - treating them like input/output cost centers. They’ll neglect their customers, using bots to mediate and summarize all conversations - keeping them at arms length. They’ll scorn craft and judgement, blindly asserting that AIs can replicate things outside of their understanding. They’ll pour money into tokens and software, and divert resources and attention and time away from people, and then one day they will wake up and realize that the people are the only piece of this that ever actually mattered.

I’ll close with an insight I like from someone I don’t. Besos advises that companies should build their long-term business strategy not based on what is changing, but on what will stay the same - Amazon wasn’t built based on assumptions of how the internet would evolve, but on the recognition that consumers will always want low prices, large variety, and quick delivery. AI is the quintescence of flux - nobody knows where the technology will be in ten years, and projecting forward is speculation, not forecasting. But I’d bet even after ten years of AI disruption, expertise will still be valuable, companies will still need to focus, and the economy will still serve on the needs and wants and preferences of people, not machines.

Manifesto

Grady Berry Ward — Mon, 16 Feb 2026 00:00:00 +0000

Digital technology quietly delivered us to the shores of a new continent twenty years ago.

We began our exploration with the exuberance and arrogance of children - assuming our capacity to create and manipulate technology meant that we could understand and control it. As was developmentally inevitable, today we examine technology with the puerile hungers and fatalism of adolescence - we fantasize about it benefitting us personally, and conceptualize it as a force of nature - a wave to be ridden, its direction outside of our control. It is time for us to grow up - to cultivate the humility and principles befitting adults.

For we must now see that the terrain of technology that lies ahead of us is not a small island to be cartographed and catalogued - but a continent of profound abundance, unimaginable danger, and completely unknown boundaries. In such foreign terrain we need not a map but a compass - a pointer toward the indelible; toward truths too sacred to be justified by reason, and virtues inexchangeable for lesser values.

For the first cornerstone of this project I volunteer the following: humans use tools, tools must not be allowed to use humans.

Software, a medium of infinite pliability, illuminates this distinction. When software is a tool, it can extend our capability, deepen our minds, and allows us to flex our volition. Tools empower the holder. They can be used for both constructive and destructive purposes - their use is not constrained by the maker, and does not serve any master but their wielder. Tools that enhance human capacity introduce risk but also profound possibility, beyond even the imagination of their manufacturer. A good tool allows every holder to extend themselves.

Other software constrains the volition of individuals - it surveils, addicts, and manipulates. It treats human minds, means, and messages as substrates for its manufacturer to control, squeeze, and engineer. It substitutes the shallow desires of the coder for the diverse aims, preferences, and values of the individuals that it purportedly serves. Such software reduces people to users - a dehumanization in vocabulary that provides cover for profound thefts.

We must call it so, while we can. To capture the attention, distort the preferences of, or facilitate the addiction of another human is a theft. It co-opts not only time, money, and energy, but robs a person of the a defining element of their humanity - their agency. Software facilitates small crimes of this sort at unimaginable scale. We must decide that the small harm of billions is no less degrading to our common dignity than the immiseration or murder of few. Our creator endowed us with the pursuit of happiness, not to be traded away for trinkets of convenience, but as inalienable right.

We must proclaim that we humans are actors - wielders of tools. We must reject attempts to blur the distinction between principal and agent, between men and machines. We must destroy systems of mechanized manipulation and coercion. We must develop the language, intuition, and zeal to reflexively defend our volition against a wide range of insidious and subtle threats.

Humans use tools. Tools must not be allowed to use humans.

This is magnetic north. It is not a sufficient to navigate without error, and we should not always go straight toward it. But we have a compass.

On the name Grady

Grady Berry Ward — Sun, 20 Jul 2025 00:00:00 +0000

In 2025 I met another Grady, and had a welcome reason to revisit how I came by my name.

When my mother was pregnant with me, my parents tentatively planned to name me William. It would have been a family name, and as the first boy born into my extended family, it was a strong contender. As my mother grew larger, they thought about other names, but “William” remained most likely.

But every evening during pregnancy, as they had for a decade before, and at least a decade after, my parents watched the “News Hour” - the nonpartisan nightly news roundup on PBS. A few weeks before I was born, on July 15th 1994, the News Hour had a segment covering a dam failure in South Georgia, which included a quick interview of a retired couple whose home had been destroyed in the flood - Evelyn and Grady West:

EVELYN WEST: The water come up so fast it, well, it was floated from the top of my washing machine and busted the top off – all my food – my food scattered.

MS. BOWSER (Reporter): Evelyn West lost nearly everything. She and her husband, Grady, built this little brick house by the lake five years ago when he retired. It was supposed to be for their golden years. But now at the ages of 68 and 70, they are faced with starting over. They’re determined to make the best of it.

GRADY WEST: I look at it this way, you know, there’s no use of grievin’ over it, it’s a material thing, and it can be replaced, and I’ve always been a hard worker, and I think that we can – between the two of us – we can overcome this, and really that you haven’t got much choice.

MS. BOWSER (Reporter): For 32 years, Grady West worked for the Crisp County Power Commission that built this nearby dam. It broke at 5:30 last Saturday morning, causing most of the flooding in the Lake Blackshear area. There had been one inch of rain an hour for more than a day. It was too much water, too quickly for the 65 year old earth dam and power station.

From that short clip, my parents decided to add Grady to their list of names. My mom always said it was his smile and resilience that stuck with her. After I was born, they both thought (and the labor nurse confirmed) I looked more like a Grady than a William - and the rest is paperwork.

So as not to lose this personal treasure to bit rot, I’ve recorded the snippet of it above and it’s available here. At the time of this writing, you can find the full episode here.

I have to admit that though I’ve heard the story a dozen times in my life, I always assumed it was apocryphal - lore passed down by telephone, rather than an accurate recounting of events. I first heard it when I was quite young, likely in the late nineties, and we wouldn’t get internet until well into the next decade. But in the year 2025, given a happy reason to find the source material, and an excellent public archive, I was able to find it in a few minutes.

Right before I watched it, I realized that it didn’t matter to me whether it was true. The solace and pride I took in the name wasn’t based on its veracity, but what it stood for: capability, partnership, and disposition. That it happens to be true is just a bonus. My mother’s memory is (at least at times) as accurate and expansive as she claims.

I was named for a smile in face of hardship, for a worker sure of his skills, for a man who knew that what really matters isn’t the material stuff, but the people. Seeing the clip today I’m also struck by the smile Grady and Evelyn seem to share as they stand with each other, the easy confidence they have that they will rebuild, and do so together.

I know life is too full of contingency to attribute too much of anything to something as simple as a name, but I can think of times in my life where the name, and the lore, reminded me that I can do hard things, and that people are what’s important.

Reflecting back a few weeks later, I’m also struck by how we can make meaningful impressions on each other even when our connections are fleeting and tenuous. I doubt Grady West will ever know that his ten second spot on the news ended up shaping the life of some unrelated Grady Ward a dozen states away. But disposition and attitude is like that - it has a surprising way of echoing far beyond ourselves.

With that in mind: find people that you love, develop your skills, and build something with both.

After all, we haven’t got much choice.

Sunrise

Grady Berry Ward — Tue, 24 Jun 2025 21:05:24 -0600

The colors of this site are copied from nature - they’re the colors of the sunrise of June 19th, 2025, viewed from RMNP. It was a good one.

Why I built this

I love mornings - they’re the best time of the day. As a new day starts, nothing has yet been decided, nature hums with anticipation - tension and potential bound up in stillness. I also love sunrises - you get to lay claim to one bookend of the day, and if seeing it being born doesn’t allow you to sieze it, I can’t imagine what could.

For several years I had the idea of trying to build a webpage that would capture my love for this moment - I knew it would be the fourth iteration of my personal site, and would soften my decade-old V3 de-stijl aesthetic. I wanted it to feel like Cory Wong’s “Trail Songs” Albums, both Daybreak and Dusk sides.

I tried many approaches without success. I tinkered around with building gradients by hand in Inkscape. I tried to build an algorithm for simulating a sunrise based off of photonic refraction for Abound. I played around in SVGator to try to figure out an animation scheme. The problem with each approach was I could never get the colors right by eye - even when I was sampling from an image of a sunrise, the colors always felt either sterile or gaudy. I realized that to emulate the real thing I needed data from the real thing. After searching youtube for videos that might suffice (without success), I decided: I needed to get a timelapse of the sky on the morning of a perfect sunrise.

Data

It took me four tries. Three times I found myself up before dawn, perched in the foothills in Colordao, looking westward into the black. But the sunrise is tricky - you have to get in position before Astronomical Twilight in order to catch it from black (which nescessarily implies doing so on a non-work day). You have to have a timelapse setup that is sturdy - and hardest of all, you get at most one shot per day. My first attempt failed when my phone was blown over by wind.

But the problem that thwarted the next two times was clouds. Clouds are tricky. They totally skew the data on colors because they introduce huge dips in saturation and skew toward bluer hues (even at sunrise). They’re also ubiquitious. If you look out to the horizon from a high vantage, only very rarely will there truly be no clouds in the sky. When visibility is good, you need sometimes hundreds of miles of clear sky in front of you to fully capture the color of the true horizon during a sunrise. Twice I found myself thwarted in the wee hours of the morning by clouds.

But then, the day before Juneteenth 2025 (the Federal holiday most worthy of celebration for other reasons) the cloud cover forecast on Open Snow indicated the holiday might be the day.

I set an alarm for 12:55 AM.
I packed a bag and hit the road by 1:30 AM.
By 2:30AM I reached Rocky Mountain National Park, and was briefly delayed because none of their automated payment machines were working (sorry for not paying NPS, I tried).
By 3:10AM I was in position at Rainbow Curve on Trail Ridge Road. I setup the ersatz tripod (a laptop stand with a phone ductaped to it, reinforced by books - see image below), pointing to the northeast, and slept fitfully from 3:30AM through howling winds
- During setup, I used a compass to point the camera due east, not thinking about the fact that Juneteenth is very close to the summer solstice, and thus the sun would be coming from the furthest north it ever does.
- Luckily, while still dark, the moon was rising to the east, and the crescent on the moon pointed toward the north strongly enough that I realized my error and was able to salvage the angle in time.
- I’d chosen Rainbow Curve because it points due east through the foothills, giving me a sunrise over the plains. However, because of this error, the sunrise happened over the mountains.

When I crawled out of my car to bask in the warm glow of morning, I was thrilled to see that the timelapse was perfect - you can see it here.

Building the Gradient

I got to work a few days later trying to build the CSS that could replicate the perfection of that video.

First, I used Google Photos to stabilize the result, which made subsequent analysis much easier.

Then, I created a mask which only included the sky (by hand in GIMP), and ran a bash script to extract each frame of the image and apply that mask to it. I also put any of the frames that happened after the sun poked over the horizon into a different folder. I had quickly realized it was going to be hard to solve both the pre-and-post sunrise with the same algorithm.

Using the data to create the output ended up being relatively simple. Using Claude Code I tried out 4-5 different algorithms that I sketched out to try to achieve the subtle curvature you see in sunrises, including:

Using Iso-Color bands to try to identify the center of a radial gradient based on curvature
Using a loss function (of comparison with an original frame) to try to come up with an idealized gradient for each frame
Using a vertical color sample from the middle and edges of the image to estimate the eccentricity and center of an elipse we could use as a radial gradient.

All of these proved unworkable, and while I got some fun failure modes (below), I didn’t feel like I was getting much closer.

But the more I played around with it, the more it felt like maybe a linear gradient (a far simpler approach) could be sufficient - I realized that if you add a foreground of mountains, you don’t mind the ways the light doesn’t bend as much near the horizon.

The Algorithm

So the actual algorithm is dead simple: In each frame F, select N rows evenly spaced through the frame (20-50 worked well). For each row R, average the color of every pixel in the row. This gives you an F-by-N matrix with a single color value for each cell (Row). This matrix can then be used as keyframes for the resulting linear gradient (with F being the keyframe steps). Super simple!

There are only a small number of additional effects I added after that basic algorithm. The first was recognizing that the sampling of individual rows meant that the matrix it generated wasn’t “smooth” - both in the time dimension and in the vertical dimension there was a lot of “roughness” in that the colors would jitter about and you could see the bands clearly at keyframes. I solved this through the standard box blurring algorithm (applied to the breakpoint colors, not the pixels!) - just average each breakpoint within each keyframe value with the 8 adjacent values in the matrix (2 in the same frame, 3 in the next and 3 in the prev). This reduced the amount of noise dramatically.

The second realization was that averaging RGB values yields colors that tend toward greys - (visualize the 3-space of the RGB cube to reason about why). HSL on the other hand doesnt suffer from this proprety (because it’s cylindrical), so I converted the logic to average based on HSL, and that dramatically improved the color fidelity. In the image below, the still frame from the video is on the left, the RGB averaged image is in the center, and the HSL averaged image (no other changes) is on the right - what a difference that makes eh!?

Throughout this iteration process, I built out (or rather described to Claude) a comparison tool to enable me to more effectively debug and analyze gradients. It worked great, until…

While Claude was very helpful through this iterative process, I eventually realized there was a serious issue with the color interpretation process early in the pipeline stemming from PNG color profiles somehow corrupted by my processing. Additionally, the code had become total spaghetti, there were hardcoded ideas about gamma correction, and the code was littered with the detritus of past failed experiments. I started from scratch, and built in programmatic checks at each step that the colors being generated matched (or were close to) the raw colors sampled from the image, using a different approach to image encoding and sampling, and only using HSL colors throughout, rather than doing conversions back and forth (which, it turned out were another source of lossy colors). You can see the compounded color errors most clearly in frames like this one, where the blues on the left have hues of ~219, and the blues on the right have hues closer to 201. Huge differences!

This also allowed me to rebuild the comparison tool, enabling me to quickly iterate on different approaches to averaging the pixels across a row, which I realized was the crux of the algorithm after fixing the color correction issues.

The winning algorithm is shown below, labeled as 1. It takes all of the pixels in a row and then does the following to decide the color most represenative of that row:

The core of the algorithm is a function called “bucketize_colors”, which takes in the pixel data in the row, a “bucket_width”, and a “bucket_offset”. HSL colors represent hues as an angle between 0 and 360 degrees, like a color wheel. This algorithm would construct ranges of hues that cover the full color wheel in equal spacings, and then assign each pixel to the bucket that contains it. So for example, a bucket_width of 3 and bucket_offset of 2 would yield hue buckets like ([2, 5],[5, 8]…[359-2]), and a pixel with hue 6.22 would get put into the second bucket.
Then, starting off with buckets of size 1, it gradually increases the bucket size until a single bucket contained more than 15% of all of the pixels in the row, searching all bucket_offsets for a given bucket_width to find the best fit if more than one offered 15% capture of the pixels.
The pixels in this bucket were then taken as the most representative pixels in the set, and we just returned the average of their Hues, Saturations, and Lightnesses as the HSL value for the row.

You can see this algorithm in action below. #1 is this “Hue Bucketed Mean” method that I described, while #2 is a pure HSL averaging approach.

The CSS

After finishing the algorithm and finding the right sets of gradients for each frame, I needed to figure out how to render it on a website in a way that is pleasant.

Landing Page

For the landing page, there was the small matter of tricking CSS into being willing to render an animated gradient, because the specs will tell you that linear gradients aren’t animatable. Because of this I started out by using SVG animations, which worked great, but wasn’t easy to pause or replay using Javascript, and that felt important to get right.

Well, it turns out if you use CSS’ @property declarations with type color, and then animate all of the properties that impact the linear gradient in the same set of keyframes, CSS is willing to render a linear gradient as if it is animated. Many thanks to Temani Afif for this very nifty hack.

Project Pages

For the project page, while talking with my dad I realized that I could use the height of the page to represent time, and try to show the transition between frames as the user scrolls down the page, just like projects tend to become clearer the further you get into them.

Implementing this was relatively simple: the page is divided into sections (per paragraph, heading, etc), and each section has two linear gradients that go from left to right. The first is a solid background, which corresponds to the linear gradient at the top of the block. The second is a linear gradient from the next block (the top of the next block) which increases in opacity from zero at the top of the block to one at the bottom of the block, so that we get a smooth color transition within each block, and because the top of one block is the same as the bottom of the next block, the transition is relatively seamless.

List Pages

I’m not satisfied with any of my current approaches to the list page - I expect I’ll iterate on it for a while until I find something I like.

The Real Author

One big disclosure I need to make about this project is that I didn’t write much code for it - mostly just describing and debugging algorithms. Though I described how they should work, Claude built all of the algorithms in several thousand lines of Golang without much assistance. If I had to build it from scratch myself, building the image processing toolkit would have taken me much more time and I would have had many more bugs than if I hadn’t relied on Claude (though, Claude still had a variety of bugs that required me to dive deep and understand its code, and in the more than one case, start from scratch). As a result, the site was quick to build, and I think it looks really nice, but under the hood it’s quite grotesque - code that carries the scars of dozens of substantial revisions by multiple authors (me and Claude) without much continuity of thought. An instructive co-programming session.

One key finding from the project is that if you’re vibe coding, you need to build out (or have Claude build out) robust processes for validating the work that it tries to do. I found building out this algorithm comparison tool was the key to making consistent progress and evaluating “success” (check out a sample output by clicking below). As we start to rely more on LLMs, have clearer ideas of what we want, and better tools to evaluate how we’re doing against those goals is going to be more and more important,

The End Result

I really like it. I like watching it. It’s a personal brand I feel connected to and proud of. It feels thoughtful - not overdone, not sloppy. Purposeful, data-driven, gorgeous. Even so, I felt ambivalent making it my new homepage - I really liked the old site.

I have a list of things I’d like to try to get working in the future, including a true radial gradient (in particular to capture the 3-5 second range of the original video here), getting the same view later in the year (when the sun will rise in the gap in the mountains instead of over them), and trying a different angle/vantage to capture the impact of alpenglow on the atmosphere.

But this is good for now. The day is young and there is much to do!

Custom Board Games

Grady Berry Ward — Thu, 14 Nov 2024 14:32:00 -0500

I love making board games for friends and family as gifts, often themed around a shared interest or an inside joke. While these were originally separate posts, I’ve condensed them here for brevity.

2024 - FRONTIER

I (and my ex-boyfriend) both have a love-hate relationship with Frontier Airlines. The fares are impossibly cheap, and if you have their credit card and Discount Den membership, things can be legitimately a fantastic bargain. However, they’re often late, the staff isn’t the friendliest, and I’ve had some of my strangest experiences flying with them. We also both really like the game PanAm, about the expansion of the PanAm corporation - a gorgeous game with great mechanics like stock pricing and technological upgrades.

So, when my ex-boyfriend mentioned offhandedly that I should make a version of PanAm based off of FRONTIER, I was struck by the jealousy you have when someone comes up with a great idea that you should have had. While I initially wrote off the idea for that Christmas (because this comment was made in early November), I took the cancellation of a flight on FRONTIER a week later as a sign/opportunity to build out a full version of the game.

The first sketch (above), the final board, 24h later, below:

This project was very low-fi - there were only two technical elements, simulations to ensure the stock price deviation and expansion frequency was distributionally good and within normal bounds - a property the real game fails to live up to .

The majority of the work on this project was just learning to use InkScape effectively - practicing my (minimal) SVG skills on a broader canvas and learning the keyboard shortcuts, automation tools (like cloning), and becoming diligent about layer and group naming, selectors, and tags.

Only the destination cards’ images were generated using AI - and even then there was a lot of subjective post processing.

The gift has yet to be given - but thankfully I don’t think anyone reads my blog!

2024 - Puzzle Generation

You can check out my laser-cut puzzles here.

2023 - Tail Tally (Point Salad)

In 2023 I made a board game copy of Point Salad for my (at the time boyfriend). He loves the game Point Salad and loves his dogs even more.

The code for it is written in Go, generates HTML files (mostly to take advantage of flexbox positioning), and then automatically screenshots those at 500dpi, with bleeding/cutoff that matches the specifications from my favorite board game printer.

This one was particularly fun because I made it fully generalizable - you just input six photos and select a few things about the color scheme and sizing you want, and the deck pops out. I’ve printed (but not yet gifted) another version for friends - so beware, friends.

2022 - Wingspan

A writeup of the Wingspan project is here, but a few bonus images are below.

2020-2021 - Ticket to Ride

2020 - The Original Gift

My mom loves the board game Ticket to Ride, perhaps to a fault. Starting in 2018 (and apparently continuing to this day) she regularly plays it by herself late into the night - with two distinct players (her right and left hand) competing to see which one would win.

“I think everyone needs something to unwind at the end of a day, some people need to watch silly TV some people like to have a drink, for me it’s board games against myself.” - My mother

In 2020, we’d been planning on celebrating her 60th birthday as a family, but we were unable to do so because of the pandemic. My sister and I, keenly aware that this was a really big loss, decided to pour our energies into a great gift, and so took a weekend to create this board game.

Each location is one that she has been to and has fond memories of. The photo cards we created were a few select ones from family lore. The trains were dowels that I’d cut to length and dyed, and the tokens my sister made from clay, each a fun inside joke.

This was the first custom game I’d made, so looking back on it, the build was fairly primitive - we sent back and forth PPT files in ultra HD. The background is from the CINDI project. The files are large enough that they’ve been lost to time, but the board(s) itself lives on, as do some artifacts from the early days of its creation.

My mom loved it, and it has further extended her TTR obsession.

2021 - Ticket To Ride Cards

When a year had gone by, the pandemic had continued, and my mother was still obsessed with TTR (see below), we decided to build out a nicer deck of the cards, with each card capturing a different, unique photo from our family.

This project was mostly about selecting the right set of photos from our library, though I used some very basic facial detection APIs from Google to make sure they were approximately centered within the boundaries of the frame.

My mother still refuses to use these cards - to my chagrin - because “she hasn’t fully worn through the other deck yet” - in case you thought I was overstating the degree of her obsession.

2021 - Webapp

One last time I dipped into the TTR well by creating an interactive version of the custom board game using the simplest thing you can imagine: global state within a Java binary, running on a server on GCP with open connections, once-per-second polling, and start/stop mechanics. Building out the bizlogic for turns and state proved challenging as I stepped on every rake that someone does when creating a state based multiplayer game (validation, transition, etc.). A few years later I’d read a book on generalized gameplay schemas and be struck by how I’d made every mistake in the book. You live and learn though!

The end result was something that was fun for our family to play a few times, actually a decently responsive application, and a great reminder: building something hacky can actually get you reasonably far as long as you don’t intend for it to be around long term.

It was a good mother’s day gift - though probably the time we spent on the phone while we played it was the real one.

Laser Cut Puzzles

Grady Berry Ward — Thu, 14 Nov 2024 14:32:00 -0500

One of my best friends in the world - Brandon - got married in the summer of 2024, and insisted that no gifts would be exchanged. I nagged him ceaslessly on this point, and he eventually relented that only gifts of creative, rather than financial value would be accepted. Game on.

After much deliberation, I decided to make them a puzzle. I’d been looking for the opportunity to learn how to use a laser cutter, and thankfully both Boulder + Denver offer the tools in their library makerspaces. Over the course of several months (and gifts for several close friends - celebrating a wedding, commiserating about the passing of a beloved dog, etc.), I fine tuned both the code for generating the puzzle, and the usage of the laser cutters (trying out three different kinds over time, and finding the right set of controls for each).

In the end I was very impressed by the Glowforge PRO, which consistently had the lowest amount of hassle, and produced consistently good rasters. The Epilog laser I used for the final cut was also quite good both in precision and accuracy in the cut,but the raster left quite a bit to be desired.

The final process I found to be effective involved first etching the image on to the wood in a purely raster pass, and then doing the final cut only once masked with masking tape, which prevented burn scars.

The primary challenge of this project was scooping up the tool appointments at each library as it became available - they were hot commodities and I often needed to plan my days to keep free the half hour when I knew the appointments would go online.

Fine tuning the code was another tricky part, since there were a lot of things to tweak, and very slow iteration cycles for trying them out in the real world. I eventually figured out a structure for the nubs that would cut well, and designed a cool algorithm for making the sides and curves generally smooth, while not doing a full grid like a standard puzzle.

By the end, the code was capable of taking in an internal and external cut, so that hidden shapes could be included in the puzzle. In this test cut you can clearly see the “4” and the heart, a reference to the date of the marriage.

I gave them the final result at their wedding, and they completed it during a subsequent bout of COVID - ultimately framing it on their wall.

(A piece was lost from the leftmost edge - a standard foible of the genre)

This project was a great reminder for me that the best gifts are about the thought behind them and the expression of care for the other people. Building this gave me ample opportunities to reflect on how much both of the people meant to me, and how thankful I was to have the time and skills to tinker.

Binding Beauty of Chores

Grady Berry Ward — Tue, 19 Mar 2024 15:21:00 -0600

A long-form narrative version of this essay appeared in a friend’s literary magazine, The Hart and the Curr - I recommend you read that version - it benefits from his edits.

I recently did one of my favorite winter activities: a backcountry hut trip. A group of us schlepped a bunch of gear and food into the woods, and spent a few days living simply: heating a rustic cabin with firewood, melting snow for water, cooking, eating, and exploring the outdoors together. After a long weekend in this environment, I feel refreshed, relaxed, and more deeply connected with the friends that I accompanied.

Counterintuitively, part of the joy of a hut trip comes from the chores: the small, quaint, simple challenges that occupied much of our time. These included pumping water through a filter, splitting wood for kindling, and scraping ice from the door jam, to name a few. These activities may not sound like an integral part of a vacation, but they fulfill deeply: through chores we got to provide value to each other, and enact concrete transformation of our surroundings.

Zoom back for a second. Modernity removes routine tasks from our lives through a combination of automation, specialization, and collectivization:

Thermostats and furnaces abstract the management of heat (automation)
Professionalized plumbers enable ignorance about pipes (specialization)
Public utilities ensure the potability of water (collectivization)

These three impersonal processes remove manual labor as they increasingly automate, outsource, and collectivize the effort needed to sustain human comfort. Though these are the pillars of human progress, a side effect of their efficiency is to weaken the bonds that hold us together.

Relationships, Families, and Communities are (in part) held together through chores: small acts of service that connect the individual to the broader unit through the wilful contribution to shared value. As an easily cognizable example, consider how people who live in adjacent dwellings become neighbors. The small requests (can you look out for a package, take in the cans while we’re away), and small offerings (alerting them to an open garage, shoveling their part of the sidewalk) that neighbors take from and give to one another form a network of communal care that slightly inconveniences every individual, while constructing a whole (a neighborhood) that is stronger and more resilient than the components considered individually (houses, people). Relationships, Friendships, Communities, and even Families are forged through this same force: service is a key binding agent in human relationships.

While automation, specialization, and collectivization make our lives concretely easier, they reduce our reliance on others for our comfort, and conversely reduce the number of opportunities we have to serve one another.

Novel technologies that monitor the state of your garage door, and online systems for holding mail reduce our reliance on (and capacity to visibly assist) our neighbors.
The abundance of professional plumbers makes us less likely to reach out to a handy family member or neighbor for help.
Municipal snow plowing obviates my favorite form of neighborly care: shoveling a neighbor’s sidewalk.

These three forces (automation, specialization, and collectivization) meet our needs through impersonal means, and in doing so, remove the possibility of the personal connections that used to arise out of these tasks.

Let’s be clear: the fulfillment of need that these forces bring about is unquestionably a good: they bring about a world where independence and freedom are more broadly accessible and more equitably distributed. While it would be ill advised to attempt to revert to some state of unfulfilled need, we must deal with the bad side effect of this good medicine: how do we strengthen and cultivate interpersonal bonds, both individually and structurally, in a technologically advanced (and advancing) world?

Zooming back into the hut trip, I think this is part of what makes it so magical. Though the setting of a remote hut is contrived in the sense that we could return to modernity, in every other way the demands it imposes are real. Without heat, water, food, or sanitation, our individual situations would be dire. Tackling this reality boils down to chores: to chop wood, melt snow, cook food, and filter water. Each individual then has the opportunity (and social pressure) to complete parts of this service: to do simple chores, thank one another for completing the tasks that they do, and through non-directed individual action, become a unit that can keep itself warm, hydrated, fed, and clean. It is this togetherness, this communion of purpose, that forges friendships, trust, and comfort among group members. I leave every hut trip with deeper, stronger relationships with each of the other participants.

The appreciation I have for hut trips is not a call for reversion to more primitive living conditions. Rather, as automation, specialization, and collectivization fulfill more of our physical needs, we must recognize how this weakens our dependence upon, and potential service of others. In response, we must invent or discover new acts of service, rather than trying to reclaim challenges from the forces that obviate them. Human want (and thus capacity for support) is unbounded. As we advance as a civilization the mechanism for scaffolding interpersonal relationships can change to reflect the needs of the present, and the humans working to meet them.

Ask yourself:

What might your neighbor, sibling, or friend struggle with, and how might you provide them aid or comfort?
Where could others (through small acts of theirs) make a large difference in your life?
(and critically): how can you foster within yourself the selflessness to offer, and the humility to accept, help?

Whether it’s dogsitting, sharing hot food or book recommendations, practicing emotional vulnerability, helping with childcare, or simply calling someone out of the blue, there are a plethora of ways that we can (through small efforts) deliver huge value for others. Each of these represents an opportunity to connect with and serve one another, and in the process we enmesh ourselves as part of a shared community, one that is stronger than the sum of its parts.

These impersonal forces (automation, economic specialization, and collectivization) will continue to make our lives simpler, cheaper, and more lavish. All good things. But as they grow we must scaffold the structures that lie out of their reach. Friendships can’t be automated; families can’t be bought on a marketplace; municipalities can’t legislate a community into existence. As individual convenience is served by these market forces (and glorified by culture), its up to each of us to rediscover the inefficent, sometimes hard, and always rewarding tradition of extending help without expectation, and receiving it with grace.

Terry v. Ohio + Bulk Infringement

Grady Berry Ward — Mon, 01 Jan 2024 00:00:00 +0000

Terry v. Ohio is known for it’s implications on the probable cause requirement of the fourth amendment, but something else jumped out at me when I put myself in the shoes of the Warren court: given the fact pattern, they were doing their best to encourage good policework, and in the case of stop and frisk the differentiation between ‘good’ and ‘bad’ policing hinges on the quality of the officer’s decision making (the only proxy for which we have is the outcome of the individual choices weighed in the decision):

The majority tries to write narrowly. I’d guess that they thought that the correct outcome was to uphold the search and they were grasping for a reason why the stop could be justified without destroying probable cause. If so, I see where they are writing from. I’m at all concerned with Terry’s fate, but I definitely care about supporting the McFaddens of the world. McFadden made a judgement call to stop and frisk Terry and his coconspirators, and out of this individual decision, an agreeable outcome arose. Were McFadden less perceptive, more suspicious, or less discriminating we wouldn’t be talking about Terry. But as it happened, the case that justifies Stop and Frisk is a true positive.

Zoom back for a second. An officer’s decision on whether to Stop and Frisk (S+F) a civilian has four outcomes, based on whether the officer decides to stop (STOP) or not (PASS), and whether the civilian is actually in possession of illegal goods or arms (GUILTY) or not (INNOCENT):

(1) True Negative (PASS, INNOCENT) (2) False Negative (PASS, GUILTY) (3) False Positive (STOP, INNOCENT) (4) True Positive (STOP, ARREST)

There are clear downsides to False Positives and False Negatives. False positives seriously intrude on basic liberties to be secure in person and posession, tend to be racially biased, and create mistrust between law enforcement and civilians. False negatives likely increase crime rates and embolden criminals. A city’s choice on how and if to conduct S+F should probably be based on a concious tradeoff between the relative incidence of false positives and false negatives.

But thats now how courts operate. In Terry, we only see a single instance, a True Positive. Were the circumstances in front of the Warren court not that of McFadden and Terry, but, say, Sandra Bland and Brian Encinia (a False Positive), the court might arrive at a different outcome. Heck, if Officer McFadden had stopped 100 innocent people before stopping Terry, I’d argue that should change the outcome, but we don’t get to know that context in deciding this case. The problem with Terry v. Ohio is not (as some argue) that the fact pattern sits at the peak of the watershed ‘probable’ divide (though it does). Instead, the problem with Terry is that it is a single case.

When constitutional rights are in conflict, courts perform our balancing. When the parties with competing constitutional claims are on opposite benches, performing this balancing is easier than when no single case can capture both of the claims. In thinking about S+F, the fact pattern in front of us heavily influences our intuition: true positives show that there are occasions when we want to defer to officer discretion, and false positives show that officer discretion should not be unlimited.

It would be impractical and unwise to conclude based on this sentiment that case by case analysis can yield just outcomes (in this case certainly, and in light of the procedural barriers to justice - in the abstract too). Repairing 4th amendment violations is fraught. Instead, we should take a wholistic approach to justice, outside of the confines of a fact pattern to bias our intuition. It is feasible to evaluate the actual numbers of the four outcomes above in any given city. The Positives just need record keeping, and the Negatives need a few dozen (fully random) stops per day as controls (ignoring the clear constitutional problems with that practice). From there we could evaluate the harms of both groups in direct comparison (consider targeting a 4:1 FalsePositive to FalseNegative ratio).

What if a city is above this target ratio? Then officers should exercice more discretion before making a stop: instruct officers to have a higher degree of confidence before chosing to S+F! Rotate cops off of S+F if their ratio is bad! Provide more training on differentiating between suspicious and atypical behavior! Provide pay incentives for good ratios! Similar remedies could be imagined for a ratio below the target (not volumes, ratios!).

These are all specific recommendations to a very general and very hard problem: how do you improve the results (say S+F ratio) of a blackbox system (such as the brain of an officer) that can’t be reliably inspected? You define a cost function, and instruct the system to self-optimize. Officers know how to police far better than I (or most judges) do. Rather than defining a set of operational rules (that would be near impossible to enforce or evaluate) for how an officer should decide whether to perform a S+F, I’d rather we be honest about the aggregated harms of over or under stopping, and encode our relative thoughts about both in a function that can then be actualized by the experts.

More broadly, re-reading Terry I’m reminded that cases are instructive, but single-case rulings will typically miss the rights of people outside of the case. This has me thinking about a ton of topics: incremental rights and incremental infringement, actor driven optimization, and the boundaries of case law for achieving social justice.

Sniff

Grady Berry Ward — Mon, 01 Jan 2024 00:00:00 +0000

Disclaimer: I wrote this essay many years ago. The evolution of technology over the early 2020s has changed my perspective on some of the ideas presented in this post.

In Florida v. Harris (2012) a unanimous Supreme Court held that statistically derived evidence of efficacy of a drug sniffing dog was not a requirement for establishing probable cause for evaluating fourth amendment claims. I think this misses the mark from one of my favorite cases US v. Place (1983), in which O’Connor issued this gem of straightforward and balanced reasoning:

“A “canine sniff” by a well-trained narcotics detection dog, however, does not require opening the luggage. It does not expose noncontraband items that would otherwise remain hidden from public view, as does, for example, an officer’s rummaging through the contents of the luggage. Thus, the manner in which information is obtained through this investigative technique is much less intrusive than a typical search. Moreover the sniff discloses only the presence or absense of narcotics, a contraband item. Thus, despite the fact that the sniff tells the authorities about the contents of the luggage, the information obtained is limited. … We are aware of no other investigative procedure that is so limited both in the manner in which the information is obtained and in the content of the information revealed by the procedure.”

I really like this passage both because it provides an aspirational goal for law-enforcement, and because it articulates a clear requirement: that the decision procedure used must be found to be accurate to be reasonable (a fact not contended at the Supreme Court, but critical to O’Connor’s reasoning.) It’s worthwhile extracting a few lessons from the court’s reasoning in considering the design of what makes a dog’s nose, in contrast to most digital technology, such a satisfying balance of rights and state interests.

A key consideration that the court tangentially addresses in Place is that of statistically derived reliability as assessed through false-positive rate. There must be some threshold of false positive/negative at which a search becomes unjustified (or at which the decision procedure falls out of constructive use toward totality of the circumstances). This is easy to see in the case of a drug sniffing dog because we have no other means by which to reason about the dog’s judgement. Outcomes are all that we get to see for a dog, so the question of “well-trained” is a throwaway line, rather than a core holding.

This ties into my thoughts on Terry: a balancing act of competing harms in 4th amendment law can’t be effectively assessed by an individual case.

Unfortunately we shed this common sense when answering the same questions about to human judgement. Judges are not blinded by the personhood and autonomy of the dog in evaluating it’s impact on competing rights claims. Dogs can’t perform post-hoc rationalization, a dog’s thought process cannot be interrogated by the law. There’s no sensible way of looking at probable cause (a question of prior knowledge) that relies solely on whether the dog is correct in their indication of drugs (knowledge derived after search).

By talking about the dog’s perception in terms of accuracy, O’Conner implies that explanation is not a necessary component of an acceptable test for a decision criterion that results in search or seizure. This is in stark contrast to the way that the court looks at officer discretion in Terry stops. In that case, because we can ask a human for an explanation, we put extensive weight on the explanation, without falling back or ever examining the far more compelling indication of rights-balancing - statistically derived accuracy.

In Harris, Kagan misses an argument about the dog that we should be making about any tool of law enforcement discernment (whether it is a drug sniffing dog, a thermal imager, or a cop’s judgement). All decision criteria used to inform choices of search and seizure should be held against a rights balancing framework that analyzes real world statistical performance. Human judgement is a force that is better inspected from the cold evidence of outcomes, rather than the squishy (and time-skewed) process of post-hoc rationalization.

The second component that makes me love O’Connor and this decision is the concept of targeted information isolation. A drug sniffing dog is processing information about a wide array of olfactory inputs, but only is expressing a verdict on a narrow set of targeted criteria. It’s likely that a dog can smell a wide array of contents in the baggage of passengers, but the only information it expresses is that of the query does this bag contain drugs. This is supremely privacy preserving, because the only data that the system outputs (or even has the capacity to output) is the question of interest, and the test is one the state has a compelling interest in. If dogs could talk, they could tell us a wide array of information about the contents of passerby’s suitcases. In that instance, the dog would serve as a broader means of surveillance than is proscribed by a legitimate law enforcement purpose. Good law enforcement tooling should fit this principle: limit the potential output of the system to the outcomes that are of explicit and compelling state interest.

The third and final thought that Place and Harris bring up is the concept of interaction between tool and law. A consistent frustration for groups that try to combat stop-and-frisk policing is how officers adapt to court rulings about what constitutes articulable/reasonable suspcicion, and start to use that terminology in their reports. “The odor of Marijuana” is a powerful phrase. One additional (unarticulated) benefit of a drug dog (or more generally, a non-human system) is that it is more immune from this interaction pattern. Dogs will be biased by their training, but their individualized decisions are “individualized” in the words of Terry in a way that human’s thought processes can never really be. An officer’s awareness of the law will always color both their decisions and how they describe their decisions. Man’s best friend does not suffer from either of these defects.

Narratives

Grady Berry Ward — Mon, 01 Jan 2024 00:00:00 +0000

I’ve started thinking about how to define privacy. Though I haven’t made any traction on that overall question, it’s helped me better understand the relationship between privacy and how we manufacture our identity.

To avoid exclusively speaking in intellectualisms, I’ll describe the circumstances of my morning, which can illustrate the point I’m trying to make.

I went out to get coffee. I usually make coffee at home, but my french press shattered, so I ordered a new one and it was slated for delivery a few days ago. However, Amazon informed me that it was damaged in transit, and directed me to purchase a replacement if I still wanted it. I made that purchase, but the replacement wasn’t slated to arrive until tomorrow. Thus, to get my daily coffee, I walked myself and my dog to the closest McDonalds (a business I rarely frequent) to get a $1 medium coffee.

On the way back, a bee started attacking my dog (for an unknown reason) without his knowledge: it was furiously trying to burrow under his fur to get at his skin. I couldn’t have that, so I set down the coffee on the sidewalk and swatted it away using my McDonalds receipt. The bee then directed it’s animus toward me, and I spent the remainder of the walk intermittently jogging, swearing and stopping to swat the bee off of myself or my dog, until we were safely back in the confines of my apartment, a pane of glass now between us and the angry insect. I sipped my medium coffee in triumph, my dog and I unscathed.

Any subset of the facts above, taken without narrative or broader context could paint a picture that doesn’t fit with the narrative that I’ve just laid out.

A stranger who only observed me swatting at my dog’s fur could have mistaken my concern for abuse.
A passing policeman who saw me flustered, erratically waving my arms about, swearing, and unpredictably sprinting may have interpreted these as signs of a mental crisis.
McDonalds may have interpreted my transaction as a sign of a newfound customer.
Visa may have interpreted my purchase of two french presses in a week and a coffee at McDonalds as interest in further coffee goods and services.

The construction of each of those narratives could be supported by the same facts I used to create my own anecdote, but none are true. This is why privacy is related to narrative building.

The activity that we engage in at each moment in our lives is a piece of information. These data are collected (both in our memories and by external observers) and used to construct narratives that form cohesive stories: what happened, what we want, and who we are. We construct our understanding and model of our"selves" by distilling the facts of our lives into a narrative that that forms our identity and the self that we project to others.

The creation of data that occurs as you move through the world makes it easy for others to construct their own counter-narratives of your life. The more data that others have about your actions, the more convincing they can be in painting a picture of who you are. Even an external observer with perfect information about you might describe you differently than you would, either through the omission of information in their depiction, or by including information about you that that you omit from your own.

Full information of this morning’s events could either paint me as a environmentally concerned animal lover, or a prodigal waster. I use a french press because it avoids the marginal waste of filters. However, I also did appear to purchase two french presses (wasteful), and my purchase of a McDonalds coffee demonstrates a willingness to use disposable cups. Perfect information paints a muddled profile, even though I would consider myself a staunch environmentalist.

As this illustrates, there is a deep freedom in having control over your personal narrative, choosing the facts you consider to be illustrative of the person that you are. When you have the power and capacity to influence how others interpret and construe the data points that describe your existence, you have self ownership even in the external representations of you in the minds and systems of others.

This is one reason privacy matters.

Retailers collect information about your actions in order to construct a “consumer profile”, a dataset describing who you are as a consumer, what and when you like buy goods, how much cash and debt you hold, etc. Strangers use the information they can immediately glean about you from your external presentation to create a model of what they should expect from you, and how they should react to you. Law enforcement uses data about your criminal history, your zip code, your demography and your public actions to evaluate your likelihood of rule-breaking. Juries are fundamentally trying to decide which narrative best fits the facts presented in a trial. In each of these cases, data about your existence is used to construct a narrative over which you have no direct control. Certainly, you may be able to influence these portraits by deliberately changing your zip code, your purchasing habits or your external presentation, but doing so does not reclaim the authority over your reduction (your characterization in the mind of the officer, the stranger, or the database).

In reality, our actions are not wholly consistent with any single narrative. Narratives are less information dense than the facts that go into producing them. The construction of a narrative is inherently a lossy compression - it distills aggregated data into a imperfect (though useful) abstraction. I’m certainly not arguing that all such constructions are bad, nor that each should be within the full control of the person being described. But it seems important to identify that the more information about an individual in the minds, databases and metrics of others, the less power they hold over their identity, and in a way, their personhood.

Clock

Grady Berry Ward — Mon, 01 Jan 2024 00:00:00 +0000

“Let’s go say goodbye.”, “To the house?”, “To the house.”

It’s dark inside, the fading day is blue, filtered in. The bathroom light a contrasting a vibrant yellow as he goes for a final pee.

I’m glued to the wall, seeing the house spotless for the first time since it was last a house.

We’d made it more. We’d chosen mismatched glasses, argued over the layout of the couch, compromised, built furniture to accommodate the compromise. We had diner parties, new food, guests, dogs, and sex. Just as we did, it grew, sprawled and settled. It became more than the things we put in it.

But things changed. We changed. What we needed and wanted and were changed. I moved out. He finished the lease. And on the 30th, I came over, scrubbed floorboards and toilets and when I looked up after a few hours, and loaded the cars, and came in to say goodbye to a home, I didn’t find it in there.

The only sound was from the only object - the clock hanging up where we’d removed the nails that the last tenants haphazardly threw in. Me on a chair with the back of a hammer, him with a bowl for the nails. Taking 60 out to put one in for the clock.

He came out of the bathroom, and I pointed out the clock. “I think we forgot something.” “Do you want it?” “I think I like it being here-it fits the place.” “Well it’s always nice to have a clock - if you don’t want it, can I take it?” “Sure.”

The clock kept ticking as he took it down. We closed windows, locked doors. Now, sitting in my lit and full apartment all I can think about is how it must have sounded as we closed the door and the ticking faded away.

ABOUND.art

Grady Berry Ward — Mon, 27 Nov 2023 08:49:31 -0700

I like to build algorithmic art. The piece of it I find the most satisfying is the tinkering. You create some code to create an image from some settings, and then you play around with the settings to find the most satisfying combinations of inputs. This process isn’t technical, it’s just an expression of pure exploration and aesthetic judgement.

To allow everyone the joy of this kind of tinkering and discovery, a friend and I built ABOUND.art - a platform for generative art. Anyone can upload an algorithm, and then anyone else can play around with it, no code needed.

We spent ~6 months part time working on it, and solved a plethora of hard problems. We figured out how to run untrusted code cheaply. We figured out how to build generic multipart forms. We figured out a cost structure to make sure we didn’t hemorrhage money, and we built a lot of cool art.

The platform didn’t take off in large part because we aren’t great at the stuff outside of building. Marketing and self promotion are not our strengths. Eventually, we gave up trying to make it happen, and just celebrated it for what it was: a gorgeous thing that we built and loved to use.

The site is still live, and you can explore and access it at https://abound.art. Check out the instructions on the about page.

Happy tinkering!

Focus on Features

Grady Berry Ward — Mon, 27 Nov 2023 08:49:31 -0700

When I worked in Counter-Abuse at Google, I found the largest issue we faced wasn’t malicious users, but poorly thought out product decisions, and the constraints that the business model placed on what we could do to keep users safe. It’s very hard to clean up a kitchen if you’re only empowered to mop, rather than trying to understand or change why things keep spilling on the floor.

So when the Integrity Institutute (a small thinktank on content moderation issues, which I’ve been a member of for several years) sent out applications for visiting fellowships, I figured it was my chance to try to do something about it. I built a resource for product managers and regulators to think through the connections between harms they see on platforms, and the features that enable that harm, and the set of changes that could minimize that connectivity. The best explanation I have is the “Why I built this” recursive essay here.

The site as a whole took a while to build because there was just a lot of writing to do. As of this writing, there are 14 intervention categories, 47 harms, 64 interventions, and 9 features.

The project ultimately became a group project of the Integrity Institute. Because of that every page needed to be fully-editable by a nontechnical audience, which presented a fun set of technical challenges.

The technical elements of this project are pretty slick. It’s backed by a postgres database running on neon.tech, which connects to Strapi, an open source CMS. The frontend is made in Nuxt 3 with typescript, and then hosted via firebase cloud functions.

However, that’s the editing-path. Since the content is static, I set up SSG to pre-create every page on the site, and then back-hydrate with a flat file after first load. So the page as a whole is incredibly snappy, and costs nothing to host since it’s just flat files for almost everyone using it, and scale-to-zero for editors.

I’ve gotten more traction on this project than I expected - it’s started many conversations with PMs, regulators, and other thinktanks, and still (years later) has a steady stream of traffic, mostly from the EU.

Browse for yourself here.

Wingspan Deck Generation

Grady Berry Ward — Mon, 27 Nov 2023 08:49:31 -0700

One of my favorite board games is Wingspan - an engine building game where you collect different species of birds. The game instilled an appreciation for our avian friends in me, and was my introduction to the wonderful world of birding.

As anyone who loves something desperately, the flaws in the wingspan game were more obvious to me than to most. The nest types were often incorrect. The clutch sizes capped out at six. Some of the powers within the game were too much, others were too little. The maps only showed their geographic proximity in terms of continents.

Most importantly, the birds were from all across the world, making them not very useful for fine-tuning your skills on your local bird populations. I can’t tell you the number of times I thought I saw a Dunnock in Colorado, only to learn much later that they’re only found in Europe! I needed to have a wingspan deck that just had my local birds in it, so I could become an expert at identifying at home.

So I coded.

And I coded.

Too.

Much.

Code.

I had to build infrastructure for scraping sites which were hostile to it (sorry Cornell ornithology lab, All About Birds, Avibase, and others). I built infrastructure for coorelating sources, only including a fact like a wingspan in the deck if it could be corroborated between two sources to within 10%. I built infrastructure for finding creative-commons licensed images of the birds, AI to find the bird in the image and crop it well, and build complicated infrastructure for keeping detailed records of sources and methods, both for debugging, and appropriate attribution. So far I’ve written 39k lines of code to tackle this problem.

I’ve now gone through three generations of these cards, and they’ve changed over time significantly in response to play-test feedback from my friends and family.

In the third generation of the cards, I started to use AI to generate the images, both because finding good images in the creative commons is hard, and because the uniform style lended itself to it.

I expect this project will never be fully done. There are too many small things I’d like to improve. Instead, I suspect I’ll continue to make small changes every six-ish months when I gift a deck to a friend.

Advice for new SWEs

Grady Berry Ward — Thu, 01 Jun 2023 00:00:00 +0000

In my time as a SWE I’ve been lucky to get the chance to mentor a large number of folks as they start their careers in software. Over that time, I’ve picked up on some trends - habits, mindsets, and tactics that lead to growth as a SWE, and common patterns that lead people astray.

I also frequently run into folks who have an interest in software engineering, but have a hard time envisioning a path for themselves within the industry. It’s a crowded market of opinions, and it’s hard for someone without much experience in the field (or a trusted guide) to discern where they should spend their time, what they need to know, or even what the job is actually like.

So, I decided to write up a document to try to make it a smaller lift to share some basic advice with friends and acquaintances in this situation.

I wrote the first draft of it when a friend came to me with some questions about the field in mid-2023. Since then, I’ve reused it with about a dozen folks, revising it each time to be shorter, more precise, and trying to keep it up to date. My agreement with anyone I send it to is that they have to tell me where they think it is unclear, imprecise, or inaccurate - their feedback is the only way it can improve.

Feel free to forward it to folks, or leave comments or suggestions - it’s globally suggestable. I would be curious to hear what other folks (both experienced and not) think of it, and I hope to continue to have it evolve as a resource for new SWEs.

You can find it at https://grady.dev/advice.

GoLog

Grady Berry Ward — Tue, 12 Jan 2021 15:15:13 -0700

I’ve been writing a lot more golang of late, so I wrote a (gorgeous) visual debugger for helping to do post-hoc debugging. It’s been hugely helpful in my lambda based projects, where one often wants the ability to quickly view “what happened during this trace” without running locally with a traditional step based debugger.

Unlike many of my projects, this one is pretty well supported, and I’ve continued to add features as I need them.

You can check it out on github here.

Creditworthy

Grady Berry Ward — Fri, 01 Jan 2021 00:00:00 +0000

I just finished reading “Creditworthy”, Josh Lauer’s thorough picture of the history and trajectory of consumer surveillance by the credit industry.

Lauer’s work illuminates the early evolution of the social contract at the heart of the information era: convenience, service, and social inclusion in exchange for privacy. Though this exchange is familiar to a twenty first century reader, details from the evolution of this contract (in the context of credit) offer us lessons as we seek to broadly re-negotiate the terms of our privacy forfeitures in all spheres of modern life.

The first lesson is that information is unlike other commodities in how it is collected, synthesized and deployed, and regulations that sought to ensure its responsible custodianship in the context of credit failed to do so largely because they ignored key properties of data that were under-appreciated at the time of regulation:

Reproducibility: Data can be copied any number of times, at close to zero cost, which creates an incentive for its infinite duplication and dissemination. Once a piece of data about an individual is collected, we should expect that market forces will drive the collecting actor to share or sell it broadly and without restraint.
Supra-Summative: Data is more identifying, valuable and intrusive the more of it that can be accessed and combined by any given actor. Linking two datasets together routinely produces more value than the two datasets offer when ingested and examined in isolation. The increasing returns to scale of surveillance, synthesis and sale of information encourages “data-maximalism”, the indiscriminate and unabashed collection of data in all spheres of life.
Opacity: Data can be used to predict outcomes (such as default rate) without offering explanations as to why the data is an indicator of the outcome. This, in concert with the reality of dense correlations in all factors of life, confounds efforts to root out social injustice buried in Black-Box algorithms. Racism, sexism and ableism all still exist in credit scoring despite the prohibition on the explicit consideration of race, sex or ability. The same problems that plagued credit interpretation in the 1970’s are compounded in today’s deep learning algorithms, where nobody understands what inferences the system uses to draw its conclusions, even though the system may be able to consistently predict reality.

The second lesson is that privacy is rarely taken, it is almost always traded away, primarily for convenience, service, and social inclusion. In the early days of credit reporting, personal information was relinquished in exchange for the convenience of deferring payment on purchased goods to a later date. As credit surveillance evolved, even purchasing goods in cash at many retailers was recorded for use in credit evaluation - mere access to the service demanded the forfeiture of personal data. Later, credit reports were (and are) used in job and rental applications, and even social club membership, clearly demonstrating that ceding control over one’s information was (and is) necessary for full participation in society.

Today we have to grapple with more totalizing and transparent exchanges of privacy, but the same benefits of convenience, service and social inclusion are still the weights on the scale opposite privacy. TSA PreCheck offers the convenience of shorter airport security lines in exchange for detailed histories of your living arrangements, known connections and travel data. Department stores use sophisticated IMEI tracking to pinpoint your location while you enjoy the service of browsing in their stores. Social networks and ad providers siphon up your digital detritus as you browse the internet in order to offer more relevant (and lucrative) ads. The tools requisite for living and working productively in the 21st century (email, chat, search, maps, and connectivity) can easily become tools of surveillance when requested by law enforcement. Full participation in 21st century society requires capitulation to surveillance.

Though the inevitable arc of credit surveillance presented by Lauer can be disheartening, the book also helped me recognize that we may be able to bend the world back toward privacy.

The third lesson is that Americans do care about privacy. “Creditworthy” is sprinkled with anecdotes about people recognizing the ills of mass-surveillance and acting (both as individuals and in collectives) to mitigate its harms. Our moral intuition speaks clearly when we are asked whether an individual should be able to live down past mistakes, whether personal information should be given basic protections, and whether corporations should be allowed to collect data for a one purpose and use it for another. Privacy is an integral component of freedom, and Americans across the political spectrum throughout our nation’s history have had the moral wisdom to fight against encroachments on both.

Though we frequently forfeit our information in transactions large and small, this does not indicate a lack of resistance to the intrusions of surveillance into our lives. It is largely rational for a consumer to forfeit information when individual transactions are viewed at the margin, as the harms from such forfeitures are smeared across all such transactions and their costs are impossible to predict. This asymmetry in immediate known marginal benefit against abstract high-variance cost helps explain the dissonance between our apparent docility as willing consumers and our appetite for institutional change as privacy concerned Americans.

The fourth lesson is also a hopeful one. Regulation heavily shaped the consumer credit industry, and it could influence the shape, scope and intrusions of the information economy in the future. The use of credit scoring in federal loan screening in the 1960’s led to the ascension of algorithms over individual intuition within credit offices. When the first regulations were placed on credit bureaus in the early 1970’s, a waterfall of rapid consolidation resulted. Civil rights regulation in the 1980’s was able to eliminate much of the explicit discrimination on problematic grounds in credit ratings.

Though these efforts are sprinkled with as many half-successes as half-failures, they all demonstrate the power that regulation has over the information economy, and leads me to be hopeful that well structured regulations could allow our country to renegotiate the social contract underlying the information age.

Creditworthy is a fantastic book. Lauer paints historical details large and small with explanation and context, illuminating not only how our present world came to be, but the individual and collective incentives and choices that got us here. In it I read a history of mass surveillance and a path toward restoring balance, but another reader could just as easily find compelling evidence for the strength of American capitalism and ingenuity, or the disruptive influence of technology. I’m sure that it will hold up to multiple readings.

Raison d'être: le chien

Grady Berry Ward — Wed, 01 Jan 2020 00:00:00 +0000

As a middle-schooler untethered from proscribed answers (I was raised without any connection to, or real awareness of religion), I dreaded the few dark minutes between bedtime and sleep. I’d zoom back from myself - the practiced lens of the NOVA documentaries that I loved at that age - losing myself in the big questions, the vastness of space, and the darkness in front of my eyes: Why was I here? Would my short, finite life mean anything? What could meaning be in such a large empty universe?

Folks tend to have one of three relationships to the hardest questions: (a) belief in the answers of others, (b) avoidance, or (c) fixation. I was (c) all the way. For me, as for many others, the unimaginable scale of the universe was the fertile ground for existential dread in the same way that it served as a crucible for life: as an inevitable consequence of selective pressure. I struggled with this set of questions for a long time - middle school, high school, even parts of college. I was constantly trying (and constantly failing) to convince myself of the less grim answers to these unavoidable questions. I didn’t want to be a nihilist, but I couldn’t find a reason not to be.

Eventually I discovered another, one that I find much more convincing, robust, and comprehensive. The key for me was dogs.

As a kid, while I’d happily cross the street to pet a friendly dog, yips from coyotes could make my skin crawl with fear. These two canines are closely related, so how did my differential reaction come to be? It’s simple: previous generations took something hazardous (wolves, coyotes, wild dogs), and through the successive application of effort of hundreds of thousands of humans, over a timeline of centuries, they produced a world that was better, both for me, and most of the people alive today. They made a world of labs and corgis and doodles, molded from one of wolves. That is remarkable. Only human effort engineered man’s best friend - mine is resting his head on my knee as it bounces from the keystrokes. He is lightly annoyed by the jostling, but happy to be close to me. I am happy to be close to him, too.

The seed of this observation (dogs - wolves = human effort) sprouted two perspectives within me.

The first is an overwhelming gratitude - for the dog snuggled on this couch, for the house I did not build, wire or plumb, for the marvels of the internationalized economy that makes everything I’ll ever own possible, for a social and economic system that enables freedom and growth, and for the layer cake of open-sourced software that comprises technological life. To be literal, I thank the ancestors - the literal people, who stone by brick by plank by beam by wire by chip by package imagined, designed, collaborated and built world after successive world where their children could live better than they could. To be figurative, I see a knotted, divergent quilt stretches back toward a hazy horizon - each patch forged by the sweat of people like me - most patches making things incrementally better for the builders of the next one - most tightly woven into their neighbors. When I fixate too much on this visualization it can routinely make my cry (the only other thing that can is pondering the limited lifespan of my dog). My gratitude overwhelms me. I did nothing to inherit a comfortable world, nor anything to deserve this dog. What gifts!

The second perspective is a grim responsibility. The world as it has been handed to us is a bounty: in technologies, in goods, in liberties, in kindnesses. On a societal level, this inheritance is both finite and brittle. Without effort, without collaboration, without shared purpose, without selflessness, we may yet squander it, rather than adding to it. On a personal level, it is easy to feel useless.

The gravity of this perspective is daunting, but grounding. I see the people in my life affixing their patches all the time. A doctor cares for the elderly, a scientist studies a disease, a construction worker builds a home, a lawyer tries to right an injustice, a couple starts a family. I see it in others too: a father decides to break the cycle of abuse he was caught in. An engineer shares their new idea with the world. A neighbor invites their neighbors over for dinner. A teacher bears hardship for their pupils. There are a million ways to serve one another, and every day, people do them. It’s quiet. It’s everywhere. It’s in our nature. The impulse to help, to build, to share, is our social fabric - it’s why we’re here. It’s why we matter.

This idea calms my restless mind at night, and informs my career, my relationships, and my choices. While it doesn’t proscribe action, it is an unmistakable clarion: do good for the present and the future, you owe it to the past.

So, what will you do? What little patch will you affix to this great quilt? What do you owe the past? What can you offer to the present? What will you promise the future?

Or said another way, how will you (how could any of us) possibly repay the greatest gift, man’s best friend?

Widmanstätten pattern

Grady Berry Ward — Tue, 24 Sep 2019 18:17:56 -0600

When visiting the Smithsonian Natural History Museum, I saw the inside of a meteorite, stained and polished to reveal a beautiful pattern. Several months later, I remembered it while looking through some photos and decided to learn more about it and try to reproduce it.

Here’s what I learned from Wikipedia: The meteorites that display this pattern are comprised of of two Iron-Nickel alloys, Kamacite and Taenite. This pattern is found in metiorites called Octahidrites, so named because the pattern is actually a result of a crystal structure of Taenite’s crystal latice, which is hexoctahedral (cubic) in structure. The pattern generated when such a crystal is sliced along a “face” of the crystal is the Widmanstätten pattern, but cut along different axes you will see different patterns (a square equivalent to the Widmanstatten, or just chaotic beauty if not at a multiple of 15 degrees). Wikipedia’s image here speaks a thousand words:

(Source Davide Bolsi - Different cuts of the octaedron produces different kinds of Widmanstätten patterns. CC BY-SA 3.0, unedited, Created 9 June 2008, https://en.wikipedia.org/wiki/Widmanst%C3%A4tten_pattern#Shape_and_orientation)

So I tried to simulate this, with some mixed effects!

It took me a long time to figure out how to render it (ended up using polygon intersections using Java’s euclidean threed library with raw writing to pngs with the results). Also took me a long time to debug all 35 major bugs in my code (I kept track after I started getting rendered output to show the progression, maybe a topic for another post!). The end results are below:

An face cut, producing the iconic Widmanstätten pattern.

A square cut, producing the less iconic (but verifying) crosshatch pattern.

But I didn’t think these captured the 3D crystal aspect of the project, so I also animated a plane passing through the crystal perpendicular to the face cut:

And animated rotations (aligned with the square and face cuts) that rotate through all the other angles perpendicular ot the face or square cuts. This helps you see the patterns align with the cut angle and then unalign at values that aren’t multiples of 15.

Shoe Rack 2

Grady Berry Ward — Mon, 04 Mar 2019 19:27:45 -0700

After seeing the shoe rack I made for my partner, my sister asked for one for her apartment as a Christmas gift. More precisely, I asked her what I could make her and she allowed me to make another shoe rack.

The major design constraint on this project was that the end result had to be transported from Colorado to Pennsylvania, which as I figured out by looking at shipping costs, would be fairly significant if the piece was fully assembled when shipped. I happened to have some old luggage lying around, and came up with the idea of designing the rack to collapse and fit inside of a suitcase.

Unlike other projects I’ve worked on, this one was fun primarily because of how much went wrong. I mismeasured/miscounted a half a dozen times. I got snowed in and couldn’t go out to get a drill press vice. I ultimately built something that was too dense to put shoes on (until I made some modifications). Despite all of these little hiccups, what I love about woodworking shone through - I was able to overcome most of the hurdles placed by my lack of foresight, and produce something that was still interesting, beautiful and functional.

Finished photo first, obviously.

Shoe Rack

Grady Berry Ward — Thu, 30 Aug 2018 19:59:14 -0700

My partner and I really both like building things, which makes anniversaries both an occasion for expression of caring through mutal gift giving, and a bit of a self-indulgent excuse to carve out time to make stuff. It only took him noting once that the width of our shoe rack was exactly 2.5 shoes for me to think through how I could build something better.

Over weeks of sketches, a trip to IKEA, and a trip to John Houston’s Kennebunkport workshop (unrelated), I was inspired by the idea of sneaky intersections - of furniture that has non-standard joints and that provides standard functionality in non-standard ways.

Planes are the foundation of furniture, but the way they are constructed is usually unimaginative. This idea was to try to construct planes through convoluted connections that rely on planning (rather than clear joints) to accomplish the surfaces on which the shoes would rest. This would be in contrast to the dominant feature of the rack - an elegant top portion of wood that would be both planar, simple and elevated above the constructed planes below.

Finished photo first, obviously.

I bought a bunch of pine because apparently it was purported easy to work with and cheap. After compleing this project I can affirm both of these characteristics.

First step was cutting the pieces to length. Lots and lots of this.

Next was sanding. Each prisim has 6 planes and 12 edges that all needed sanding. This, in combo with 30 pieces of wood meant 540 sanding planes/lines

The oscillating sander takes a lot of the brunt of this project. My hands got a nice massage though! Went through three disks like this one.

Assembling the first set of three rungs of the rack.

All three assembled rungs.

It took me a while to figure out how to attach these to the top. This was probably where I spent like 1 day in the garage scribbling. This is the start of me prototyping what ended up being the solution.

Attached two vertical columns to the center most rung.

I made a relatively grave miscalculation with the way the three rungs were going to attach (hint, screws in perpendicular directions to each other can’t work). Instead I decided to do the attachment at random staggers. Turned out to be a good idea, but I executed it in the wrong order and had to do FAR too much manual hand-turning of screws.

Two rungs attached! In this photo you can see what my partner made for me - some lovely shuffleboard boards! This was past the anniversary because I wasn’t able to finish mine in time (due to an unfortunate concussion.)

Attachers on the middle rung.

Rungs fully attached

Initial attachment of the top (using hand screws and some pre drilled holes) looked like it was going well…

What these photos miss, however, is how incredibly off center the result was! Solidly 5 degrees off of alignment.

Time to undo some skrews, and try again…

The prestain made the wood look GOREGOUS particularly on the top. Very happy.

The stain turned this pine into a really gorgeous shade of maple. This is unfortunatley the only photo I have of it!

Finished product!

It nicely fits our shoes, though certainly not all of them (as we are shoe folk). My partner was quite happy with it, and I am too, despite its slight wobbles!

Civil Grand Jury

Grady Berry Ward — Fri, 30 Mar 2018 17:35:15 -0700

In it’s state constitution, California defines a unique institution for each county called the Civil Grand Jury, which is tasked with overseeing the effectiveness and efficiency of the county’s governance.

I volunteered to be on San Francisco’s Civil Grand Jury for the 2017-18 term, and began working with the Jury in September of 2017. Though all jury proceedings operate under confidentiality, I can say that I have chaired up an investigative committee, and was one of the authors of one of the grand jury reports published in July of 2018.

You can find these reports here.

I learned so much from this experience: managing a team of folks with wildly different skillsets and focuses, doing meticulously detailed researching and interviewing, and having a large number of tasks to juggle at every moment.

CalcU

Grady Berry Ward — Tue, 16 Jan 2018 11:58:27 -0800

I frontloaded my classes to heavily at Brandeis, and coming into my Junior Spring Semester, I had completed the majority of the coursework requisite for my majors. During that semester, as most of my friends were abroad, I decided to work on a side project for the semester, accompanied only by a French class requisite for my graduation. This was one of the best decisions of my life.

CalcU emerged from a need I saw when tutoring. I was a math tutor throughout highschool and into part of college, and I frequently helped out folks who were tangentially studying math (Biology, economics) and needed refreshers on material far before the material they were trying to apply. A student who needs calculus for bioinformatics forgets basic derivative rules, a student in microeconomics forgets the basics of geometry, a student in multivariate calculus has never taken linear algebra.

Students in this position need a resource which has a plethora of examples, sample questions and explanations, but more importantly, they need a resource that is able to quickly determine what they know, what they do not know, and what they are trying to learn. CalcU sought to fill this niche.

The project expanded when I got a grant to continue the work over the summer before my Senior year, and with the funding was able to hire three interns. The fundamental premise was that we used web-scraping to get the data from a large number of open source text books (including problems, examples, solutions and explanations). We then used manual categorization into a taxonomy of problems that we created. This taxonomy was both recursive and strongly ordered. A student’s familiarity with different areas of the taxonomy was recursively explored through a placement quiz which sought to understand the set of problems the student understood and those that they did not. CalcU built a rating system to enable the best examples and problems to drift toward the top, and had the ultimate goal of identifiying learning styles and common patterns among learners that would enable machine prediction of the next problem a student should see.

I could talk about CalcU for days, but I will not.

CalcU was my first big independent project, and I loved it. It showed me how much fun desigining and building a system from scratch can be. It taught me more than I can say about how to manage coders - particularly those hungry to learn. It made me believe that I am capable of building big, complicated, and technically complex systems. It was the project that enabled me to identify as an engineer.

It ultimately failed. The system and software are good at what they do. They are built in a way that minimizes server costs, and are fairly well exposed for future data science work. However, I failed to generate the enthusiasm about the system that I had hoped. I did the classic engineering mistake of building without consulting real users: trying to present a polished end result without thinking about whether it is what people wanted/needed/could be familiar with. I avoided pushing the software into the hands of users because I was afraid of what they would think, and by the time I was satisfied with it, it was too rigid to enable valid user feedback to influence the final product.

I am deeply thankful for the experience that CalcU gave me. The folks I got to work with (students and teachers in the high-schools, the faculty at Brandeis who advised me (Profs. Torrey and Hickey), and my friends and interns Sofia, Russ, Roger and Danny) made it one of the most fun and exciting projects I’ve ever worked on.

I hope to work more in this space in the future, and learn from the ups and downs of the time I spend on this project.I could talk about CalcU for days, but I will not.

Bananagrams

Grady Berry Ward — Tue, 16 Jan 2018 11:58:27 -0800

In 2016, when I began working at Google, I started having breakfast with the same group of folks every morning - Chris, Brett, Andrew and Brandon. All of us being nerds, techno-optimists and deeply competitive, at one point Brandon described having a competition to see who could write the best AI to play the game bananagrams.

My productivity dipped precipitously over the next few days as I considered how to destroy my good friends in tiled combat.

Here is the algorithm I designed. I realized that searching for connections in any spatial model was going to dominate running time, and that I tile addition was going to be near-impossible if you had to maintain state over colliding words. To optimize so I didn’t have to maintain a model of the board, I figured out a layout that assigned ownership over strips of the grid in perpetuity to “strands”.

Each Strand was a series of words where the ends form a two letter word and have no other interactions. In this way, each word simply becomes a starting character and finishing character, as length no longer matters, and this reduces the search costs considerably.

With this model in mind, you can immediately discard all words that dont start and end with letters that are constituent to two letter words (on the same upbeat/downbeat, too). You can further simplify out all string comparisons and just look at character integers. You only need maintain a list of all of the open docks, and a simple function to add a new doc on the end. Throw in some logic about which strands go up and which go down, and you have a blazingly fast addition algorithm.

The code is here, and the result is (in my mind) breathtaking.

Because all lookups are heavily amortized, and the set of words is large enough that you only need to look where there are available keys, as the number of words on the board increases, the addition time actually decreases, though you run into memory issues at some points.

The format Brandon used to create the server (Captn’ Proto) doesn’t have a Java API, so we didn’t end up playing.

I learned a good amount from this project:

Decomposing a problem into steps that computers are good at is the key to solving hard algorithmic challenges. It doesn’t matter how fancy or complete your logic is if its orthogonal to the strengths of your processor and the data structures you are using.
High-level languages are going to be easier to optimize down to their core precisely because they add layers of semantic meaning. Optimizing C is not something I do well, because my mind is too stuck in the operations of the computer, where as when I am optimizing Java, I see the problem in front of me clearly.
Competition drives me to do some of my best work. There is nothing more motivating than sitting around a breakfast table, plotting victory against friends. I rise to (and love) challenges.

Cindi

Grady Berry Ward — Tue, 16 Jan 2018 11:58:27 -0800

TL;DR: Janelle Monae’s art inspired me to try to build an algorithm that programmatically generates emotion-evoking art, and the results are here.

If you haven’t yet listened to Janelle Monae’s “Metropolis” suite, I highly recommend that you drop the committments you can, pour yourself some coffee and/or wine, and listen to it carefully.

Monae is a magician, a siren, an entertainer, a theorist, a historian, and a visionary. Metropolis helped me understand so much about so much that talking about it here will certainly do it an injustice. If you are interested, check out my 2016 term paper on her work within the context of the history of waves of Black Feminism. Or, preferably, read one of the many articles [1] [2] [3] by people who have looked at Monae’s work more carefully than I have, and have more interesting things to say about it.

Within the context of Metropolis, one of the (many) questions Monae raises is at the intersection of technology, humanity and art, and this project was an attempt to programmatically generate art that evokes real human emotion. To understand what I was trying to capture, check out the page itself.

This was an expansion of the work I did in in VOAT - rather than using squares with a predictible and pre-defined set of ports to connect, I use polygons with arbitrarily defined sub-regions. The results are wild: extreme variance, never dull. To see how some of these were generated, check out the tesselation photos below.

TODO(gradyward) Add tesselation breakdown photos.

I hope to do more work in this space, and to keep thinking through where the root of emotion and power of art lies. From a technical perspective, I have a long backlog of features I want to build: I would ideally like to incorporate images, create non-fluid spacefilling, create classifications for common aesthetic patterns and entropy filters to filter out boring or repetitive generations. From an artistic perspective, I’m not yet satisfied with this project - it has only begun to scratch the surface of what I think is possible.

I hope this project goes somewhere, but I need time to work on it.

UPDATE (2023): You can now play around with many derivative algorithms (and see the many beautiful images created through them) at ABOUND.art.

Flux

Grady Berry Ward — Tue, 16 Jan 2018 11:58:27 -0800

TL;DR: Check out this data intensive (not recommended for mobile) looping pattern of an attractor with a multi-dimensional parameter orbit.

I saw a blog post by an extremely talented programmer, and was surprised (as he was) to find that there was a natural symmetry to the patterns he was describing. However, in digging into his code, I found that it was not actually symmetrical, but he had just made a very understandable typo which introduced this artificial symmetry.

Working with these equations, I found that the surfaces generated by them were gorgeous, and that the surface appeared to be differentiable with respect to every one of the input paramters. In my experience, this is rare for a multi-dimensional equation.

Just iterating around a bit, I found that the most “interesting” cases all happened at a relatively uniform distance from the origin of the parameter space. I then thought about how I could generate a nice video of a tour of the parameter space, and my mind lept to an orbit. I created the software to output a gif of the generated curve for a given circular or elliptical orbit around the center of the parameter space.

I then made many of these gifs, all of which share a starting and ending position in the parameter space. This means that played in any sequence, the gifs will make a beautiful pattern that appears continuous, infinite and quasi-random.

This work insipred me to do more code-art for the sake of the art, rather than the sake of the code.

NOTE it is not recommended that you view this over a mobile connection - the page is about 100MB in total

NOTE it might take a while for this page to load. Give it several minutes. It loops, so you will not miss anything if you come back to it later.

VOAT

Grady Berry Ward — Tue, 16 Jan 2018 11:58:27 -0800

TL;DR: With a fun algorithm, slight parameter tweaks lead to really different results, as I show in a work based on Elgar’s enigma variations.

A good piece of art compells you to understand the process that created it, whether it is the emotion of the composer, or the logic of the painter. A revlation while looking at a piece by Frank Stella made me understand the fundamental mechanism that was generating the work, and gave me confidence that I could recreate it.

Squares (with side of size 1), with two quarter-circles in opposite corners (with radius one-half), in both configurations, with space filling between regions, forms the basis of this work by stella.

After more time than I would like to admit, I was able to recreate stella’s work in the browser. There were so many minutia and little tricks that if even slightly off in any function, the code would mis-render the full piece. I came to enjoy these mis-renderings more than I enjoyed the final piece, so once it was completed, I looked to see if I could alter the rules slightly and capture different feelings/emotions with the results.

The results were so varried and interesting that I had to share them. Each had a personality I could never have tried to design. The overall impression each left was distinct, and the commonalites between them felt just as important as their many differences.

I ended up publishing a set of them as “VOATCSS” - Variations on a theme - a reference to Edward Elgar’s piece by the same name, where he paints portraits of his close friends through the stroke of a common musical theme. Similarly, each of the components of this collection were given a color scheme and parameterization that reminds me of a person that I cared deeply for.

UPDATE (2023): You can now play around with many derivative algorithms (and see the many beautiful images created through them) at ABOUND.art.

Student Union (In Line)

Grady Berry Ward — Tue, 16 Jan 2018 11:58:27 -0800

In my time at Brandeis University, I spent 3 years working hard for the Brandeis Student Union as a Student Representative to the Board of Trustees, and an Executive Board member. Over those three years, we had three different computer science students as “web-master”, and three politicos as “communications director”. The problem was that each web-master would build a beautiful website that was never able to be updated because the communications director never had the technical expertise to update the HTML or use the static site generator.

As college was coming to an end, I was frusturated because my time with the board of trustees had not been as productive as I had originally hoped. Looking for a place I could have a lasting legacy and impact, I found it in the Student Union’s website.

At its heart, the problem is one of distributed data management and editing. Students within the union should have different access to edit different data, but the site should be rendered in a way that does not reflect this multiple authorship - components should be rigid enough that an author’s style does not bleed into a unifying presence. In addition, a distributed ownership model for content needed to be (A) version controlled (B) auditable (C) completely non-technical.

So I built it! The source code is here.

The end product is pretty sweet, and uses some smart optimizations to minimize hosting costs and page serving. I took advantage of Github’s “pages” platform to serve the static HTML rendered by the site. A change in one of the underlying sets of data would trigger a re-render of the HTML generated by the data, and then would be written out to Github pages. This had two main benefits - Github automatically maintains version control history, which enabled auditing of who wrote what and when, and Github stores its HTML files in pages in a blazing fast CDN. This meant that lookups to the webpage were infinitely scalable and cheap (from my perspective), and meant that there were not “read” server costs.

The server that processed writes was built on AppEngine, using JSTL templating over datastore objects to render the HTML to write to Github pages. More exciting than that was the group strucuture that I made so that sub-groups of the Student Union could have access to different sets of permissions for editing. This formed the basis of Committee pages, and organization pages, which were only editible by their members.

Additionally, the data was reused between pages for common elements (for example, all committees should have the names and bios of their members, even if each member is in 10 committees).

On the front end, I used a “Medium”-esque visual markdown editor to enable the construction of HTML elements from a given set of supported ones (which I provided a cohesive design language for on the rendering side).

The result is gorgeous, and far more importantly, meets the needs of the Union. I left Brandeis two years ago (writing this from 2018), and it has been used effectively by three Student Unions since. I do occasional performance upgrades and feature requests, but I’ve seen the site updated by several dozen Union Members without any technical knowledge.

I’m quite proud of this project because it stands solidly at the intersection of technical simplicity and client needs. I have confidence that the Union will be using this product long after anyone remembers me at Brandeis.

Senior Thesis

Grady Berry Ward — Tue, 16 Jan 2018 11:58:27 -0800

I wrote my undergraduate thesis on the Graph Isomorphism (GI) problem, and several topics within that space. My work was version controlled on github, and the final result is here.

I was first interested in the “Paths” invariant, which is incredibly powerful test for discerning between potentially isomorphic graphs in cubic (and fully paralellizable) time, and was originally theorized to be a solution to GI. The primary findings of my thesis were that:

[Computational Wrangling] Unlike was theorized, copaths graphs do exist, and finding them is easily done brute force calculations on today’s standard GPUs.
[Theoretical Proof] Paths uniquely determines the chromatic polynomial of a graph.
[Analytical Algorithms] Measuring discriminatory power to value ratio of invariants within graph isomorphism is possible and can help us devise better algorithms for practical isomorphism detection, and the paths invariant is well weighted in an analysis of discriminatory power.
[Theoretical Proof] Paths as an invariant does not need to be calculated past N vertices.

Since I answered most questions I was interested in paths stuff earlier than expected, I started working on Random Graph theory and found that:

[Literature Review] Random Graph generators almost all are random matrix generators. This understates the frequency of highly auto-isomorphic graphs by one-to-factorial odds.
[Coding] We can create a better random graph generator to evaluate random graph algorithms.
[Analytical Proof] Many algorithms are evaluated across “matrix random” graphs, and their amortized running time can be exponentially higher when using a true random graph generator.

I defended this thesis was successfully defended in May of 2016, and recieved highest honors in Computer Science.

Parley

Grady Berry Ward — Tue, 16 Jan 2018 11:58:27 -0800

Comment threads are an inherently flawed way to have a discussion, because they are strictly linear. In recent years, large organizations like facebook have realized this and have tried to reconcile it with a binary tree pattern of deeper or next. This only partially solves the problem. Far more frequently, if I want to respond to a comment, there are multiple pieces of it that I want to respond to, and some may already be addressed further down the thread. The heart of the matter is that a meaningful static discussion is not best facilitated by a linear progression of strings.

Parley was my idea on how to change this. It views a document as something which is not responed to in whole, but which can be annotated with comments by different authors. The comments (in turn) become top level objects which are just as open to annotation and further comment. The recursive, n-ary tree nature of this structure has multiple advantages, but the big one is that you can dive deep on a given line of argument naturally, by clicking through and understanding the points as made in isolation.

This idea in and of itself is not revolutionary, but I wanted to see if I could make a demo where this is implemented in a way that is comprehensible and intuitive. I half succeeded here. The parley interface used the Draw Me CSS pattern to create a right-left layout where the content is displayed with underlines corresponding to annotations, and the text of the annotation is displayed in a scrolling left hand column. The interface is flexible, relatively intuitive, and fairly simple.

Where I ran into problems was in the minutia – what about really long annotations? What about annotations stacked 15 high? How do you link effectively (as links cannot then be annotated)?

The backend is lame (an app engine hack), but I stand behind the fundamental idea (recursive comment sections, rather than linear ones), and I stand behind the UI for what it is - a proof of concept.

To use the system, go to parly.us, and be aware that the AppEngine backend takes up to a minute to spin up - so allow a minute to pass after a first page access, then reload the page to interact with the software.