This is a place for me to document my own guidelines for using AI. These are written with the core assumption that doing something yourself slowly is better than an AI doing it quickly. This ultimately means that I prefer to not use AI at all, however sometimes the speed of task completion takes precedence over my own value of my labor, creativity, and learning. That is a particularly difficult thing for me to admit, and my hope is that codifying these guidelines will make me more intentional in my use of AI.
To clarify, by AI I mean the usage of large language models (LLM), especially through chat or agentic interfaces. These guidelines don't strictly apply to non LLM-based tools. (FSMs, Markov chains, etc.)
There are currently three guidelines which, if not adhered to, constitute an irresponsible usage of AI.
These guidelines are generalizable, but their power comes from their specific applications. I tried to avoid caveats or edge-cases which I might exploit.
This guideline relies on technical language to describe processes which can either be perfectly unscientific or completely contained within code. In most applications of this guideline, I mean that any AI generated code must pass an organic test suite, also written in code. This ensures that any AI generated code satisfies a human-written list of important behaviors: output formats, expected edge cases, etcetera.
I have recently been struggling over whether or not to include visual or manual tests as valid examples of organic tests. For example, if I ask an AI agent to update the style sheet of a particular page on my website to get a format I want, am I allowed to only visually confirm that the layout matches what I want? What would a code-only test look like for this, anyway? Of course I can automate the checking of pixel locations on a screen, but again I encounter the trade-off of time vs. value. The main advantage of having automated tests is the ability to ensure that new features don't break existing ones across your entire application. The "visual test" (or in the case of interfaces, manually confirming the new interaction works) doesn't have this advantage. (Although perhaps an interface could be designed to ensure that it does?)
Avoiding organic and automated tests when using AI quickly leads to laziness and eventually the altogether abandonment of the usage guidelines. The speed of development and of change is simply too fast for me to keep up with and remain critical: to reflect on whether or not I find something interesting, engaging, creative, and well-designed.
My use of the "visual test" is further complicated by my rejection of the same kind of test applied to writing, in particular when drafting new text. The trouble arises when one begins to conflate the benefits of the process of writing and the final, edited text. My point on this isn't particularly original but remains strong: allowing AI to participate in any stage of writing neuters ideas, weakens the writing and editorial skills of the organic writer, and robs the writer of the joy of reflecting, failing, and re-writing. In some specific kinds of technical writing -- e.g. the methodology section of a technical writeup -- the argument can be made that the benefit of the process of writing are so small so as to be discarded entirely in favor of AI generated writing. I suppose the argument would go that no reflection or insights into methodology are typically gained when completing a technical writeup at the end of the project, but in fact I suspect that the problem arises from the misconception that in strict technical writing, the descriptive and the critical are quarantined from each other. Nevertheless, I am suspicious of the straw-man argument I have put forth, and prefer to never use AI when drafting new text.
The same criticisms more or less apply to editing: a struggle between process and output, time and value. According to this guideline, AI should not be allowed to edit text larger than a single sentence at a time. Anything larger is too wide for me to evaluate the implications of for the rest of the piece.
The above discussion mostly assumed that an AI system is editing an artifact directly, either through a code editor or interface. Another important case this guideline doesn't immediately address is when someone doesn't allow AI to edit their object, but instead adopts a "show-and-tell" model: presenting an isolated copy of their work to an AI and asks for advice or edits on this isolated copy. A common use case that follows this description but isn't directly addressed by the above guideline is asking an AI for an outline: either in the case of drafting or editing a piece of writing, or an architecture for a piece of code. Asking an AI for a diagram of what should be done, rather than asking an AI to do it.
I tend to encourage this kind of interaction with AI systems more than direct intervention or editing, as it still requires a stage of critical thinking of what to include or regent from the AI's plan. But do I want to be a manager, or a researcher?
Plan mode, a common feature in most AI systems, originated from a body of research which took advantage of the auto-regressive design of attention mechanisms to have LLMs write relevant text for themselves rather than always have it provided organically. It was not originally designed for the use case I described earlier: the quarantined evaluation or diagramming of work. But it can be used for it.
I must be able to fully describe the input data to another person. This guideline relies on a few ambiguous terms which are best defined on a case-to-case basis. This is the weakest of the three guidelines, as I can't verify it on my own. I don't typically work on projects alone, but I do work on individual tasks alone. More importantly, satisfying this guideline requires getting into the habit of stopping work to reflect on my own assumptions and decisions. I believe this stage in the research process is paramount, but this is a habit which is difficult to train and easy to lose.
Using AI for data analysis tasks (or for analysis tasks more broadly) runs into the same problem as their application to education: if an LLM can complete this task as well as the version of me which fully comprehends the relevant material (as many proponents of AI argue), how could I possibly use the output of an LLM (a class assignment and its grade) to evaluate my own comprehension?
This is the simplest and easiest guideline to verify. If an object was created by AI, or created by code written by AI, it must clearly say so alongside the object. For example, if a slide contains content created by AI or the result of AI generated code, it must say so on that slide. If text (reports, software) was at any point altered by or changed due to the suggestions of an AI model, that must be stated in a dedicated "AI Usage Statement" section at the beginning of the text.