aicoded.dev

What cannot be built

AI assistants make the same few mistakes that leaked data from well-known AI-built apps. The framework and the delivery pipeline refuse each one: with a code the builder sees, or before the request reaches any of the app's own code.

Each case below says what goes wrong and what refuses it, and links the tests that make the attempt on purpose, in small apps like those an AI assistant writes. Where a test also runs the same app without the mistake, that app works, so the mistake alone is refused. aicoded publish runs aicoded check before it sends an app, and the delivery pipeline runs it again on every publish. The linked tests are part of the open framework, for anyone to read and run.

A table anyone can read

What goes wrong: the rules that say who may see which data are missing or wrong, so anyone with the link reads every row.

What refuses it:

A database key in the app

What goes wrong: an app that holds its database's address and password, or another key, hands them to whoever gets its code.

What refuses it:

And the runner hands the app no key to leak. The runner logs in to the database for the app. In the test, an app whose permission list declares a database looks for a key where a runner could put one: its only environment variable names the folder of its sockets, and that folder holds nothing but sockets. Yet the app writes a row to its database and reads it back. aicoded check would refuse this app's code, and the test runs it as if it had got past the checks. Test: TestNothingToLeak, app: key-hunt.

A request that skips the login

What goes wrong: an app that trusts the browser, or forgets to check the session on one form, lets in a request that never passed the login.

What refuses it: the app does not write this check. Every request carries a viewer token that the runner signs for this one app, valid for at most a minute, and the framework refuses a request without a valid one with 401 before it reaches any of the app's own code: a page, a form and a live connection alike.

In the test, requests go straight to a notes page that only staff may open, past the company login, with no token, a malformed, forged or expired one, or one for another app. Each gets 401 and nothing of the page, the app logs why, and the form adds no note. The same requests with a token signed for a member of staff get the page, add the note and open the live connection. Test: TestRequestThatSkipsTheLogin, app: staff-notes.

Markdown · Source on GitHub · framework f46741f