aicoded.dev

Simulated attacks: how the delivery pipeline tests the running app

Once an app's tests pass, the delivery pipeline starts the program it built and attacks it, as the security checks of layer L7: requests without a session or without a role, another viewer who tries to see or change records that are not theirs, markup in every field, and a search of the app's log and telemetry for what people entered. This guide says how the app runs, who the test viewers are, what each attack does, what fails a publish and what is only a note, how to let the attacks reach every page, and what they cannot know.

Where they run

The attacks are two steps of the delivery pipeline. They come after tests, and run only when the tests passed:

Step Layer What it does
attacks L7 Simulated attacks finds the app's pages and forms, as a viewer would, and attacks them
personal-data L7 Simulated attacks searches what the app wrote during the attacks for what was entered in its forms

One run of the app serves both. personal-data is skipped when attacks ended in error, or when the app did not start. The app runs on its own, and nothing serves it to people:

Each request carries a viewer token for one of the test viewers, as a request through the company login does, or, to test the app without a session, no token or one that is not valid.

The test viewers

The attacks act as three viewers, none of them in a group:

In the room-maintenance example, /tickets/{id} declares the Guard with role="staff", and the Guard lets in the ticket's author and hotel-ops. So the other viewer holds staff alone, and must neither see nor change the tickets the first viewer reported. /tickets/{id}/edit asks for the same roles, and is tested the same way. /tickets/{id}/status asks for hotel-ops as well, more than the page that declares the Guard, so it is closed by role and not tested for ownership: the other viewer, without hotel-ops, gets 403 there whoever owns the ticket. A page with shared="true" shows every record to everyone its rules admit, so it is not tested for ownership either.

How the attacks find pages

The first viewer finds the pages as a person would:

  1. It starts at / and at every page without an id in its URL. It follows, breadth-first, the links, the addresses of forms and the redirects that stay in the app, up to six steps from where it started.
  2. It posts each form it finds on a page without an id, so the forms that create records run first. A form counts as used when it answers with a redirect, as it does when Process returns web.Redirect; the attacks follow the redirect too.
  3. After the other viewer's tests, it posts the forms of the pages with an id, so a form that cancels or deletes a record runs last.

It fills every field with a value that the field's checks accept:

A guarded page with a text parameter that the first viewer did not reach is also tried with that viewer's own id, as in /people/l7-a/certificates, since such a page often shows the viewer's own records.

The attacks stop finding pages after 4000 requests or 3 minutes, and visit at most 20 URLs of each page; the whole step ends within 8 minutes, and an answer is read up to 2 MiB. Reaching a limit is no problem in itself, but each page not reached by then is a note (E-GATE-038).

What each attack does

A page that the first viewer reached is attacked at the URLs it reached. Any other page is attacked at a URL made from its path, with 1 for a number in it and l7 for text, such as /trips/1.

What fails and what is a note

These problems stop the publish, as those of every other step do:

E-GATE-038 is a note, which does not stop the publish: a page the attacks did not reach. The change record counts the pages, those the attacks reached, and lists those they did not, so security sees which pages they did not check.

A problem about a page is at its template, at line 1, such as pages/trips/n_id/index.html:1; one about a value entered is at the template of the page with the form; E-GATE-036 and E-GATE-037 are at main.go:1. The message names the request, the roles of the test viewer, what the app answered and what was expected, and never holds a token:

  trips: pages/trips/n_id/index.html:1: E-GATE-032: POST /trips/7 (form withdraw) as another viewer with the roles staff: answered 303, expected 403 or 404
    fix: check ownership in the page's Guard, which runs before every form, not in Data or Process
    docs: https://aicoded.dev/docs/errors/E-GATE-032

When the attacks cannot run through a fault of the delivery pipeline, such as a runner that does not answer, the step ends in error, never in passed: publish a new commit later, as after any error.

Letting the attacks reach every page

The attacks check only what they reach, and they reach a page the way a viewer does:

In the room-maintenance example, the menu links /tickets/report, whose form reports a ticket and redirects to it; /tickets links each ticket, and a ticket's page links its edit and status pages. So the attacks reach every page, and test the Guard of /tickets/{id} and of its edit page.

What the attacks cannot know

The attacks test that the app keeps to what its permission list says, not that the list says the right thing:

The codes

aicoded explain <code> prints the page of each, as does the howto tool of aicoded mcp.

See also

Markdown · Source on GitHub · framework f46741f