Second Look

What Second Look stores and sends

A review means sending code to an AI model. This page says what goes where. It describes this deployment as the code in the repository is written; a team that runs its own copy can check every line of it.

Sent to the model, for each review

  • The pull request's diff, without lockfiles, generated files, and binaries.
  • The full contents of the changed files, up to a size limit, for context.
  • The pull request's title and description, and the repository's .second-look.md from the main branch.
  • On a later commit: the earlier findings the reviewer confirmed or rejected, with rejection reasons.

The model is Anthropic's, called through its API, and Anthropic's terms for API use apply to what is sent. Nothing is sent anywhere else.

Sent to a chat channel, if one is configured

A team can set a webhook so reviewers are told when a review is waiting. The message holds the repository, the pull request's title, number and author, the number of findings by severity, and a link. It never holds code or the text of a finding. This deployment has no webhook configured unless its operator has added one.

Kept in the database

  • Repository owner and name, pull request number, title, and author's GitHub login, and the commits reviewed.
  • The first pass's summary and findings: file, lines, severity, and the model's explanation and suggested fix. These can quote short pieces of the code.
  • Reviewers' decisions, rewrites, rejection reasons, notes to the author, and the GitHub login of whoever made each one, with the time.
  • Token counts per review, for cost.

Not kept

  • The diff and the file contents. They are fetched from GitHub each time a review page is opened, and held in memory for a few minutes.
  • Code pasted into the playground. It is sent to the model once and not stored; only the token count is.
  • GitHub tokens. Signing in puts a signed cookie in your browser with your GitHub login and the name, email and picture link from your GitHub profile. Nothing about you is written to the database until you make a decision, and then only your login.
  • IP addresses. The playground's rate limit keys on a salted hash of the address.

Logs

Server logs hold event names, ids, counts, and error codes. They never hold code, diffs, file contents, model output, or tokens.

On GitHub

Nothing is posted to a pull request until a reviewer publishes. Then one review is posted with only the findings the reviewer confirmed, and the check is updated. Rejected findings and reasons stay in Second Look.

Back to the playground