Beyond coding. We forge. https://forgejo.org
Find a file
Gergely Nagy e1fe3bbdc0
feat(quota): Humble beginnings of a quota engine
This is an implementation of a quota engine, and the API routes to
manage its settings. This does *not* contain any enforcement code: this
is just the bedrock, the engine itself.

The goal of the engine is to be flexible and future proof: to be nimble
enough to build on it further, without having to rewrite large parts of
it.

It might feel a little more complicated than necessary, because the goal
was to be able to support scenarios only very few Forgejo instances
need, scenarios the vast majority of mostly smaller instances simply do
not care about. The goal is to support both big and small, and for that,
we need a solid, flexible foundation.

There are thee big parts to the engine: counting quota use, setting
limits, and evaluating whether the usage is within the limits. Sounds
simple on paper, less so in practice!

Quota counting
==============

Quota is counted based on repo ownership, whenever possible, because
repo owners are in ultimate control over the resources they use: they
can delete repos, attachments, everything, even if they don't *own*
those themselves. They can clean up, and will always have the permission
and access required to do so. Would we count quota based on the owning
user, that could lead to situations where a user is unable to free up
space, because they uploaded a big attachment to a repo that has been
taken private since. It's both more fair, and much safer to count quota
against repo owners.

This means that if user A uploads an attachment to an issue opened
against organization O, that will count towards the quota of
organization O, rather than user A.

One's quota usage stats can be queried using the `/user/quota` API
endpoint. To figure out what's eating into it, the
`/user/repos?order_by=size`, `/user/quota/attachments`,
`/user/quota/artifacts`, and `/user/quota/packages` endpoints should be
consulted. There's also `/user/quota/check?subject=<...>` to check
whether the signed-in user is within a particular quota limit.

Quotas are counted based on sizes stored in the database.

Setting quota limits
====================

There are different "subjects" one can limit usage for. At this time,
only size-based limits are implemented, which are:

- `size:all`: As the name would imply, the total size of everything
  Forgejo tracks.
- `size:repos:all`: The total size of all repositories (not including
  LFS).
- `size:repos:public`: The total size of all public repositories (not
  including LFS).
- `size:repos:private`: The total size of all private repositories (not
  including LFS).
- `size:git:all`: The total size of all git data (including all
  repositories, and LFS).
- `size:git:lfs`: The size of all git LFS data (either in private or
  public repos).
- `size:assets:all`: The size of all assets tracked by Forgejo.
- `size:assets:attachments:all`: The size of all kinds of attachments
  tracked by Forgejo.
- `size:assets:attachments:issues`: Size of all attachments attached to
  issues, including issue comments.
- `size:assets:attachments:releases`: Size of all attachments attached
  to releases. This does *not* include automatically generated archives.
- `size:assets:artifacts`: Size of all Action artifacts.
- `size:assets:packages:all`: Size of all Packages.
- `size:wiki`: Wiki size

Wiki size is currently not tracked, and the engine will always deem it
within quota.

These subjects are built into Rules, which set a limit on *all* subjects
within a rule. Thus, we can create a rule that says: "1Gb limit on all
release assets, all packages, and git LFS, combined". For a rule to
stand, the total sum of all subjects must be below the rule's limit.

Rules are in turn collected into groups. A group is just a name, and a
list of rules. For a group to stand, all of its rules must stand. Thus,
if we have a group with two rules, one that sets a combined 1Gb limit on
release assets, all packages, and git LFS, and another rule that sets a
256Mb limit on packages, if the user has 512Mb of packages, the group
will not stand, because the second rule deems it over quota. Similarly,
if the user has only 128Mb of packages, but 900Mb of release assets, the
group will not stand, because the combined size of packages and release
assets is over the 1Gb limit of the first rule.

Groups themselves are collected into Group Lists. A group list stands
when *any* of the groups within stand. This allows an administrator to
set conservative defaults, but then place select users into additional
groups that increase some aspect of their limits.

To top it off, it is possible to set the default quota groups a user
belongs to in `app.ini`. If there's no explicit assignment, the engine
will use the default groups. This makes it possible to avoid having to
assign each and every user a list of quota groups, and only those need
to be explicitly assigned who need a different set of groups than the
defaults.

If a user has any quota groups assigned to them, the default list will
not be considered for them.

The management APIs
===================

This commit contains the engine itself, its unit tests, and the quota
management APIs. It does not contain any enforcement.

The APIs are documented in-code, and in the swagger docs, and the
integration tests can serve as an example on how to use them.

Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
2024-08-02 11:10:34 +02:00
.devcontainer Update ghcr.io/devcontainers/features/git-lfs Docker tag to v1.2.1 2024-06-20 00:01:42 +00:00
.forgejo fix(release-notes-assistant): categorize multiline drafts & cleanup 2024-08-01 20:56:34 +02:00
assets Merge pull request '[CHORE] Use github.com/ProtonMail/go-crypto' (#4506) from gusted/proton-openpgp into forgejo 2024-07-15 16:49:05 +00:00
build Add testifylint to lint checks (#4535) 2024-07-30 19:41:10 +00:00
cmd Add testifylint to lint checks (#4535) 2024-07-30 19:41:10 +00:00
contrib Lock file maintenance 2024-07-01 00:03:45 +00:00
custom/conf Add signature support for the RPM module (#4780) 2024-08-02 05:56:57 +00:00
docker fix(Dockerfile.rootless): revert to default path for app.ini 2024-04-26 21:30:10 +02:00
models feat(quota): Humble beginnings of a quota engine 2024-08-02 11:10:34 +02:00
modules feat(quota): Humble beginnings of a quota engine 2024-08-02 11:10:34 +02:00
options feat(UI): fix links, add labels for releases on repo activity page 2024-08-02 07:56:03 +02:00
public Improvements to English locale (#4453) 2024-07-12 11:58:50 +00:00
release-notes chore(release-notes): weekly cherry-pick week 2024-31 2024-07-28 09:00:29 +02:00
releases/images [DOCS] RELEASE-NOTES.md 2024-02-05 14:44:32 +01:00
routers feat(quota): Humble beginnings of a quota engine 2024-08-02 11:10:34 +02:00
services feat(quota): Humble beginnings of a quota engine 2024-08-02 11:10:34 +02:00
templates feat(quota): Humble beginnings of a quota engine 2024-08-02 11:10:34 +02:00
tests feat(quota): Humble beginnings of a quota engine 2024-08-02 11:10:34 +02:00
tools Add lint-go-gopls (#30729) 2024-06-09 11:13:39 +02:00
web_src Merge pull request 'Implement external release assets' (#1445) from maltejur/forgejo:forgejo-external-attachments into forgejo 2024-07-30 15:50:57 +00:00
.air.toml Reduce air verbosity (#31417) 2024-06-23 12:30:09 +02:00
.deadcode-out chore: update .deadcode.out 2024-07-28 09:00:29 +02:00
.dockerignore Add /public/assets/img/webpack to ignore files again (#30451) 2024-04-15 20:01:36 +02:00
.editorconfig fixed indentation style in editorconfig for go.mod 2024-05-14 00:24:18 +02:00
.envrc Enable direnv (#31672) 2024-07-28 07:18:24 +02:00
.eslintrc.yaml [PORT] Enable no-jquery/no-parse-html-literal and fix violation (gitea#31684) 2024-07-28 16:52:02 +02:00
.gitattributes Add interface{} to any replacement to make fmt, exclude *.pb.go (#30461) 2024-04-15 20:01:36 +02:00
.gitignore Enable direnv (#31672) 2024-07-28 07:18:24 +02:00
.gitmodules cleanup(tests): remove manual testing submodule 2024-04-21 10:13:51 +02:00
.gitpod.yml Remove sqlite-viewer and using database client (#31223) 2024-06-09 11:13:39 +02:00
.golangci.yml Add testifylint to lint checks (#4535) 2024-07-30 19:41:10 +00:00
.ignore Add /options/license and /options/gitignore to .ignore (#30219) 2024-04-07 15:40:31 +02:00
.markdownlint.yaml Update JS dependencies (#28537) 2023-12-30 05:29:03 +00:00
.npmrc Upgrade to npm lockfile v3 and explicitely set it (#23561) 2023-03-18 19:38:10 +01:00
.release-notes-assistant.yaml feat(release-notes-assistant): diff of the change in preview 2024-07-25 22:32:14 +02:00
.spectral.yaml
.yamllint.yaml fully replace drone with actions (#27556) 2023-10-11 06:39:32 +00:00
BSDmakefile Fix build errors on BSD (in BSDMakefile) (#27594) 2023-10-13 15:38:27 +00:00
build.go User/Org Feed render description as per web (#23887) 2023-04-04 04:39:47 +01:00
CODEOWNERS I feel responsible … (Codeowners) 2024-04-27 02:22:05 +02:00
CONTRIBUTING.md docs: contributing: avoid information duplication (#3454) 2024-04-25 19:10:43 +00:00
DCO Remove address from DCO (#22595) 2023-01-24 18:52:38 +00:00
Dockerfile Fix deprecated Dockerfile ENV format (#31450) 2024-06-23 13:20:40 +02:00
Dockerfile.rootless Fix deprecated Dockerfile ENV format (#31450) 2024-06-23 13:20:40 +02:00
flake.lock Fix update flake (#31626) 2024-07-14 11:35:15 +02:00
flake.nix Fix update flake (#31626) 2024-07-14 11:35:15 +02:00
go.mod Add signature support for the RPM module (#4780) 2024-08-02 05:56:57 +00:00
go.sum Add signature support for the RPM module (#4780) 2024-08-02 05:56:57 +00:00
LICENSE [DOCS] LICENSE: add Forgejo Authors 2024-02-05 14:44:32 +01:00
main.go [RELEASE] decouple the release name from the version number 2024-02-17 15:27:35 +01:00
Makefile chore(renovate): use mirror image 2024-07-30 09:23:44 +02:00
package-lock.json Update dependency vue to v3.4.35 2024-08-01 00:02:22 +00:00
package.json Update dependency vue to v3.4.35 2024-08-01 00:02:22 +00:00
playwright.config.js Enforce trailing comma in JS on multiline (#30002) 2024-03-26 19:04:27 +01:00
poetry.lock Lock file maintenance 2024-07-01 00:03:45 +00:00
poetry.toml Clean up pyproject.toml and package.json, fix poetry options (#25327) 2023-06-18 18:13:08 +00:00
pyproject.toml Enable poetry non-package mode (#31282) 2024-06-09 16:04:57 +02:00
README.md [skip ci] IGNORE (#4106) 2024-06-11 16:06:50 +00:00
release-notes-assistant.sh fix(release-notes-assistant): categorize multiline drafts & cleanup 2024-08-01 20:56:34 +02:00
RELEASE-NOTES.md docs(release-notes): 8.0.0 & 7.0.6 - updates 2024-07-30 16:28:02 +02:00
renovate.json chore(renovate): use mirror image 2024-07-30 09:23:44 +02:00
stylelint.config.js Merge pull request 'Port "Enable declaration-block-no-redundant-longhand-properties (#30950)' (#3769) from beowulf/gitea-port-pull-30950 into forgejo 2024-05-14 22:23:54 +00:00
tailwind.config.js [FEAT] folding results for repo search (#4134) 2024-06-15 20:16:18 +00:00
vitest.config.js Switch to happy-dom for testing (#29948) 2024-03-26 19:04:26 +01:00
webpack.config.js Merge pull request '[CHORE] Remove AGPL-1.0 as allowed license' (#4673) from gusted/forgejo-rm-agpl into forgejo 2024-07-25 07:40:19 +00:00

Welcome to Forgejo

Hi there! Tired of big platforms playing monopoly? Providing Git hosting for your project, friends, company or community? Forgejo (/for'd͡ʒe.jo/ inspired by forĝejo the Esperanto word for forge) has you covered with its intuitive interface, light and easy hosting and a lot of builtin functionality.

Forgejo was created in 2022 because we think that the project should be owned by an independent community. If you second that, then Forgejo is for you! Our promise: Independent Free/Libre Software forever!

What does Forgejo offer?

If you like any of the following, Forgejo is literally meant for you:

  • Lightweight: Forgejo can easily be hosted on nearly every machine. Running on a Raspberry? Small cloud instance? No problem!
  • Project management: Besides Git hosting, Forgejo offers issues, pull requests, wikis, kanban boards and much more to coordinate with your team.
  • Publishing: Have something to share? Use releases to host your software for download, or use the package registry to publish it for docker, npm and many other package managers.
  • Customizable: Want to change your look? Change some settings? There are many config switches to make Forgejo work exactly like you want.
  • Powerful: Organizations & team permissions, CI integration, Code Search, LDAP, OAuth and much more. If you have advanced needs, Forgejo has you covered.
  • Privacy: From update checker to default settings: Forgejo is built to be privacy first for you and your crew.
  • Federation: (WIP) We are actively working to connect software forges with each other through ActivityPub, and create a collaborative network of personal instances.

Learn more

Dive into the documentation, subscribe to releases and blog post on our website, find us on the Fediverse or hop into our Matrix room if you have any questions or want to get involved.

Get involved

If you are interested in making Forgejo better, either by reporting a bug or by changing the governance, please take a look at the contribution guide.