Commit graph

1512 commits

Author SHA1 Message Date
Gusted
d5a1188086 Merge pull request 'feat: add partial quoting' (#5677) from gusted/forgejo-partial-qouting into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/5677
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Reviewed-by: Otto <otto@codeberg.org>
2024-11-05 20:13:04 +00:00
wxiaoguang
171de4d107
[PORT] Fix git error handling (gitea#32401)
---
Conflict resolution: Trivial, for `repo_attributes.go` move where the
`IsErrCanceledOrKilled` needs to happen because of other changes that
happened in this file.

To add some words to this change: It seems to be mostly simplifying the
error handling of git operations.

(cherry picked from commit e524f63d58900557d7d57fc3bcd19d9facc8b8ee)
2024-11-03 16:47:44 +01:00
JakobDev
86546fe63e
feat: Add Search to Releases Page 2024-11-02 10:24:35 +01:00
Tom Neuber
68d5cf0e92
Add branch auto deletion for scheduled PRs 2024-10-31 03:49:15 +01:00
Earl Warren
485db0a3ba Merge pull request '[gitea] week 2024-44 cherry pick (gitea/main -> forgejo)' (#5714) from algernon/wcp/2024-44 into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/5714
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
2024-10-29 09:05:27 +00:00
Gusted
266e0b2ce9
security: add permission check to 'delete branch after merge'
- Add a permission check that the doer has write permissions to the head
repository if the the 'delete branch after merge' is enabled when
merging a pull request.
- Unify the checks in the web and API router to `DeleteBranchAfterMerge`.
- Added integration tests.
2024-10-28 05:48:10 +01:00
Zettat123
8c79008d6f
Add DISABLE_ORGANIZATIONS_PAGE and DISABLE_CODE_PAGE settings for explore pages and fix an issue related to user search (#32288)
These settings can allow users to only display the repositories explore page.

Thanks to yp05327 and wxiaoguang !

---------

Co-authored-by: Giteabot <teabot@gitea.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit 9206fbb55fd28f21720072fce6a36cc22277934c)

Conflicts:
	 - templates/explore/navbar.tmpl
	   Resolved by manually applying the last hunk to our template.
2024-10-27 11:27:40 +01:00
Gusted
2c2ac80030 feat: Add partial quoting
- If you select a portion of the comment, `Quote reply` will not only
quote that portion and not copy paste the whole text as it previously
did. This is achieved by using the `@github/quote-selection` package.
- There's preprocessing to ensure Forgejo-flavored markdown syntax is
preserved.
- e2e test added.
- Resolves #1342
2024-10-26 19:15:43 +02:00
Earl Warren
c1a08156f8 Merge pull request '[gitea] week 2024-43 cherry pick (gitea/main -> forgejo)' (#5621) from algernon/wcp/2024-43 into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/5621
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
2024-10-22 17:37:03 +00:00
Gusted
f298bf125a Merge pull request 'fix: use ValidateEmail as binding across web forms' (#5158) from solomonv/consolidate-email-validation into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/5158
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
2024-10-21 14:31:32 +00:00
dragon
95c7599db5 fix arch pkg 2024-10-21 10:08:57 +08:00
6543
f4c2c401ec
API: enhance SearchIssues swagger docs (#32208)
this will result in better api clients generated out of the openapi docs
... for SearchIssues

---
*Sponsored by Kithara Software GmbH*

(cherry picked from commit d638067d3cb0a7f69b4d899f65b9be4940bd3e41)
2024-10-20 09:44:18 +02:00
Earl Warren
e2354703ed Merge pull request '[gitea] week 2024-42 cherry pick (gitea/main -> forgejo)' (#5543) from earl-warren/wcp/2024-42 into forgejo
Some checks are pending
/ release (push) Waiting to run
testing / backend-checks (push) Waiting to run
testing / frontend-checks (push) Waiting to run
testing / security-check (push) Blocked by required conditions
testing / test-unit (push) Blocked by required conditions
testing / test-e2e (push) Blocked by required conditions
testing / test-remote-cacher (map[image:docker.io/bitnami/redis:7.2 port:6379]) (push) Blocked by required conditions
testing / test-remote-cacher (map[image:docker.io/bitnami/valkey:7.2 port:6379]) (push) Blocked by required conditions
testing / test-remote-cacher (map[image:ghcr.io/microsoft/garnet-alpine:1.0.14 port:6379]) (push) Blocked by required conditions
testing / test-remote-cacher (map[image:registry.redict.io/redict:7.3.0-scratch port:6379]) (push) Blocked by required conditions
testing / test-mysql (push) Blocked by required conditions
testing / test-pgsql (push) Blocked by required conditions
testing / test-sqlite (push) Blocked by required conditions
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/5543
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
2024-10-19 13:54:24 +00:00
Earl Warren
e299a7f4cd Merge pull request 'fix: don't cancel schedule workflows on push to main branch' (#5470) from Kwonunn/forgejo:dont-cancel-schedule-workflow into forgejo
Some checks are pending
/ release (push) Waiting to run
testing / backend-checks (push) Waiting to run
testing / frontend-checks (push) Waiting to run
testing / test-unit (push) Blocked by required conditions
testing / test-e2e (push) Blocked by required conditions
testing / test-remote-cacher (map[image:docker.io/bitnami/redis:7.2 port:6379]) (push) Blocked by required conditions
testing / test-remote-cacher (map[image:docker.io/bitnami/valkey:7.2 port:6379]) (push) Blocked by required conditions
testing / test-remote-cacher (map[image:ghcr.io/microsoft/garnet-alpine:1.0.14 port:6379]) (push) Blocked by required conditions
testing / test-remote-cacher (map[image:registry.redict.io/redict:7.3.0-scratch port:6379]) (push) Blocked by required conditions
testing / test-mysql (push) Blocked by required conditions
testing / test-pgsql (push) Blocked by required conditions
testing / test-sqlite (push) Blocked by required conditions
testing / security-check (push) Blocked by required conditions
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/5470
2024-10-13 12:06:02 +00:00
6543
d0af8fe4dc
Allow filtering PRs by poster in the ListPullRequests API (#32209)
as title

---
*Sponsored by Kithara Software GmbH*

(cherry picked from commit bdd655f2bde5facada4394f36fe54e364787de7a)
2024-10-13 07:58:01 +03:00
Lunny Xiao
a052d2b602
Fix bug when a token is given public only
Port of https://github.com/go-gitea/gitea/pull/32204

(cherry picked from commit d6d3c96e6555fc91b3e2ef21f4d8d7475564bb3e)

Conflicts:
  routers/api/v1/api.go
	services/context/api.go
  trivial context conflicts
2024-10-10 10:41:42 +03:00
yp05327
af901ac7bb
Add support for searching users by email (#30908)
Fix #30898

we have an option `SearchByEmail`, so enable it, then we can search user
by email.
Also added a test for it.

(cherry picked from commit 5d6d025c9b8d2abca9ec2bfdc795d1f0c1c6592d)
2024-10-06 08:10:22 +02:00
Job
1dfe58ad11
Fix PR creation on forked repositories (#31863)
Resolves #20475

(cherry picked from commit 7e68bc88238104d2ee8b5a877fc1ad437f1778a4)

Conflicts:
	tests/integration/pull_create_test.go
  add missing testPullCreateDirectly from
  c63060b130d34e3f03f28f4dccbf04d381a95c17 Fix code owners will not be mentioned when a pull request comes from a forked repository (#30476)
2024-10-06 08:08:40 +02:00
Zettat123
b67b7c1238
Fix the logic of finding the latest pull review commit ID (#32139)
Fix #31423

(cherry picked from commit f4b8f6fc40ce2869135372a5c6ec6418d27ebfba)

Conflicts:
	models/fixtures/comment.yml
  comment fixtures have to be shifted because there is one more in Forgejo
2024-10-06 07:44:28 +02:00
Kwonunn
b20c0b1469
fix: don't cancel schedule workflows on push to main branch 2024-10-05 12:14:44 +02:00
ChristopherHX
8f0a05a7e4
Fix artifact v4 upload above 8MB (#31664)
Multiple chunks are uploaded with type "block" without using
"appendBlock" and eventually out of order for bigger uploads.
8MB seems to be the chunk size

This change parses the blockList uploaded after all blocks to get the
final artifact size and order them correctly before calculating the
sha256 checksum over all blocks

Fixes #31354

(cherry picked from commit b594cec2bda6f861effedb2e8e0a7ebba191c0e9)

Conflicts:
	routers/api/actions/artifactsv4.go
  conflict because of Refactor AppURL usage (#30885) 67c1a07285008cc00036a87cef966c3bd519a50c
    that was not cherry-picked in Forgejo
    the resolution consist of removing the extra ctx argument
2024-09-29 09:24:15 +02:00
Exploding Dragon
89742c4913 feat: add architecture-specific removal support for arch package (#5351)
Some checks are pending
/ release (push) Waiting to run
testing / backend-checks (push) Waiting to run
testing / frontend-checks (push) Waiting to run
testing / test-unit (push) Blocked by required conditions
testing / test-remote-cacher (map[image:docker.io/bitnami/redis:7.2 port:6379]) (push) Blocked by required conditions
testing / test-remote-cacher (map[image:docker.io/bitnami/valkey:7.2 port:6379]) (push) Blocked by required conditions
testing / test-remote-cacher (map[image:ghcr.io/microsoft/garnet-alpine:1.0.14 port:6379]) (push) Blocked by required conditions
testing / test-remote-cacher (map[image:registry.redict.io/redict:7.3.0-scratch port:6379]) (push) Blocked by required conditions
testing / test-mysql (push) Blocked by required conditions
testing / test-pgsql (push) Blocked by required conditions
testing / test-sqlite (push) Blocked by required conditions
testing / security-check (push) Blocked by required conditions
- [x] add architecture-specific removal support
- [x] Fix upload competition
- [x] Fix not checking input when downloading

docs: https://codeberg.org/forgejo/docs/pulls/874

### Release notes

- [ ] I do not want this change to show in the release notes.

Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/5351
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Exploding Dragon <explodingfkl@gmail.com>
Co-committed-by: Exploding Dragon <explodingfkl@gmail.com>
2024-09-27 08:21:22 +00:00
Lunny Xiao
f709de2403
Fix wrong last modify time (#32102)
(cherry picked from commit a802508f88e546bf18990559e44bf27a09c869ee)
2024-09-27 08:42:48 +02:00
KN4CK3R
526054332a
Fix incorrect /tokens api (#32085)
Fixes #32078

- Add missing scopes output.
- Disallow empty scope.

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
(cherry picked from commit 08adbc468f8875fd4763c3656b334203c11adc0a)
2024-09-27 08:42:48 +02:00
Zettat123
961766744b
Check if the due_date is nil when editing issues (#32035)
(cherry picked from commit 3a51c37672d2fbad1f222922e75ce704d5a1ac71)
2024-09-22 09:05:15 +02:00
Kemal Zebari
eb765dabfd
Handle invalid target when creating releases using API (#31841)
A 500 status code was thrown when passing a non-existent target to the
create release API. This snapshot handles this error and instead throws
a 404 status code.

Discovered while working on #31840.

(cherry picked from commit f05d9c98c4cb95e3a8a71bf3e2f8f4529e09f96f)
2024-09-14 18:45:18 +02:00
Adam Majer
5f3a2aafdf
Return 404 instead of error when commit not exist (#31977)
Fix #31976

(cherry picked from commit bf7ae0429cdf236510aa6702e499878bc2345a6e)
2024-09-09 20:54:47 +02:00
Solomon Victorino
471567b3ba fix: use ValidateEmail as binding across web forms 2024-08-28 22:25:17 -06:00
Gusted
5a871f6095
[SEC] Ensure propagation of API scopes for Conan and Container authentication
- The Conan and Container packages use a different type of
authentication. It first authenticates via the regular way (api tokens
or user:password, handled via `auth.Basic`) and then generates a JWT
token that is used by the package software (such as Docker) to do the
action they wanted to do. This JWT token didn't properly propagate the
API scopes that the token was generated for, and thus could lead to a
'scope escalation' within the Conan and Container packages, read
access to write access.
- Store the API scope in the JWT token, so it can be propagated on
subsequent calls that uses that JWT token.
- Integration test added.
- Resolves #5128
2024-08-28 10:33:32 +02:00
Gusted
d623126103 Merge pull request '[PORT] Refactor the usage of batch catfile (gitea#31754)' (#5122) from gusted/forgejo-port-gt-31754 into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/5122
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
2024-08-26 16:04:47 +00:00
Gusted
1a68d14cf8
[FIX] Don't allow SSH authentication without ssh executable
- Follow up of #4819
- When no `ssh` executable is present, disable the UI and backend bits
that allow the creation of push mirrors that use SSH authentication. As
this feature requires the usage of the `ssh` binary.
- Integration test added.
2024-08-26 08:58:35 +02:00
Gusted
b44dcf553c
[TESTS] Fix usage of LoadRepoCommit
It loads the Commit with a temporary open GitRepo. This is incorrect,
the GitRepo should be open as long as the Commit can be used. This
mainly removes the usage of this function as it's not needed.
2024-08-26 08:03:48 +02:00
Philip Peterson
03508b33a8
[FEAT] Allow pushmirror to use publickey authentication
- Continuation of https://github.com/go-gitea/gitea/pull/18835 (by
@Gusted, so it's fine to change copyright holder to Forgejo).
- Add the option to use SSH for push mirrors, this would allow for the
deploy keys feature to be used and not require tokens to be used which
cannot be limited to a specific repository. The private key is stored
encrypted (via the `keying` module) on the database and NEVER given to
the user, to avoid accidental exposure and misuse.
- CAVEAT: This does require the `ssh` binary to be present, which may
not be available in containerized environments, this could be solved by
adding a SSH client into forgejo itself and use the forgejo binary as
SSH command, but should be done in another PR.
- CAVEAT: Mirroring of LFS content is not supported, this would require
the previous stated problem to be solved due to LFS authentication (an
attempt was made at forgejo/forgejo#2544).
- Integration test added.
- Resolves #4416
2024-08-22 17:05:07 +02:00
Gusted
fe18428806
Fix linting issues 2024-08-18 16:25:13 +02:00
Otto
b77f45f46b Merge pull request 'Revert "Prevent allow/reject reviews on merged/closed PRs"' (#4907) from caesar/forgejo:revert-no-closed-pr-review into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4907
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Reviewed-by: Otto <otto@codeberg.org>
2024-08-13 23:25:54 +00:00
Otto Richter
e330c88411 fix(api): Correct descriptions for quota calls 2024-08-12 22:05:56 +02:00
Caesar Schinas
65c2595f26
Revert "Prevent allow/reject reviews on merged/closed PRs"
This reverts commit 4ed372af13.
This change from Gitea was not considered by the Forgejo UI team and there is a consensus that it feels like a regression.

The test which was added in that commit is kept and modified to test that reviews can successfully be submitted on closed and merged PRs.

Closes forgejo/design#11
2024-08-12 12:24:52 +01:00
Earl Warren
44002a6399 Merge pull request 'chore(refactor): split repo_service.ForkRepository in two' (#4879) from earl-warren/forgejo:wip-fork-split into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4879
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
2024-08-11 15:50:52 +00:00
Earl Warren
cfefe2b6c9
chore(refactor): split repo_service.ForkRepository in two
ForkRepository performs two different functions:

* The fork itself, if it does not already exist
* Updates and notifications after the fork is performed

The function is split to reflect that and otherwise unmodified.

The two function are given different names to:

* clarify which integration tests provides coverage
* distinguish it from the notification method by the same name
2024-08-11 12:40:34 +02:00
Exploding Dragon
87d50eca87 feat: support grouping by any path for arch package (#4903)
Previous arch package grouping was not well-suited for complex or multi-architecture environments. It now supports the following content:

- Support grouping by any path.
- New support for packages in `xz` format.
- Fix clean up rules

<!--start release-notes-assistant-->

## Draft release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Features
  - [PR](https://codeberg.org/forgejo/forgejo/pulls/4903): <!--number 4903 --><!--line 0 --><!--description c3VwcG9ydCBncm91cGluZyBieSBhbnkgcGF0aCBmb3IgYXJjaCBwYWNrYWdl-->support grouping by any path for arch package<!--description-->
<!--end release-notes-assistant-->

Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4903
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Exploding Dragon <explodingfkl@gmail.com>
Co-committed-by: Exploding Dragon <explodingfkl@gmail.com>
2024-08-11 10:35:11 +00:00
Yaroslav Halchenko
5ae2dbcb14 Adjust codespell config + make it fix few typos which sneaked in since addition of codespell support (#4857)
Now that my colleague just posted a wonderful blog post https://blog.datalad.org/posts/forgejo-runner-podman-deployment/ on forgejo runner, some time I will try to add that damn codespell action to work on CI here ;)  meanwhile some typos managed to sneak in and this PR should address them (one change might be functional in a test -- not sure if would cause a fail or not)

### Release notes

- [ ] I do not want this change to show in the release notes.
- [ ] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.

Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4857
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Yaroslav Halchenko <debian@onerussian.com>
Co-committed-by: Yaroslav Halchenko <debian@onerussian.com>
2024-08-08 16:07:35 +00:00
Exploding Dragon
1bc986423d fix: rpm sign resource leak (#4878)
Fixed the resource leak  in #4780.

Related:  [go-gitea/gitea#31794](https://github.com/go-gitea/gitea/pull/31794)
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4878
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Exploding Dragon <explodingfkl@gmail.com>
Co-committed-by: Exploding Dragon <explodingfkl@gmail.com>
2024-08-08 07:28:09 +00:00
Earl Warren
517637137c Merge pull request '[gitea] week 2024-32 cherry pick (gitea/main -> forgejo)' (#4801) from earl-warren/wcp/2024-32 into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4801
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
2024-08-06 05:47:20 +00:00
Gergely Nagy
f121e87aa6
activitypub: Implement an instance-wide actor
An instance-wide actor is required for outgoing signed requests that are
done on behalf of the instance, rather than on behalf of other actors.
Such things include updating profile information, or fetching public
keys.

Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
2024-08-05 10:50:26 +02:00
Jason Song
92fbc8e216
Set owner id to zero when GetRegistrationToken for repo (#31725)
Fix #31707.

It's split from #31724.

Although #31724 could also fix #31707, it has change a lot so it's not a
good idea to backport it.

(cherry picked from commit 81fa471119a6733d257f63f8c2c1f4acc583d21b)
2024-08-04 10:14:34 +02:00
Bo-Yi Wu
6a4d1dfab8
fix(api): owner ID should be zero when created repo secret (#31715)
- Change condition to include `RepoID` equal to 0 for organization
secrets

---------

Signed-off-by: Bo-Yi Wu <appleboy.tw@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
(cherry picked from commit d39bce7f003cf2137a5a561ed488c7b638e52275)

Conflicts:
	routers/api/v1/repo/action.go
  trivial context conflict (PathParams vs Params)
2024-08-04 10:14:34 +02:00
Jason Song
6e63afe31f
Fix API endpoint for registration-token (#31722)
Partially fix #31707. Related to #30656

(cherry picked from commit bf5ae79c5163b8dd6a3185711ad11893b1270f62)
2024-08-04 10:14:34 +02:00
Exploding Dragon
f17194ca91 Arch packages implementation (#4785)
This PR is from https://github.com/go-gitea/gitea/pull/31037

This PR was originally created by @d1nch8g , and the original source code comes from https://ion.lc/core/gitea.

This PR adds a package registry for [Arch Linux](https://archlinux.org/) packages with support for package files, [signatures](https://wiki.archlinux.org/title/Pacman/Package_signing), and automatic [pacman-database](https://archlinux.org/pacman/repo-add.8.html) management.

Features:

1. Push any ` tar.zst ` package and Gitea sign it.
2. Delete endpoint for specific package version and all related files
3. Supports trust levels with `SigLevel = Required`.
4. Package UI with instructions to connect to the new pacman database and visualised package metadata

![](/attachments/810ca6df-bd20-44c2-bdf7-95e94886d750)

You can follow [this tutorial](https://wiki.archlinux.org/title/Creating_packages) to build a *.pkg.tar.zst package for testing

docs pr: https://codeberg.org/forgejo/docs/pulls/791

Co-authored-by: d1nch8g@ion.lc
Co-authored-by: @KN4CK3R
Co-authored-by: @mahlzahn
Co-authored-by: @silverwind
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4785
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Exploding Dragon <explodingfkl@gmail.com>
Co-committed-by: Exploding Dragon <explodingfkl@gmail.com>
2024-08-04 06:16:29 +00:00
Gergely Nagy
67fa52dedb
feat(quota): Quota enforcement
The previous commit laid out the foundation of the quota engine, this
one builds on top of it, and implements the actual enforcement.

Enforcement happens at the route decoration level, whenever possible. In
case of the API, when over quota, a 413 error is returned, with an
appropriate JSON payload. In case of web routes, a 413 HTML page is
rendered with similar information.

This implementation is for a **soft quota**: quota usage is checked
before an operation is to be performed, and the operation is *only*
denied if the user is already over quota. This makes it possible to go
over quota, but has the significant advantage of being practically
implementable within the current Forgejo architecture.

The goal of enforcement is to deny actions that can make the user go
over quota, and allow the rest. As such, deleting things should - in
almost all cases - be possible. A prime exemption is deleting files via
the web ui: that creates a new commit, which in turn increases repo
size, thus, is denied if the user is over quota.

Limitations
-----------

Because we generally work at a route decorator level, and rarely
look *into* the operation itself, `size:repos:public` and
`size:repos:private` are not enforced at this level, the engine enforces
against `size:repos:all`. This will be improved in the future.

AGit does not play very well with this system, because AGit PRs count
toward the repo they're opened against, while in the GitHub-style fork +
pull model, it counts against the fork. This too, can be improved in the
future.

There's very little done on the UI side to guard against going over
quota. What this patch implements, is enforcement, not prevention. The
UI will still let you *try* operations that *will* result in a denial.

Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
2024-08-02 11:10:34 +02:00
Gergely Nagy
e1fe3bbdc0
feat(quota): Humble beginnings of a quota engine
This is an implementation of a quota engine, and the API routes to
manage its settings. This does *not* contain any enforcement code: this
is just the bedrock, the engine itself.

The goal of the engine is to be flexible and future proof: to be nimble
enough to build on it further, without having to rewrite large parts of
it.

It might feel a little more complicated than necessary, because the goal
was to be able to support scenarios only very few Forgejo instances
need, scenarios the vast majority of mostly smaller instances simply do
not care about. The goal is to support both big and small, and for that,
we need a solid, flexible foundation.

There are thee big parts to the engine: counting quota use, setting
limits, and evaluating whether the usage is within the limits. Sounds
simple on paper, less so in practice!

Quota counting
==============

Quota is counted based on repo ownership, whenever possible, because
repo owners are in ultimate control over the resources they use: they
can delete repos, attachments, everything, even if they don't *own*
those themselves. They can clean up, and will always have the permission
and access required to do so. Would we count quota based on the owning
user, that could lead to situations where a user is unable to free up
space, because they uploaded a big attachment to a repo that has been
taken private since. It's both more fair, and much safer to count quota
against repo owners.

This means that if user A uploads an attachment to an issue opened
against organization O, that will count towards the quota of
organization O, rather than user A.

One's quota usage stats can be queried using the `/user/quota` API
endpoint. To figure out what's eating into it, the
`/user/repos?order_by=size`, `/user/quota/attachments`,
`/user/quota/artifacts`, and `/user/quota/packages` endpoints should be
consulted. There's also `/user/quota/check?subject=<...>` to check
whether the signed-in user is within a particular quota limit.

Quotas are counted based on sizes stored in the database.

Setting quota limits
====================

There are different "subjects" one can limit usage for. At this time,
only size-based limits are implemented, which are:

- `size:all`: As the name would imply, the total size of everything
  Forgejo tracks.
- `size:repos:all`: The total size of all repositories (not including
  LFS).
- `size:repos:public`: The total size of all public repositories (not
  including LFS).
- `size:repos:private`: The total size of all private repositories (not
  including LFS).
- `size:git:all`: The total size of all git data (including all
  repositories, and LFS).
- `size:git:lfs`: The size of all git LFS data (either in private or
  public repos).
- `size:assets:all`: The size of all assets tracked by Forgejo.
- `size:assets:attachments:all`: The size of all kinds of attachments
  tracked by Forgejo.
- `size:assets:attachments:issues`: Size of all attachments attached to
  issues, including issue comments.
- `size:assets:attachments:releases`: Size of all attachments attached
  to releases. This does *not* include automatically generated archives.
- `size:assets:artifacts`: Size of all Action artifacts.
- `size:assets:packages:all`: Size of all Packages.
- `size:wiki`: Wiki size

Wiki size is currently not tracked, and the engine will always deem it
within quota.

These subjects are built into Rules, which set a limit on *all* subjects
within a rule. Thus, we can create a rule that says: "1Gb limit on all
release assets, all packages, and git LFS, combined". For a rule to
stand, the total sum of all subjects must be below the rule's limit.

Rules are in turn collected into groups. A group is just a name, and a
list of rules. For a group to stand, all of its rules must stand. Thus,
if we have a group with two rules, one that sets a combined 1Gb limit on
release assets, all packages, and git LFS, and another rule that sets a
256Mb limit on packages, if the user has 512Mb of packages, the group
will not stand, because the second rule deems it over quota. Similarly,
if the user has only 128Mb of packages, but 900Mb of release assets, the
group will not stand, because the combined size of packages and release
assets is over the 1Gb limit of the first rule.

Groups themselves are collected into Group Lists. A group list stands
when *any* of the groups within stand. This allows an administrator to
set conservative defaults, but then place select users into additional
groups that increase some aspect of their limits.

To top it off, it is possible to set the default quota groups a user
belongs to in `app.ini`. If there's no explicit assignment, the engine
will use the default groups. This makes it possible to avoid having to
assign each and every user a list of quota groups, and only those need
to be explicitly assigned who need a different set of groups than the
defaults.

If a user has any quota groups assigned to them, the default list will
not be considered for them.

The management APIs
===================

This commit contains the engine itself, its unit tests, and the quota
management APIs. It does not contain any enforcement.

The APIs are documented in-code, and in the swagger docs, and the
integration tests can serve as an example on how to use them.

Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
2024-08-02 11:10:34 +02:00