mataroa - GitLite

KO | EN

gitlite — search

#python #typescript #llm #golang #go #machine-learning #rust #linux #docker #cli #pytorch #security #openai #open-source

mataroa

(★ 361)

naked blogging platform

파일 탐색기

최종 버전 다운로드 (.zip)

- - check.yml
  - deploy.yml
  - image.yml
- - caddy.service
  - Caddyfile
  - mataroa-backup.service
  - mataroa-backup.timer
  - mataroa-dailysummary.service
  - mataroa-dailysummary.timer
  - mataroa-exports.service
  - mataroa-exports.timer
  - mataroa-notifications.service
  - mataroa-notifications.timer
  - mataroa-renewal.service
  - mataroa-renewal.timer
  - mataroa.env
  - mataroa.service
- .envrc.example
- backup-database.sh
- deploy.sh
- generate-systemd-files.sh
- install-caddy.sh
- provision.sh
- setup-database.sh
- setup-rclone.sh
- container.xml
- content.opf
- mimetype
- toc.ncx
- toc.xhtml
- 404.html
- baseof.html
- config.toml
- index.html
- list.html
- single.html
- style.css
- theme.toml
- 404.html
- _index.md
- config.toml
- index.html
- post.html
- style.css
- - 0001_initial.py
  - 0002_user_blog_title.py
  - 0003_post.py
  - 0004_auto_20200530_0046.py
  - 0005_auto_20200530_1206.py
  - 0006_auto_20200531_1619.py
  - 0007_user_blog_byline.py
  - 0008_auto_20200601_2326.py
  - 0009_auto_20200604_2327.py
  - 0010_user_cname.py
  - 0011_auto_20200606_1903.py
  - 0012_auto_20200606_1903.py
  - 0013_auto_20200606_2048.py
  - 0014_auto_20200607_0017.py
  - 0015_auto_20200607_0023.py
  - 0016_auto_20200607_0024.py
  - 0017_post_published_at.py
  - 0018_auto_20200607_2049.py
  - 0019_auto_20200607_2131.py
  - 0020_auto_20200608_1915.py
  - 0021_image_slug.py
  - 0022_auto_20200608_2117.py
  - 0023_auto_20200608_2141.py
  - 0024_auto_20200608_2304.py
  - 0025_page.py
  - 0026_auto_20200610_1854.py
  - 0027_auto_20200610_1904.py
  - 0028_auto_20200610_2248.py
  - 0029_user_footer_note.py
  - 0030_auto_20200613_1726.py
  - 0031_auto_20200620_1344.py
  - 0032_auto_20200620_1431.py
  - 0033_auto_20200626_1947.py
  - 0034_analytic.py
  - 0035_auto_20200812_2020.py
  - 0036_auto_20200812_2102.py
  - 0037_auto_20200812_2126.py
  - 0038_auto_20200812_2152.py
  - 0039_auto_20200816_1543.py
  - 0040_postnotification_postnotificationrecord.py
  - 0041_auto_20200820_2107.py
  - 0042_auto_20200821_1342.py
  - 0043_user_notifications_on.py
  - 0044_postnotificationrecord_post.py
  - 0045_auto_20200821_1659.py
  - 0046_auto_20200821_1820.py
  - 0047_auto_20200830_1057.py
  - 0048_auto_20201218_1351.py
  - 0049_user_redirect_domain.py
  - 0050_auto_20210101_1509.py
  - 0051_auto_20210111_2111.py
  - 0052_auto_20210124_1932.py
  - 0053_notification_is_active.py
  - 0054_auto_20210312_1643.py
  - 0055_user_theme_zialucia.py
  - 0056_auto_20210317_2313.py
  - 0057_auto_20210317_2321.py
  - 0058_remove_analytic_referer.py
  - 0059_auto_20210409_1320.py
  - 0060_auto_20210429_1506.py
  - 0061_auto_20210503_0035.py
  - 0062_auto_20210516_2310.py
  - 0063_exportrecord.py
  - 0064_user_export_unsubscribe_key.py
  - 0065_remove_uuid_null_export_unsubscribe.py
  - 0066_add_uuid_field_export_unsubscribe.py
  - 0067_rename_analytic_analyticpost.py
  - 0068_analyticpage.py
  - 0069_alter_analyticpage_path.py
  - 0070_notificationrecord_is_canceled.py
  - 0071_user_monero_address.py
  - 0072_alter_user_username.py
  - 0073_user_api_key.py
  - 0074_populate_api_key_values.py
  - 0075_remove_api_key_null.py
  - 0076_alter_user_footer_note.py
  - 0077_comment_is_approved.py
  - 0078_alter_user_theme_zialucia.py
  - 0079_user_theme_sansserif_alter_user_theme_zialucia.py
  - 0080_alter_user_theme_sansserif.py
  - 0081_comment_is_author_alter_comment_is_approved.py
  - 0082_snapshot.py
  - 0083_user_post_backups_on.py
  - 0084_alter_user_post_backups_on.py
  - 0085_alter_user_footer_note.py
  - 0086_alter_user_blog_byline.py
  - 0087_user_is_approved.py
  - 0088_alter_page_is_hidden.py
  - 0089_user_stripe_subscription_id.py
  - 0090_onboard.py
  - 0091_onboard_created_at.py
  - 0092_alter_onboard_user.py
  - 0093_alter_onboard_problems_alter_onboard_quality.py
  - 0094_onboard_code.py
  - 0095_auto_20231109_2111.py
  - 0096_auto_20231109_2111.py
  - 0097_user_subscribe_note.py
  - 0098_alter_user_notifications_on.py
  - 0099_alter_user_subscribe_note.py
  - 0100_onboard_seo.py
  - 0101_post_broadcasted_at.py
  - 0102_remove_notificationrecord_is_canceled.py
  - 0103_alter_post_broadcasted_at.py
  - 0104_alter_onboard_problems_alter_onboard_quality_and_more.py
  - 0105_merge_subscribe_footer_notes.py
  - 0106_remove_subscribe_note.py
  - 0107_user_post_altpath_on.py
  - 0108_alter_user_custom_domain.py
  - 0109_remove_onboard_problems_remove_onboard_quality_and_more.py
  - 0110_analyticpost_created_at_index.py
  - 0111_analyticpage_indexes.py
  - 0112_remove_user_about.py
  - 0113_post_owner_published_at_idx.py
  - 0114_user_is_delisted.py
  - __init__.py
- - favicon.png
  - logo.svg
- - 400.html
  - 403.html
  - 404.html
  - 500.html
- - __init__.py
  - test_analytics.py
  - test_api.py
  - test_billing.py
  - test_blog.py
  - test_checkstripe.py
  - test_comments.py
  - test_feeds.py
  - test_images.py
  - test_management.py
  - test_pages.py
  - test_posts.py
  - test_sitemap.py
  - test_snapshots.py
  - test_static.py
  - test_text_processing.py
  - test_users.py
  - test_webring.py
- - __init__.py
  - api.py
  - billing.py
  - export.py
  - general.py
  - moderation.py
- __init__.py
- admin.py
- apps.py
- denylist.py
- feeds.py
- forms.py
- middleware.py
- models.py
- scheme.py
- sitemaps.py
- text_processing.py
- urls.py
- validators.py
- __init__.py
- asgi.py
- settings.py
- urls.py
- wsgi.py
- Makefile
- .gitkeep
.build.yml
.envrc.example
.gitignore
AGENTS.md
CHANGELOG.md
docker-compose.yml
Dockerfile
LICENSE
manage.py
pyproject.toml
README.md
reset-database-localdev.sh
SECURITY.md
uv.lock

# 설치 가이드

1. 코드 내려받기

git clone https://github.com/mataroablog/mataroa

깃허브에서 프로젝트 코드 전체를 내 컴퓨터로 내려받습니다.

cd mataroa

방금 내려받은 프로젝트 폴더 안으로 이동합니다.

2. Docker

쉬움 추천

사전 준비물

Git GitHub에서 프로젝트 코드를 내려받으려면 필요합니다.
Docker Desktop 컨테이너를 빌드하고 실행하려면 필요합니다. 설치 후 실행해서 백그라운드에 켜두세요.

docker compose up

정의된 모든 컨테이너(서버, DB 등)를 한 번에 빌드하고 백그라운드에서 실행합니다.

├── docker-compose.yml

compose 설정 파일에 정의된 서비스들을 대상으로 명령을 실행합니다.

- [Docker](#docker)

이 명령어를 터미널에 그대로 입력해 실행하세요.

✅ 터미널에 docker compose ps 를 입력해 컨테이너들이 Up 상태인지 확인하세요. README에 포트 번호가 적혀있다면 브라우저에서 http://localhost:포트번호 로 접속해보세요.

이 레포의 README에 적힌 실제 명령어를 그대로 가져왔습니다.

3. Python

쉬움

사전 준비물

Git GitHub에서 프로젝트 코드를 내려받으려면 필요합니다.
Python 3 설치 시 'Add Python to PATH' 옵션을 꼭 체크하세요 (Windows).

pip install .

PyPI에 배포된 패키지를 바로 설치합니다. 소스 클론이 필요 없습니다.

python manage.py runserver

장고(Django) 개발 서버를 실행합니다.

✅ 에러 메시지 없이 실행되고 터미널에 안내 문구가 출력되면 정상입니다.

4. Make

보통

사전 준비물

Git GitHub에서 프로젝트 코드를 내려받으려면 필요합니다.
Make Linux/macOS는 보통 기본 설치되어 있습니다. Windows는 별도 설치(예: MSYS2, WSL)가 필요합니다.

cd postgresql

이 프로젝트의 관련 파일이 하위 폴더 안에 있어서, 먼저 그 폴더로 이동합니다.

make

생성된 빌드 설정을 바탕으로 실제 컴파일을 진행해 실행 파일을 만듭니다.

✅ 에러 없이 끝나면 성공입니다. 생성된 실행 파일을 직접 실행해보세요.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# mataroa

Naked blogging platform.

## Table of Contents

- [Community](#community)
    - [Tools](#tools)
- [Local Development](#local-development)
    - [Subdomains](#subdomains)
    - [Docker](#docker)
    - [Dependencies](#dependencies)
    - [Environment variables](#environment-variables)
    - [Database](#database)
    - [Serve](#serve)
    - [Testing](#testing)
    - [Code linting and formatting](#code-linting-and-formatting)
- [Contributing](#contributing)
    - [Directory structure](#directory-structure)
    - [Coding Conventions](#coding-conventions)
    - [Git Commit Message Guidelines](#git-commit-message-guidelines)
    - [Feature Release Checklist](#feature-release-checklist)
    - [Dependency Policy](#dependency-policy)
- [Operations](#operations)
    - [Deployment](#deployment)
    - [Billing](#billing)
    - [Recurring Tasks](#recurring-tasks)
    - [Post by email](#post-by-email)
    - [Database Backup](#database-backup)
    - [Server Migration Checklist](#server-migration-checklist)
    - [On Server Outage](#on-server-outage)
- [License](#license)

## Community

Main repository on GitHub:
[github.com/mataroablog/mataroa](https://github.com/mataroablog/mataroa)

Mirror on sr.ht:
[git.sr.ht/~sirodoht/mataroa](https://git.sr.ht/~sirodoht/mataroa)

Bug tracking on GitHub:
[github.com/mataroablog/mataroa/issues](https://github.com/mataroablog/mataroa/issues)

Community mailing list on sr.ht:
[lists.sr.ht/~sirodoht/mataroa-community](https://lists.sr.ht/~sirodoht/mataroa-community)

### Tools

* [mataroa-cli](https://github.com/mataroablog/mataroa-cli)
* [Mataroa Telegram Bot](https://github.com/jorphex/Mataroa-Telegram-Bot)

## Local Development

This is a [Django](https://www.djangoproject.com/) codebase. Check out the
[Django docs](https://docs.djangoproject.com/) for general technical
documentation.

The Django project is [`mataroa`](mataroa). There is one Django app,
[`main`](main), with all business logic. Application CLI commands are generally
divided into two categories, those under `python manage.py` and those under
`make`.

### Subdomains

Mataroa works primarily with subdomain, thus one cannot access the basic web app
using the standard `http://127.0.0.1:8000` or `http://localhost:8000` URLs. What we do
for local development is add a few custom entries on our `/etc/hosts` system file.

Important note: there needs to be an entry of each user account created in the local
development environment, so that the web server can respond to it.

The first line is the main needed: `mataroalocal.blog`. The rest are included as
examples of other users one can create in their local environment. The
easiest way to create them is to go through the sign up page
(`http://mataroalocal.blog:8000/accounts/create/` using default values).

```
# /etc/hosts

127.0.0.1 mataroalocal.blog

127.0.0.1 paul.mataroalocal.blog
127.0.0.1 random.mataroalocal.blog
127.0.0.1 anyusername.mataroalocal.blog
```

This will enable us to access mataroa locally (once we start the web server) at
[http://mataroalocal.blog:8000/](http://mataroalocal.blog:8000/)
and if we make a user account with username `paul`, then we will be able to access it at
[http://paul.mataroalocal.blog:8000/](http://paul.mataroalocal.blog:8000/)

### Docker

> [!NOTE]
> This is the last step for initial Docker setup. See the "Environment variables"
> section below, for further configuration details.

To set up a development environment with Docker and Docker Compose, run the following
to start the web server and database:

```sh
docker compose up
```

If you have also configured hosts as described above in the "Subdomains"
section, mataroa should now be locally accessible at
[http://mataroalocal.blog:8000/](http://mataroalocal.blog:8000/)

Note: The database data are saved in the git-ignored `docker-postgres-data` docker
volume, located in the root of the project.

### Dependencies

We use `uv` for dependency management and virtual environments.

```sh
uv sync --all-groups
```

See [Dependency Policy](#dependency-policy) for more details on adding/upgrading dependencies.

### Environment variables

A file named `.envrc` is used to define the environment variables required for
this project to function. One can either export it directly or use
[direnv](https://github.com/direnv/direnv). There is an example environment
file one can copy as base:

```sh
cp .envrc.example .envrc
```

Set `SIGNUPS_ENABLED=0` to close both signup steps. Signups are enabled by
default.

When on Docker, to change or populate environment variables, edit the `environment`
key of the `web` service either directly on `docker-compose.yml` or by overriding it
using the standard named git-ignored `docker-compose.override.yml`.

```yaml
# docker-compose.override.yml

services:
  web:
    environment:
      EMAIL_HOST_USER: smtp-user
      EMAIL_HOST_PASSWORD: smtp-password
```

Finally, stop and start `docker compose up` again. It should pick up the override file
as it has the default name `docker-compose.override.yml`.

### Database

This project uses one PostgreSQL database for persistence.

The reset script drops and recreates the local `mataroa` role and database.
The default role password is `mataroa`, and the role receives the `CREATEDB`
permission required by Django's test runner:

```sh
./reset-database-localdev.sh
```

The defaults match `docker-compose.yml`. They can be overridden with
`POSTGRES_HOST`, `POSTGRES_PORT`, `POSTGRES_ADMIN_USER`,
`POSTGRES_ADMIN_PASSWORD`, `DB_NAME`, `DB_USER`, and `DB_PASSWORD`.

After setting the `DATABASE_URL` ([see above](#environment-variables)), create
the database schema with:

```sh
uv run python manage.py migrate
```

To generate a deterministic development dataset without deleting existing data, run:

```sh
uv run python manage.py devdata
```

This creates 10 users with 25 posts each, plus sample pages, comments, and
newsletter subscribers. The generated superuser credentials are `admin` /
`admin`; other generated users have usernames such as `devuser001` and use the
same password. Counts can be adjusted with `--users`,
`--posts-per-user`, `--pages-per-user`, `--comments-per-post`, and
`--subscribers-per-user`. The command is idempotent and only runs when `DEBUG`
or `LOCALDEV` is enabled.

### Serve

To run the Django development server:

```sh
uv python manage.py runserver
```

If you have also configured hosts as described above in the "Subdomains"
section, mataroa should now be locally accessible at
[http://mataroalocal.blog:8000/](http://mataroalocal.blog:8000/)

### Testing

Using the Django test runner:

```sh
uv run python manage.py test
```

For coverage, run:

```sh
uv run coverage run --source='.' --omit '.venv/*' manage.py test
uv run coverage report -m
```

### Code linting and formatting

We use [ruff](https://github.com/astral-sh/ruff) for Python linting and code formatting.

To lint:

```sh
uv run ruff check
```

To apply automatic fixes if they exist:

```sh
uv run ruff check --fix
```

To format:

```sh
uv run ruff format --check
```

## Contributing

Main repository on GitHub:
[github.com/mataroablog/mataroa](https://github.com/mataroablog/mataroa)

Mirror repository on sr.ht:
[git.sr.ht/~sirodoht/mataroa](https://git.sr.ht/~sirodoht/mataroa)

Report bugs on GitHub:
[github.com/mataroablog/mataroa/issues](https://github.com/mataroablog/mataroa/issues)

Contribute on GitHub with Pull Requests:
[github.com/mataroablog/mataroa/pulls](https://github.com/mataroablog/mataroa/pulls)

Contribute with email patches on sr.ht:
[~sirodoht/public-inbox@lists.sr.ht](mailto:~sirodoht/public-inbox@lists.sr.ht).
See how to contribute using email patches on [git-send-email.io](https://git-send-email.io/).

### Directory structure

Here, an overview of the project's code sources is presented. The purpose is
for the reader to understand what kind of functionality is located where in
the source code.

All business logic of the application is in one Django app: [`main`](main).

Condensed and commented sources file tree:

```
.
├── .build.yml # SourceHut CI build config
├── .envrc.example # example direnv file
├── .github/ # GitHub Actions config files
├── Caddyfile # configuration for Caddy webserver
├── Dockerfile
├── docker-compose.yml
├── export_base_epub/ # base sources for epub export functionality
├── export_base_hugo/ # base sources for hugo export functionality
├── export_base_zola/ # base sources for zola export functionality
├── main/
│   ├── admin.py
│   ├── apps.py
│   ├── denylist.py # list of various keywords allowed and denied
│   ├── feeds.py # django rss functionality
│   ├── forms.py
│   ├── management/ # commands under `python manage.py`
│   │   └── commands/
│   │       ├── checkstripe.py
│   │       ├── devdata.py
│   │       ├── mailexports.py
│   │       ├── mailsummary.py
│   │       ├── processnotifications.py
│   │       └── testbulkmail.py
│   ├── middleware.py # mostly subdomain routing
│   ├── migrations/
│   ├── models.py
│   ├── sitemaps.py
│   ├── static/
│   ├── templates
│   │   ├── main/ # HTML templates for most pages
│   │   ├── assets/
│   │   │   ├── drag-and-drop-upload.js
│   │   │   └── style.css
│   │   ├── partials/
│   │   │   ├── footer.html
│   │   │   ├── footer_blog.html
│   │   │   └── webring.html
│   │   └── registration/
│   ├── tests/
│   │   ├── test_api.py
│   │   ├── test_billing.py
│   │   ├── test_blog.py
│   │   └── testdata/
│   ├── text_processing.py # markdown and text utilities
│   ├── urls.py
│   ├── validators.py # custom form and field validators
│   └── views/
│       ├── api.py
│       ├── billing.py
│       ├── general.py
├── manage.py
└── mataroa
    ├── asgi.py
    ├── settings.py # django configuration file
    ├── urls.py
    └── wsgi.py
```

#### [`main/urls.py`](main/urls.py)

All urls are in this module. They are visually divided into several sections:

* general, includes index, dashboard, static pages
* user system, includes signup, settings, logout
* blog posts, the CRUD operations of
* blog extras, includes rss and newsletter features
* comments, related to the blog post comments
* billing, subscription and card related
* blog import, export, webring
* images CRUD
* analytics list and details
* pages CRUD

#### [`main/views/`](main/views/)

The majority of business logic is organized in the `views/` directory, split across
several modules for better organization.

Generally,
[Django class-based generic views](https://docs.djangoproject.com/en/3.2/topics/class-based-views/generic-display/)
are used most of the time as they provide useful functionality abstracted away.

The Django source code [for generic views](https://github.com/django/django/tree/main/django/views/generic)
is also extremely readable:

* [base.py](https://github.com/django/django/blob/main/django/views/generic/base.py): base `View` and `TemplateView`
* [list.py](https://github.com/django/django/blob/main/django/views/generic/list.py): `ListView`
* [edit.py](https://github.com/django/django/blob/main/django/views/generic/edit.py): `UpdateView`, `DeleteView`, `FormView`
* [detail.py](https://github.com/django/django/blob/main/django/views/generic/detail.py): `DetailView`

[Function-based views](https://docs.djangoproject.com/en/3.2/intro/tutorial01/#write-your-first-view)
are used in cases where the CRUD/RESTful design pattern is not clear such as
`notification_unsubscribe_key` where we unsubscribe an email via a GET operation.

##### [`main/views/general.py`](main/views/general.py)

The main view module containing most of the application's business logic:

* indexes, dashboard, static pages
* user CRUD and login/logout
* posts CRUD
* comments CRUD
* images CRUD
* pages CRUD
* webring
* analytics
* notifications subscribe/unsubscribe

##### [`main/views/api.py`](main/views/api.py)

This module contains all API related views. These views have their own
api key based authentication.

##### [`main/views/export.py`](main/views/export.py)

This module contains all views related to the export capabilities of mataroa.

The way the exports work is by reading the base files from the repository root:
[export_base_hugo](export_base_hugo/), [export_base_zola](export_base_zola/),
[export_base_epub](export_base_epub/) for Hugo, Zola, and epub respectively.
After reading, we replace some strings on the configurations, generate posts
as markdown strings, and zip-archive everything in-memory. Finally, we respond
using the appropriate content type (`application/zip` or `application/epub`) and
[Content-Disposition](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Disposition)
`attachment`.

##### [`main/views/billing.py`](main/views/billing.py)

This module contains all billing and subscription related views. It's designed to
support one payment processor, Stripe.

##### [`main/views/moderation.py`](main/views/moderation.py)

This module contains all views related to moderating the content of the platform.

#### [`main/tests/`](main/tests/)

All tests are under this directory. They are divided into several modules,
based on the functionality and the views they test.

Everything uses the built-in Python `unittest` module along with standard
Django testing facilities.

#### [`main/models.py`](main/models.py) and [`main/migrations/`](main/migrations/)

`main/models.py` is where the database schema is defined, translated into
Django ORM-speak. This always displays the latest schema.

`main/migrations/` includes all incremental migrations required to reach
the schema defined in `main/models.py` starting from an empty database.

We use the built-in Django commands to generate and execute migrations, namely
`makemigrations` and `migrate`. For example, the steps to make a schema change
would be something like:

1. Make the change in `main/models.py`. See
[Django Model field reference](https://docs.djangoproject.com/en/3.2/ref/models/fields/).
1. Run `python manage.py makemigrations` to auto-generate the migrations.
1. Potentially refactor the auto-generated migration file (located at `main/migrations/XXXX_auto_XXXXXXXX.py`)
1. Run `python manage.py migrate` to execute migrations.
1. Also `make format` before committing.

#### [`main/forms.py`](main/forms.py)

Here a collection of Django-based forms resides, mostly in regards to user creation,
upload functionalities (for post import or image upload), and card details
submission.

See [Django Form fields reference](https://docs.djangoproject.com/en/3.2/ref/forms/fields/).

#### [`main/text_processing.py`](main/text_processing.py)

This module contains utilities for processing text content, including markdown
rendering, syntax highlighting, and text transformations used throughout the
application.

#### [`main/templates/assets/style.css`](main/templates/assets/style.css)

On mataroa, a user can enable an option, Theme Zia Lucia, and get a higher font
size by default. Because we need to change the body font-size value, we render
the CSS. It is not static. This is why it lives inside the templates directory.

### Coding Conventions

1. All files should end with a new line character.
1. Python code should be formatted with [ruff](https://github.com/astral-sh/ruff).

### Git Commit Message Guidelines

We follow some simple non-austere git commit message guidelines.

* Start with a verb
    * `add`
    * `change`
    * `delete`
    * `fix`
    * `refactor`
    * `tweak`
    * et al.
* Start with a lowercase letter
    * eg. `change analytic page path to the same of page slug`
* Do not end with a fullstop

### Feature Release Checklist

This document provides a comprehensive checklist for releasing a new version of Mataroa.

1. Finish with all commits, push to main
1. Verify `CHANGELOG.md` is up-to-date and add release date
1. Verify CI is green
1. Bump in in `pyproject.toml` with `git commit -m "release v1.x"`
1. Run `uv lock`
1. Create Git tag: `git tag -a v1.x -m "v1.x"`
1. Push tag: `git push origin --tags` and `git push srht --tags`

### Dependency Policy

The mataroa project has an unusually strict yet usually unclear dependency policy.

Vague rules include:

* No third-party Django apps.
* All Python / PyPI packages should be individually vetted.
    * Packages should be published from community-trusted organisations or developers.
    * Packages should be actively maintained (though not necessarily actively developed).
    * Packages should hold a high quality of coding practices.
* No JavaScript libraries / dependencies.

Current list of top-level PyPI dependencies (source at [`pyproject.toml`](pyproject.toml)):

* [Django](https://pypi.org/project/Django/)
* [psycopg](https://pypi.org/project/psycopg/)
* [gunicorn](https://pypi.org/project/gunicorn/)
* [Markdown](https://pypi.org/project/Markdown/)
* [Pygments](https://pypi.org/project/Pygments/)
* [bleach](https://pypi.org/project/bleach/)
* [stripe](https://pypi.org/project/stripe/)

#### Adding a new dependency

After approving a dependency, add it using `uv`:

1. Ensure `uv` is installed and a virtualenv exists (managed by `uv`).
1. Add the dependency to `pyproject.toml` and lockfile with:
   - Runtime: `uv add PACKAGE`
   - Dev-only: `uv add --dev PACKAGE`
1. Install/sync dependencies: `uv sync`

#### Upgrading dependencies

When a new Django version is out it’s a good idea to upgrade everything.

Steps:

1. Update the lockfile: `uv lock --upgrade` and review output changes.
1. Spot non-patch level version bumps and examine release notes.
1. Install updated deps: `uv sync`
1. Unless something comes up, make sure tests and smoke tests pass.
1. Deploy new dependency versions.

## Operations

Details for people hosting their own mataroa instance.

### Deployment

#### 1. Choose domain name

For the purposes of this guide let us assume `example.com` is the domain name of choice.

#### 2. Setup DNS

Both `example.com` and `*.example.com` should point to the mataroa server IP. Verify changes have
propagated before provisioning:

```sh
dig example.com A
dig *.example.com A
dig randomsubdomain.example.com A
```

#### 3. Configuration

Set up the config environment for provisioning. The config environment remains on your local machine.

```sh
cd deploy/

# make a copy of the example envrc file
cp .envrc.example .envrc

# edit parameters as required
vim .envrc

# load variables into environment
source .envrc
```

#### 4. Provision

Finally, run the provisioning script:

```sh
cd deploy/
./provision.sh
```

This script will:

* install essential packages (gcc, git, rclone, postgresql, etc.)
* install and configure Caddy web server
* create the deploy user with proper permissions
* set up the postgres database
* install uv
* clone the mataroa repository
* configure all systemd services and timers
* run initial Django migrations and collect static files
* enable and start all systemd services

Note: Caddy will automatically obtain and manage SSL certificates for your domain and all
subdomains using Let's Encrypt. The certificate generation for each domain happens when that domain
is first accessed, which leads to increased latency for that first page visit.

#### 5. Future deployments

To deploy new changes:

```sh
cd deploy/
./deploy.sh
```

This will connect to the server and:

* pull the latest code from git
* update python dependencies
* run database migrations
* collect static files
* reload the gunicorn/django service

#### Useful Commands

To reload the gunicorn process:

```sh
sudo systemctl reload mataroa
```

To reload Caddy:

```sh
systemctl restart caddy  # root only
```

gunicorn logs:

```sh
journalctl -fb -u mataroa
```

Caddy logs:

```sh
journalctl -fb -u caddy
```

Get an overview with systemd status:

```sh
systemctl status caddy
systemctl status mataroa
```

### Billing

One can deploy mataroa without setting up billing functionalities. This is
the default case. To handle payments and subscriptions this project uses
[Stripe](https://stripe.com/). To enable Stripe and payments, one needs to have
a Stripe account with a single
[Product](https://stripe.com/docs/billing/prices-guide) (eg. "Mataroa Premium
Plan").

To configure, add the following variables from your Stripe account to your
`.envrc`:

```sh
export STRIPE_API_KEY="sk_test_XXX"
export STRIPE_PUBLIC_KEY="pk_test_XXX"
export STRIPE_WEBHOOK_SECRET="whsec_XXX"
export STRIPE_PRICE_ID="price_XXX"
```

### Post by email

Postmark authenticates requests to the inbound webhook with HTTP Basic
Authentication. The username is fixed as `postmark`; configure a dedicated
password in `deploy/.envrc`:

```sh
export POSTMARK_WEBHOOK_PASSWORD="a-long-random-password"
```

Then configure the Inbound Message Stream webhook URL in Postmark, using the
same credentials:

```text
https://postmark:a-long-random-password@mataroa.blog/webhook/postmark/
```

The endpoint rejects all requests if the password environment variable is
missing.

### Recurring Tasks

We don't use cron but systemd timers for jobs that need to run recurringly.

#### Process email notifications

```sh
python manage.py processnotifications
```

Sends notification emails for new blog posts.

Triggers daily at 10AM server time.

#### Email blog exports

```sh
python manage.py mailexports
```

Emails users their blog exports.

Triggers monthly, first day of the month, 6AM server time.

#### Daily summary

```sh
python manage.py mailsummary
```

Sends mataroa daily moderation summary.

Triggers daily at 00:15 server time.

### Database Backup

We use the script [`backup-database.sh`](./deploy/backup-database.sh) to dump the database and
upload it into an S3-compatible object storage cloud using [rclone](https://rclone.org/).

To create a database dump run:

```sh
pg_dump -Fc --no-acl mataroa -h localhost -U mataroa -f /home/deploy/mataroa.dump -W
```

To restore a database dump run:

```sh
pg_restore --disable-triggers -j 4 -v -h localhost -cO --if-exists -d mataroa -U mataroa -W mataroa.dump
```

### Server Migration Checklist

Nothing lasts forever. One day you might want to migrate your mataroa platform to another server.
Mataroa stores everything, images including, in the PostgreSQL database. Naughty indeed, yet it
makes it much easier to backup and migrate.

> [!CAUTION]
> This migration process involves downtime.

To start with, one a migrator has setup their new server we recommend that they they test everything
in another domain, other than the main (existing) one:

1. Find a test domain name and set up DNS pointing to the new server
1. Set up `deploy/.envrc` for test domain name
1. Run `deploy/provision.sh` for new server via test domain name
1. Verify everything in the new server with the test domain works
1. Install old server's public key to new server's authorized keys so that we can scp the database dump

Then, for the migration:

1. Change Caddyfile to not reverse proxy to mataroa in the new server (so that we switch it on when we are ready)
1. Change Caddyfile to not reverse proxy to mataroa in the old server. This means down time begins.
1. Disconnect old server from public IP (we assume a floating IP is used)
1. Start the final database dump in the old server. Important: this assumes we are connected to the old server via another non-floating IP.
1. Change `/etc/systemd/system/mataroa.env` to point to the proper domain name (not the test one)
1. Install floating IP to new server (restart server to verify)
1. Once database dump finishes, secure copy it to the new server
1. Restore database dump in new server
1. Restore Caddyfile and restart Caddy

Finally, once the new server everything works don't forget to verify DEBUG is 0.

The above assume the migrator has a floating IP that they can move around. If
not, there are two problems. The migrator needs to change DNS records but much
more problematically all custom domains stop working :/ For this reason we
should implement CNAME custom domains. However, CNAME custom domains do not
support root domains, so what's the point anyway you ask. Good question. I don't
know. I only hope I never decide to switch away from Hetzner.

### On Server Outage

So, mataroa is down. What do we do?

First, we panic. We run around in circles with our hands up in despair. It's important to do this
first, don't think of it as a joke. Once that's done:

#### 1. Check Caddy

Caddy is the first point of contact inside the server from the outside world.

First ssh into server:

```sh
ssh root@mataroa.blog
```

Caddy runs as a systemd service. Check status with:

```sh
systemctl status caddy
```

Exit with `q`. If the service is not running and is errored restart with:

```sh
systemctl restart caddy
```

If restart does not work, check logs:

```sh
journalctl -u caddy -r
```

`-r` is for reverse. Use `-f` to follow logs real time:

```sh
journalctl -u caddy -f
```

To search within all logs do slash and then the keyword itself, eg: `/keyword-here`, then hit enter.

The config for Caddy is:

```sh
cat /etc/caddy/Caddyfile
```

The systemd config for Caddy is:

```sh
cat /etc/systemd/system/multi-user.target.wants/caddy.service
```

#### 2. Check gunicorn

After Caddy receives the request, it forwards it to gunicorn. Gunicorn is what manages the mataroa
Django instances, so it's named `mataroa`. It runs as a systemd service.

To see status:

```sh
systemctl status mataroa
```

To restart:

```sh
systemctl restart mataroa
```

To see logs:

```sh
journalctl -u mataroa -r
```

and to follow them:

```sh
journalctl -u mataroa -f
```

The systemd config for mataroa/gunicorn is:

```sh
cat /etc/systemd/system/multi-user.target.wants/mataroa.service
```

The env variables in a separate file:

```sh
cat /etc/systemd/system/mataroa.env
```

#### 3. How to hotfix code

Here's where the code lives and how to access it:

```sh
sudo -i -u deploy
cd /var/www/mataroa/

# load env variables from the systemd env file:
# /etc/systemd/system/mataroa.env
# note: this file is root permissioned

uv run python manage.py
```

If you make a change in the source code files (inside `/var/www/mataroa`) you need to restart the
service for the changes to take effect:

```sh
systemctl restart mataroa
```

## License

Copyright Mataroa Contributors

This program is free software: you can redistribute it and/or modify it under
the terms of the GNU Affero General Public License as published by the Free
Software Foundation, version 3.