E621

Installation (Easy mode - For development environments)

Prerequisites

• Latest version of Docker (download).
• Latest version of Docker Compose (download)
• Git (download)

If you are on Windows Docker Compose is already included, you do not need to install it yourself. If you are on Linux/MacOS you can probably use your package manager.

Installation

1. Download and install the prerequisites.
2. Clone the repo with git clone https://github.com/e621ng/e621ng.git.
3. cd into the repo.
4. Copy the sample environment file with cp .env.sample .env.
5. WSL Only: Run the following commands:

   git config core.fileMode false
   cp -ru hooks/ .git
   

   This will resolve permission issues, and set up a hook that will reset file permissions to what they are supposed to be in the future.
   If you are not using WSL, this is likely not a problem for you.
6. Run the following commands:

   docker compose run --rm e621 /app/bin/setup
   docker compose up
   

   After running the commands once only docker compose up is needed to bring up the containers.
7. To confirm the installation worked, open the web browser of your choice and enter http://localhost:3000 into the address bar and see if the website loads correctly. An admin account has been created automatically, the username and password are admin and hexerade respectively.
8. By default, the site will lack any content. For testing purposes, you can generate some using the following command:

   docker exec -it e621ng-e621-1 /app/bin/populate
   

   The command can be run multiple times to generate more content.
   Environmental variables are available to customize what kind of content is generated.

Note: When gems or js packages are updated you need to execute docker compose build to reflect them in the container.

Local DText gem

You may want to test changes made to the DText gem on a local instance. You are recommended to reconsider and rethink your life choices.

If you are sure that you want to do this, follow these steps.

1. Clone the repo into a vendor directory. Example: ~/e621ng/vendor/dtext/.
   1. cd ~/e621ng
   2. mkdir vendor && cd vendor
   3. git clone https://github.com/e621ng/dtext.git (substitute your local fork as needed)
2. Rebuild the container
   1. cd ~/e621ng
   2. docker compose build --no-cache
3. Reset the Gemfile.lock: git checkout HEAD -- Gemfile.lock
   This is not required, but it will prevent you from accidentally committing bad changes.
4. Set LOCAL_DTEXT=true in the .env file.

At this point, the DText repository is set up. It will be automatically compiled whenever the docker container is started.

It is recommended to set LOCAL_DTEXT to false whenever you are not actively working on anything related to the DText repo, and then rebuild the container.

Development environment

This repo provides a Dev Container configuration. You can use something like the Dev Container extension for VSCode to make use of it. Simply install it, open the folder in VSCode, and click the button in the bottom right to open the folder in the Dev Container.

I followed the above instructions but it doesn't work, what should I do?

Try this:

1. docker compose down -v to remove all volumes.
2. docker compose build --no-cache to rebuild the image from scratch.
3. Follow the instructions starting from step 5.

Why are there a bunch of changes I can't revert?

You're most likely using Windows. Give this a shot, it tells Git to stop tracking file mode changes:

git config core.fileMode false

TrueNAS / Local Server Installation

If you decide to deploy this docker image to an external / local server, you do need to remember to change the DANBOORU_HOST variable in the docker-compose.yml file to the IP of your server. Otherwise, you will not be able to access it, or the image links will be broken.

Specifically for TrueNAS/NAS boxes users: you need to use the shell itself to set the repo up, you can then manage the images/variable/config with Portainer/Dockge after it's set up.

Testing and Linting

Testing Suite

The test suite runs in parallel using parallel_tests. Before running tests for the first time (or after a schema change), set up the parallel test databases:

docker compose run --rm --entrypoint bin/rake rspec parallel:create
docker compose run --rm --entrypoint bin/rake rspec parallel:load_schema
docker compose run --rm --entrypoint bin/rake rspec parallel:seed

Then run the suite:

docker compose run --rm rspec

This defaults to 4 parallel workers. Override with PARALLEL_TEST_PROCESSORS:

PARALLEL_TEST_PROCESSORS=8 docker compose run --rm rspec

Linters

docker compose run --rm rubocop to lint the Ruby code docker compose run --rm eslint to lint JavaScript/TypeScript code

Database

The postgres server accepts outside connections which you can use to access it with a local client. Use localhost:34517 to connect to a database named e621_development with the user e621. Leave the password blank, anything will work.

Contributing

Follow the PR naming conventions

Prefix your PR with the primary subsystem being changed in square brackets, and use the imperative present tense.

Good: [Posts] Tweak post search UI

Bad:

• Tweak post search UI: no subsystem prefix
• [Users] Tweak post search UI: unrelated subsystem prefix
• [Posts] Tweaked post search UI: wrong tense

Run the required checks locally prior to submission

Ensuring our checks pass prior to submission greatly increases the chances that your PR will be reviewed in a timely manner. There are 4 checks run on PRs; their local equivalents are:

• docker compose run --rm rspec: test suite for Ruby/Rails
• docker compose run --rm rubocop: linter for Ruby/Rails
• docker compose run --rm vitest: test suite for JavaScript/TypeScript
• docker compose run --rm eslint: linter for JavaScript/TypeScript

If addressing a preexisting issue, please reference it

For issues already submitted to GitHub, use closing keywords in the PR description to let GitHub automatically link the two. This will allow us to both see the full context of the problem and close the issue as solved automatically upon merging.

If the issue is not on GitHub, linking to discussion about the issue lets our maintainers gauge the interest in the feature and the likelihood the PR sufficiently addresses the demands.

Limit the scope of your changes

If you are undertaking a large project, please submit/reference an issue describing the overall project, and break it down into smaller logical units that are submitted separately. This makes for dramatically easier and faster review (and, if need be, iteration). Plus, it reduces the burden of dealing with merge conflicts for other contributors.

Add or update relevant tests

If you create a new public class or function, or alter the behavior of preexisting ones, adding or updating the relevant tests allows maintainers to have far more confidence in your code with far less review, increasing the chances at a speedy turnaround.

We currently have RSpec suite for our Ruby/Rails code, and Vitest for JS/TS. We do have code coverage analytics, and while we don't strictly require that the overall coverage never decreases, that is an ideal we'd like to strive for.

Do not commit changes to rubocop_todo.yml, nor excessively use inline comments to silence lint errors/warnings

We have a workflow to properly update the to-do file without causing frustrating conflicts. You should not be hiding lint errors/warnings with this file.

We ask that you avoid using in-line config comments to bypass the linter, although it is permissible in cases where the code's structure makes it impossible to do so without a substantial or needlessly convoluted refactoring effort (i.e. TagQuery's parse_query method).

Similarly to test coverage, while we don't strictly require that the overall number of lints ignored – through both inline comments and rubocop_todo.yml – never increases, that is an ideal we'd like to strive for.

Check for and review prior discussion regarding the goal of your PR, and any attempts to fix the same issues

This ensures you have a solid idea of the requirements for acceptance and the overall desire for the PR before starting work on it and submitting for approval.

Production Setup

Installation follows the same steps as the docker compose file. Ubuntu 20.04 is the current installation target. There is no script that performs these steps for you, as you need to split them up to match your infrastructure. Running a single machine install in production is possible, but is likely to be somewhat sluggish due to contention in disk between PostgreSQL and OpenSearch. Minimum RAM is 4GB. You will need to adjust values in config files to match how much RAM is available. If you are targeting more than a hundred thousand posts and reasonable user volumes, you probably want to procure yourself a database server. See tuning guides for PostgreSQL and OpenSearch for help planning these requirements.

Production Troubleshooting

These instructions won't work for everyone. If your setup is not working, here are the steps I usually recommend to people:

1. Test the database. Make sure you can connect to it using psql. Make sure the tables exist. If this fails, you need to work on correctly installing PostgreSQL, importing the initial schema, and running the migrations.

2. Test the Rails database connection by using rails console. Run Post.count to make sure Rails can connect to the database. If this fails, you need to make sure your Danbooru configuration files are correct.

3. Test Nginx to make sure it's working correctly. You may need to debug your Nginx configuration file.

4. Check all log files.