Merge branch 'update-everything' into 'master'

Update all the deps See merge request mangadex-pub/mangadex_at_home!103
Update all the deps
2024-01-19 02:48:37 +00:00 · 2023-08-07 22:09:31 +00:00 · 2023-08-07 22:09:31 +00:00 · 2023-04-12 00:51:20 +00:00 · 2023-04-12 01:42:28 +01:00 · 2023-04-07 20:51:29 +00:00
32 changed files with 570 additions and 339 deletions
--- a/.gitattributes
+++ b/.gitattributes
@ -1 +1,2 @@
 CHANGELOG.md merge=union
+* text=auto eol=lf
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@ -1,36 +1,42 @@
 stages:
  - build
  - publish
-  - publish_latest
-  - publish_docker
+  - docker
+  - push

-build:
+Gradle Build:
  image: openjdk:8
  stage: build
+  only:
+    - branches
+    - tags
+    - merge_requests
+  before_script:
+    - export VERSION="${CI_COMMIT_TAG:-$CI_COMMIT_SHORT_SHA}"
  script:
-    - export VERSION="$CI_COMMIT_REF_NAME"
    - ./gradlew build
-    - "ls -lah build/libs"
-  cache:
-    key: "mangadex_at_home-build"
-    paths:
-      - /root/.gradle
+  variables:
+    KUBERNETES_MEMORY_REQUEST: 3Gi
+    KUBERNETES_MEMORY_LIMIT: 3Gi
  artifacts:
    name: "mangadex_at_home"
    paths:
      - "build/libs/mangadex_at_home-*-all.jar"

-publish:
+Publish Artifacts:
  image: alpine
  stage: publish
+  needs:
+    - Gradle Build
+  variables:
+    GIT_STRATEGY: none
+
  before_script:
-    - apk update && apk add git zip
-    - export VERSION="$CI_COMMIT_REF_NAME"
+    - apk update && apk add zip
+    - export VERSION="${CI_COMMIT_TAG:-$CI_COMMIT_SHORT_SHA}"
  script:
    - cp build/libs/mangadex_at_home-${VERSION}-all.jar ./
    - zip -r9 mangadex_at_home-${VERSION}.zip mangadex_at_home-${VERSION}-all.jar settings.sample.yaml
-  dependencies:
-    - build
  artifacts:
    name: "mangadex_at_home"
    paths:
@ -38,19 +44,58 @@ publish:
      - "mangadex_at_home-*.zip"
      - "settings.sample.yaml"

-publish_docker:
-  image: docker:git
-  stage: publish
-  only:
-    - tags
+Docker Build:
+  image: docker:20.10.8
  services:
-    - docker:dind
+    - docker:20.10.8-dind
+  stage: docker
+  needs:
+    - Gradle Build
+
  before_script:
    - echo "$CI_REGISTRY_PASSWORD" | docker login -u "$CI_REGISTRY_USER" --password-stdin ${CI_REGISTRY}
-    - export VERSION="$CI_COMMIT_REF_NAME"
-  dependencies:
-    - build
+    - export VERSION="${CI_COMMIT_TAG:-$CI_COMMIT_SHORT_SHA}"
+    - export BASE_TAG="git-$CI_COMMIT_SHORT_SHA"
+
  script:
    - mv build/libs/mangadex_at_home-${VERSION}-all.jar build/libs/mangadex_at_home.jar
-    - docker build . -t ${CI_REGISTRY_IMAGE}:${VERSION}
-    - docker push ${CI_REGISTRY_IMAGE}:${VERSION}
+    - docker build . -t $CI_REGISTRY_IMAGE:$BASE_TAG
+    - docker push $CI_REGISTRY_IMAGE:$BASE_TAG
+
+
+.docker_push: &docker_push
+  image: docker:20.10.8
+  services:
+    - docker:20.10.8-dind
+  stage: push
+  variables:
+    GIT_STRATEGY: none
+
+  before_script:
+    - echo "$CI_REGISTRY_PASSWORD" | docker login -u "$CI_REGISTRY_USER" --password-stdin ${CI_REGISTRY}
+    - export BASE_TAG="git-$CI_COMMIT_SHORT_SHA"
+    - export SHORT_TAG="$(echo $CI_COMMIT_TAG | cut -d "." -f1)"
+
+  script:
+    - docker pull $CI_REGISTRY_IMAGE:$BASE_TAG
+    - docker tag $CI_REGISTRY_IMAGE:$BASE_TAG $CI_REGISTRY_IMAGE:$NEW_TAG
+    - docker tag $CI_REGISTRY_IMAGE:$BASE_TAG $CI_REGISTRY_IMAGE:$SHORT_TAG
+    - docker push $CI_REGISTRY_IMAGE --all-tags
+
+Push Latest:
+  <<: *docker_push
+  needs:
+    - Docker Build
+  only:
+    - master
+  variables:
+    NEW_TAG: latest
+
+Push Tags:
+  <<: *docker_push
+  needs:
+    - Docker Build
+  only:
+    - tags
+  variables:
+    NEW_TAG: $CI_COMMIT_TAG
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@ -17,6 +17,27 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

 ### Security

+## [2.0.4] - 2023-08-07
+### Changed
+- [2023-08-07] Updated dependencies [@carbotaniuman].
+- [2023-04-06] Fixed DB contention issues [@carbotaniuman].
+- [2023-04-06] Make errors more useful [@carbotaniuman].
+
+## [2.0.3] - 2022-02-17
+### Changed
+- [2022-02-17] Updated dependencies [@carbotaniuman].
+
+### Fixed
+- [2022-02-17] Fix possible race condition in DB handling code [@carbotaniuman].
+- [2022-02-17] Missing ISO code no longer fails request [@carbotaniuman].
+
+## [2.0.2] - 2022-02-16
+### Removed
+- [2022-02-16] Remove TLS 1.0 and 1.1 support [@carbotaniuman].
+
+### Fixed
+- [2022-02-16] Fix uncatched exceptions killing threads and not being logged [@carbotaniuman].
+
 ## [2.0.1] - 2021-05-27
 ### Added
 - [2021-05-27] Added SNI check to prevent people from simply scanning nodes [@carbotaniuman].
@ -396,7 +417,10 @@ This release contains many breaking changes! Of note are the changes to the cach
 ### Fixed
 - [2020-06-11] Tweaked logging configuration to reduce log file sizes by [@carbotaniuman].

-[Unreleased]: https://gitlab.com/mangadex/mangadex_at_home/-/compare/2.0.0...HEAD
+[Unreleased]: https://gitlab.com/mangadex/mangadex_at_home/-/compare/2.0.4...HEAD
+[2.0.4]: https://gitlab.com/mangadex/mangadex_at_home/-/compare/2.0.3...2.0.4
+[2.0.3]: https://gitlab.com/mangadex/mangadex_at_home/-/compare/2.0.2...2.0.3
+[2.0.2]: https://gitlab.com/mangadex/mangadex_at_home/-/compare/2.0.1...2.0.2
 [2.0.1]: https://gitlab.com/mangadex/mangadex_at_home/-/compare/2.0.0...2.0.1
 [2.0.0]: https://gitlab.com/mangadex/mangadex_at_home/-/compare/2.0.0-rc14...2.0.0
 [2.0.0-rc14]: https://gitlab.com/mangadex/mangadex_at_home/-/compare/2.0.0-rc13...2.0.0-rc14
--- a/2
+++ b/2
@ -1,4 +1,4 @@
-FROM adoptopenjdk:15
+FROM eclipse-temurin:17-jre

 WORKDIR /mangahome
 ADD /build/libs/mangadex_at_home.jar /mangahome/mangadex_at_home.jar
--- a/build.gradle
+++ b/build.gradle
@ -1,21 +1,26 @@
+import org.jetbrains.kotlin.gradle.tasks.KotlinCompile
+
 plugins {
    id "jacoco"
    id "java"
-    id "org.jetbrains.kotlin.jvm" version "1.4.30"
-    id "org.jetbrains.kotlin.kapt" version "1.4.0"
+    id "org.jetbrains.kotlin.jvm" version "1.8.0"
+    id "org.jetbrains.kotlin.kapt" version "1.8.0"
    id "application"
-    id "com.github.johnrengelman.shadow" version "5.2.0"
+    id "com.github.johnrengelman.shadow" version "7.0.0"
    id "com.diffplug.spotless" version "5.8.2"
-    id "dev.afanasev.sekret" version "0.0.7"
+    id "net.afanasev.sekret" version "0.1.1-RC3"
+    id "com.palantir.git-version" version "0.12.3"
 }

 group = "com.mangadex"
-version = System.getenv().getOrDefault("VERSION", 'git describe --tags --dirty'.execute().text.trim())
-mainClassName = "mdnet.MainKt"
+version = System.getenv("VERSION") ?: gitVersion()
+
+application {
+    mainClass = "mdnet.MainKt"
+}

 repositories {
    mavenCentral()
-    jcenter()
 }

 configurations {
@ -23,35 +28,36 @@ configurations {
 }

 dependencies {
-    compileOnly group: "dev.afanasev", name: "sekret-annotation", version: "0.0.7"
+    implementation "org.jetbrains.kotlin:kotlin-reflect"

-    implementation group: "commons-io", name: "commons-io", version: "2.8.0"
-    implementation group: "org.apache.commons", name: "commons-compress", version: "1.20"
-    implementation group: "ch.qos.logback", name: "logback-classic", version: "1.3.0-alpha4"
+    implementation group: "commons-io", name: "commons-io", version: "2.11.0"
+    implementation group: "org.apache.commons", name: "commons-compress", version: "1.22"
+    implementation group: "ch.qos.logback", name: "logback-classic", version: "1.3.6"

-    implementation group: "io.micrometer", name: "micrometer-registry-prometheus", version: "1.6.2"
-    implementation group: "com.maxmind.geoip2", name: "geoip2", version: "2.15.0"
+    implementation group: "io.micrometer", name: "micrometer-registry-prometheus", version: "1.8.3"
+    implementation group: "com.maxmind.geoip2", name: "geoip2", version: "2.16.1"

-    implementation platform(group: "org.http4k", name: "http4k-bom", version: "4.3.5.4")
-    implementation platform(group: "com.fasterxml.jackson", name: "jackson-bom", version: "2.12.1")
-    implementation platform(group: "io.netty", name: "netty-bom", version: "4.1.60.Final")
+    implementation platform(group: "org.http4k", name: "http4k-bom", version: "4.41.3.0")
+    implementation platform(group: "com.fasterxml.jackson", name: "jackson-bom", version: "2.14.2")
+    implementation platform(group: "io.netty", name: "netty-bom", version: "4.1.91.Final")

    implementation group: "org.http4k", name: "http4k-core"
    implementation group: "org.http4k", name: "http4k-resilience4j"
-    implementation group: "io.github.resilience4j", name: "resilience4j-micrometer", version: "1.6.1"
+    implementation group: "io.github.resilience4j", name: "resilience4j-micrometer", version: "1.7.1"
    implementation group: "org.http4k", name: "http4k-format-jackson"
    implementation group: "com.fasterxml.jackson.dataformat", name: "jackson-dataformat-yaml"
    implementation group: "com.fasterxml.jackson.datatype", name: "jackson-datatype-jsr310"
    implementation group: "org.http4k", name: "http4k-client-okhttp"
    implementation group: "org.http4k", name: "http4k-metrics-micrometer"
    implementation group: "org.http4k", name: "http4k-server-netty"
+    implementation group: "io.netty", name: "netty-codec-haproxy"
    implementation group: "io.netty", name: "netty-transport-native-epoll", classifier: "linux-x86_64"
-    implementation group: "io.netty.incubator", name: "netty-incubator-transport-native-io_uring", version: "0.0.3.Final", classifier: "linux-x86_64"
+    implementation group: "io.netty.incubator", name: "netty-incubator-transport-native-io_uring", version: "0.0.19.Final", classifier: "linux-x86_64"
    testImplementation group: "org.http4k", name: "http4k-testing-kotest"
-    runtimeOnly group: "io.netty", name: "netty-tcnative-boringssl-static", version: "2.0.36.Final"
+    runtimeOnly group: "io.netty", name: "netty-tcnative-boringssl-static", version: "2.0.59.Final"

-    implementation group: "com.zaxxer", name: "HikariCP", version: "4.0.2"
-    implementation group: "org.xerial", name: "sqlite-jdbc", version: "3.34.0"
+    implementation group: "com.zaxxer", name: "HikariCP", version: "4.0.3"
+    implementation group: "org.xerial", name: "sqlite-jdbc", version: "3.41.2.1"
    implementation "org.ktorm:ktorm-core:$ktorm_version"
    implementation "org.ktorm:ktorm-jackson:$ktorm_version"

@ -60,14 +66,16 @@ dependencies {

    testImplementation "io.kotest:kotest-runner-junit5:$kotest_version"
    testImplementation "io.kotest:kotest-assertions-core:$kotest_version"
-    testImplementation "io.mockk:mockk:1.10.4"
+    testImplementation "io.mockk:mockk:1.13.4"
 }

-test {
+tasks.withType(Test).configureEach {
    useJUnitPlatform()
+    javaLauncher = javaToolchains.launcherFor {
+        languageVersion = JavaLanguageVersion.of(8)
+    }
 }
-
-task testDev(type: Test) {
+tasks.register("testDev", Test) {
    group = "verification"
    useJUnitPlatform()
    filter {
@ -83,24 +91,18 @@ kapt {
 }

 java {
-    sourceCompatibility = JavaVersion.VERSION_1_8
-    targetCompatibility = JavaVersion.VERSION_1_8
+    toolchain {
+        languageVersion.set(JavaLanguageVersion.of(8))
+    }
 }

-tasks.withType(org.jetbrains.kotlin.gradle.tasks.KotlinCompile).all {
-    kotlinOptions {
-        jvmTarget = "1.8"
+tasks.withType(KotlinCompile).configureEach {
+    compilerOptions {
+        freeCompilerArgs = ["-Xjsr305=strict"]
    }
 }

 spotless {
-    java {
-        targetExclude("build/generated/**/*")
-        eclipse()
-        removeUnusedImports()
-        trimTrailingWhitespace()
-        endWithNewline()
-    }
    kotlin {
        ktlint("0.40.0").userData(["disabled_rules": "no-wildcard-imports"])
        licenseHeaderFile "license_header"
@ -116,9 +118,10 @@ tasks.register("generateVersion", Copy) {
    into "$buildDir/generated/java"
    expand templateContext
 }
+compileJava.dependsOn generateVersion
+sourceSets.main.java.srcDir generateVersion

-sourceSets.main.java.srcDir generateVersion.outputs.files
-
+// https://gist.github.com/medvedev/968119d7786966d9ed4442ae17aca279
 tasks.register("depsize") {
    description = 'Prints dependencies for "default" configuration'
    doLast() {
--- a/docker/README.md
+++ b/docker/README.md
@ -12,6 +12,8 @@ Once installed, you can check that it works by opening a command prompt and runn

 ## Run as a standalone container

+*Note* Changes to `the docker-compose.yml` are coming, and as such, this instruction page will get reworked a bit.
+
 Use either a specific image, preferrably the [latest image published](https://gitlab.com/mangadex-pub/mangadex_at_home/container_registry/1200259)

 > While it might work, using `registry.gitlab.com/mangadex-pub/mangadex_at_home:latest` is a bad idea as we do not guarantee forward-compatibility
--- a/docker/docker-compose.yml
+++ b/docker/docker-compose.yml
@ -4,7 +4,7 @@ services:

  mangadex-at-home:
    container_name: mangadex-at-home
-    image: "registry.gitlab.com/mangadex-pub/mangadex_at_home:<version>"
+    image: "registry.gitlab.com/mangadex-pub/mangadex_at_home:2"
    ports:
      - 443:443
    volumes:
@ -12,7 +12,6 @@ services:
      - ./data/cache/:/mangahome/data/
    environment:
      JAVA_TOOL_OPTIONS: "-Xms1G -Xmx1G -XX:+UseShenandoahGC -Xss512K"
-    privileged: true
    command: [
        "bash",
        "-c",
@ -31,7 +30,7 @@ services:

  prometheus:
    container_name: prometheus
-    image: prom/prometheus:v2.24.1
+    image: prom/prometheus:v2.34.0
    user: "root"
    group_add:
      - 0
@ -50,7 +49,7 @@ services:

  grafana:
    container_name: grafana
-    image: grafana/grafana:7.4.0
+    image: grafana/grafana:8.4.3
    user: "root"
    group_add:
      - 0
--- a/gradle.properties
+++ b/gradle.properties
@ -1,4 +1,3 @@
-http_4k_version=4.3.0.0
-kotest_version=4.4.1
-ktorm_version=3.3.0
-picocli_version=4.6.1
+kotest_version=5.5.5
+ktorm_version=3.6.0
+picocli_version=4.7.1
--- a/gradle/wrapper/gradle-wrapper.jar
+++ b/gradle/wrapper/gradle-wrapper.jar
--- a/gradle/wrapper/gradle-wrapper.properties
+++ b/gradle/wrapper/gradle-wrapper.properties
@ -1,6 +1,5 @@
-#Thu Jul 02 11:52:16 CDT 2020
-distributionUrl=https\://services.gradle.org/distributions/gradle-6.5.1-all.zip
 distributionBase=GRADLE_USER_HOME
 distributionPath=wrapper/dists
-zipStorePath=wrapper/dists
+distributionUrl=https\://services.gradle.org/distributions/gradle-8.0.2-bin.zip
 zipStoreBase=GRADLE_USER_HOME
+zipStorePath=wrapper/dists
--- a/55
+++ b/55
@ -1,5 +1,21 @@
 #!/usr/bin/env sh

+#
+# Copyright 2015 the original author or authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
 ##############################################################################
 ##
 ##  Gradle start up script for UN*X
@ -28,7 +44,7 @@ APP_NAME="Gradle"
 APP_BASE_NAME=`basename "$0"`

 # Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
-DEFAULT_JVM_OPTS='"-Xmx64m"'
+DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'

 # Use the maximum available, or set MAX_FD != -1 to use that value.
 MAX_FD="maximum"
@ -56,7 +72,7 @@ case "`uname`" in
  Darwin* )
    darwin=true
    ;;
-  MINGW* )
+  MSYS* | MINGW* )
    msys=true
    ;;
  NONSTOP* )
@ -66,6 +82,7 @@ esac

 CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar

+
 # Determine the Java command to use to start the JVM.
 if [ -n "$JAVA_HOME" ] ; then
    if [ -x "$JAVA_HOME/jre/sh/java" ] ; then
@ -109,10 +126,11 @@ if $darwin; then
    GRADLE_OPTS="$GRADLE_OPTS \"-Xdock:name=$APP_NAME\" \"-Xdock:icon=$APP_HOME/media/gradle.icns\""
 fi

-# For Cygwin, switch paths to Windows format before running java
-if $cygwin ; then
+# For Cygwin or MSYS, switch paths to Windows format before running java
+if [ "$cygwin" = "true" -o "$msys" = "true" ] ; then
    APP_HOME=`cygpath --path --mixed "$APP_HOME"`
    CLASSPATH=`cygpath --path --mixed "$CLASSPATH"`
+
    JAVACMD=`cygpath --unix "$JAVACMD"`

    # We build the pattern for arguments to be converted via cygpath
@ -138,19 +156,19 @@ if $cygwin ; then
        else
            eval `echo args$i`="\"$arg\""
        fi
-        i=$((i+1))
+        i=`expr $i + 1`
    done
    case $i in
-        (0) set -- ;;
-        (1) set -- "$args0" ;;
-        (2) set -- "$args0" "$args1" ;;
-        (3) set -- "$args0" "$args1" "$args2" ;;
-        (4) set -- "$args0" "$args1" "$args2" "$args3" ;;
-        (5) set -- "$args0" "$args1" "$args2" "$args3" "$args4" ;;
-        (6) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" ;;
-        (7) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" ;;
-        (8) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" ;;
-        (9) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" "$args8" ;;
+        0) set -- ;;
+        1) set -- "$args0" ;;
+        2) set -- "$args0" "$args1" ;;
+        3) set -- "$args0" "$args1" "$args2" ;;
+        4) set -- "$args0" "$args1" "$args2" "$args3" ;;
+        5) set -- "$args0" "$args1" "$args2" "$args3" "$args4" ;;
+        6) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" ;;
+        7) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" ;;
+        8) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" ;;
+        9) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" "$args8" ;;
    esac
 fi

@ -159,14 +177,9 @@ save () {
    for i do printf %s\\n "$i" | sed "s/'/'\\\\''/g;1s/^/'/;\$s/\$/' \\\\/" ; done
    echo " "
 }
-APP_ARGS=$(save "$@")
+APP_ARGS=`save "$@"`

 # Collect all arguments for the java command, following the shell quoting and substitution rules
 eval set -- $DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS "\"-Dorg.gradle.appname=$APP_BASE_NAME\"" -classpath "\"$CLASSPATH\"" org.gradle.wrapper.GradleWrapperMain "$APP_ARGS"

-# by default we should be in the correct project dir, but when run from Finder on Mac, the cwd is wrong
-if [ "$(uname)" = "Darwin" ] && [ "$HOME" = "$PWD" ]; then
-  cd "$(dirname "$0")"
-fi
-
 exec "$JAVACMD" "$@"
--- a/gradlew.bat
+++ b/gradlew.bat
@ -1,3 +1,19 @@
+@rem
+@rem Copyright 2015 the original author or authors.
+@rem
+@rem Licensed under the Apache License, Version 2.0 (the "License");
+@rem you may not use this file except in compliance with the License.
+@rem You may obtain a copy of the License at
+@rem
+@rem      https://www.apache.org/licenses/LICENSE-2.0
+@rem
+@rem Unless required by applicable law or agreed to in writing, software
+@rem distributed under the License is distributed on an "AS IS" BASIS,
+@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+@rem See the License for the specific language governing permissions and
+@rem limitations under the License.
+@rem
+
@if "%DEBUG%" == "" @echo off
@rem ##########################################################################
@rem
@ -13,15 +29,18 @@ if "%DIRNAME%" == "" set DIRNAME=.
 set APP_BASE_NAME=%~n0
 set APP_HOME=%DIRNAME%

+@rem Resolve any "." and ".." in APP_HOME to make it shorter.
+for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi
+
@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
-set DEFAULT_JVM_OPTS="-Xmx64m"
+set DEFAULT_JVM_OPTS="-Xmx64m" "-Xms64m"

@rem Find java.exe
 if defined JAVA_HOME goto findJavaFromJavaHome

 set JAVA_EXE=java.exe
 %JAVA_EXE% -version >NUL 2>&1
-if "%ERRORLEVEL%" == "0" goto init
+if "%ERRORLEVEL%" == "0" goto execute

 echo.
 echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
@ -35,7 +54,7 @@ goto fail
 set JAVA_HOME=%JAVA_HOME:"=%
 set JAVA_EXE=%JAVA_HOME%/bin/java.exe

-if exist "%JAVA_EXE%" goto init
+if exist "%JAVA_EXE%" goto execute

 echo.
 echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME%
@ -45,28 +64,14 @@ echo location of your Java installation.

 goto fail

-:init
-@rem Get command-line arguments, handling Windows variants
-
-if not "%OS%" == "Windows_NT" goto win9xME_args
-
-:win9xME_args
-@rem Slurp the command line arguments.
-set CMD_LINE_ARGS=
-set _SKIP=2
-
-:win9xME_args_slurp
-if "x%~1" == "x" goto execute
-
-set CMD_LINE_ARGS=%*
-
 :execute
@rem Setup the command line

 set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar

+
@rem Execute Gradle
-"%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" org.gradle.wrapper.GradleWrapperMain %CMD_LINE_ARGS%
+"%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" org.gradle.wrapper.GradleWrapperMain %*

 :end
@rem End local scope for the variables with windows NT shell
--- a/settings.gradle
+++ b/settings.gradle
@ -1,16 +1 @@
-pluginManagement {
-    repositories {
-        gradlePluginPortal()
-        jcenter()
-        google()
-    }
-    resolutionStrategy {
-        eachPlugin {
-            if (requested.id.id == "com.squareup.sqldelight") {
-                useModule("com.squareup.sqldelight:gradle-plugin:${requested.version}")
-            }
-        }
-    }
-}
-
 rootProject.name = 'mangadex_at_home'
--- a/settings.sample.yaml
+++ b/settings.sample.yaml
@ -80,3 +80,20 @@ server_settings:
  # 0 defaults to (2 * your available processors)
  # https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/Runtime.html#availableProcessors()
  threads: 0
+  # Whether to enable support for HAProxy Proxy Protocol
+  # If using a reverse proxy to forward requests to MD@H via
+  # ssl passthrough, you can use Proxy Protocol to preserve
+  # original IP if your reverse proxy supports it. This
+  # will allow geo location metrics to work correctly.
+  # https://www.haproxy.com/blog/haproxy/proxy-protocol/
+  enable_proxy_protocol: false
+
+
+# Settings intended for advanced use cases or tinkering
+dev_settings:
+  # The url to override the MD@H backend with
+  dev_url: ~
+  # Whether to disable the sni check for mangadex.network and localhost
+  disable_sni_check: false
+  # Whether to send the server header or not, defaults to false
+  send_server_header: false
--- a/src/main/java/mdnet/cache/CachingInputStream.java
+++ b/src/main/java/mdnet/cache/CachingInputStream.java
@ -1,118 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements.  See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License.  You may obtain a copy of the License at
- *
- *      http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package mdnet.cache;
-
-import org.apache.commons.io.IOUtils;
-import org.apache.commons.io.input.ProxyInputStream;
-
-import java.io.IOException;
-import java.io.InputStream;
-import java.io.OutputStream;
-import java.util.concurrent.ExecutorService;
-
-import static org.apache.commons.io.IOUtils.EOF;
-
-public class CachingInputStream extends ProxyInputStream {
-	private final OutputStream cache;
-	private final ExecutorService executor;
-	private final Runnable onClose;
-	private boolean eofReached = false;
-
-	public CachingInputStream(InputStream response, ExecutorService executor, OutputStream cache, Runnable onClose) {
-		super(response);
-		this.executor = executor;
-		this.cache = cache;
-		this.onClose = onClose;
-	}
-
-	@Override
-	public void close() throws IOException {
-		if (eofReached) {
-			try {
-				in.close();
-			} catch (IOException ignored) {
-			}
-			try {
-				cache.close();
-			} catch (IOException ignored) {
-			}
-			onClose.run();
-		} else {
-			executor.submit(() -> {
-				try {
-					IOUtils.copy(in, cache);
-				} catch (IOException ignored) {
-				} finally {
-					try {
-						in.close();
-					} catch (IOException ignored) {
-					}
-					try {
-						cache.close();
-					} catch (IOException ignored) {
-					}
-					onClose.run();
-				}
-			});
-		}
-	}
-
-	@Override
-	public int read() throws IOException {
-		final int ch = super.read();
-		if (ch != EOF) {
-			try {
-				cache.write(ch);
-			} catch (IOException ignored) {
-				// don't let write failures affect the image loading
-			}
-		} else {
-			eofReached = true;
-		}
-		return ch;
-	}
-
-	@Override
-	public int read(final byte[] bts, final int st, final int end) throws IOException {
-		final int n = super.read(bts, st, end);
-		if (n != EOF) {
-			try {
-				cache.write(bts, st, n);
-			} catch (IOException ignored) {
-				// don't let write failures affect the image loading
-			}
-		} else {
-			eofReached = true;
-		}
-		return n;
-	}
-
-	@Override
-	public int read(final byte[] bts) throws IOException {
-		final int n = super.read(bts);
-		if (n != EOF) {
-			try {
-				cache.write(bts, 0, n);
-			} catch (IOException ignored) {
-				// don't let write failures affect the image loading
-			}
-		} else {
-			eofReached = true;
-		}
-		return n;
-	}
-}
--- a/src/main/kotlin/mdnet/BackendApi.kt
+++ b/src/main/kotlin/mdnet/BackendApi.kt
@ -124,7 +124,7 @@ class BackendApi(private val settings: ClientSettings) {
                try {
                    PING_FAILURE_LENS(response)
                } catch (e: LensFailure) {
-                    PingFailure(response.status.code, response.status.description)
+                    PingFailure(response.status.code, response.bodyString())
                }
            }
        }
--- a/src/main/kotlin/mdnet/Constants.kt
+++ b/src/main/kotlin/mdnet/Constants.kt
@ -21,7 +21,7 @@ package mdnet
 import java.time.Duration

 object Constants {
-    const val CLIENT_BUILD = 31
+    const val CLIENT_BUILD = 32

    @JvmField val MAX_AGE_CACHE: Duration = Duration.ofDays(14)

--- a/src/main/kotlin/mdnet/ServerManager.kt
+++ b/src/main/kotlin/mdnet/ServerManager.kt
@ -24,7 +24,6 @@ import io.micrometer.prometheus.PrometheusConfig
 import io.micrometer.prometheus.PrometheusMeterRegistry
 import mdnet.cache.ImageStorage
 import mdnet.data.Statistics
-import mdnet.logging.error
 import mdnet.logging.info
 import mdnet.logging.warn
 import mdnet.metrics.DefaultMicrometerMetrics
@ -188,7 +187,7 @@ class ServerManager(
                        }
                    }
                } catch (e: Exception) {
-                    LOGGER.error(e) { "Main loop failed" }
+                    LOGGER.warn(e) { "Main loop failed" }
                }
            },
            5, 5, TimeUnit.SECONDS
--- a/src/main/kotlin/mdnet/cache/CachingInputStream.kt
+++ b/src/main/kotlin/mdnet/cache/CachingInputStream.kt
@ -0,0 +1,114 @@
+/*
+Mangadex@Home
+Copyright (c) 2020, MangaDex Network
+This file is part of MangaDex@Home.
+
+MangaDex@Home is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+MangaDex@Home is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this MangaDex@Home.  If not, see <http://www.gnu.org/licenses/>.
+*/
+package mdnet.cache
+
+import org.apache.commons.io.IOUtils
+import org.apache.commons.io.input.ProxyInputStream
+import java.io.IOException
+import java.io.InputStream
+import java.io.OutputStream
+import java.lang.Runnable
+import java.util.concurrent.ExecutorService
+import kotlin.Throws
+
+class CachingInputStream(
+    response: InputStream?,
+    private val executor: ExecutorService,
+    private val cache: OutputStream,
+    private val onClose: Runnable
+) : ProxyInputStream(response) {
+    private var eofReached = false
+
+    @Throws(IOException::class)
+    override fun close() {
+        if (eofReached) {
+            try {
+                `in`.close()
+            } catch (ignored: IOException) {
+            }
+            try {
+                cache.close()
+            } catch (ignored: IOException) {
+            }
+            onClose.run()
+        } else {
+            executor.submit {
+                try {
+                    IOUtils.copy(`in`, cache)
+                } catch (ignored: IOException) {
+                } finally {
+                    try {
+                        `in`.close()
+                    } catch (ignored: IOException) {
+                    }
+                    try {
+                        cache.close()
+                    } catch (ignored: IOException) {
+                    }
+                    onClose.run()
+                }
+            }
+        }
+    }
+
+    @Throws(IOException::class)
+    override fun read(): Int {
+        val ch = super.read()
+        if (ch != IOUtils.EOF) {
+            try {
+                cache.write(ch)
+            } catch (ignored: IOException) {
+                // don't let write failures affect the image loading
+            }
+        } else {
+            eofReached = true
+        }
+        return ch
+    }
+
+    @Throws(IOException::class)
+    override fun read(bts: ByteArray, st: Int, end: Int): Int {
+        val n = super.read(bts, st, end)
+        if (n != IOUtils.EOF) {
+            try {
+                cache.write(bts, st, n)
+            } catch (ignored: IOException) {
+                // don't let write failures affect the image loading
+            }
+        } else {
+            eofReached = true
+        }
+        return n
+    }
+
+    @Throws(IOException::class)
+    override fun read(bts: ByteArray): Int {
+        val n = super.read(bts)
+        if (n != IOUtils.EOF) {
+            try {
+                cache.write(bts, 0, n)
+            } catch (ignored: IOException) {
+                // don't let write failures affect the image loading
+            }
+        } else {
+            eofReached = true
+        }
+        return n
+    }
+}
--- a/src/main/kotlin/mdnet/cache/ImageStorage.kt
+++ b/src/main/kotlin/mdnet/cache/ImageStorage.kt
@ -25,6 +25,7 @@ import com.fasterxml.jackson.module.kotlin.jacksonObjectMapper
 import com.fasterxml.jackson.module.kotlin.readValue
 import mdnet.logging.info
 import mdnet.logging.trace
+import mdnet.logging.warn
 import org.apache.commons.io.file.PathUtils
 import org.ktorm.database.Database
 import org.ktorm.dsl.*
@ -35,6 +36,7 @@ import java.sql.SQLException
 import java.time.Instant
 import java.util.UUID
 import java.util.concurrent.*
+import java.util.concurrent.locks.ReentrantLock

@JsonNaming(PropertyNamingStrategies.SnakeCaseStrategy::class)
 data class ImageMetadata(
@ -58,10 +60,11 @@ class ImageStorage(
    private val cacheDirectory: Path,
    private val database: Database,
    autoPrune: Boolean = true
-) {
+) : AutoCloseable {
    private val tempCacheDirectory = cacheDirectory.resolve("tmp")
+    private val databaseLock = ReentrantLock()

-    private val evictor: ScheduledExecutorService = Executors.newScheduledThreadPool(2)
+    private val evictor: ScheduledExecutorService = Executors.newScheduledThreadPool(1)
    private val queue = LinkedBlockingQueue<String>(1000)

    /**
@ -89,36 +92,55 @@ class ImageStorage(

        evictor.scheduleWithFixedDelay(
            {
-                val toUpdate = HashSet<String>()
-                queue.drainTo(toUpdate)
-                val now = Instant.now()
+                try {
+                    val toUpdate = HashSet<String>()
+                    queue.drainTo(toUpdate)

-                LOGGER.info { "Updating LRU times for ${toUpdate.size} entries" }
-                synchronized(database) {
-                    database.batchUpdate(DbImage) {
-                        for (id in toUpdate) {
-                            item {
-                                set(DbImage.accessed, now)
-                                where {
-                                    DbImage.id eq id
+                    if (toUpdate.isEmpty()) {
+                        LOGGER.info { "Updating LRU times for ${toUpdate.size} entries" }
+                    } else {
+                        LOGGER.info { "Skipping empty LRU update" }
+                    }
+
+                    val now = Instant.now()
+
+                    if (databaseLock.tryLock(500, TimeUnit.MILLISECONDS)) {
+                        try {
+                            database.batchUpdate(DbImage) {
+                                for (id in toUpdate) {
+                                    item {
+                                        set(DbImage.accessed, now)
+                                        where {
+                                            DbImage.id eq id
+                                        }
+                                    }
                                }
                            }
+                        } finally {
+                            databaseLock.unlock()
                        }
+                    } else {
+                        LOGGER.warn { "High contention for database lock, bailing LRU update" }
                    }
+                    calculateSize()
+                } catch (e: Exception) {
+                    LOGGER.warn(e) { "Error updating LRU $this" }
                }
-                calculateSize()
            },
-            1, 1, TimeUnit.MINUTES
+            15, 30, TimeUnit.SECONDS
        )

-        // evict LRU cache every 3 minutes
        if (autoPrune) {
            evictor.scheduleWithFixedDelay(
                {
-                    calculateSize()
-                    pruneImages()
+                    try {
+                        calculateSize()
+                        pruneImages()
+                    } catch (e: Exception) {
+                        LOGGER.warn(e) { "Error pruning images" }
+                    }
                },
-                0, 3, TimeUnit.MINUTES
+                0, 1, TimeUnit.MINUTES
            )
        }
    }
@ -230,14 +252,24 @@ class ImageStorage(
            )

            Files.deleteIfExists(path)
-        } catch (e: IOException) {
-            // a failure means the image did not exist
-        } finally {
-            synchronized(database) {
+        } catch (_: IOException) {
+        }
+
+        // it is safe, but not optimal, for the
+        // DB write to fail after we've grabbed the file,
+        // as that just inflates the count.
+        // it will get resolved when the file gets grabbed again,
+        // or if the cache gets pruned.
+        if (databaseLock.tryLock(500, TimeUnit.MILLISECONDS)) {
+            try {
                database.delete(DbImage) {
                    DbImage.id eq id
                }
+            } finally {
+                databaseLock.unlock()
            }
+        } else {
+            LOGGER.warn { "High contention for database lock, bailing image delete write" }
        }
    }

@ -255,7 +287,7 @@ class ImageStorage(
        }
    }

-    fun close() {
+    override fun close() {
        evictor.shutdown()
        evictor.awaitTermination(10, TimeUnit.SECONDS)
    }
@ -311,23 +343,27 @@ class ImageStorage(

            Files.createDirectories(getPath(id).parent)

-            try {
-                synchronized(database) {
+            if (databaseLock.tryLock(500, TimeUnit.MILLISECONDS)) {
+                try {
                    database.insert(DbImage) {
                        set(DbImage.id, id)
                        set(DbImage.accessed, Instant.now())
                        set(DbImage.size, metadataSize + bytes)
                    }
+                } catch (e: SQLException) {
+                    // someone got to us before this (TOCTOU)
+                    // there are 2 situations here
+                    // one is that the
+                    // other write died in between writing the DB and
+                    // moving the file
+                    // the other is that we have raced and the other
+                    // is about to write the file
+                    // we handle this below
+                } finally {
+                    databaseLock.unlock()
                }
-            } catch (e: SQLException) {
-                // someone got to us before this (TOCTOU)
-                // there are 2 situations here
-                // one is that the
-                // other write died in between writing the DB and
-                // moving the file
-                // the other is that we have raced and the other
-                // is about to write the file
-                // we handle this below
+            } else {
+                LOGGER.warn { "High contention for database lock, bailing DB write" }
            }

            try {
@ -366,7 +402,6 @@ class ImageStorage(
        private val JACKSON: ObjectMapper = jacksonObjectMapper()

        private fun String.toCachePath() =
-            this.substring(0, 3).replace(".(?!$)".toRegex(), "$0 ").split(" ".toRegex()).reversed()
-                .plus(this).joinToString(File.separator)
+            this.substring(0, 3).split("").reversed().drop(1).joinToString(File.separator) + this
    }
 }
--- a/src/main/kotlin/mdnet/metrics/GeoIpMetricsFilter.kt
+++ b/src/main/kotlin/mdnet/metrics/GeoIpMetricsFilter.kt
@ -62,7 +62,10 @@ class GeoIpMetricsFilter(
                val inetAddress = InetAddress.getByName(sourceIp)
                if (!inetAddress.isLoopbackAddress && !inetAddress.isAnyLocalAddress) {
                    val country = databaseReader!!.country(inetAddress)
-                    recordCountry(country.country.isoCode)
+
+                    if (country.country.isoCode != null) {
+                        recordCountry(country.country.isoCode)
+                    }
                }
            } catch (e: GeoIp2Exception) {
                // do not disclose ip here, for privacy of logs
--- a/src/main/kotlin/mdnet/metrics/PostTransactionLabeler.kt
+++ b/src/main/kotlin/mdnet/metrics/PostTransactionLabeler.kt
@ -28,7 +28,7 @@ class PostTransactionLabeler : HttpTransactionLabeler {
                "method" to transaction.request.method.toString(),
                "status" to transaction.response.status.code.toString(),
                "path" to transaction.routingGroup,
-                "cache" to (transaction.response.header("X-Cache") ?: "MISS").toUpperCase()
+                "cache" to (transaction.response.header("X-Cache") ?: "MISS").uppercase()
            )
        )
    }
--- a/src/main/kotlin/mdnet/netty/ApplicationNetty.kt
+++ b/src/main/kotlin/mdnet/netty/ApplicationNetty.kt
@ -19,6 +19,7 @@ along with this MangaDex@Home.  If not, see <http://www.gnu.org/licenses/>.
 package mdnet.netty

 import io.netty.bootstrap.ServerBootstrap
+import io.netty.buffer.ByteBuf
 import io.netty.channel.*
 import io.netty.channel.epoll.Epoll
 import io.netty.channel.epoll.EpollEventLoopGroup
@ -27,6 +28,12 @@ import io.netty.channel.nio.NioEventLoopGroup
 import io.netty.channel.socket.SocketChannel
 import io.netty.channel.socket.nio.NioServerSocketChannel
 import io.netty.handler.codec.DecoderException
+import io.netty.handler.codec.ProtocolDetectionResult
+import io.netty.handler.codec.ProtocolDetectionState
+import io.netty.handler.codec.haproxy.HAProxyMessage
+import io.netty.handler.codec.haproxy.HAProxyMessageDecoder
+import io.netty.handler.codec.haproxy.HAProxyProtocolVersion
+import io.netty.handler.codec.http.FullHttpRequest
 import io.netty.handler.codec.http.HttpObjectAggregator
 import io.netty.handler.codec.http.HttpServerCodec
 import io.netty.handler.codec.http.HttpServerKeepAliveHandler
@ -43,7 +50,10 @@ import io.netty.handler.traffic.TrafficCounter
 import io.netty.incubator.channel.uring.IOUring
 import io.netty.incubator.channel.uring.IOUringEventLoopGroup
 import io.netty.incubator.channel.uring.IOUringServerSocketChannel
+import io.netty.util.AttributeKey
+import io.netty.util.AttributeMap
 import io.netty.util.DomainWildcardMappingBuilder
+import io.netty.util.ReferenceCountUtil
 import io.netty.util.concurrent.DefaultEventExecutorGroup
 import io.netty.util.internal.SystemPropertyUtil
 import mdnet.Constants
@ -104,7 +114,7 @@ sealed class NettyTransport(threads: Int) {
        private fun defaultNumThreads() = Runtime.getRuntime().availableProcessors() * 2

        fun bestForPlatform(threads: Int): NettyTransport {
-            val name = SystemPropertyUtil.get("os.name").toLowerCase(Locale.UK).trim { it <= ' ' }
+            val name = SystemPropertyUtil.get("os.name").lowercase(Locale.US).trim { it <= ' ' }

            val threadsToUse = if (threads == 0) defaultNumThreads() else threads
            LOGGER.info { "Choosing a transport with $threadsToUse threads" }
@ -165,7 +175,7 @@ class Netty(
            val certs = getX509Certs(tls.certificate)
            val sslContext = SslContextBuilder
                .forServer(getPrivateKey(tls.privateKey), certs)
-                .protocols("TLSv1.3", "TLSv1.2", "TLSv1.1", "TLSv1")
+                .protocols("TLSv1.3", "TLSv1.2")
                .build()

            val bootstrap = ServerBootstrap()
@ -173,6 +183,35 @@ class Netty(
                .channelFactory(transport.factory)
                .childHandler(object : ChannelInitializer<SocketChannel>() {
                    public override fun initChannel(ch: SocketChannel) {
+                        if (serverSettings.enableProxyProtocol) {
+                            ch.pipeline().addLast(
+                                "proxyProtocol",
+                                object : ChannelInboundHandlerAdapter() {
+                                    override fun channelRead(ctx: ChannelHandlerContext, msg: Any) {
+                                        if (msg is ByteBuf) {
+                                            // Since the builtin `HAProxyMessageDecoder` will break non Proxy-Protocol requests
+                                            // we need to use its detection capabilities to only add it when needed.
+                                            val result: ProtocolDetectionResult<HAProxyProtocolVersion> = HAProxyMessageDecoder.detectProtocol(msg)
+                                            if (result.state() == ProtocolDetectionState.DETECTED) {
+                                                ctx.pipeline().addAfter("proxyProtocol", null, HAProxyMessageDecoder())
+                                                ctx.pipeline().remove(this)
+                                            }
+                                        }
+                                        super.channelRead(ctx, msg)
+                                    }
+                                }
+                            )
+                            ch.pipeline().addLast(
+                                "saveOriginalIp",
+                                object : SimpleChannelInboundHandler<HAProxyMessage>() {
+                                    override fun channelRead0(ctx: ChannelHandlerContext, msg: HAProxyMessage) {
+                                        // Store proxy IP in an attribute for later use after HTTP request is extracted.
+                                        // Using an attribute ensures the value is scoped to this channel.
+                                        (ctx as AttributeMap).attr(HAPROXY_SOURCE).set(msg.sourceAddress())
+                                    }
+                                }
+                            )
+                        }
                        ch.pipeline().addLast(
                            "ssl",
                            SniHandler(DomainWildcardMappingBuilder(sslContext).build())
@ -188,7 +227,8 @@ class Netty(
                                override fun userEventTriggered(ctx: ChannelHandlerContext, evt: Any) {
                                    if (evt is SniCompletionEvent) {
                                        if (!devSettings.disableSniCheck) {
-                                            if (!evt.hostname().endsWith(hostToTest) &&
+                                            if (evt.hostname() != null &&
+                                                !evt.hostname().endsWith(hostToTest) &&
                                                !evt.hostname().endsWith("localhost")
                                            ) {
                                                ctx.close()
@ -205,6 +245,26 @@ class Netty(
                        ch.pipeline().addLast("keepAlive", HttpServerKeepAliveHandler())
                        ch.pipeline().addLast("aggregator", HttpObjectAggregator(65536))

+                        if (serverSettings.enableProxyProtocol) {
+                            ch.pipeline().addLast(
+                                "setForwardHeader",
+                                object : SimpleChannelInboundHandler<FullHttpRequest>(false) {
+                                    override fun channelRead0(ctx: ChannelHandlerContext, request: FullHttpRequest) {
+                                        // The geo-location code already supports the `Forwarded` header so setting
+                                        // it is the easiest way to introduce the original IP downstream.
+                                        if ((ctx as AttributeMap).hasAttr(HAPROXY_SOURCE)) {
+                                            val addr = (ctx as AttributeMap).attr(HAPROXY_SOURCE).get()
+                                            request.headers().set("Forwarded", addr)
+                                        }
+                                        // Since we're modifying the request without handling it, we must
+                                        // call retain to ensure it will still be available downstream.
+                                        ReferenceCountUtil.retain(request)
+                                        ctx.fireChannelRead(request)
+                                    }
+                                }
+                            )
+                        }
+
                        ch.pipeline().addLast("burstLimiter", burstLimiter)

                        ch.pipeline().addLast(
@ -255,6 +315,7 @@ class Netty(

    companion object {
        private val LOGGER = LoggerFactory.getLogger(Netty::class.java)
+        private val HAPROXY_SOURCE = AttributeKey.newInstance<String>("haproxy_source")
    }
 }

--- a/src/main/kotlin/mdnet/server/ImageHandler.kt
+++ b/src/main/kotlin/mdnet/server/ImageHandler.kt
@ -174,7 +174,7 @@ class ImageServer(

    companion object {
        private val LOGGER = LoggerFactory.getLogger(ImageServer::class.java)
-        private fun String.isImageMimetype() = this.toLowerCase().startsWith("image/")
+        private fun String.isImageMimetype() = this.lowercase().startsWith("image/")

        private fun baseHandler(): Filter =
            CachingFilters.Response.MaxAge(Clock.systemUTC(), Constants.MAX_AGE_CACHE)
--- a/src/main/kotlin/mdnet/server/ImageServer.kt
+++ b/src/main/kotlin/mdnet/server/ImageServer.kt
@ -86,7 +86,7 @@ fun getServer(

    val circuited = ResilienceFilters.CircuitBreak(
        circuitBreaker,
-        isError = { r: Response -> !r.status.successful }
+        isError = { r: Response -> r.status.serverError }
    )

    val upstream = ClientFilters.MicrometerMetrics.RequestTimer(registry)
@ -102,14 +102,19 @@ fun getServer(

    FunctionCounter.builder(
        "client.sent",
-        statistics,
-        { it.bytesSent.get().toDouble() }
-    ).baseUnit(BaseUnits.BYTES).register(registry)
+        statistics
+    ) { it.bytesSent.get().toDouble() }
+        .baseUnit(BaseUnits.BYTES).register(registry)

    val verifier = TokenVerifier(
        tokenKey = remoteSettings.tokenKey,
+        isDisabled = devSettings.disableTokenValidation,
    )

+    if (devSettings.disableTokenValidation) {
+        LOGGER.warn { "Token validation has been explicitly disabled. This should only be used for testing!" }
+    }
+
    return timeRequest()
        .then(addCommonHeaders(devSettings.sendServerHeader))
        .then(catchAllHideDetails())
--- a/src/main/kotlin/mdnet/server/TokenVerifier.kt
+++ b/src/main/kotlin/mdnet/server/TokenVerifier.kt
@ -37,11 +37,15 @@ import org.slf4j.LoggerFactory
 import java.time.OffsetDateTime
 import java.util.Base64

-class TokenVerifier(tokenKey: ByteArray) : Filter {
+class TokenVerifier(tokenKey: ByteArray, private val isDisabled: Boolean) : Filter {
    private val box = TweetNaclFast.SecretBox(tokenKey)

    override fun invoke(next: HttpHandler): HttpHandler {
        return then@{
+            if (isDisabled) {
+                return@then next(it)
+            }
+
            val chapterHash = Path.of("chapterHash")(it)

            val cleanedUri = it.uri.path.replaceBefore("/data", "/{token}")
--- a/src/main/kotlin/mdnet/settings/ClientSettings.kt
+++ b/src/main/kotlin/mdnet/settings/ClientSettings.kt
@ -20,7 +20,6 @@ package mdnet.settings

 import com.fasterxml.jackson.databind.PropertyNamingStrategies
 import com.fasterxml.jackson.databind.annotation.JsonNaming
-import dev.afanasev.sekret.Secret

@JsonNaming(PropertyNamingStrategies.SnakeCaseStrategy::class)
 data class ClientSettings(
@ -33,7 +32,7 @@ data class ClientSettings(

@JsonNaming(PropertyNamingStrategies.SnakeCaseStrategy::class)
 data class ServerSettings(
-    @field:Secret val secret: String,
+    val secret: String,
    val externalPort: Int = 0,
    val gracefulShutdownWaitSeconds: Int = 60,
    val hostname: String = "0.0.0.0",
@ -43,17 +42,27 @@ data class ServerSettings(
    val externalIp: String? = null,
    val port: Int = 443,
    val threads: Int = 0,
-)
+    val enableProxyProtocol: Boolean = false,
+) {
+    override fun toString(): String {
+        return "ServerSettings(secret=<redacted>, externalPort=$externalPort, gracefulShutdownWaitSeconds=$gracefulShutdownWaitSeconds, hostname='$hostname', maxKilobitsPerSecond=$maxKilobitsPerSecond, externalMaxKilobitsPerSecond=$externalMaxKilobitsPerSecond, maxMebibytesPerHour=$maxMebibytesPerHour, externalIp=$externalIp, port=$port, threads=$threads, enableProxyProtocol=$enableProxyProtocol)"
+    }
+}

@JsonNaming(PropertyNamingStrategies.SnakeCaseStrategy::class)
 data class DevSettings(
    val devUrl: String? = null,
    val disableSniCheck: Boolean = false,
    val sendServerHeader: Boolean = false,
+    val disableTokenValidation: Boolean = false,
 )

@JsonNaming(PropertyNamingStrategies.SnakeCaseStrategy::class)
 data class MetricsSettings(
    val enableGeoip: Boolean = false,
-    @field:Secret val geoipLicenseKey: String = "none"
-)
+    val geoipLicenseKey: String = "none"
+) {
+    override fun toString(): String {
+        return "MetricsSettings(enableGeoip=$enableGeoip, geoipLicenseKey=<redacted>)"
+    }
+}
--- a/src/main/kotlin/mdnet/settings/PingResult.kt
+++ b/src/main/kotlin/mdnet/settings/PingResult.kt
@ -20,7 +20,6 @@ package mdnet.settings

 import com.fasterxml.jackson.databind.PropertyNamingStrategies
 import com.fasterxml.jackson.databind.annotation.JsonNaming
-import dev.afanasev.sekret.Secret
 import org.http4k.core.Uri

 sealed class PingResult
@ -37,12 +36,13 @@ data class RemoteSettings(
    val latestBuild: Int,
    val url: Uri,
    val clientId: String,
-    @field:Secret val tokenKey: ByteArray,
+    val tokenKey: ByteArray,
    val compromised: Boolean,
    val paused: Boolean,
    val disableTokens: Boolean = false,
    val tls: TlsCert?
 ) : PingResult() {
+
    override fun equals(other: Any?): Boolean {
        if (this === other) return true
        if (javaClass != other?.javaClass) return false
@ -74,11 +74,19 @@ data class RemoteSettings(
        result = 31 * result + (tls?.hashCode() ?: 0)
        return result
    }
+
+    override fun toString(): String {
+        return "RemoteSettings(imageServer=$imageServer, latestBuild=$latestBuild, url=$url, clientId='$clientId', tokenKey=<redacted>, compromised=$compromised, paused=$paused, disableTokens=$disableTokens, tls=$tls)"
+    }
 }

@JsonNaming(PropertyNamingStrategies.SnakeCaseStrategy::class)
 data class TlsCert(
    val createdAt: String,
-    @field:Secret val privateKey: String,
-    @field:Secret val certificate: String
-)
+    val privateKey: String,
+    val certificate: String
+) {
+    override fun toString(): String {
+        return "TlsCert(createdAt='$createdAt', privateKey=<redacted>, certificate=<redacted>)"
+    }
+}
--- a/src/main/kotlin/mdnet/settings/SettingsRequest.kt
+++ b/src/main/kotlin/mdnet/settings/SettingsRequest.kt
@ -20,20 +20,27 @@ package mdnet.settings

 import com.fasterxml.jackson.databind.PropertyNamingStrategies
 import com.fasterxml.jackson.databind.annotation.JsonNaming
-import dev.afanasev.sekret.Secret

@JsonNaming(PropertyNamingStrategies.SnakeCaseStrategy::class)
 data class SettingsRequest(
-    @field:Secret val secret: String,
+    val secret: String,
    val ipAddress: String?,
    val port: Int,
    val diskSpace: Long,
    val networkSpeed: Long,
    val buildVersion: Int,
    val tlsCreatedAt: String?,
-)
+) {
+    override fun toString(): String {
+        return "SettingsRequest(secret=<redacted>, ipAddress=$ipAddress, port=$port, diskSpace=$diskSpace, networkSpeed=$networkSpeed, buildVersion=$buildVersion, tlsCreatedAt=$tlsCreatedAt)"
+    }
+}

@JsonNaming(PropertyNamingStrategies.SnakeCaseStrategy::class)
 data class LogoutRequest(
-    @field:Secret val secret: String,
-)
+    val secret: String,
+) {
+    override fun toString(): String {
+        return "LogoutRequest(secret=<redacted>)"
+    }
+}
--- a/src/test/kotlin/mdnet/cache/ImageStorageTest.kt
+++ b/src/test/kotlin/mdnet/cache/ImageStorageTest.kt
@ -32,18 +32,20 @@ import io.kotest.matchers.nulls.shouldNotBeNull
 import io.kotest.matchers.shouldBe
 import org.apache.commons.io.IOUtils
 import org.ktorm.database.Database
+import kotlin.time.Duration.Companion.minutes
 import kotlin.time.ExperimentalTime
-import kotlin.time.minutes

 class ImageStorageTest : FreeSpec() {
    override fun isolationMode() = IsolationMode.InstancePerTest

    init {
-        val imageStorage = ImageStorage(
-            maxSize = 5,
-            cacheDirectory = tempdir().toPath(),
-            database = Database.connect("jdbc:sqlite:${tempfile()}"),
-            autoPrune = false,
+        val imageStorage = autoClose(
+            ImageStorage(
+                maxSize = 5,
+                cacheDirectory = tempdir().toPath(),
+                database = Database.connect("jdbc:sqlite:${tempfile()}"),
+                autoPrune = false,
+            )
        )

        val testMeta = ImageMetadata("a", "a", 123)
@ -81,7 +83,8 @@ class ImageStorageTest : FreeSpec() {
                writer.stream.write(ByteArray(12))
                writer.abort()

-                "should not update size" {
+                "should not update size even if calculated" {
+                    imageStorage.calculateSize()
                    imageStorage.size.shouldBeZero()
                }
            }
@ -157,14 +160,17 @@ class ImageStorageSlowTest : FreeSpec() {
    override fun isolationMode() = IsolationMode.InstancePerTest

    init {
-        val imageStorage = ImageStorage(
-            maxSize = 4097,
-            cacheDirectory = tempdir().toPath(),
-            database = Database.connect("jdbc:sqlite:${tempfile()}"),
+        val imageStorage = autoClose(
+            ImageStorage(
+                maxSize = 4097,
+                cacheDirectory = tempdir().toPath(),
+                database = Database.connect("jdbc:sqlite:${tempfile()}"),
+            )
        )

        "autoPrune" - {
            "should update size eventually" {
+                println("1 - $imageStorage")
                val writer = imageStorage.storeImage("test", ImageMetadata("a", "a", 4096))
                writer.shouldNotBeNull()

@ -177,6 +183,7 @@ class ImageStorageSlowTest : FreeSpec() {
            }

            "should prune if insufficient size eventually" {
+                println("2 - $imageStorage")
                imageStorage.maxSize = 10000

                val writer = imageStorage.storeImage("test", ImageMetadata("a", "a", 123))
@ -185,6 +192,7 @@ class ImageStorageSlowTest : FreeSpec() {
                writer.stream.write(ByteArray(8192))
                writer.commit(8192).shouldBeTrue()

+                imageStorage.calculateSize()
                eventually(5.minutes) {
                    imageStorage.size.shouldBeZero()
                }
--- a/src/test/kotlin/mdnet/server/ImageServerTest.kt
+++ b/src/test/kotlin/mdnet/server/ImageServerTest.kt
@ -116,11 +116,13 @@ class ImageServerTest : FreeSpec() {
            }

            "with real cache" - {
-                val storage = ImageStorage(
-                    maxSize = 100000,
-                    cacheDirectory = tempdir().toPath(),
-                    database = Database.connect("jdbc:sqlite:${tempfile()}"),
-                    autoPrune = false,
+                val storage = autoClose(
+                    ImageStorage(
+                        maxSize = 100000,
+                        cacheDirectory = tempdir().toPath(),
+                        database = Database.connect("jdbc:sqlite:${tempfile()}"),
+                        autoPrune = false,
+                    )
                )

                val server = ImageServer(
@ -161,11 +163,13 @@ class ImageServerTest : FreeSpec() {
        "failed upstream responses" - {
            val client = mockk<HttpHandler>()

-            val storage = ImageStorage(
-                maxSize = 100000,
-                cacheDirectory = tempdir().toPath(),
-                database = Database.connect("jdbc:sqlite:${tempfile()}"),
-                autoPrune = false,
+            val storage = autoClose(
+                ImageStorage(
+                    maxSize = 100000,
+                    cacheDirectory = tempdir().toPath(),
+                    database = Database.connect("jdbc:sqlite:${tempfile()}"),
+                    autoPrune = false,
+                )
            )

            val server = ImageServer(
--- a/src/test/kotlin/mdnet/server/TokenVerifierTest.kt
+++ b/src/test/kotlin/mdnet/server/TokenVerifierTest.kt
@ -31,7 +31,7 @@ class TokenVerifierTest : FreeSpec() {
        val clientKeys = TweetNaclFast.Box.keyPair()
        val box = TweetNaclFast.Box(clientKeys.publicKey, remoteKeys.secretKey)

-        val backend = TokenVerifier(box.before()).then {
+        val backend = TokenVerifier(box.before(), false).then {
            Response(Status.OK)
        }
Author	SHA1	Message	Date
carbotaniuman	754c1de51d	Merge branch 'update-everything' into 'master' Update all the deps See merge request mangadex-pub/mangadex_at_home!103	2023-08-07 22:09:31 +00:00
carbotaniuman	04f5addfaa	Update all the deps	2023-08-07 22:09:31 +00:00
Tristan	4078d07053	Merge branch 'fix-ci' into 'master' Raise minimum guaranteed memory for gradle build to 3GB See merge request mangadex-pub/mangadex_at_home!105	2023-04-12 00:51:20 +00:00
Tristan	a37f82e21e	Raise minimum guaranteed memory for gradle build to 3GB	2023-04-12 01:42:28 +01:00
carbotaniuman	36ffe204a1	Merge branch 'docker-changes' into 'master' Docker compose changes See merge request mangadex-pub/mangadex_at_home!102	2023-04-07 20:51:29 +00:00
Georgi Yankov	1984a994a4	Docker compose changes	2023-04-07 20:51:29 +00:00
carbotaniuman	aca092a141	Merge branch 'fix-contention' into 'master' Try and fix contention issues See merge request mangadex-pub/mangadex_at_home!101	2023-04-06 21:27:06 +00:00
carbotaniuman	ac246e5449	Try and fix contention issues	2023-04-06 21:27:06 +00:00
carbotaniuman	c9a5548770	Merge branch 'fix-error-string' into 'master' Get the body of the error See merge request mangadex-pub/mangadex_at_home!98	2022-02-21 17:16:08 +00:00
carbotaniuman	aa0931dddc	Get the body of the error	2022-02-21 17:16:08 +00:00
carbotaniuman	acda98ae55	Merge branch 'update-and-fix' into 'master' Update deps and fix See merge request mangadex-pub/mangadex_at_home!100	2022-02-18 03:20:02 +00:00
carbotaniuman	08f72be1b0	Update deps and fix	2022-02-17 21:02:37 -06:00
carbotaniuman	85dce402df	Merge branch 'fix-tls-and-crashes' into 'master' Fix tls and crashes See merge request mangadex-pub/mangadex_at_home!99	2022-02-17 06:27:44 +00:00
carbotaniuman	d198c27de3	Fix tls and crashes	2022-02-17 06:27:43 +00:00
carbotaniuman	c7830161c1	Merge branch 'fix-nits' into 'master' Really fix CI See merge request mangadex-pub/mangadex_at_home!97	2022-02-07 03:59:51 +00:00
carbotaniuman	f651409e87	Really fix CI	2022-02-07 03:59:51 +00:00
carbotaniuman	0cf608a66b	Update .gitlab-ci.yml	2021-10-19 04:55:53 +00:00
carbotaniuman	0d490b3d5b	Merge branch 'modernize-ci-again' into 'master' Fix a few more nits with CI See merge request mangadex-pub/mangadex_at_home!96	2021-10-19 01:24:53 +00:00
carbotaniuman	0a7e738253	Fix a few more nits with CI	2021-10-01 23:13:34 -05:00
carbotaniuman	15f13eb7c3	Merge branch 'ci-test' into 'master' Update CI See merge request mangadex-pub/mangadex_at_home!95	2021-10-02 03:54:10 +00:00
carbotaniuman	ce94b21b03	Update CI	2021-10-02 03:54:09 +00:00
carbotaniuman	baecff51be	Merge branch 'ci-update' into 'master' Ci update See merge request mangadex-pub/mangadex_at_home!94	2021-10-01 06:10:07 +00:00
carbotaniuman	7dc9c21a8b	Ci update	2021-10-01 06:10:07 +00:00
carbotaniuman	7498ed0a4d	Merge branch 'disable-token-validation' into 'master' Add support for skipping token validation See merge request mangadex-pub/mangadex_at_home!92	2021-10-01 02:21:31 +00:00
carbotaniuman	ad392b30d8	Merge branch 'minor-cleanup' into 'master' Minor cleanup See merge request mangadex-pub/mangadex_at_home!93	2021-10-01 02:19:59 +00:00
carbotaniuman	85668ff207	Minor cleanup	2021-10-01 02:19:58 +00:00
carbotaniuman	f93481975c	Merge branch 'master' into 'master' Add support for proxy protocol IP forwarding Closes #85 See merge request mangadex-pub/mangadex_at_home!88	2021-07-19 16:38:10 +00:00
melink14	f3d35b60eb	Add support for proxy protocol IP forwarding	2021-07-19 16:38:09 +00:00
Edward Shen	7f8b406164	Add support for skipping token validation	2021-07-19 00:34:50 -04:00
carbotaniuman	10e394db1e	Merge branch 'better-cache-path' into 'master' Remove regex use in cache path generation See merge request mangadex-pub/mangadex_at_home!91	2021-07-11 20:36:39 +00:00
Edward Shen	d35d5b386e	Remove regex use in cache path generation	2021-07-11 16:16:26 -04:00
carbotaniuman	dd8e434024	Merge branch 'update-gradle' into 'master' Update to Gradle 7 and Kotlin 1.5.10 See merge request mangadex-pub/mangadex_at_home!90	2021-07-11 19:27:49 +00:00
carbotaniuman	a12d251bf4	Update to Gradle 7 and Kotlin 1.5.10	2021-07-06 14:40:02 -05:00