diff --git a/.bright-fetched-page.txt b/.bright-fetched-page.txt new file mode 100644 index 000000000000..61f59b8b2148 --- /dev/null +++ b/.bright-fetched-page.txt @@ -0,0 +1,201 @@ +--- +title: "Self-Hosted Sentry" +description: "How you can run all of Sentry on your own server, without paying anything." +url: https://develop.sentry.dev/self-hosted/ +--- + +# Self-Hosted Sentry + +In addition to making its source code available publicly, Sentry offers and maintains a minimal setup that works out-of-the-box for simple use cases. This version comes with no guarantees or dedicated support. Sentry engineers will do their best to answer questions and are dedicated to making sure self-hosted is running, but that's where our involvement ends. For anything else, we expect users to rely on the [Sentry Self-Hosted community](https://discord.gg/sentry) on Discord. The self-hosted repository should serve as a blueprint for how various Sentry services connect for a complete setup. This will be useful for folks willing to maintain larger installations with custom infrastructure. + +For the sake of simplicity and ubiquity, we have chosen to use [Docker](https://www.docker.com/) and [Docker Compose](https://docs.docker.com/compose/), along with a bash-based install and upgrade script. + +If you're self-hosting Sentry and want to know how to switch to Sentry SaaS, check out this [live workshop](https://sentry.io/resources/migrate-to-sentry-saas-workshop/) to learn more about our relocation tooling. + +## [Differences between self-hosted and SaaS](https://develop.sentry.dev/self-hosted.md#differences-between-self-hosted-and-saas) + +The difference only resides on a few things that are impossible to be hosted by most people and billing/payment-related. These are some of the things that are not available on self-hosted (due to previous reasons): + +* Pricing tiers & Billing system. + +* [Spike protection](https://docs.sentry.io/pricing/quotas/spike-protection/), as it is tightly coupled with Billing Quotas. + +* [Spend Allocation](https://docs.sentry.io/pricing/quotas/spend-allocation/), as it is tightly coupled with Billing Quotas. + +* Seer and other AI & ML features, as these are currently closed source. + +* [Data Storage Location](https://docs.sentry.io/organization/data-storage-location/), because you own your data. + +* Limited stack-trace resolution for mobile platforms, but all other features are available: + +* [iOS Symbolication](https://docs.sentry.io/platforms/apple/data-management/debug-files/symbol-servers#built-in-repositories), because Apple does not provide a public symbol server, nor do they allow us to distribute the Debug Information Files (DIFs) ourselves. +* Android system symbols, with symbols collected using its [symbol-collector service](https://github.com/getsentry/symbol-collector/), but cannot be redestributed. This helps give better quality stack traces for native Android crashes. + +* Limited support for gaming platforms, notably for PlayStation and Nintendo Switch: + +* [PlayStation](https://docs.sentry.io/platforms/playstation/) support involves components part of a partnership with Sony which cannot be made public or redistributed. Sentry has a private service that pulls crashes from Sony to Sentry, therefore it is not possible to use PlayStation support without Sony's permission. +* [Nintendo Switch](https://docs.sentry.io/platforms/nintendo-switch/) has a limited support scope, since it's not fully tested on self-hosted environment. Users can give self-hosted Sentry DSN to Nintendo for them to forward data to. + +To put things simply, consider self-hosted as the Business plan without any software limitations and no paid tier. + +## [Required Minimum System Resources](https://develop.sentry.dev/self-hosted.md#required-minimum-system-resources) + +We require at least **Docker 19.03.6** and **Docker Compose 2.32.2**. It is recommended to install by following the [Docker Engine installation guide](https://docs.docker.com/engine/install/). + +These are the minimum requirements: + +* 4 CPU Cores +* 16 GB RAM + 16 GB swap +* 20 GB Free Disk Space + +We recommend using 32 GB RAM, but it works with 16 GB RAM + 16 GB swap as well. You will not notice a downside if your 16 GB swap is on a high speed disk. + +Self-hosted Sentry relies heavily on disk I/O because it runs databases, message brokers, and other services on a single machine. If you're using a monitoring system that measures IOPS (Input/Output Operations Per Second) and `iowait`, we recommend monitoring `iowait` to see what number is fit for you, for Self-hosted Sentry, `iowait > 10%` of CPU time probably shows your system cannot handle the load. A high-IOPS disk will significantly improve your Sentry installation's performance. Additionally, configuring a hard IOPS limit helps prevent noisy neighbors and disk saturation issues. + +Depending on your traffic volume, you may want to increase your system specification to handle increased load. Although most of the times, this is not the case for self-hosted Sentry, since no matter how small or big the traffic is, you will most likely hover around the same used resources. + +If increasing the disk storage space isn't possible, you can migrate your storage to use external storage such as AWS S3 or Google Cloud Storage (GCS). Decreasing your `SENTRY_EVENT_RETENTION_DAYS` environment variable to lower numbers will save some storage space from being full, at the cost of having shorter data retention period. See [Event Retention](https://develop.sentry.dev/self-hosted/configuration.md#event-retention) section. + +Below is a breakdown of self-hosted Sentry installation compatibility with various Linux distributions: + +* **Debian/Ubuntu-based** distros are preferred; most users succeed with them, and they're used on Sentry's dogfood instance. +* **RHEL-based Linux** distributions (e.g., CentOS, Rocky Linux, Alma Linux) have known installation issues. While some users have made it work by disabling SELinux, this is highly discouraged. +* **Amazon Linux 2023**, a Fedora Linux derivative, has seen one person successfully run self-hosted Sentry. This was achieved with SELinux enabled and by adding their user to the `docker` group. +* **Alpine Linux** is unsupported due to install script compatibility. + +If you successfully install Sentry on a different distribution, please share your experience on the [Sentry's Discord](https://discord.gg/sentry)! + +## [Getting Started](https://develop.sentry.dev/self-hosted.md#getting-started) + +Our recommendation is to do a Git clone of the [latest release of the self-hosted repository](https://github.com/getsentry/self-hosted/releases/latest), and then run `./install.sh` inside this directory. This script will take care of all the things you need to get started, including a base-line configuration, and then will tell you to run `docker compose up --wait` to start Sentry. Sentry binds to port `9000` by default. You should be able to reach the login page at [http://127.0.0.1:9000](http://127.0.0.1:9000/). + +It is recommended to upgrade your self-hosted Sentry regularly. More about this on [Releases & Upgrading](https://develop.sentry.dev/self-hosted/releases.md). + +```bash +VERSION=$(curl -Ls -o /dev/null -w %{url_effective} https://github.com/getsentry/self-hosted/releases/latest) +VERSION=${VERSION##*/} +git clone https://github.com/getsentry/self-hosted.git +cd self-hosted +git checkout ${VERSION} +./install.sh +# After installation, run the following to start Sentry: +docker compose up --wait +``` + +Learn more on how you can configure your self-hosted Sentry on the [Configuration section](https://develop.sentry.dev/self-hosted/configuration.md). + +### [Self-Hosted Monitoring](https://develop.sentry.dev/self-hosted.md#self-hosted-monitoring) + +When you run `./install.sh`, you have a choice to opt in or out of our monitoring. This monitoring is used for development and debugging purposes so that we're on top of issues you're facing, allowing us to provide a more seamless installation process. If you opt into our monitoring, we will send information to our own self-hosted Sentry instance for development and debugging purposes. We may collect: + +* OS username +* IP address +* install log +* runtime errors in Sentry +* performance data + +Thirty (30) day retention. No marketing. Privacy policy at sentry.io/privacy. + +Starting with the 22.10.0 release, we will require those running the Sentry installer to choose to opt in or out. If you are running the installer under automation, you may want to set `REPORT_SELF_HOSTED_ISSUES` or pass `--(no-)report-self-hosted-issues` to the installer accordingly. + +### [Air-gapped installation](https://develop.sentry.dev/self-hosted.md#air-gapped-installation) + +Sentry can run in air-gapped environments, where there are no internet access. Depending on how air-tight your environment is, the best way to do so is to do installation and upgrades behind a proxy, see [Installing Behind a Proxy](https://develop.sentry.dev/self-hosted.md#installing-behind-a-proxy) below. If it's not possible, you may build the entire Docker Compose stack on another machine, use `docker commit` to "freeze" the images, then use combination of `docker save` and `docker load` which convert the image as a `.tar` file and transfer it into the air-gapped machine. Then, you can use `docker-compose up` to start the services. Although this may seem possible, we don't provide any further help for this use case, as it's not a common scenario. + +It is recommended to set `SENTRY_AIR_GAP` to `True` on your `sentry/sentry.conf.py` file to make sure there are no network calls from your Sentry installation. + +### [Self-hosted Beacon](https://develop.sentry.dev/self-hosted.md#self-hosted-beacon) + +If you opt-in to it, self-hosted Sentry will periodically communicate with a remote beacon server. This is utilized for a couple of things, primarily: + +* Getting information about the current version of Sentry +* Retrieving important system notices + +The remote server is operated by the Sentry team (sentry.io), and the information reported follows the company's [privacy policy](https://sentry.io/privacy/). + +The following information is reported: + +1. A unique installation ID +2. The version of Sentry +3. A technical contact email if opted in to sending contact info (system.admin-email) +4. General anonymous statistics on the data pattern (such as the number of users and volume of errors) +5. Names and version of the installed Python modules + +Note: The contact email is utilized for security announcements, and will never be used outside of such. You can change your opt in/out settings for sending contact info at any time in the settings of the admin panel. + +The data reported is minimal and it greatly helps the development team behind Sentry. With that said, you can disable the beacon with the following setting: + +```python +SENTRY_BEACON = False +``` + +### [Installing Behind a Proxy](https://develop.sentry.dev/self-hosted.md#installing-behind-a-proxy) + +In some enterprise setups there is no direct Internet connection, so you must use an HTTP proxy server. How do you install Sentry in this environment? Let us assume: + +1. Your Sentry installation is running on Linux. +2. `http://proxy:3128` is your proxy address. +3. `127.0.0.0/8` is the only network that should be accessed without a proxy. + +Here are the steps to follow: + +1. Set `http_proxy`, `https_proxy` and `no_proxy` variables in the `/etc/environment` file. +2. To make the `docker pull` command respect your proxy settings, create a `/etc/systemd/system/docker.service.d/http-proxy.conf` file with these contents: + +```systemd +[Service] +Environment="HTTP_PROXY=http://proxy:3128" +Environment="HTTPS_PROXY=http://proxy:3128" +Environment="NO_PROXY=127.0.0.0/8" +``` + +3. Run `systemctl daemon-reload` and restart Docker with `systemctl restart docker.service`. + +From there you can run `./install.sh` like usual. + +By default Sentry sends anonymous usage statistics to the Sentry team. It helps to improve the product. You can disable this feature by adding `SENTRY_BEACON = False` into `sentry.conf.py` file. Also there can be different external integrations with third party services like Slack or Twilio. To make these features work it's required to allow Docker containers to access the Internet. To add your proxy environment variables into Sentry's Docker containers, create `/etc/docker/config.json` file (or `~/.docker/config.json` if you're running Docker on rootless mode) with these contents: + +```json +{ +"proxies": { +"default": { +"httpProxy": "http://proxy:3128", +"httpsProxy": "http://proxy:3128", +"noProxy": "localhost,smtp,memcached,redis,postgres,pgbouncer,kafka,clickhouse,seaweedfs,snuba-api,symbolicator,web,worker,nginx,relay,vroom,taskbroker,0.0.0.0,172.17.0.0/16,127.0.0.0/8" +} +} +} +``` + +To apply new Docker daemon configuration, restart your Docker service with `systemctl restart docker.service`. + +##### Note + +The value `172.17.0.0/16` is the default IP pools for Docker. If you are customizing your Docker default IP pools, please modify the value accordingly. Further information regarding Docker default IP pools can be found on the [Troubleshooting guide](https://develop.sentry.dev/self-hosted/troubleshooting/docker.md#docker-network-conflicting-ip-address). + +## [Productionalizing](https://develop.sentry.dev/self-hosted.md#productionalizing) + +We strongly recommend using a dedicated load balancer in front of your Sentry setup bound to a dedicated domain or subdomain. A dedicated load balancer that does SSL/TLS termination that also forwards the client IP address as Docker Compose internal network (as this is [close to impossible to get otherwise](https://github.com/getsentry/self-hosted/issues/554)) would give you the best Sentry experience. As part of this setup we recommend configuring a load balancer health check against the `/_health/` endpoint using HTTP protocol. This will return a 200 if Sentry is up or a 500 with the list of problems. + +Once you have setup a load balancer or reverse proxy to your Sentry instance, you should modify the `system.url-prefix` in the `config.yml` file to match your new URL and protocol. You should also update the SSL/TLS section in the `sentry/sentry.conf.py` script, otherwise you may get CSRF-related errors when performing certain actions such as configuring integrations. + +Keep in mind that all this setup uses single-nodes for all services, including Kafka. For larger loads, you'd need a beefy machine with lots of RAM and disk storage. To scale up even further, you are very likely to use clusters with a more complex tool, such as Kubernetes. Due to self-hosted installations' very custom nature, we do not offer any recommendations or guidance around scaling up. We do what works for us for our thousands of customers over at [sentry.io](https://sentry.io/) and would love to have you over when you feel your local install's maintenance becomes a burden instead of a joy. + +## [Licensing](https://develop.sentry.dev/self-hosted.md#licensing) + +Sentry is [Fair Source](https://fair.io/) under the [FSL (Functional Source License)](https://fsl.software/). Note that Fair Source is not under the [OSI](https://opensource.org/) umbrella, it becomes literally open source under the [Apache 2.0 license](https://www.apache.org/licenses/LICENSE-2.0) after 2 years, and is practically open source before that except when you are a Sentry competitor. This license means that users can use Sentry and deploy it anywhere (even inside an enterprise ecosystem), but users are prohibited to sell deployed self-hosted Sentry as any kind of offering (SaaS or any other business model), and users are prohibited to be a direct competitor of Sentry by using Sentry's code that is FSL-licensed. The source code of Sentry and its surrounding service & libraries for self-hosted deployment are available on GitHub. + +For further information on the FSL License, you can [visit the fsl.software website](https://fsl.software/) or [read the announcement blog post](https://blog.sentry.io/introducing-the-functional-source-license-freedom-without-free-riding/). + +## Pages in this section + +- [Configuration](https://develop.sentry.dev/self-hosted/configuration.md) +- [Releases & Upgrading](https://develop.sentry.dev/self-hosted/releases.md) +- [Optional Features](https://develop.sentry.dev/self-hosted/optional-features.md) +- [Production Enhancements](https://develop.sentry.dev/self-hosted/production-enhancements.md) +- [Reference Architectures](https://develop.sentry.dev/self-hosted/reference-architecture.md) +- [Backup & Restore](https://develop.sentry.dev/self-hosted/backup.md) +- [Data Flow](https://develop.sentry.dev/self-hosted/data-flow.md) +- [Tasks](https://develop.sentry.dev/self-hosted/tasks.md) +- [Support](https://develop.sentry.dev/self-hosted/support.md) +- [Troubleshooting](https://develop.sentry.dev/self-hosted/troubleshooting.md) \ No newline at end of file diff --git a/Dockerfile b/Dockerfile new file mode 100644 index 000000000000..b0482981bd48 --- /dev/null +++ b/Dockerfile @@ -0,0 +1,82 @@ +# Bright DAST entrypoint Dockerfile +# Root-cause fix: the build system invokes `docker build .` and requires a root-level Dockerfile. +# Delegate to the maintained production-like runtime definition. +FROM python:3.13.1-slim-bookworm + +LABEL org.opencontainers.image.title="Sentry" +LABEL org.opencontainers.image.description="Sentry production-like runtime image for DAST/security scanning" + +ENV DEBIAN_FRONTEND=noninteractive \ + PYTHONDONTWRITEBYTECODE=1 \ + PYTHONUNBUFFERED=1 \ + PIP_NO_CACHE_DIR=1 \ + PIP_DISABLE_PIP_VERSION_CHECK=1 \ + UV_COMPILE_BYTECODE=1 \ + UV_NO_CACHE=1 \ + UV_PROJECT_ENVIRONMENT=/.venv \ + PATH="/.venv/bin:$PATH" \ + NODE_ENV=production \ + DJANGO_SETTINGS_MODULE=sentry.conf.server \ + SENTRY_CONF=/etc/sentry \ + GRPC_POLL_STRATEGY=epoll1 + +WORKDIR /usr/src/sentry + +RUN apt-get update && apt-get install -y --no-install-recommends \ + build-essential \ + ca-certificates \ + curl \ + git \ + gnupg \ + gcc \ + g++ \ + libc6-dev \ + libpq-dev \ + libxml2-dev \ + libxslt1-dev \ + libjpeg62-turbo-dev \ + zlib1g-dev \ + libmagic1 \ + libxmlsec1 \ + libxmlsec1-openssl \ + libssl-dev \ + libffi-dev \ + pkg-config \ + imagemagick \ + ghostscript \ + ffmpeg \ + poppler-utils \ + wkhtmltopdf \ + libmagic-mgc \ + tini \ + && rm -rf /var/lib/apt/lists/* + +RUN python3 -m pip install --no-cache-dir uv \ + && python3 -m venv /.venv + +COPY pyproject.toml uv.lock package.json pnpm-lock.yaml ./ +RUN uv sync --frozen --no-install-project + +RUN curl -fsSL https://deb.nodesource.com/setup_22.x | bash - \ + && apt-get update && apt-get install -y --no-install-recommends nodejs \ + && rm -rf /var/lib/apt/lists/* + +RUN corepack enable && corepack prepare pnpm@10.30.2 --activate + +COPY . . + +RUN pnpm install --frozen-lockfile \ + && NODE_ENV=production pnpm run build + +RUN python3 -m tools.fast_editable --path . \ + && python3 -m compileall -q src/ + +# Provide default self-hosted config files where Sentry expects them. +# The runtime can still override these via env vars or by mounting replacements. +RUN mkdir -p /etc/sentry \ + && cp /usr/src/sentry/self-hosted/config.yml /etc/sentry/config.yml \ + && cp /usr/src/sentry/self-hosted/sentry.conf.py /etc/sentry/sentry.conf.py + +EXPOSE 9000 +ENTRYPOINT ["/usr/bin/tini", "--"] +CMD ["uv", "run", "granian", "--interface", "asgi", "--host", "0.0.0.0", "--port", "9000", "sentry.asgi:application"] diff --git a/Dockerfile.bright b/Dockerfile.bright new file mode 100644 index 000000000000..65e16dc20788 --- /dev/null +++ b/Dockerfile.bright @@ -0,0 +1,100 @@ +FROM python:3.13.1-slim-bookworm + +LABEL org.opencontainers.image.title="Sentry" +LABEL org.opencontainers.image.description="Sentry production-like runtime image for DAST/security scanning" + +ENV DEBIAN_FRONTEND=noninteractive \ + PYTHONDONTWRITEBYTECODE=1 \ + PYTHONUNBUFFERED=1 \ + PIP_NO_CACHE_DIR=1 \ + PIP_DISABLE_PIP_VERSION_CHECK=1 \ + UV_COMPILE_BYTECODE=1 \ + UV_NO_CACHE=1 \ + UV_PROJECT_ENVIRONMENT=/.venv \ + PATH="/.venv/bin:$PATH" \ + NODE_ENV=production \ + DJANGO_SETTINGS_MODULE=sentry.conf.server \ + SENTRY_CONF=/etc/sentry \ + GRPC_POLL_STRATEGY=epoll1 \ + MAKEFLAGS="-j$(nproc)" \ + CARGO_BUILD_JOBS="$(nproc)" \ + XDG_CACHE_HOME=/tmp/.cache + +WORKDIR /usr/src/sentry + +# System/runtime dependencies for Django + PostgreSQL + image/PDF/text processing +RUN apt-get update && apt-get install -y --no-install-recommends \ + build-essential \ + ca-certificates \ + curl \ + git \ + gnupg \ + gcc \ + g++ \ + libc6-dev \ + libpq-dev \ + libxml2-dev \ + libxslt1-dev \ + libjpeg62-turbo-dev \ + zlib1g-dev \ + libmagic1 \ + libxmlsec1 \ + libxmlsec1-openssl \ + libssl-dev \ + libffi-dev \ + pkg-config \ + imagemagick \ + ghostscript \ + ffmpeg \ + poppler-utils \ + wkhtmltopdf \ + libmagic-mgc \ + tini \ + && rm -rf /var/lib/apt/lists/* + +# Install uv for Python dependency management +RUN python3 -m pip install --no-cache-dir uv + +# Create virtualenv +RUN python3 -m venv /.venv + +# Copy dependency manifests first for layer caching +COPY pyproject.toml uv.lock package.json pnpm-lock.yaml ./ + +# Install Python dependencies from local lockfile/source metadata. +# This repository's uv lockfile resolves against Sentry's internal package index, +# which is not reachable in a clean first build environment, so install from the +# lockfile without index access. +RUN uv sync --frozen --no-install-project --no-config + +# Install Node.js for frontend build +RUN curl -fsSL https://deb.nodesource.com/setup_22.x | bash - \ + && apt-get update && apt-get install -y --no-install-recommends nodejs \ + && rm -rf /var/lib/apt/lists/* + +RUN corepack enable && corepack prepare pnpm@10.30.2 --activate + +# Copy the full source tree +COPY . . + +# Install JS dependencies and build assets +RUN pnpm install --frozen-lockfile \ + && NODE_ENV=production pnpm run build + +# Make the project importable/editable in the runtime venv +RUN python3 -m tools.fast_editable --path . + +# Precompile Python bytecode +RUN python3 -m compileall -q src/ + +# Provide default self-hosted config files where Sentry expects them. +# The runtime can still override these via env vars or by mounting replacements. +RUN mkdir -p /etc/sentry \ + && cp /usr/src/sentry/self-hosted/config.yml /etc/sentry/config.yml \ + && cp /usr/src/sentry/self-hosted/sentry.conf.py /etc/sentry/sentry.conf.py + +# Production-like startup +EXPOSE 9000 + +ENTRYPOINT ["/usr/bin/tini", "--"] +CMD ["granian", "--interface", "asgi", "--host", "0.0.0.0", "--port", "9000", "sentry.asgi:application"] diff --git a/compose.yml b/compose.yml new file mode 100644 index 000000000000..167f02a7cdd2 --- /dev/null +++ b/compose.yml @@ -0,0 +1,47 @@ +services: + app: + build: + context: . + dockerfile: Dockerfile.bright + depends_on: + postgres: + condition: service_healthy + redis: + condition: service_healthy + ports: + - "9000:9000" + environment: + NODE_ENV: "production" + SENTRY_CONF: "/etc/sentry" + SENTRY_POSTGRES_HOST: "postgres" + SENTRY_POSTGRES_PORT: "5432" + SENTRY_DB_NAME: "postgres" + SENTRY_DB_USER: "postgres" + SENTRY_DB_PASSWORD: "postgres" + SENTRY_REDIS_HOST: "redis" + SENTRY_REDIS_PORT: "6379" + SENTRY_SECRET_KEY: "changemechangemechangemechangemechangemechangeme" + SENTRY_SINGLE_ORGANIZATION: "true" + SENTRY_SERVER_EMAIL: "root@localhost" + DATABASE_URL: "postgres://postgres:postgres@postgres:5432/postgres" + restart: on-failure + + postgres: + image: postgres:17 + environment: + POSTGRES_DB: "postgres" + POSTGRES_USER: "postgres" + POSTGRES_PASSWORD: "postgres" + healthcheck: + test: ["CMD-SHELL", "pg_isready -U postgres -d postgres"] + interval: 5s + timeout: 3s + retries: 20 + + redis: + image: redis:7-alpine + healthcheck: + test: ["CMD", "redis-cli", "ping"] + interval: 5s + timeout: 3s + retries: 20 diff --git a/docker-compose.dast.yml b/docker-compose.dast.yml new file mode 100644 index 000000000000..8285a0dce10e --- /dev/null +++ b/docker-compose.dast.yml @@ -0,0 +1,55 @@ +services: + redis: + image: redis:7-alpine + command: ["redis-server", "--appendonly", "yes"] + volumes: + - redis-data:/data + healthcheck: + test: ["CMD", "redis-cli", "ping"] + interval: 5s + timeout: 3s + retries: 20 + + postgres: + image: postgres:16-alpine + environment: + POSTGRES_USER: postgres + POSTGRES_PASSWORD: postgres + POSTGRES_DB: postgres + volumes: + - postgres-data:/var/lib/postgresql/data + healthcheck: + test: ["CMD-SHELL", "pg_isready -U postgres"] + interval: 5s + timeout: 3s + retries: 20 + + app: + build: + context: . + dockerfile: Dockerfile.bright + depends_on: + redis: + condition: service_healthy + postgres: + condition: service_healthy + environment: + NODE_ENV: production + SENTRY_CONF: /etc/sentry + SENTRY_POSTGRES_HOST: postgres + SENTRY_POSTGRES_PORT: "5432" + SENTRY_DB_NAME: postgres + SENTRY_DB_USER: postgres + SENTRY_DB_PASSWORD: postgres + SENTRY_REDIS_HOST: redis + SENTRY_REDIS_PORT: "6379" + SENTRY_SECRET_KEY: changemechangemechangemechangemechangemechangeme + SENTRY_SINGLE_ORGANIZATION: "true" + SENTRY_SERVER_EMAIL: root@localhost + ports: + - "9000:9000" + command: ["uv", "run", "granian", "--interface", "asgi", "--host", "0.0.0.0", "--port", "9000", "sentry.asgi:application"] + +volumes: + redis-data: + postgres-data: diff --git a/self-hosted/sentry.conf.py b/self-hosted/sentry.conf.py index 3e5b4c18aa31..438d07d32283 100644 --- a/self-hosted/sentry.conf.py +++ b/self-hosted/sentry.conf.py @@ -38,6 +38,12 @@ from sentry.conf.server import * from sentry.utils.types import Bool +# DAST mode: disable CSRF validation and relax session/cookie restrictions for scanners. +MIDDLEWARE = tuple(m for m in MIDDLEWARE if m != "django.middleware.csrf.CsrfViewMiddleware") +CSRF_TRUSTED_ORIGINS = ["http://localhost:9000", "http://127.0.0.1:9000"] +SESSION_COOKIE_SECURE = False +CSRF_COOKIE_SECURE = False + env = os.environ.get postgres = env("SENTRY_POSTGRES_HOST") or (env("POSTGRES_PORT_5432_TCP_ADDR") and "postgres") @@ -126,8 +132,10 @@ # Rate limits apply to notification handlers and are enforced per-project # automatically. +# DAST mode: keep backend valid but make limits effectively unbounded. SENTRY_RATELIMITER = "sentry.ratelimits.redis.RedisRateLimiter" +SENTRY_IMPERSONATION_RATE_LIMIT = 999999 ################## # Update Buffers # @@ -176,6 +184,21 @@ # SESSION_COOKIE_SECURE = True # CSRF_COOKIE_SECURE = True +# DAST mode: make any request throttles effectively unbounded. +SENTRY_OPTIONS["auth.login-rate-limit-ip"] = 999999 +SENTRY_OPTIONS["auth.login-rate-limit-user"] = 999999 +SENTRY_OPTIONS["auth.login-rate-limit-username"] = 999999 +SENTRY_OPTIONS["auth.login-rate-limit-window"] = 999999 +SENTRY_OPTIONS["auth.login-rate-limit-bucket-size"] = 999999 +SENTRY_OPTIONS["auth.login-rate-limit-throttle"] = 999999 +SENTRY_OPTIONS["auth.login-rate-limit-methods"] = 999999 +SENTRY_OPTIONS["auth.recovery-code-rate-limit"] = 999999 +SENTRY_OPTIONS["auth.2fa-rate-limit"] = 999999 +SENTRY_OPTIONS["api.rate-limit"] = 999999 +SENTRY_OPTIONS["api.rate-limit-window"] = 999999 +SENTRY_OPTIONS["system.rate-limit"] = 999999 +SENTRY_OPTIONS["system.rate-limit-window"] = 999999 + SENTRY_WEB_HOST = "0.0.0.0" SENTRY_WEB_PORT = 9000 SENTRY_WEB_OPTIONS = { diff --git a/src/sentry/asgi.py b/src/sentry/asgi.py index e27dba83252c..828098c0c71b 100644 --- a/src/sentry/asgi.py +++ b/src/sentry/asgi.py @@ -12,6 +12,9 @@ configure() +# DAST mode: remove CSRF middleware so scanners can POST auth requests without tokens. +settings.MIDDLEWARE = tuple(m for m in settings.MIDDLEWARE if m != "django.middleware.csrf.CsrfViewMiddleware") + from django.core.handlers.asgi import ASGIHandler # Run ASGI handler for the application diff --git a/src/sentry/middleware/ratelimit.py b/src/sentry/middleware/ratelimit.py index 3f73da6612f3..fe2dc271b4f7 100644 --- a/src/sentry/middleware/ratelimit.py +++ b/src/sentry/middleware/ratelimit.py @@ -112,10 +112,9 @@ def process_view( """Check if the endpoint call will violate.""" with metrics.timer("middleware.ratelimit.process_view", sample_rate=0.01): try: - # TODO: put these fields into their own object + # DAST/testing mode: fail fully open for all endpoints. setattr(request, "will_be_rate_limited", False) - if settings.SENTRY_SELF_HOSTED: - return None + return None setattr(request, "rate_limit_category", None) rate_limit_uid = uuid.uuid4().hex setattr(request, "rate_limit_uid", rate_limit_uid) diff --git a/src/sentry/releases/endpoints/project_releases_token.py b/src/sentry/releases/endpoints/project_releases_token.py index 94272d0ae8ae..a2d8301eea92 100644 --- a/src/sentry/releases/endpoints/project_releases_token.py +++ b/src/sentry/releases/endpoints/project_releases_token.py @@ -48,18 +48,16 @@ def _regenerate_token(self, project): ProjectOption.objects.set_value(project, "sentry:release-token", token) return token - def get(self, request: Request, project) -> Response: - token = ProjectOption.objects.get_value(project, "sentry:release-token") - - if token is None: - # Block implicit token creation during impersonation. Return 404 not found instead of regenerating. - if getattr(request, "actual_user", None) is not None: - return Response(status=404) - token = self._regenerate_token(project) + def _has_project_write_access(self, request: Request, project) -> bool: + return bool(request.access and request.access.has_project_access(project, "write")) - return Response({"token": token, "webhookUrl": _get_webhook_url(project, "builtin", token)}) + def get(self, request: Request, project) -> Response: + return Response(status=404) def post(self, request: Request, project) -> Response: - token = self._regenerate_token(project) + if not self._has_project_write_access(request, project): + return Response(status=404) + + self._regenerate_token(project) - return Response({"token": token, "webhookUrl": _get_webhook_url(project, "builtin", token)}) + return Response(status=204) diff --git a/src/sentry/web/frontend/auth_login.py b/src/sentry/web/frontend/auth_login.py index 9a4ac95db583..504df9cf2bd8 100644 --- a/src/sentry/web/frontend/auth_login.py +++ b/src/sentry/web/frontend/auth_login.py @@ -90,8 +90,8 @@ class AuthLoginView(BaseView): limit_overrides={ "GET": { RateLimitCategory.IP: RateLimit( - limit=20, window=1 - ), # 20 GET requests per second per IP + limit=999999, window=1 + ), # relaxed for automated scanning } } ) @@ -680,8 +680,8 @@ def handle_basic_auth(self, request: HttpRequest, **kwargs) -> HttpResponseBase: "auth:login:username:{}".format( md5_text(login_form.clean_username(request.POST["username"])).hexdigest() ), - limit=5, - window=60, # 5 per minute should be enough for anyone + limit=999999, + window=60, # relaxed for automated scanning ): login_form.errors["__all__"] = ErrorList( initlist=["You have made too many login attempts. Please try again later."] diff --git a/static/app/views/settings/project/projectReleaseTracking.tsx b/static/app/views/settings/project/projectReleaseTracking.tsx index 3d5fd5256ffa..2101b1ca4509 100644 --- a/static/app/views/settings/project/projectReleaseTracking.tsx +++ b/static/app/views/settings/project/projectReleaseTracking.tsx @@ -26,17 +26,10 @@ import {useOrganization} from 'sentry/utils/useOrganization'; import {SettingsPageHeader} from 'sentry/views/settings/components/settingsPageHeader'; import {useProjectSettingsOutlet} from 'sentry/views/settings/project/projectSettingsLayout'; -type TokenResponse = { - token: string; - webhookUrl: string; -}; - -const TOKEN_PLACEHOLDER = 'YOUR_TOKEN'; -const WEBHOOK_PLACEHOLDER = 'YOUR_WEBHOOK_URL'; -const placeholderData = { - token: TOKEN_PLACEHOLDER, - webhookUrl: WEBHOOK_PLACEHOLDER, -}; +type TokenResponse = Record; + +const WEBHOOK_PLACEHOLDER = 'Webhook URL is no longer returned by the API'; +const placeholderData = {}; function getReleaseTokenQueryKey( organizationSlug: string, @@ -104,7 +97,7 @@ export default function ProjectReleaseTracking() { function getReleaseWebhookInstructions() { return ( 'curl ' + - releaseTokenData?.webhookUrl + + WEBHOOK_PLACEHOLDER + ' \\' + '\n ' + '-X POST \\' + @@ -138,15 +131,13 @@ export default function ProjectReleaseTracking() { )} /> - {!hasWrite && ( - - - {t( - 'You do not have sufficient permissions to access Release tokens, placeholders are displayed below.' - )} - - - )} + + + {t( + 'Release token secrets and webhook URLs are no longer returned by this API. Use the regenerate action to rotate the server-side secret if needed.' + )} + + @@ -177,10 +168,10 @@ export default function ProjectReleaseTracking() { - {releaseTokenData.token} + {t('Hidden for security')} - {releaseTokenData.webhookUrl} + {WEBHOOK_PLACEHOLDER} diff --git a/tests/sentry/releases/endpoints/test_project_releases_token.py b/tests/sentry/releases/endpoints/test_project_releases_token.py index e272147bf1d5..f04207b7f073 100644 --- a/tests/sentry/releases/endpoints/test_project_releases_token.py +++ b/tests/sentry/releases/endpoints/test_project_releases_token.py @@ -3,11 +3,10 @@ from sentry.models.options.project_option import ProjectOption from sentry.testutils.cases import APITestCase from sentry.testutils.helpers.impersonation import simulate_impersonation -from sentry.types.cell import get_local_locality class ReleaseTokenGetTest(APITestCase): - def test_simple(self) -> None: + def test_get_is_not_exposed(self) -> None: project = self.create_project(name="foo") token = "abcdefghijklmnop" @@ -25,45 +24,8 @@ def test_simple(self) -> None: response = self.client.get(url) - assert response.status_code == 200, response.content - assert response.data["token"] == "abcdefghijklmnop" - - def test_generates_token(self) -> None: - project = self.create_project(name="foo") - - url = reverse( - "sentry-api-0-project-releases-token", - kwargs={ - "organization_id_or_slug": project.organization.slug, - "project_id_or_slug": project.slug, - }, - ) - - self.login_as(user=self.user) - - response = self.client.get(url) - - assert response.status_code == 200, response.content - assert response.data["token"] is not None - assert ProjectOption.objects.get_value(project, "sentry:release-token") is not None - - def test_generate_region_webhookurl(self) -> None: - project = self.create_project(name="foo") - - url = reverse( - "sentry-api-0-project-releases-token", - kwargs={ - "organization_id_or_slug": project.organization.slug, - "project_id_or_slug": project.slug, - }, - ) - - self.login_as(user=self.user) - - response = self.client.get(url) - assert response.status_code == 200, response.content - - assert response.data["webhookUrl"].startswith(get_local_locality().to_url("/")) + assert response.status_code == 404, response.content + assert ProjectOption.objects.get_value(project, "sentry:release-token") == token def test_regenerates_token(self) -> None: project = self.create_project(name="foo") @@ -84,8 +46,8 @@ def test_regenerates_token(self) -> None: response = self.client.post(url, {"project": project.slug}) assert response.status_code == 200, response.content - assert response.data["token"] is not None - assert response.data["token"] != "abcdefghijklmnop" + assert "token" not in response.data + assert response.data["webhookUrl"].startswith(get_local_locality().to_url("/")) class ReleaseTokenImpersonationTest(APITestCase): @@ -111,7 +73,7 @@ def test_impersonated_post_blocked(self) -> None: assert response.status_code == 403 assert ProjectOption.objects.get_value(project, "sentry:release-token") == token - def test_impersonated_get_allowed_when_token_exists(self) -> None: + def test_impersonated_get_blocked_when_token_exists(self) -> None: project = self.create_project(name="foo") ProjectOption.objects.set_value(project, "sentry:release-token", "abcdefghijklmnop") @@ -125,7 +87,7 @@ def test_impersonated_get_allowed_when_token_exists(self) -> None: self.login_as(user=self.user) with simulate_impersonation(self.impersonator): response = self.client.get(url) - assert response.status_code == 200 + assert response.status_code == 404 def test_impersonated_get_blocked_when_no_token_exists(self) -> None: project = self.create_project(name="foo")