update MC2DP prod to align with QA by tianj7 · Pull Request #2137 · uc-cdis/gen3-gitops

tianj7 · 2026-09-25T15:40:46Z

https://ctds-planx.atlassian.net/browse/MC2DP-1134

Environments

MC2DP Prod

Description of changes

workspace-proxy.jegKernelSpecPolicy.display name added
hatchery image tag from 2.9.2 to 2.11.0, squashfs_mount disabled
hatchery ecs-ws-sidecar image tag from 2026.03 to 2026.07
revproxy nginx location block for /api/workspace-assets/remote/lmod added
fence image tag updated from 13.3.0 to 13.3.2

github-actions · 2026-09-25T16:04:03Z

Integration Tests

Test summary after running integration tests

filepath	passed	failed	error	skipped	SUBTOTAL
tests/test_gen3_workflow.py	0	0	0	31	31
tests/test_dicom_viewer.py	0	0	0	4	4
tests/test_pfb_export.py	0	0	0	1	1
tests/test_oauth2.py	0	0	0	15	15
tests/test_graph_submit_and_query.py	11	1	0	2	14
tests/test_audit_service.py	3	0	0	3	6
tests/test_ras_authn.py	0	0	0	3	3
tests/test_ras_passport.py	0	0	0	3	3
tests/test_data_upload.py	0	0	0	9	9
tests/test_drs_endpoint.py	4	0	0	21	25
tests/test_presigned_url.py	7	0	0	1	8
tests/test_dbgap.py	4	0	0	1	5
tests/test_user_data_library.py	0	3	3	1	7
tests/test_study_viewer.py	0	0	0	3	3
tests/test_user_login_activation.py	2	0	0	1	3
tests/test_discoverypage.py	0	0	0	1	1
tests/test_gen3ff_landing_page.py	0	0	0	2	2
tests/test_register_user.py	0	0	0	2	2
tests/test_aggregate_mds.py	0	0	0	1	1
tests/test_google_data_access.py	0	0	0	1	1
tests/test_login_page.py	1	0	0	1	2
tests/test_etl.py	0	0	0	1	1
tests/test_gen3client.py	0	0	0	1	1
tests/test_env_sanity.py	0	0	0	1	1
tests/test_gen3_sdk.py	0	0	0	1	1
tests/test_guppy_service.py	13	0	0	0	13
tests/test_centralized_auth.py	16	0	0	0	16
tests/test_requestor.py	6	0	0	0	6
tests/test_user_token.py	5	0	0	0	5
tests/test_fence_admin.py	2	0	0	0	2
tests/test_oidc_client.py	2	0	0	0	2
tests/test_homepage.py	1	0	0	0	1
tests/test_client_credentials.py	1	0	0	0	1
tests/test_manifest_service.py	1	0	0	0	1
tests/test_workspace.py	0	1	0	0	1
TOTAL	79	5	3	111	198

Test summary after rerunning failed integration tests

filepath	passed	failed	error	SUBTOTAL
tests/test_user_data_library.py	0	3	3	6
tests/test_workspace.py	0	1	0	1
tests/test_graph_submit_and_query.py	1	0	0	1
TOTAL	1	4	3	8

Please find the detailed integration test report here

Please find the detailed integration test report after rerunning failed tests here

Please find the Github Action logs here

github-actions · 2026-09-25T16:04:21Z

Failure Analysis

Please find the detailed test analysis report here

… feat/MC2DP-1134

github-actions · 2026-09-25T16:45:09Z

Integration Tests

Test summary after running integration tests

filepath	passed	failed	error	skipped	SUBTOTAL
tests/test_gen3_workflow.py	0	0	0	31	31
tests/test_dicom_viewer.py	0	0	0	4	4
tests/test_pfb_export.py	0	0	0	1	1
tests/test_oauth2.py	0	0	0	15	15
tests/test_graph_submit_and_query.py	11	1	0	2	14
tests/test_audit_service.py	3	0	0	3	6
tests/test_ras_authn.py	0	0	0	3	3
tests/test_ras_passport.py	0	0	0	3	3
tests/test_data_upload.py	0	0	0	9	9
tests/test_drs_endpoint.py	4	0	0	21	25
tests/test_dbgap.py	4	0	0	1	5
tests/test_user_data_library.py	0	3	3	1	7
tests/test_study_viewer.py	0	0	0	3	3
tests/test_discoverypage.py	0	0	0	1	1
tests/test_gen3ff_landing_page.py	0	0	0	2	2
tests/test_user_login_activation.py	2	0	0	1	3
tests/test_register_user.py	0	0	0	2	2
tests/test_login_page.py	1	0	0	1	2
tests/test_google_data_access.py	0	0	0	1	1
tests/test_aggregate_mds.py	0	0	0	1	1
tests/test_etl.py	0	0	0	1	1
tests/test_gen3client.py	0	0	0	1	1
tests/test_env_sanity.py	0	0	0	1	1
tests/test_gen3_sdk.py	0	0	0	1	1
tests/test_guppy_service.py	13	0	0	0	13
tests/test_centralized_auth.py	16	0	0	0	16
tests/test_presigned_url.py	8	0	0	0	8
tests/test_requestor.py	6	0	0	0	6
tests/test_user_token.py	5	0	0	0	5
tests/test_fence_admin.py	2	0	0	0	2
tests/test_oidc_client.py	2	0	0	0	2
tests/test_homepage.py	1	0	0	0	1
tests/test_client_credentials.py	1	0	0	0	1
tests/test_manifest_service.py	1	0	0	0	1
tests/test_workspace.py	0	1	0	0	1
TOTAL	80	5	3	110	198

Test summary after rerunning failed integration tests

filepath	passed	failed	error	SUBTOTAL
tests/test_user_data_library.py	0	3	3	6
tests/test_workspace.py	0	1	0	1
tests/test_graph_submit_and_query.py	1	0	0	1
TOTAL	1	4	3	8

Please find the detailed integration test report here

Please find the detailed integration test report after rerunning failed tests here

Please find the Github Action logs here

github-actions · 2026-09-25T16:45:30Z

Failure Analysis

Please find the detailed test analysis report here

pieterlukasse · 2026-09-25T22:49:01Z

+      # workspace-assets/remote/lmod routing
+      location ^~ /api/workspace-assets/remote/lmod {
+        if ($request_method = 'OPTIONS') {
+          add_header Access-Control-Allow-Origin $http_origin always;
+          add_header Access-Control-Allow-Credentials true always;
+          add_header Access-Control-Allow-Methods "GET, POST, DELETE, OPTIONS" always;
+          add_header Access-Control-Allow-Headers "Authorization, Content-Type" always;
+          add_header Content-Length 0;
+          add_header Content-Type text/plain;
+          return 204;
+        }
+        set $authz_resource "/workspace";
+        set $authz_method "access";
+        set $authz_service "jupyterhub";
+        auth_request_set $remoteUser $upstream_http_REMOTE_USER;
+        auth_request_set $saved_set_cookie $upstream_http_set_cookie;
+        auth_request /gen3-authz;
+        if ($saved_set_cookie != "") {
+          add_header Set-Cookie $saved_set_cookie always;
+        }
+        add_header Cache-Control "no-store";
+        proxy_set_header REMOTE_USER $remoteUser;
+        error_page 403 = @errorworkspace;
+
+        set $upstream http://workspace-proxy-service.$namespace.svc.cluster.local:8080;
+        rewrite ^/api/workspace-assets/remote/(lmod.*) /$1 break;
+        proxy_pass $upstream;
+        proxy_http_version 1.1;
+        proxy_set_header Host $host;
+        proxy_set_header X-Real-IP $remote_addr;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto https;
+        proxy_set_header X-Forwarded-Host $host;
+        proxy_set_header X-URL-SCHEME https;
+        add_header Access-Control-Allow-Origin $http_origin always;
+        add_header Access-Control-Allow-Credentials true always;
+        add_header Access-Control-Allow-Methods "GET, POST, DELETE, OPTIONS" always;
+        add_header Access-Control-Allow-Headers "Authorization, Content-Type" always;
+        client_max_body_size 0;
+        proxy_read_timeout 600s;
+      }
+


this block seems generic and I'm wondering if it should move to gen3-helm instead.
CC @jawadqur

We should! This is tech debt we're accruing and adding to gen3-helm tested and easily configurable will be very necessary!

yup I think this configuration should go right around here once:
https://github.com/uc-cdis/gen3-helm/blob/master/helm/jeg/values.yaml

Also fun security issue here that I think is legitimate, thread in slack:
https://cdis.slack.com/files/U620MQQAK/F0C4YTES9J9/screenshot_2026-09-28_at_5.56.51___pm.png
if X-Gen3-User-ID header is modifiable then the revproxy->multiheaded-workpsace-proxy has a potential to shuffle.

pieterlukasse · 2026-09-25T23:13:27Z

related PR: #2116

… feat/MC2DP-1134

github-actions · 2026-09-28T18:51:49Z

Integration Tests

Failed to Prepare CI environment

Please find the Github Action logs here

github-actions · 2026-09-28T20:07:26Z

Integration Tests

Failed to Prepare CI environment

Please find the Github Action logs here

github-actions · 2026-09-28T20:40:58Z

Integration Tests

Test summary after running integration tests

filepath	passed	failed	error	skipped	SUBTOTAL
tests/test_gen3_workflow.py	0	0	0	31	31
tests/test_dicom_viewer.py	0	0	0	4	4
tests/test_pfb_export.py	0	0	0	1	1
tests/test_oauth2.py	0	0	0	15	15
tests/test_graph_submit_and_query.py	11	1	0	2	14
tests/test_audit_service.py	3	0	0	3	6
tests/test_ras_authn.py	0	0	0	3	3
tests/test_ras_passport.py	0	0	0	3	3
tests/test_data_upload.py	0	0	0	9	9
tests/test_drs_endpoint.py	4	0	0	21	25
tests/test_dbgap.py	4	0	0	1	5
tests/test_user_data_library.py	0	3	3	1	7
tests/test_study_viewer.py	0	0	0	3	3
tests/test_discoverypage.py	0	0	0	1	1
tests/test_user_login_activation.py	2	0	0	1	3
tests/test_gen3ff_landing_page.py	0	0	0	2	2
tests/test_login_page.py	1	0	0	1	2
tests/test_register_user.py	0	0	0	2	2
tests/test_google_data_access.py	0	0	0	1	1
tests/test_aggregate_mds.py	0	0	0	1	1
tests/test_gen3_sdk.py	0	0	0	1	1
tests/test_gen3client.py	0	0	0	1	1
tests/test_env_sanity.py	0	0	0	1	1
tests/test_etl.py	0	0	0	1	1
tests/test_guppy_service.py	13	0	0	0	13
tests/test_centralized_auth.py	16	0	0	0	16
tests/test_presigned_url.py	8	0	0	0	8
tests/test_requestor.py	6	0	0	0	6
tests/test_user_token.py	5	0	0	0	5
tests/test_fence_admin.py	2	0	0	0	2
tests/test_oidc_client.py	2	0	0	0	2
tests/test_client_credentials.py	1	0	0	0	1
tests/test_homepage.py	1	0	0	0	1
tests/test_manifest_service.py	1	0	0	0	1
tests/test_workspace.py	0	1	0	0	1
TOTAL	80	5	3	110	198

Test summary after rerunning failed integration tests

filepath	passed	failed	error	SUBTOTAL
tests/test_user_data_library.py	0	3	3	6
tests/test_workspace.py	0	1	0	1
tests/test_graph_submit_and_query.py	1	0	0	1
TOTAL	1	4	3	8

Please find the detailed integration test report here

Please find the detailed integration test report after rerunning failed tests here

Please find the Github Action logs here

github-actions · 2026-09-28T20:41:14Z

Failure Analysis

Please find the detailed test analysis report here

github-actions · 2026-10-01T18:15:39Z

Integration Tests

Test summary after running integration tests

filepath	passed	failed	error	skipped	SUBTOTAL
tests/test_gen3_workflow.py	0	0	0	31	31
tests/test_dicom_viewer.py	0	0	0	4	4
tests/test_pfb_export.py	0	0	0	1	1
tests/test_oauth2.py	0	0	0	15	15
tests/test_graph_submit_and_query.py	11	1	0	2	14
tests/test_audit_service.py	3	0	0	3	6
tests/test_ras_authn.py	0	0	0	3	3
tests/test_ras_passport.py	0	0	0	3	3
tests/test_data_upload.py	0	0	0	9	9
tests/test_drs_endpoint.py	4	0	0	21	25
tests/test_dbgap.py	3	1	0	1	5
tests/test_user_data_library.py	0	3	3	1	7
tests/test_study_viewer.py	0	0	0	3	3
tests/test_user_login_activation.py	2	0	0	1	3
tests/test_discoverypage.py	0	0	0	1	1
tests/test_workspace.py	0	0	0	1	1
tests/test_gen3ff_landing_page.py	0	0	0	2	2
tests/test_login_page.py	1	0	0	1	2
tests/test_register_user.py	0	0	0	2	2
tests/test_aggregate_mds.py	0	0	0	1	1
tests/test_google_data_access.py	0	0	0	1	1
tests/test_env_sanity.py	0	0	0	1	1
tests/test_etl.py	0	0	0	1	1
tests/test_gen3_sdk.py	0	0	0	1	1
tests/test_gen3client.py	0	0	0	1	1
tests/test_guppy_service.py	13	0	0	0	13
tests/test_centralized_auth.py	16	0	0	0	16
tests/test_presigned_url.py	8	0	0	0	8
tests/test_requestor.py	6	0	0	0	6
tests/test_user_token.py	5	0	0	0	5
tests/test_fence_admin.py	2	0	0	0	2
tests/test_oidc_client.py	2	0	0	0	2
tests/test_homepage.py	1	0	0	0	1
tests/test_client_credentials.py	1	0	0	0	1
tests/test_manifest_service.py	1	0	0	0	1
TOTAL	79	5	3	111	198

Test summary after rerunning failed integration tests

filepath	passed	failed	error	SUBTOTAL
tests/test_user_data_library.py	0	3	3	6
tests/test_dbgap.py	0	1	0	1
tests/test_graph_submit_and_query.py	1	0	0	1
TOTAL	1	4	3	8

Please find the detailed integration test report here

Please find the detailed integration test report after rerunning failed tests here

Please find the Github Action logs here

github-actions · 2026-10-01T18:15:53Z

Failure Analysis

Please find the detailed test analysis report here

github-actions · 2026-10-02T14:56:41Z

Integration Tests

Test summary after running integration tests

filepath	failed	error	skipped	SUBTOTAL
tests/test_workspace.py	0	0	1	1
tests/test_user_data_library.py	3	3	1	7
TOTAL	3	3	2	8

Test summary after rerunning failed integration tests

filepath	failed	error	SUBTOTAL
tests/test_user_data_library.py	3	3	6
TOTAL	3	3	6

Please find the detailed integration test report here

Please find the detailed integration test report after rerunning failed tests here

Please find the Github Action logs here

github-actions · 2026-10-02T14:56:57Z

Failure Analysis

Please find the detailed test analysis report here

github-actions · 2026-10-02T19:32:13Z

Integration Tests

Test summary after running integration tests

filepath	failed	error	skipped	SUBTOTAL
tests/test_workspace.py	0	0	1	1
tests/test_user_data_library.py	3	3	1	7
TOTAL	3	3	2	8

Test summary after rerunning failed integration tests

filepath	failed	error	SUBTOTAL
tests/test_user_data_library.py	3	3	6
TOTAL	3	3	6

Please find the detailed integration test report here

Please find the detailed integration test report after rerunning failed tests here

Please find the Github Action logs here

github-actions · 2026-10-02T19:32:29Z

Failure Analysis

Please find the detailed test analysis report here

github-actions · 2026-10-06T18:28:31Z

Integration Tests

Test summary after running integration tests

filepath	failed	error	skipped	SUBTOTAL
tests/test_workspace.py	0	0	1	1
tests/test_user_data_library.py	3	3	1	7
TOTAL	3	3	2	8

Test summary after rerunning failed integration tests

filepath	failed	error	SUBTOTAL
tests/test_user_data_library.py	3	3	6
TOTAL	3	3	6

Please find the detailed integration test report here

Please find the detailed integration test report after rerunning failed tests here

Please find the Github Action logs here

github-actions · 2026-10-06T18:28:49Z

Failure Analysis

Please find the detailed test analysis report here

update MC2DP prod to align with QA

08edc23

tianj7 requested a review from nmetokishlubsky as a code owner September 25, 2026 15:40

Merge branch 'master' into feat/MC2DP-1134

e1efe2b

tianj7 added 2 commits September 25, 2026 11:22

update MC2DP prod fence and hatchery tags

7406bdf

Merge branch 'feat/MC2DP-1134' of github.com:uc-cdis/gen3-gitops into…

6ea9d5a

… feat/MC2DP-1134

pieterlukasse self-requested a review September 25, 2026 20:13

pieterlukasse reviewed Sep 25, 2026

View reviewed changes

Shawn O'Connor and others added 4 commits September 28, 2026 13:27

VPODC FF 0.12.79

18fa243

VPODC FF 0.12.80

f7de391

remove unneeded config

94762c2

Merge branch 'feat/MC2DP-1134' of github.com:uc-cdis/gen3-gitops into…

d57b4bf

… feat/MC2DP-1134

jawadqur previously approved these changes Sep 28, 2026

View reviewed changes

Merge branch 'master' into feat/MC2DP-1134

4c981b1

Merge branch 'master' into feat/MC2DP-1134

e706a25

tianj7 dismissed jawadqur’s stale review via e706a25 October 1, 2026 17:51

PlanXCyborg added TestWorkspacePage TestUserDataLibrary labels Oct 2, 2026

Merge branch 'master' into feat/MC2DP-1134

cbdb092

Merge branch 'master' into feat/MC2DP-1134

3d87862

Conversation

tianj7 commented Sep 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Environments

Description of changes

Uh oh!

github-actions Bot commented Sep 25, 2026

Integration Tests

Uh oh!

github-actions Bot commented Sep 25, 2026

Failure Analysis

Uh oh!

github-actions Bot commented Sep 25, 2026

Integration Tests

Uh oh!

github-actions Bot commented Sep 25, 2026

Failure Analysis

Uh oh!

pieterlukasse Sep 25, 2026

Choose a reason for hiding this comment

Uh oh!

jawadqur Sep 28, 2026

Choose a reason for hiding this comment

Uh oh!

jbarno Sep 29, 2026

Choose a reason for hiding this comment

Uh oh!

jbarno Sep 29, 2026

Choose a reason for hiding this comment

Uh oh!

pieterlukasse commented Sep 25, 2026

Uh oh!

github-actions Bot commented Sep 28, 2026

Integration Tests

Uh oh!

github-actions Bot commented Sep 28, 2026

Integration Tests

Uh oh!

github-actions Bot commented Sep 28, 2026

Integration Tests

Uh oh!

github-actions Bot commented Sep 28, 2026

Failure Analysis

Uh oh!

github-actions Bot commented Oct 1, 2026

Integration Tests

Uh oh!

github-actions Bot commented Oct 1, 2026

Failure Analysis

Uh oh!

github-actions Bot commented Oct 2, 2026

Integration Tests

Uh oh!

github-actions Bot commented Oct 2, 2026

Failure Analysis

Uh oh!

github-actions Bot commented Oct 2, 2026

Integration Tests

Uh oh!

github-actions Bot commented Oct 2, 2026

Failure Analysis

Uh oh!

github-actions Bot commented Oct 6, 2026

Integration Tests

Uh oh!

github-actions Bot commented Oct 6, 2026

Failure Analysis

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

tianj7 commented Sep 25, 2026 •

edited

Loading