HBASE-29838 Run Hadoop Check as a GitHub Action #7651

ndimiduk · 2026-01-20T07:05:05Z

No description provided.

ndimiduk · 2026-01-20T14:09:02Z

Looks like we occupied the runner for 6h and then it was aborted.

Apache9 · 2026-01-20T16:27:28Z

https://infra.apache.org/github-actions-policy.html

The policy here does not say about the 6 hours timeout...

We can ask infra about the rules and the size of the github runners, our jenkins runners finished in "343m 23s", which was very close to 6 hours, so if the machine of the github runner is weaker, the build will be very easy to cost more than 6 hours...

ndimiduk · 2026-01-20T16:35:32Z

My action itself has timeout-minutes: 600 . i'll try bumping this up.

ndimiduk · 2026-01-20T16:38:31Z

or not. 6h is GH's hard limit, https://docs.github.com/en/actions/reference/limits

Apache9 · 2026-01-21T01:52:22Z

Then maybe we should try self hosted github runners? For self hosted runners the execution time limit is 5 days...

ndimiduk · 2026-01-23T07:07:12Z

Then maybe we should try self hosted github runners? For self hosted runners the execution time limit is 5 days...

Yes we should bring this back to our CI discussions with Infra. Maybe we can borrow from the pool of new Jenkins workers while we continue to build this out.

Yetus is supposed to provide smart, selective detection of module changes when it decided which tests to run. I think the new .github directory broke that for this run, so I've pushed a change to exclude it, maybe that will help.

I'm also going to see if I can manually parallelize the unit test runs -- maybe break out three separate checks for the three main unit test groups or something like that.

Apache9 · 2026-01-23T09:05:27Z

Then maybe we should try self hosted github runners? For self hosted runners the execution time limit is 5 days...

Yes we should bring this back to our CI discussions with Infra. Maybe we can borrow from the pool of new Jenkins workers while we continue to build this out.

Yetus is supposed to provide smart, selective detection of module changes when it decided which tests to run. I think the new .github directory broke that for this run, so I've pushed a change to exclude it, maybe that will help.

I'm also going to see if I can manually parallelize the unit test runs -- maybe break out three separate checks for the three main unit test groups or something like that.

Maybe we could split the UTs run as a seperated github check? Or even more, we could split the UTs run as two seperated check, one has -PrunDevTests(for small/medium tests) and one has -PrunLargeTests(for large tests).

ndimiduk · 2026-01-23T12:37:28Z

Maybe we could split the UTs run as a seperated github check? Or even more, we could split the UTs run as two seperated check, one has -PrunDevTests(for small/medium tests) and one has -PrunLargeTests(for large tests).

Yep, that's exactly my thinking as well. Landing these other cleanup issues and I'll be back.

Apache-HBase · 2026-01-23T13:02:10Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7651/4/console in case of problems.

Apache-HBase · 2026-01-23T13:02:22Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7651/4/console in case of problems.

ndimiduk · 2026-01-23T13:20:52Z

Okay this is better. Module selection chose only hbase-examples for running the unit tests.

-1 overall
  
  | Vote |      Subsystem |  Runtime   | Comment
  ============================================================================
  |   0  |        reexec  |   1m 13s   | Docker mode activated. 
  |  -0  |         yetus  |   0m  4s   | Unprocessed flag(s): --brief-report-file 
  |      |                |            | --spotbugs-strict-precheck
  |      |                |            | --author-ignore-list
  |      |                |            | --blanks-eol-ignore-file
  |      |                |            | --blanks-tabs-ignore-file
  |      |                |            | --quick-hadoopcheck
  +---------------------------------------------------------------------------
  |      |                |            | Prechecks 
  +---------------------------------------------------------------------------
  +---------------------------------------------------------------------------
  |      |                |            | master Compile Tests 
  +---------------------------------------------------------------------------
  |  +1  |    mvninstall  |   3m 25s   | master passed 
  |  +1  |       compile  |   0m 19s   | master passed 
  |  +1  |       javadoc  |   0m 17s   | master passed 
  |  +1  |    shadedjars  |   4m 47s   | branch has no errors when building our 
  |      |                |            | shaded downstream artifacts.
  +---------------------------------------------------------------------------
  |      |                |            | Patch Compile Tests 
  +---------------------------------------------------------------------------
  |  +1  |    mvninstall  |   3m  7s   | the patch passed 
  |  +1  |       compile  |   0m 18s   | the patch passed 
  |  +1  |         javac  |   0m 18s   | the patch passed 
  |  +1  |       javadoc  |   0m 15s   | the patch passed 
  |  +1  |    shadedjars  |   4m 51s   | patch has no errors when building our 
  |      |                |            | shaded downstream artifacts.
  +---------------------------------------------------------------------------
  |      |                |            | Other Tests 
  +---------------------------------------------------------------------------
  |  -1  |          unit  |   0m 24s   | hbase-examples in the patch failed. 
  |      |                |  19m 12s   |
  
  
  || Subsystem || Report/Notes ||
  ============================================================================
  |  unit  | /home/runner/work/hbase/hbase/yetus-jdk17-hadoop3-check/output/patch-unit-hbase-examples.txt |
  | Docker | ClientAPI=1.48 ServerAPI=1.48 base: /home/runner/work/hbase/hbase/yetus-jdk17-hadoop3-check/output/Dockerfile |
  | GITHUB PR | https://github.com/apache/hbase/pull/7651 |
  | Optional Tests | javac javadoc unit compile shadedjars |
  | uname | Linux 9b6b1363e88a 6.11.0-1018-azure #18~24.04.1-Ubuntu SMP Sat Jun 28 04:46:03 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux |
  | Build tool | maven |
  | Personality | dev-support/hbase-personality.sh |
  | git revision | master / adfd0560c941109283928946bd671a5836604e46 |
  | Default Java | Eclipse Adoptium-17.0.11+9 |
  | Max. process+thread count | 72 (vs. ulimit of 30000) |
  | modules | C: hbase-examples U: hbase-examples |
  | versions | git=2.34.1 maven=3.9.8 |
  | Powered by | Apache Yetus 0.15.0 https://yetus.apache.org/ |

Apache-HBase · 2026-01-23T13:30:29Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 29s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+0 🆗	yamllint	0m 0s		yamllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ master Compile Tests _
+1 💚	mvninstall	2m 59s		master passed
+1 💚	compile	0m 26s		master passed
+1 💚	checkstyle	0m 10s		master passed
+1 💚	spotbugs	0m 29s		master passed
+1 💚	spotless	0m 44s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+1 💚	mvninstall	2m 57s		the patch passed
+1 💚	compile	0m 26s		the patch passed
+1 💚	javac	0m 26s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 10s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	spotbugs	0m 36s		the patch passed
+1 💚	hadoopcheck	11m 23s		Patch does not cause any errors with Hadoop 3.3.6 3.4.1.
+1 💚	spotless	0m 45s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 💚	asflicense	0m 9s		The patch does not generate ASF License warnings.
		29m 2s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7651/4/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#7651
Optional Tests	dupname asflicense codespell detsecrets shellcheck shelldocs spotless yamllint javac spotbugs checkstyle compile hadoopcheck hbaseanti
uname	Linux 8dc126e0a92b 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `adfd056`
Default Java	Eclipse Adoptium-17.0.11+9
Max. process+thread count	87 (vs. ulimit of 30000)
modules	C: hbase-examples U: hbase-examples
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7651/4/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3 shellcheck=0.8.0
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2026-01-23T13:35:39Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 50s		Docker mode activated.
-0 ⚠️	yetus	0m 3s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
+1 💚	mvninstall	5m 5s		master passed
+1 💚	compile	0m 39s		master passed
+1 💚	javadoc	0m 31s		master passed
+1 💚	shadedjars	8m 58s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	4m 30s		the patch passed
+1 💚	compile	0m 32s		the patch passed
+1 💚	javac	0m 32s		the patch passed
+1 💚	javadoc	0m 22s		the patch passed
+1 💚	shadedjars	10m 3s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
-1 ❌	unit	1m 11s	/patch-unit-hbase-examples.txt	hbase-examples in the patch failed.
		34m 4s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7651/4/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#7651
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux b987b1104538 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `adfd056`
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7651/4/testReport/
Max. process+thread count	86 (vs. ulimit of 30000)
modules	C: hbase-examples U: hbase-examples
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7651/4/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

ndimiduk · 2026-01-23T15:51:15Z

Looking over out last successful nightly on master, the Large tests on hbase-server still took 7h. I'm looking for other ways to partition this up.

Apache-HBase · 2026-01-23T16:48:40Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7651/5/console in case of problems.

Apache-HBase · 2026-01-23T16:48:41Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7651/5/console in case of problems.

Apache-HBase · 2026-01-23T17:33:46Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 31s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+0 🆗	yamllint	0m 0s		yamllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
			_ master Compile Tests _
+1 💚	mvninstall	3m 28s		master passed
+1 💚	compile	8m 42s		master passed
+1 💚	spotless	0m 43s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+1 💚	mvninstall	2m 57s		the patch passed
+1 💚	compile	8m 38s		the patch passed
+1 💚	javac	8m 38s		the patch passed
+1 💚	blanks	0m 1s		The patch has no blanks issues.
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	xmllint	0m 1s		No new issues.
+1 💚	hadoopcheck	11m 36s		Patch does not cause any errors with Hadoop 3.3.6 3.4.1.
+1 💚	spotless	0m 58s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 💚	asflicense	0m 15s		The patch does not generate ASF License warnings.
		45m 54s

Subsystem	Report/Notes
Docker	ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7651/5/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#7651
Optional Tests	dupname asflicense codespell detsecrets shellcheck shelldocs spotless yamllint javac xmllint hadoopcheck compile
uname	Linux 93830e5ec19a 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `ad51411`
Default Java	Eclipse Adoptium-17.0.11+9
Max. process+thread count	195 (vs. ulimit of 30000)
modules	C: . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7651/5/console
versions	git=2.34.1 maven=3.9.8 shellcheck=0.8.0 xmllint=20913
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2026-01-23T21:09:18Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 31s		Docker mode activated.
-0 ⚠️	yetus	0m 3s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
+1 💚	mvninstall	3m 32s		master passed
+1 💚	compile	2m 24s		master passed
+1 💚	javadoc	2m 9s		master passed
+1 💚	shadedjars	5m 45s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	2m 57s		the patch passed
+1 💚	compile	2m 27s		the patch passed
+1 💚	javac	2m 27s		the patch passed
+1 💚	javadoc	1m 58s		the patch passed
+1 💚	shadedjars	5m 34s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
-1 ❌	unit	228m 26s	/patch-unit-root.txt	root in the patch failed.
		261m 28s

Subsystem	Report/Notes
Docker	ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7651/5/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#7651
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 186ed56d4904 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `ad51411`
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7651/5/testReport/
Max. process+thread count	5474 (vs. ulimit of 30000)
modules	C: . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7651/5/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

This comment has been minimized.

Sign in to view

ndimiduk force-pushed the 29838-jdk-checks-master branch from 1829212 to 352a9a1 Compare January 23, 2026 07:03

This comment has been minimized.

Sign in to view

HBASE-29838 Run Hadoop Check as a GitHub Action

a199ad1

ndimiduk force-pushed the 29838-jdk-checks-master branch from 352a9a1 to adfd056 Compare January 23, 2026 12:48

try something crazy

0c39346

force test whole project

ad51411

ndimiduk force-pushed the 29838-jdk-checks-master branch from adfd056 to ad51411 Compare January 23, 2026 16:28

HBASE-29838 Run Hadoop Check as a GitHub Action #7651

Are you sure you want to change the base?

HBASE-29838 Run Hadoop Check as a GitHub Action #7651

Conversation

ndimiduk commented Jan 20, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

ndimiduk commented Jan 20, 2026

Uh oh!

Apache9 commented Jan 20, 2026

Uh oh!

ndimiduk commented Jan 20, 2026

Uh oh!

ndimiduk commented Jan 20, 2026

Uh oh!

Apache9 commented Jan 21, 2026

Uh oh!

ndimiduk commented Jan 23, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Apache9 commented Jan 23, 2026

Uh oh!

ndimiduk commented Jan 23, 2026

Uh oh!

Apache-HBase commented Jan 23, 2026

Uh oh!

Apache-HBase commented Jan 23, 2026

Uh oh!

ndimiduk commented Jan 23, 2026

Uh oh!

Apache-HBase commented Jan 23, 2026

Uh oh!

Apache-HBase commented Jan 23, 2026

Uh oh!

ndimiduk commented Jan 23, 2026

Uh oh!

Apache-HBase commented Jan 23, 2026

Uh oh!

Apache-HBase commented Jan 23, 2026

Uh oh!

Apache-HBase commented Jan 23, 2026

Uh oh!

Apache-HBase commented Jan 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants