TEZ-4617 prevent resource leaks for Closeables #402

dk2k · 2025-04-01T07:54:09Z

No description provided.

tez-yetus · 2025-04-01T09:11:16Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	13m 49s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ master Compile Tests _
+0 🆗	mvndep	1m 56s	Maven dependency ordering for branch
+1 💚	mvninstall	7m 45s	master passed
+1 💚	compile	0m 45s	master passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚	compile	0m 47s	master passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~22.04-b06
+1 💚	checkstyle	1m 1s	master passed
+1 💚	javadoc	0m 40s	master passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚	javadoc	0m 29s	master passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~22.04-b06
+0 🆗	spotbugs	0m 30s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	1m 34s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 8s	Maven dependency ordering for patch
+1 💚	mvninstall	0m 31s	the patch passed
+1 💚	compile	0m 29s	the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚	javac	0m 29s	the patch passed
+1 💚	compile	0m 28s	the patch passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~22.04-b06
+1 💚	javac	0m 28s	the patch passed
+1 💚	checkstyle	0m 11s	tez-dag: The patch generated 0 new + 16 unchanged - 2 fixed = 16 total (was 18)
-0 ⚠️	checkstyle	0m 16s	tez-tests: The patch generated 3 new + 570 unchanged - 11 fixed = 573 total (was 581)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	javadoc	0m 14s	the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚	javadoc	0m 13s	the patch passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~22.04-b06
+1 💚	findbugs	1m 4s	the patch passed
		_ Other Tests _
+1 💚	unit	4m 16s	tez-dag in the patch passed.
+1 💚	unit	38m 49s	tez-tests in the patch passed.
+1 💚	asflicense	0m 21s	The patch does not generate ASF License warnings.
		76m 40s

Subsystem	Report/Notes
Docker	ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-402/1/artifact/out/Dockerfile
GITHUB PR	#402
Optional Tests	dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
uname	Linux a4ee3ca29fc9 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/tez.sh
git revision	master / `7c24f25`
Default Java	Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~22.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~22.04-b06
checkstyle	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-402/1/artifact/out/diff-checkstyle-tez-tests.txt
Test Results	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-402/1/testReport/
Max. process+thread count	1221 (vs. ulimit of 5500)
modules	C: tez-dag tez-tests U: .
Console output	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-402/1/console
versions	git=2.34.1 maven=3.6.3 findbugs=3.0.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

tez-tests/src/main/java/org/apache/tez/mapreduce/examples/Join.java

tez-tests/src/main/java/org/apache/tez/mapreduce/examples/RandomTextWriter.java

tez-tests/src/main/java/org/apache/tez/mapreduce/examples/RandomWriter.java

tez-dag/src/main/java/org/apache/tez/dag/app/web/AMWebController.java

abstractdog · 2026-01-16T14:25:01Z

thanks for this patch @dk2k, left some comments

# Conflicts: # tez-dag/src/main/java/org/apache/tez/dag/app/web/AMWebController.java

dk2k · 2026-01-17T10:20:58Z

@abstractdog Please review, I fixed comments and merge conflict

abstractdog · 2026-01-17T11:11:35Z

tez-tests/src/main/java/org/apache/tez/mapreduce/examples/RandomTextWriter.java

    long numBytesToWritePerMap = conf.getLong(BYTES_PER_MAP,
-                                             1*1024*1024*1024);
+            1 * 1024 * 1024 * 1024);


I'm wondering if we need this line break here, it could be:

long numBytesToWritePerMap = conf.getLong(BYTES_PER_MAP, 1*1024*1024*1024);

the same applies to:

numBytesToWrite = conf.getLong(BYTES_PER_MAP, 1*1024*1024*1024);

abstractdog · 2026-01-17T11:13:01Z

@abstractdog Please review, I fixed comments and merge conflict

thanks @dk2k , I left 1 more minor, otherwise it looks good to me +1

tez-yetus · 2026-01-17T11:30:39Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	14m 20s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ master Compile Tests _
+0 🆗	mvndep	1m 46s		Maven dependency ordering for branch
+1 💚	mvninstall	7m 53s		master passed
+1 💚	compile	0m 48s		master passed
+1 💚	checkstyle	0m 47s		master passed
+1 💚	javadoc	0m 39s		master passed
-1 ❌	spotbugs	1m 2s	/branch-spotbugs-tez-dag.txt	tez-dag in master failed.
-1 ❌	spotbugs	0m 23s	/branch-spotbugs-tez-tests.txt	tez-tests in master failed.
-0 ⚠️	patch	3m 13s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 5s		Maven dependency ordering for patch
+1 💚	mvninstall	0m 36s		the patch passed
+1 💚	codespell	0m 15s		No new issues.
+1 💚	compile	0m 33s		the patch passed
+1 💚	javac	0m 33s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 7s	/results-checkstyle-tez-tests.txt	tez-tests: The patch generated 6 new + 573 unchanged - 8 fixed = 579 total (was 581)
+1 💚	javadoc	0m 18s		the patch passed
-1 ❌	spotbugs	0m 20s	/patch-spotbugs-tez-dag.txt	tez-dag in the patch failed.
-1 ❌	spotbugs	0m 14s	/patch-spotbugs-tez-tests.txt	tez-tests in the patch failed.
			_ Other Tests _
+1 💚	unit	4m 58s		tez-dag in the patch passed.
+1 💚	unit	33m 39s		tez-tests in the patch passed.
+1 💚	asflicense	0m 19s		The patch does not generate ASF License warnings.
		70m 8s

Subsystem	Report/Notes
Docker	ClientAPI=1.52 ServerAPI=1.52 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-402/3/artifact/out/Dockerfile
GITHUB PR	#402
Optional Tests	dupname asflicense javac javadoc unit spotbugs checkstyle codespell detsecrets compile
uname	Linux 1ef178e8020c 5.15.0-164-generic #174-Ubuntu SMP Fri Nov 14 20:25:16 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	/home/jenkins/jenkins-home/workspace/tez-multibranch_PR-402/src/.yetus/personality.sh
git revision	master / `323e289`
Default Java	Ubuntu-21.0.9+10-Ubuntu-124.04
Test Results	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-402/3/testReport/
Max. process+thread count	1032 (vs. ulimit of 5500)
modules	C: tez-dag tez-tests U: .
Console output	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-402/3/console
versions	git=2.43.0 maven=3.8.7 codespell=2.0.0
Powered by	Apache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

abstractdog · 2026-01-17T11:32:11Z

spotbugs is broken at the moment (TEZ-4669), there some checkstyle problems in the affected code, I would appreciate if you can handle those
https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-402/3/artifact/out/results-checkstyle-tez-tests.txt

tez-yetus · 2026-01-17T12:13:12Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 22s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ master Compile Tests _
+0 🆗	mvndep	1m 43s		Maven dependency ordering for branch
+1 💚	mvninstall	8m 27s		master passed
+1 💚	compile	0m 45s		master passed
+1 💚	checkstyle	0m 52s		master passed
+1 💚	javadoc	0m 36s		master passed
-1 ❌	spotbugs	1m 1s	/branch-spotbugs-tez-dag.txt	tez-dag in master failed.
-1 ❌	spotbugs	0m 20s	/branch-spotbugs-tez-tests.txt	tez-tests in master failed.
-0 ⚠️	patch	3m 13s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 5s		Maven dependency ordering for patch
+1 💚	mvninstall	0m 33s		the patch passed
+1 💚	codespell	0m 18s		No new issues.
+1 💚	compile	0m 33s		the patch passed
+1 💚	javac	0m 33s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 7s	/results-checkstyle-tez-tests.txt	tez-tests: The patch generated 6 new + 573 unchanged - 8 fixed = 579 total (was 581)
+1 💚	javadoc	0m 16s		the patch passed
-1 ❌	spotbugs	0m 17s	/patch-spotbugs-tez-dag.txt	tez-dag in the patch failed.
-1 ❌	spotbugs	0m 13s	/patch-spotbugs-tez-tests.txt	tez-tests in the patch failed.
			_ Other Tests _
+1 💚	unit	4m 45s		tez-dag in the patch passed.
+1 💚	unit	34m 32s		tez-tests in the patch passed.
+1 💚	asflicense	0m 20s		The patch does not generate ASF License warnings.
		57m 15s

Subsystem	Report/Notes
Docker	ClientAPI=1.52 ServerAPI=1.52 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-402/4/artifact/out/Dockerfile
GITHUB PR	#402
Optional Tests	dupname asflicense javac javadoc unit spotbugs checkstyle codespell detsecrets compile
uname	Linux a17a31d9e2f9 5.15.0-164-generic #174-Ubuntu SMP Fri Nov 14 20:25:16 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	/home/jenkins/jenkins-home/workspace/tez-multibranch_PR-402@2/src/.yetus/personality.sh
git revision	master / `27a4828`
Default Java	Ubuntu-21.0.9+10-Ubuntu-124.04
Test Results	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-402/4/testReport/
Max. process+thread count	1057 (vs. ulimit of 5500)
modules	C: tez-dag tez-tests U: .
Console output	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-402/4/console
versions	git=2.43.0 maven=3.8.7 codespell=2.0.0
Powered by	Apache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

tez-yetus · 2026-01-17T12:32:42Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 22s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ master Compile Tests _
+0 🆗	mvndep	1m 49s		Maven dependency ordering for branch
+1 💚	mvninstall	8m 17s		master passed
+1 💚	compile	0m 46s		master passed
+1 💚	checkstyle	0m 49s		master passed
+1 💚	javadoc	0m 37s		master passed
-1 ❌	spotbugs	1m 3s	/branch-spotbugs-tez-dag.txt	tez-dag in master failed.
-1 ❌	spotbugs	0m 23s	/branch-spotbugs-tez-tests.txt	tez-tests in master failed.
-0 ⚠️	patch	3m 13s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 6s		Maven dependency ordering for patch
+1 💚	mvninstall	0m 36s		the patch passed
+1 💚	codespell	0m 21s		No new issues.
+1 💚	compile	0m 33s		the patch passed
+1 💚	javac	0m 33s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 9s		The patch passed checkstyle in tez-dag
+1 💚	checkstyle	0m 9s		tez-tests: The patch generated 0 new + 573 unchanged - 8 fixed = 573 total (was 581)
+1 💚	javadoc	0m 19s		the patch passed
-1 ❌	spotbugs	0m 20s	/patch-spotbugs-tez-dag.txt	tez-dag in the patch failed.
-1 ❌	spotbugs	0m 14s	/patch-spotbugs-tez-tests.txt	tez-tests in the patch failed.
			_ Other Tests _
+1 💚	unit	5m 4s		tez-dag in the patch passed.
+1 💚	unit	31m 55s		tez-tests in the patch passed.
+1 💚	asflicense	0m 19s		The patch does not generate ASF License warnings.
		55m 11s

Subsystem	Report/Notes
Docker	ClientAPI=1.52 ServerAPI=1.52 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-402/5/artifact/out/Dockerfile
GITHUB PR	#402
Optional Tests	dupname asflicense javac javadoc unit spotbugs checkstyle codespell detsecrets compile
uname	Linux 179edd0110f8 5.15.0-164-generic #174-Ubuntu SMP Fri Nov 14 20:25:16 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	/home/jenkins/jenkins-home/workspace/tez-multibranch_PR-402/src/.yetus/personality.sh
git revision	master / `2c61e30`
Default Java	Ubuntu-21.0.9+10-Ubuntu-124.04
Test Results	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-402/5/testReport/
Max. process+thread count	1125 (vs. ulimit of 5500)
modules	C: tez-dag tez-tests U: .
Console output	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-402/5/console
versions	git=2.43.0 maven=3.8.7 codespell=2.0.0
Powered by	Apache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

TEZ-4617 prevent resource leaks for Closeables

0cda5d1