Add integration tests and notes to comparison.md#1
Open
ivzhelev wants to merge 1090 commits intofeature/go-migrationfrom
Open
Add integration tests and notes to comparison.md#1ivzhelev wants to merge 1090 commits intofeature/go-migrationfrom
ivzhelev wants to merge 1090 commits intofeature/go-migrationfrom
Conversation
…-fix Dockerfile ruby fix
Quarterly Java release note updates
- the bin/{compile,detect,finalize,release} executables have been
wrapped in a small shim that makes sure a Ruby interpreter is
available on the PATH before exec-ing the existing buildpack
entrypoints
- works both online and offline
- currently pinned to the latest Ruby 3.1.x version available in the
Ruby Buildpack repository
Bring your own Ruby
This reverts commit 5c0a32d.
Updated Contrast agent version
- the bin/{compile,detect,finalize,release} executables have been
wrapped in a small shim that makes sure a Ruby interpreter is
available on the PATH before exec-ing the existing buildpack
entrypoints
- works both online and offline
- currently pinned to the latest Ruby 3.1.x version available in the
Ruby Buildpack repository
Bring your own Ruby (whoops)
Fixes rubocop violations
Default to empty string if CF_STACK unbound
Adds CI/Rubocop support for Ruby 3.1
Don't use lsb_release to find platform
Update Ruby check for cflinuxfs4 stack
fix start command and options error
…eadme Add Splunk to the README
…de-store-1-13-7 Update geode_store version to 1.13.7
…ibraries Add runtime CLASSPATH modification to DistZip container to support framework-provided JARs (APM agents, JDBC drivers, etc.) following the Ruby buildpack's approach. Changes: - Remove JVMKill installation from DistZip Supply phase (now handled by JRE component) - Implement augmentStartupScript() to modify startup scripts directly with framework libraries - Add collectAdditionalLibraries() to scan deps directory for framework JARs - Add buildClasspathPrefix() to construct runtime-relative CLASSPATH entries - Support both distZip formats: declare -r app_classpath and CLASSPATH= patterns - Write profile.d script to set DIST_ZIP_HOME, DIST_ZIP_BIN, and PATH at runtime - Update Release() to use absolute $HOME-based paths for startup commands The startup script augmentation happens during Finalize phase to ensure all framework libraries installed during Supply are available to the application at runtime. This matches the Ruby buildpack behavior where framework JARs are prepended to the application's CLASSPATH. Fixes CLASSPATH handling for applications using frameworks with DistZip packaging.
Problem: JVMKill agent was being installed redundantly by each container (Tomcat, JavaMain, Groovy, DistZip) and used staging paths in -agentpath, causing startup failures. Solution: - Remove duplicate JVMKill installation from all containers - JVMKill is now exclusively managed by JRE component - Fix runtime path conversion: staging paths -> runtime paths (/tmp/contents.../deps/0/jre/bin/jvmkill-1.16.0.so -> /home/vcap/deps/0/jre/bin/jvmkill-1.16.0.so) Changes: - Added JVMKillAgent.convertToRuntimePath() method - Removed installJVMKillAgent() from 4 containers - Removed duplicate JAVA_OPTS configuration from containers - Use absolute runtime paths (not $DEPS_DIR) since startup scripts run before .profile.d scripts are sourced Impact: Single source of truth for JVMKill, eliminates duplication, fixes agent loading at runtime.
Problem: Tomcat 10.x requires Java 11+ (Jakarta EE 9), while Tomcat 9.x supports Java 8-22 (Java EE 8). Using a fixed Tomcat version causes compatibility issues with different Java versions. Solution: - Detect Java major version from JAVA_HOME at supply time - Select Tomcat 10.x for Java 11+ (Jakarta EE namespace migration) - Select Tomcat 9.x for Java 8-10 (javax.* namespace) - Resolve version patterns (10.x, 9.x) using libbuildpack.FindMatchingVersion() - Fallback to default version if Java version detection fails Changes: - Added Java version detection via jres.DetermineJavaVersion() - Implemented version pattern selection logic in TomcatContainer.Supply() - Added logging for version selection decisions (Debug, Info, Warning levels) - Maintains backward compatibility with fallback to Tomcat 9.0.98 Impact: Applications automatically get the correct Tomcat version for their Java runtime, eliminating Jakarta EE namespace compatibility issues.
…lity PROBLEM: The buildpack was setting absolute staging-time paths in env/ files and release commands, which don't match runtime paths in Cloud Foundry. At runtime, dependencies are mounted at $DEPS_DIR (e.g., /home/vcap/deps) not at staging paths like /tmp/app/.cloudfoundry. SOLUTION: Migrate to Cloud Foundry's .profile.d pattern where environment variables are set dynamically at app startup using $DEPS_DIR runtime variables. CHANGES: 1. bin/compile (NEW) - Backwards compatibility wrapper for single-buildpack deployments - Creates .profile.d/ directory and 0000_set-deps-dir.sh bootstrap - Calls supply + finalize phases sequentially 2. JRE Environment Setup (jre.go, openjdk.go) - SetupJavaHome(): Use $DEPS_DIR/0/jre/<subdirectory> in profile.d - WriteJavaOpts(): Changed from env/JAVA_OPTS file to .profile.d/java_opts.sh - Preserves absolute paths for staging-time process environment - Runtime JAVA_HOME points to $DEPS_DIR (e.g., $DEPS_DIR/0/jre/jdk-17.0.13+11) 3. Tomcat Container (tomcat.go) - Supply: Creates .profile.d/tomcat.sh with CATALINA_HOME/CATALINA_BASE - findTomcatHome(): Detects nested apache-tomcat-X.Y.Z/ directory - Release: Uses $CATALINA_HOME variable (set by profile.d) in startup command - Eliminates hardcoded paths in catalina.sh invocation 4. Spring Boot CLI Container (spring_boot_cli.go) - Supply: Creates .profile.d/spring-boot-cli.sh with SPRING_BOOT_CLI_HOME - Release: Uses $SPRING_BOOT_CLI_HOME variable in startup command - Changes .additional_libs/* paths to relative (from absolute staging paths) BENEFITS: - Runtime paths work correctly in Cloud Foundry environments - Supports both single-buildpack (bin/compile) and multi-buildpack scenarios - Environment variables set dynamically at app startup, not hardcoded - Follows standard Cloud Foundry buildpack patterns (same as Ruby/Python buildpacks) TESTING: - All 129 unit tests pass - Integration tests validate runtime environment variable availability
**Problem:** The Java buildpack was using a custom bash wrapper script ($HOME/.java-buildpack/start.sh) that sourced profile.d scripts and executed the container command. This pattern: - Required maintaining custom bash logic for environment setup - Didn't follow Cloud Foundry buildpack conventions - Made it harder to integrate with CF lifecycle hooks **Solution:** Adopt the standard release YAML pattern used by all reference buildpacks (Ruby, Go, Node.js, Python): 1. **Finalize phase** writes `tmp/java-buildpack-release-step.yml` with direct container commands (e.g., `bin/application` for DistZip, `java -jar app.jar` for Spring Boot) 2. **Release phase** reads and outputs this YAML to stdout for Cloud Foundry to parse 3. **Cloud Foundry runtime** handles profile.d sourcing automatically before executing the command **Changes:** - src/java/finalize/cli/main.go: Added RunAfterCompile() and SetLaunchEnvironment() lifecycle hooks - src/java/finalize/finalize.go: Replaced generateStartupScript() with writeReleaseYaml() - src/java/release/release.go: Changed from hardcoded bash wrapper to YAML passthrough - src/java/release/release_test.go: Updated test to simulate finalize phase creating YAML file **Benefits:** - Aligns with Cloud Foundry buildpack conventions - Simplifies maintenance (no custom bash wrapper logic) - Better integration with CF lifecycle hooks - Profile.d scripts handled by platform (from Commit 1: 23bd35c) **Related:** - Depends on Commit 1 (23bd35c) which migrated environment setup to .profile.d/*.sh scripts - Part of 3-commit refactoring series to align with CF runtime standards
- Enhanced Spring Boot detection with proper manifest validation - Fixed Java Main container JAR path resolution - Improved test cleanup: skip deployment deletion on test failure - Fixed test fixtures: use correct fixtures for container types - Added staged Spring Boot app support (bin/ + lib/ pattern)
Consolidates JAVA_HOME environment setup across all 7 JRE providers by replacing individual implementations with a centralized WriteJavaHomeProfileD() function. Changes: - Remove old SetupJavaHome() from jre.go (63 lines) - Add new WriteJavaHomeProfileD() helper with better documentation - Update all JRE providers (OpenJDK, Zulu, SAP Machine, GraalVM, IBM, Oracle, Zing) to delegate to shared function - Each provider now has simple writeProfileDScript() method - Improve error handling with warnings instead of failures - Add clearer comments explaining staging vs. runtime behavior Benefits: - Single source of truth for environment variable setup - Easier maintenance and future extensions - Consistent behavior across all JRE types - Better logging and error messages
- Add tomcat-lifecycle-support 3.4.0 from Maven Central - Add tomcat-logging-support 3.4.0 from Maven Central - Add tomcat-access-logging-support 3.4.0 from Maven Central - Fix dependency name in tomcat.go (underscore → hyphen) - Improve Tomcat test assertions (ContainSubstring → Not(BeEmpty)) - Fix Java Main test fixture path (integration_valid → container_main) - Add integration_valid BOOT-INF test fixtures All changes enable proper Tomcat integration test execution with Maven Central dependencies replacing previously unavailable CF buildpack URLs.
- Add failed test detection with logging in it.After() blocks - Add --keep-failed-containers flag to preserve failed containers for debugging - Update all 10 integration test files with consistent cleanup pattern - Remove verbose debug logging from tomcat_test.go - Cleanup only happens when test succeeds or keep-failed-containers is not set
Create separate test fixtures for javax.servlet (Tomcat 9) and jakarta.servlet (Tomcat 10+) to properly test the buildpack's smart Tomcat version selection. The original fixture was compiled with javax.servlet, causing test failures with Tomcat 10+ which requires the jakarta.servlet namespace (Jakarta EE 9+). Changes: - Add container_tomcat_javax fixture: Java 8 bytecode, javax.servlet, Java EE 4.0 web.xml - Add container_tomcat_jakarta fixture: Java 11 bytecode, jakarta.servlet, Jakarta EE 5.0 web.xml - Update tomcat_test.go to use appropriate fixtures per Java version - Add Tomcat version assertions to verify correct version selection - Improve test assertions with specific response validation This enables proper testing of both Tomcat 9 (Java 8-10) and Tomcat 10+ (Java 11+) deployment scenarios, matching the smart version selection logic from commit ed099f3.
Enhance Tomcat container to properly configure exploded WAR deployments and support libraries, plus refactor DistZip to use environment-based CLASSPATH configuration instead of modifying startup scripts. Tomcat improvements: - Implement configureContextDocBase() to create ROOT.xml pointing to BuildDir - Add configureTomcatSupport() to include lifecycle support JAR via setenv.sh - Follow immutable BuildDir pattern: app stays in place, Tomcat serves from there - Enhance unit tests with proper Tomcat directory structure mocking DistZip refactoring: - Replace augmentStartupScript() with buildRuntimeClasspath() - Configure additional libraries via CLASSPATH environment variable in profile.d - Convert staging paths to runtime paths ($DEPS_DIR references) - Remove complex regex-based script modification (168 lines removed) - Simpler, more maintainable approach that respects startup script integrity Test improvements: - Add comprehensive Tomcat finalization tests with context file verification - Update DistZip unit tests to expect $HOME-prefixed paths - Move DistZip test earlier in suite for better organization These changes align with Cloud Foundry's immutable staging directory pattern and make the buildpack more robust by avoiding direct modification of application files.
Update libbuildpack to v0.0.0-20251203175254-7be530ec9fef which includes InstallDependencyWithStrip support. This allows archives that extract to top-level directories (e.g., apache-tomcat-9.0.98/ from apache-tomcat-9.0.98.tar.gz) to be installed directly to the target directory by stripping the first path component. Refactor Tomcat, Groovy, and Spring Boot CLI containers to use InstallDependencyWithStrip with stripComponents=1, eliminating the need for findTomcatHome() and similar post-extraction directory traversal logic. This matches the behavior of the original Ruby buildpack which used `tar --strip 1`.
…ction Complete Play Framework container implementation supporting all four Play application types: Pre22Staged (2.0-2.1 staged), Post22Staged (2.2+ staged), Pre22Dist (2.0-2.1 dist), and Post22Dist (2.2+ dist). Detection now prioritizes staged apps over dist apps to avoid ambiguous matches. Key improvements: - Reorder detection to check staged apps first (more specific) before dist apps - Make start scripts optional in detection - apps can use java command directly - Implement finalize phase that collects framework libraries (JVMKill, JDBC drivers, etc.) and injects them into CLASSPATH via profile.d script - Configure JAVA_OPTS with Play-specific settings (-Dhttp.port, etc.) - Generate proper release commands using start scripts or java NettyServer fallback Fix Play 2.1 dist fixture start script to use wildcard classpath pattern (lib/*) instead of hardcoded JAR list, matching the working Play 2.0 approach. This resolves integration test failures caused by missing JARs in the hardcoded classpath. Update all Play test fixtures with proper JAR files and start scripts for comprehensive testing coverage.
Introduce RegisterStandardContainers() method to ensure Supply and Finalize phases use identical container detection order. Previously, the registration order was duplicated in both phases, which risked divergence and made the critical ordering logic harder to maintain. The registration order is critical because containers are checked in sequence: 1. Spring Boot - BOOT-INF or Spring Boot JAR markers 2. Spring Boot CLI - Groovy files with POGO/beans (NO main/shebang) 3. Tomcat - WEB-INF or WAR files 4. Groovy - Groovy files with main method OR shebang 5. Play - Play Framework structure 6. DistZip - bin/ and lib/ directories 7. JavaMain - executable JAR with Main-Class manifest Also improved Spring Boot container tests by adding proper manifest files to fixtures, ensuring detection logic is properly tested.
Add smart Groovy script detection matching Ruby buildpack behavior. When multiple .groovy files exist, FindMainGroovyScript() prioritizes: 1. Files with static void main() method 2. Non-POGO files (simple scripts without class definitions) 3. Files with shebang (#!/usr/bin/env groovy) Returns the single candidate if exactly one matches these criteria, otherwise falls back to the first script found. Also refactored groovy_utils.go with comprehensive script analysis: - HasMainMethod() - detects static void main() - IsPOGO() - identifies Plain Old Groovy Objects (class definitions) - HasShebang() - checks for shebang lines - isValidGroovyFile() - filters out binary/invalid files Updated Groovy test fixtures to run Undertow web servers instead of simple print statements, enabling proper integration testing.
Ratpack fixtures: - Added functional Ratpack.groovy with GET handler returning test response - Replaced broken CLASSPATH-only start scripts with proper executable bash scripts - Added ratpack-test-app.jar to lib/ directories for both staged and dist layouts - Start scripts now properly set APP_HOME, construct CLASSPATH, and exec java Spring Boot CLI fixtures: - Converted simple script fixtures to proper REST controllers using @RestController - Added @RequestMapping/@GetMapping endpoints for HTTP testing - Ensured all files are valid POGOs without main methods or shebangs - This matches Spring Boot CLI detection rules (POGO/beans only, no main/shebang) Also improved spring_boot_cli.go to use isValidGroovyFile() for filtering invalid/binary files during detection.
- Upgrade from Ginkgo v1.16.5 to v2.27.2 - Update unit test imports to use github.com/onsi/ginkgo/v2 - Remove duplicate RunSpecs from play_test.go - Fix JRE test stager initialization to use NewStager - Update WriteJavaOpts tests to check .profile.d/java_opts.sh - Update script flags for Ginkgo v2 (--skip-package) - Vendor Ginkgo v2 dependencies All unit tests passing (133 specs).
- Remove Ruby fixtures from spec/fixtures/ (306 files) - Add Go fixtures to src/integration/testdata/ organized by type: - apps/: Application test fixtures - containers/: Container detection test fixtures - frameworks/: Framework/agent test fixtures - Update integration test imports to Ginkgo v2 - Refactor filepath.Join to idiomatic Go style (separate path segments) - Fix fixtures base path in init_test.go to point to new location - Correct fixture path mappings (remove prefixes, add subdirectories) Result: All integration tests passing with proper Go conventions
- Add stager.SetStagingEnvironment() call before finalize logic - Add stager.StagingComplete() call at end of finalize - Remove unnecessary calls (CheckBuildpackValid, SetAppCacheDir, WriteConfigYml, CleanupAppCache) - Align exit codes with reference buildpacks (go-buildpack, ruby-buildpack) Fixes ensure proper CloudFoundry lifecycle signaling
… bash Replace Go-based bash wrappers with pure bash implementations for bin/detect and bin/release scripts to fix buildpack release phase failures when deployed via git URL. The bug: When using git URL buildpacks (e.g., --buildpack https://github.com/...), Cloud Foundry uses the bash wrapper scripts which compiled Go binaries on-the-fly. These wrappers contained echo statements like 'echo "-----> Running go build release"' that polluted stdout. During the release phase, CF expects pure YAML on stdout but received echo output first, causing: 'Failed to build droplet release: buildpack's release output invalid: yaml: unmarshal errors: line 1: cannot unmarshal !!str `-----> ...`' Changes: - Convert bin/detect to pure bash with all Java detection logic inline - Convert bin/release to simple 'cat' of YAML file generated by finalize - Remove Go source code for detect and release (src/java/detect/, src/java/release/) - Add regression test using git URL to ensure bash scripts output clean YAML This only affects git URL deployments. Pre-packaged buildpacks already used pre-compiled Go binaries and were unaffected.
Spring Boot 3.x changed the JarLauncher class path from: - org.springframework.boot.loader.JarLauncher (2.x) to: - org.springframework.boot.loader.launch.JarLauncher (3.x) This fix adds automatic detection of the correct launcher class by: 1. Reading META-INF/MANIFEST.MF Main-Class entry (most accurate) 2. Checking Spring-Boot-Version and determining from version number 3. Defaulting to 3.x launcher if version cannot be determined This resolves the error: Error: Could not find or load main class org.springframework.boot.loader.JarLauncher Caused by: java.lang.ClassNotFoundException: org.springframework.boot.loader.JarLauncher Fixes support for Spring Boot 3.x applications deployed to Cloud Foundry. Tested with: - Spring Boot 2.x applications (still work with old path) - Spring Boot 3.x applications (now work with new path)
BREAKING CHANGE: Spring Auto-reconfiguration is now DISABLED by default.
Background:
- Spring Cloud Connectors has been in maintenance mode since July 2019
- Originally planned to be disabled by default after August 2022
- Originally planned to be removed after March 2024
- Both deadlines have passed
- The Ruby buildpack attempted to disable it in Sept 2022 but reverted
after user pushback and never attempted again
Changes:
- config/spring_auto_reconfiguration.yml: Set enabled: false
- src/java/frameworks/spring_auto_reconfiguration.go:
* Updated isEnabled() to return false by default
* Improved deprecation warnings to reflect disabled-by-default status
* Updated code comments to indicate DEPRECATED status
- docs/framework-spring_auto_reconfiguration.md:
* Added prominent deprecation notice
* Documented how to re-enable if necessary
* Added migration guidance to java-cfenv
- tests: Updated to explicitly enable framework when testing
Impact:
- Spring Boot 3.x apps: Already use java-cfenv (no change)
- Apps with java-cfenv in dependencies: Already skip auto-reconfig (no change)
- Apps without bound services: Never used auto-reconfig (no change)
- Spring Boot 2.x apps relying on auto-reconfig: NEED to explicitly enable
To re-enable (not recommended):
cf set-env my-app JBP_CONFIG_SPRING_AUTO_RECONFIGURATION '{enabled: true}'
Migration path:
Use java-cfenv instead: https://github.com/pivotal-cf/java-cfenv
See docs/framework-java_cf_env.md for details
Tests: All unit tests passing (58/58 specs)
Updated note for Java Opts to indicate that the configuration does not work.
ivzhelev
force-pushed
the
feature/go-migration
branch
from
e183ee8 to
00445e0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.