Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Target java17 #5045

Merged
merged 13 commits into from
Oct 31, 2024
4 changes: 2 additions & 2 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ jobs:
strategy:
fail-fast: false
matrix:
java_version: [11, 17, 23]
java_version: [17, 23]

steps:
- name: Environment
Expand Down Expand Up @@ -133,7 +133,7 @@ jobs:
strategy:
fail-fast: false
matrix:
java_version: [11, 23]
java_version: [17, 23]
test_mode: ["test_integration", "test_docs", "test_aws", "test_azure", "test_google", "test_wave"]
steps:
- name: Checkout
Expand Down
7 changes: 3 additions & 4 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -73,12 +73,12 @@ allprojects {
}

compileJava {
options.release.set(11)
options.release.set(17)
}

tasks.withType(GroovyCompile) {
sourceCompatibility = '11'
targetCompatibility = '11'
sourceCompatibility = '17'
targetCompatibility = '17'
}

idea {
Expand Down Expand Up @@ -147,7 +147,6 @@ allprojects {
// Required to run tests on Java 9 and higher in compatibility mode
tasks.withType(Test) {
jvmArgs ([
'-Dorg.spockframework.mock.ignoreByteBuddy=true',
'--enable-preview',
'--add-opens=java.base/java.lang=ALL-UNNAMED',
'--add-opens=java.base/java.io=ALL-UNNAMED',
Expand Down
6 changes: 5 additions & 1 deletion docs/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,16 @@

## Requirements

Nextflow can be used on any POSIX-compatible system (Linux, macOS, etc), and on Windows through [WSL](https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux). It requires Bash 3.2 (or later) and [Java 11 (or later, up to 23)](http://www.oracle.com/technetwork/java/javase/downloads/index.html). You can see which version you have using the following command:
Nextflow can be used on any POSIX-compatible system (Linux, macOS, etc), and on Windows through [WSL](https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux). It requires Bash 3.2 (or later) and [Java 17 (or later, up to 23)](http://www.oracle.com/technetwork/java/javase/downloads/index.html) to be installed. You can see which version you have using the following command:

```bash
java -version
```

:::{versionchanged} 24.11.0-edge
Support for Java versions prior to 17 was dropped.
:::

If you don't have a compatible version of Java installed in your computer, it is recommended that you install it through [SDKMAN!](https://sdkman.io/), and that you use the latest LTS version of Temurin. See [this website](https://whichjdk.com/) for more information.

To install Java with SDKMAN:
Expand Down
6 changes: 3 additions & 3 deletions launch.sh
Original file line number Diff line number Diff line change
Expand Up @@ -66,13 +66,13 @@ fi
JAVA_VER=$(echo "$JAVA_VER" | awk '/version/ {gsub(/"/, "", $3); print $3}')
major=${BASH_REMATCH[1]}
minor=${BASH_REMATCH[2]}
version_check="^(1.8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23)"
version_check="^(17|18|19|20|21|23)"
if [[ ! $JAVA_VER =~ $version_check ]]; then
echo "Error: cannot find Java or it's a wrong version -- please make sure that Java 8 or higher is installed"
echo "Error: cannot find Java or it's a wrong version -- please make sure that Java 17 or higher is installed"
exit 1
fi
JVM_ARGS+=" -Dfile.encoding=UTF-8 -XX:+TieredCompilation -XX:TieredStopAtLevel=1"
[[ $JAVA_VER =~ ^(9|10|11|12|13|14|15|16|17|18|19|20|21|22|23) ]] && JVM_ARGS+=" --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio.file.spi=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.locks=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.fs=ALL-UNNAMED --add-opens=java.base/sun.net.www.protocol.http=ALL-UNNAMED --add-opens=java.base/sun.net.www.protocol.https=ALL-UNNAMED --add-opens=java.base/sun.net.www.protocol.ftp=ALL-UNNAMED --add-opens=java.base/sun.net.www.protocol.file=ALL-UNNAMED --add-opens=java.base/jdk.internal.misc=ALL-UNNAMED --add-opens=java.base/jdk.internal.vm=ALL-UNNAMED --add-opens=java.base/java.util.regex=ALL-UNNAMED"
JVM_ARGS+=" --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio.file.spi=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.locks=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.fs=ALL-UNNAMED --add-opens=java.base/sun.net.www.protocol.http=ALL-UNNAMED --add-opens=java.base/sun.net.www.protocol.https=ALL-UNNAMED --add-opens=java.base/sun.net.www.protocol.ftp=ALL-UNNAMED --add-opens=java.base/sun.net.www.protocol.file=ALL-UNNAMED --add-opens=java.base/jdk.internal.misc=ALL-UNNAMED --add-opens=java.base/jdk.internal.vm=ALL-UNNAMED --add-opens=java.base/java.util.regex=ALL-UNNAMED"
[[ $NXF_ENABLE_VIRTUAL_THREADS == 'true' ]] && [[ "$JAVA_VER" =~ ^(19|20) ]] && JVM_ARGS+=" --enable-preview"
[[ "$JAVA_VER" =~ ^(21) ]] && [[ ! "$NXF_ENABLE_VIRTUAL_THREADS" ]] && NXF_ENABLE_VIRTUAL_THREADS=true

Expand Down
8 changes: 4 additions & 4 deletions modules/nextflow/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -54,10 +54,10 @@ dependencies {

// test configuration
testFixturesApi ("org.apache.groovy:groovy-test:4.0.23") { exclude group: 'org.apache.groovy' }
testFixturesApi ("cglib:cglib-nodep:3.3.0")
testFixturesApi ("org.objenesis:objenesis:3.2")
testFixturesApi ("org.spockframework:spock-core:2.3-groovy-4.0") { exclude group: 'org.apache.groovy'; exclude group: 'net.bytebuddy' }
testFixturesApi ('org.spockframework:spock-junit4:2.3-groovy-4.0') { exclude group: 'org.apache.groovy'; exclude group: 'net.bytebuddy' }
testFixturesApi ("org.objenesis:objenesis:3.4")
testFixturesApi ("net.bytebuddy:byte-buddy:1.14.17")
pditommaso marked this conversation as resolved.
Show resolved Hide resolved
testFixturesApi ("org.spockframework:spock-core:2.3-groovy-4.0") { exclude group: 'org.apache.groovy' }
testFixturesApi ('org.spockframework:spock-junit4:2.3-groovy-4.0') { exclude group: 'org.apache.groovy' }
testFixturesApi 'com.google.jimfs:jimfs:1.2'

}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -241,8 +241,15 @@ abstract class RepositoryProvider {
}
}

@Memoized
protected <T> List<T> invokeAndResponseWithPaging(String request, Closure<T> parse) {
// this is needed because apparently bytebuddy used by testing framework is not able
// to handle properly this method signature using both generics and `@Memoized` annotation.
// therefore the `@Memoized` has been moved to the inner method invocation
return invokeAndResponseWithPaging0(request, parse)
}

@Memoized
protected List invokeAndResponseWithPaging0(String request, Closure parse) {
int page = 0
final result = new ArrayList()
while( true ) {
Expand Down
61 changes: 24 additions & 37 deletions nextflow
Original file line number Diff line number Diff line change
Expand Up @@ -313,7 +313,7 @@ if [ ! -x "$JAVA_CMD" ] ; then
JAVA_CMD="$JAVA_HOME/bin/java"
fi
elif [ -x /usr/libexec/java_home ]; then
JAVA_CMD="$(/usr/libexec/java_home -v 11+ 2>/dev/null)/bin/java" || JAVA_CMD=java
JAVA_CMD="$(/usr/libexec/java_home -v 17+ 2>/dev/null)/bin/java" || JAVA_CMD=java
else
JAVA_CMD="$(which java)" || JAVA_CMD=java
fi
Expand Down Expand Up @@ -344,14 +344,8 @@ else
fi
major=${BASH_REMATCH[1]}
minor=${BASH_REMATCH[2]}
# legacy version - Java 7/8 only
if [ $major -eq 0 ] && [ $minor -lt 26 ]; then
version_check="^(1.7|1.8)"
version_message="Java 7 or 8"
else
version_check="^(1.8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23)"
version_message="Java 8 or later (up to 22)"
fi
version_check="^(17|18|19|20|21|22|23)"
version_message="Java 17 or later (up to 23)"
if [[ ! $JAVA_VER =~ $version_check ]]; then
echo_red "ERROR: Cannot find Java or it's a wrong version -- please make sure that $version_message is installed"
if [[ "$NXF_JAVA_HOME" ]]; then
Expand All @@ -361,9 +355,6 @@ else
fi
exit 1
fi
if [[ ! $JAVA_VER =~ ^(11|12|13|14|15|16|17|18|19|20|21|22|23) ]]; then
echo_yellow "NOTE: Nextflow is not tested with Java $JAVA_VER -- It's recommended the use of version 11 up to 23\n"
fi
mkdir -p "$(dirname "$JAVA_KEY")"
[[ -f $JAVA_VER ]] && echo $JAVA_VER > "$JAVA_KEY"
fi
Expand Down Expand Up @@ -461,33 +452,29 @@ else
[[ "$NXF_JVM_ARGS" ]] && launcher+=($NXF_JVM_ARGS)
[[ "$remote_debug" ]] && launcher+=(-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=$NXF_REMOTE_DEBUG_PORT)

if [[ "$JAVA_VER" =~ ^(9|10|11|12|13|14|15|16|17|18|19|20|21|22|23) ]]; then
launcher+=(--add-opens=java.base/java.lang=ALL-UNNAMED)
launcher+=(--add-opens=java.base/java.io=ALL-UNNAMED)
launcher+=(--add-opens=java.base/java.nio=ALL-UNNAMED)
launcher+=(--add-opens=java.base/java.net=ALL-UNNAMED)
launcher+=(--add-opens=java.base/java.util=ALL-UNNAMED)
launcher+=(--add-opens=java.base/java.util.concurrent.locks=ALL-UNNAMED)
launcher+=(--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED)
launcher+=(--add-opens=java.base/java.nio.file.spi=ALL-UNNAMED)
launcher+=(--add-opens=java.base/sun.nio.ch=ALL-UNNAMED)
launcher+=(--add-opens=java.base/sun.nio.fs=ALL-UNNAMED)
launcher+=(--add-opens=java.base/sun.net.www.protocol.http=ALL-UNNAMED)
launcher+=(--add-opens=java.base/sun.net.www.protocol.https=ALL-UNNAMED)
launcher+=(--add-opens=java.base/sun.net.www.protocol.ftp=ALL-UNNAMED)
launcher+=(--add-opens=java.base/sun.net.www.protocol.file=ALL-UNNAMED)
launcher+=(--add-opens=java.base/jdk.internal.misc=ALL-UNNAMED)
launcher+=(--add-opens=java.base/jdk.internal.vm=ALL-UNNAMED)
launcher+=(--add-opens=java.base/java.util.regex=ALL-UNNAMED)
if [[ "$NXF_ENABLE_VIRTUAL_THREADS" == 'true' ]]; then
if [[ "$JAVA_VER" =~ ^(19|20) ]]; then launcher+=(--enable-preview)
elif [[ ! "$JAVA_VER" =~ ^(21|22|23) ]]; then die "Virtual threads require Java 19 or later - current version $JAVA_VER"
fi
launcher+=(--add-opens=java.base/java.lang=ALL-UNNAMED)
launcher+=(--add-opens=java.base/java.io=ALL-UNNAMED)
launcher+=(--add-opens=java.base/java.nio=ALL-UNNAMED)
launcher+=(--add-opens=java.base/java.net=ALL-UNNAMED)
launcher+=(--add-opens=java.base/java.util=ALL-UNNAMED)
launcher+=(--add-opens=java.base/java.util.concurrent.locks=ALL-UNNAMED)
launcher+=(--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED)
launcher+=(--add-opens=java.base/java.nio.file.spi=ALL-UNNAMED)
launcher+=(--add-opens=java.base/sun.nio.ch=ALL-UNNAMED)
launcher+=(--add-opens=java.base/sun.nio.fs=ALL-UNNAMED)
launcher+=(--add-opens=java.base/sun.net.www.protocol.http=ALL-UNNAMED)
launcher+=(--add-opens=java.base/sun.net.www.protocol.https=ALL-UNNAMED)
launcher+=(--add-opens=java.base/sun.net.www.protocol.ftp=ALL-UNNAMED)
launcher+=(--add-opens=java.base/sun.net.www.protocol.file=ALL-UNNAMED)
launcher+=(--add-opens=java.base/jdk.internal.misc=ALL-UNNAMED)
launcher+=(--add-opens=java.base/jdk.internal.vm=ALL-UNNAMED)
launcher+=(--add-opens=java.base/java.util.regex=ALL-UNNAMED)
if [[ "$NXF_ENABLE_VIRTUAL_THREADS" == 'true' ]]; then
if [[ "$JAVA_VER" =~ ^(19|20) ]]; then launcher+=(--enable-preview)
elif [[ ! "$JAVA_VER" =~ ^(21|22|23) ]]; then die "Virtual threads require Java 19 or later - current version $JAVA_VER"
fi
launcher+=("${cmd_tail[@]}")
else
launcher+=("${cmd_tail[@]}")
fi
launcher+=("${cmd_tail[@]}")

# create the launch file only when using the legacy launcher (capsule)
if [[ $NXF_LEGACY_LAUNCHER ]]; then
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -527,16 +527,23 @@ class AzFileSystem extends FileSystem {
* @param cond A predicate that determines when a retry should be triggered
* @return The {@link dev.failsafe.RetryPolicy} instance
*/
@Memoized
protected <T> RetryPolicy<T> retryPolicy(Predicate<? extends Throwable> cond) {
// this is needed because apparently bytebuddy used by testing framework is not able
// to handle properly this method signature using both generics and `@Memoized` annotation.
// therefore the `@Memoized` has been moved to the inner method invocation
return (RetryPolicy<T>) retryPolicy0(cond)
}

@Memoized
protected RetryPolicy retryPolicy0(Predicate<? extends Throwable> cond) {
final cfg = AzConfig.getConfig().retryConfig()
final listener = new EventListener<ExecutionAttemptedEvent>() {
@Override
void accept(ExecutionAttemptedEvent event) throws Throwable {
log.debug("Azure I/O exception - attempt: ${event.attemptCount}; cause: ${event.lastFailure?.message}")
}
}
return RetryPolicy.<T>builder()
return RetryPolicy.builder()
.handleIf(cond)
.withBackoff(cfg.delay.toMillis(), cfg.maxDelay.toMillis(), ChronoUnit.MILLIS)
.withMaxAttempts(cfg.maxAttempts)
Expand Down