mirror of https://github.com/apache/zeppelin synced 2026-05-24 09:38:26 +00:00

[ZEPPELIN-6118] Remove unnecessary hadoop3 profile

### What is this PR for?

Given that hadoop2 support was removed, hadoop3 is the only option, and it's unlikely that hadoop4 has existed in predictable years. We can remove such a profile to simplify the `pom.xml` files.

### What type of PR is it?

Improvement

### Todos

- [x] remove profile `hadoop3` and update CI workflow yamls and docs
- [x] rename profile `hadoop3-aws` to `hadoop-aws`
- [x] rename profile `hadoop3-azure` to `hadoop-azure`
- [x] given hadoop deps are defined at root `pom.xml`'s `dependencyManagement` block, simplify sub-projects' `pom.xml` by removing redundant attributes.

### What is the Jira issue?

ZEPPELIN-6118

### How should this be tested?

Pass GHA.
### Screenshots (if appropriate)

### Questions:
* Does the license files need to update? No.
* Is there breaking changes for older versions? No.
* Does this needs documentation? No.


Closes #4861 from pan3793/ZEPPELIN-6118.

Signed-off-by: Cheng Pan <chengpan@apache.org>

2024-10-05 20:35:31 +08:00

3 KiB

Raw Permalink Blame History

layout	title	description	group
page	Useful Developer Tools		development/contribution

{% include JB/setup %}

Useful Developer Tools

Developing `zeppelin-web`

Check zeppelin-web: Local Development.

Tools

SVM: Scala Version Manager

svm would be useful when changing scala version frequently.

JDK change script: OSX

this script would be helpful when changing JDK version frequently.

function setjdk() {
  if [ $# -ne 0 ]; then
  # written based on OSX. 
  # use diffrent base path for other OS
  removeFromPath '/System/Library/Frameworks/JavaVM.framework/Home/bin'
  if [ -n "${JAVA_HOME+x}" ]; then
    removeFromPath $JAVA_HOME
  fi
  export JAVA_HOME=`/usr/libexec/java_home -v $@`
  export PATH=$JAVA_HOME/bin:$PATH
  fi
}
function removeFromPath() {
  export PATH=$(echo $PATH | sed -E -e "s;:$1;;" -e "s;$1:?;;")
}

you can use this function like setjdk 1.8 / setjdk 1.7

Building Submodules Selectively

# build `zeppelin-web` only
./mvnw clean -pl 'zeppelin-web' package -DskipTests;

# build `zeppelin-server` and its dependencies only
./mvnw clean package -pl 'spark,spark-dependencies,python,markdown,zeppelin-server' --am -DskipTests

# build spark related modules with default profiles
./mvnw clean package -pl 'spark,spark-dependencies,zeppelin-server' --am -DskipTests

# build spark related modules with profiles: scala 2.13, spark 3.5
./mvnw clean package -Pspark-scala-2.13 -Pspark-3.5 \
-pl 'spark,spark-dependencies,zeppelin-server' --am -DskipTests

# build `zeppelin-server` and `markdown` with dependencies
./mvnw clean package -pl 'markdown,zeppelin-server' --am -DskipTests

Running Individual Tests

# run the `HeliumBundleFactoryTest` test class
./mvnw test -pl 'zeppelin-server' --am -DfailIfNoTests=false -Dtest=HeliumBundleFactoryTest

Running Selenium Tests

Make sure that Zeppelin instance is started to execute integration tests (= selenium tests).

# run the `SparkParagraphIT` test class
TEST_SELENIUM="true" ./mvnw test -pl 'zeppelin-server' --am \
-DfailIfNoTests=false -Dtest=SparkParagraphIT

# run the `testSqlSpark` test function only in the `SparkParagraphIT` class
# but note that, some test might be dependent on the previous tests
TEST_SELENIUM="true" ./mvnw test -pl 'zeppelin-server' --am \
-DfailIfNoTests=false -Dtest=SparkParagraphIT#testSqlSpark

3 KiB Raw Permalink Blame History