mirror of
https://github.com/apache/zeppelin
synced 2026-05-24 09:38:26 +00:00
4ac577f711
### What is this PR for? Take 2 of #1618 because I had some earlier problems with rebasing. Since, then I have added some new features, namely: - Matplotlib integration tests for pyspark - `install_external_dependencies.sh` which conditionally installs the R or python dependencies based on the specified build profile in `.travis.yml`. This saves several minutes of time for a few of the build profiles since the R dependencies are compiled from source and therefore take quite a bit of time to install. - The extra python unit tests which require external dependencies (`matplotlib` and `pandas`) are now relegated to two separate build profiles. This is done primarily to efficiently test both Python 2 and 3. - Some minor bugs in the python and pyspark interpreters (mostly with respect to python 3 compatibility) were caught as a result of these tests, and are also fixed in this PR. ### What type of PR is it? Improvement and Bugfix ### What is the Jira issue? [ZEPPELIN-1639](https://issues.apache.org/jira/browse/ZEPPELIN-1639) ### How should this be tested? CI tests should be green! ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Alex Goodman <agoodm@users.noreply.github.com> Closes #1632 from agoodm/ZEPPELIN-1639 and squashes the following commits:01380c2[Alex Goodman] Make sure python 3 profile uses scala 2.11363019e[Alex Goodman] Use spark 2.0 with python 3a4f43af[Alex Goodman] Update comments in .travis.yml5a60181[Alex Goodman] Isolate python tests73663f6[Alex Goodman] Update tests for new InterpreterContext constructor5709c5d[Alex Goodman] Re-add pyspark to build profileee95d67[Alex Goodman] Move python 3 tests to all modules with spark 2.03a76958[Alex Goodman] Travis42da31c[Alex Goodman] Shorten python versionb6b88be[Alex Goodman] Add python dependencies to .travis.yml
46 lines
1.7 KiB
Bash
Executable file
46 lines
1.7 KiB
Bash
Executable file
#!/bin/bash
|
|
|
|
# Licensed to the Apache Software Foundation (ASF) under one or more
|
|
# contributor license agreements. See the NOTICE file distributed with
|
|
# this work for additional information regarding copyright ownership.
|
|
# The ASF licenses this file to You under the Apache License, Version 2.0
|
|
# (the "License"); you may not use this file except in compliance with
|
|
# the License. You may obtain a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing, software
|
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
# See the License for the specific language governing permissions and
|
|
# limitations under the License.
|
|
#
|
|
|
|
# Script for installing R / Python dependencies for Travis CI
|
|
set -ev
|
|
touch ~/.environ
|
|
|
|
# Install R dependencies if R profiles are used
|
|
if [[ ${PROFILE/"-Pr "} != $PROFILE ]] || [[ ${PROFILE/"-Psparkr "} != $PROFILE ]] ; then
|
|
echo "R_LIBS=~/R" > ~/.Renviron
|
|
echo "export R_LIBS=~/R" >> ~/.environ
|
|
source ~/.environ
|
|
if [[ ! -d "$HOME/R/knitr" ]] ; then
|
|
mkdir -p ~/R
|
|
R -e "install.packages('knitr', repos = 'http://cran.us.r-project.org', lib='~/R')"
|
|
fi
|
|
fi
|
|
|
|
# Install Python dependencies for Python specific tests
|
|
if [[ -n "$PYTHON" ]] ; then
|
|
wget https://repo.continuum.io/miniconda/Miniconda${PYTHON}-latest-Linux-x86_64.sh -O miniconda.sh
|
|
bash miniconda.sh -b -p $HOME/miniconda
|
|
echo "export PATH='$HOME/miniconda/bin:$PATH'" >> ~/.environ
|
|
source ~/.environ
|
|
hash -r
|
|
conda config --set always_yes yes --set changeps1 no
|
|
conda update -q conda
|
|
conda info -a
|
|
conda config --add channels conda-forge
|
|
conda install -q matplotlib pandasql
|
|
fi
|