Installing Kubeflow on MicroK8s/AWS for testing and development
Kubeflow can be installed on any compatible version of Kubernetes, but with some setups there are a few extra steps to ensure everything runs smoothly. This page documents how to install Kubeflow on top of MicroK8s running in an AWS instance - a useful setup for testing, developing Kubeflow or short-lived experiments.
- The latest version of the Juju OLM. On Ubuntu or other snap enabled operating systems, just run
sudo snap install juju --classic
- You should have an active AWS account, with the credentials already stored in Juju. See the Juju documentation if you need to add credentials now.
1. Bootstrap AWS instance
Juju requires a controller instance to manage the deployment. If you already have a controller for AWS, you can skip to step 2. To bootstrap a controller on AWS, run the following command on your local machine:
juju bootstrap \ --bootstrap-constraints "cores=2 mem=4G" \ --config mongo-memory-profile=low \ aws/us-east-1 \ mk8s-controller
You can substitute in a different region instead of
aws/us-east-1, though bear in mind that not all AWS instances are available in some regions. You can also use a different name for the controller (
mk8s-controller is used here).
2. Deploy a host instance
The host instance will run MicroK8s, and will require enough resources to deploy the Kubeflow components. If you want to run GPU workloads, you can also choose a GPU enabled instance. For example:
- Standard deployment
juju deploy ubuntu demo --constraints="root-disk=100G instance-type=t2.2xlarge" --series focal && juju wait -wv
- GPU deployment
juju deploy ubuntu demo --constraints="root-disk=100G instance-type=g4dn.xlarge" --series focal && juju wait -wv
3. Enable access to the instance
Run these commands to be able to connect to the running instance from your local machine (or elsewhere).
juju run --unit demo/0 "open-port 80" juju expose demo
4. Connect to AWS instance
You can now use Juju to connect to the AWS instance using SSH:
juju ssh demo/0
or if you prefer, directly from another terminal on the local machine:
ssh ubuntu@(juju status --format yaml | yq e '.applications.demo.units["demo/0"]'.public-address -)
Note: The rest of these steps should be performed on the AWS instance, through the SSH connection which has just been made.
5. Install requirements on AWS instance
To run and deploy Kubeflow, the remote instance will need some software installed. As well as MicroK8s itself and Juju, some additional applications will be needed for use as a development system.
for snap in microk8s juju juju-wait juju-kubectl juju-bundle; do sudo snap install $snap --classic; done sudo usermod -a -G microk8s ubuntu
For a GPU enabled system, you should also install and initialise the required drivers:
sudo apt install -y nvidia-headless-460 sudo modprobe nvidia
To complete the setup, you will need to log in again (to pick up group changes). You can just run
exit from the SSH terminal, and re-run the Juju or SSH command from step 4 to re-connect.
6. Deploy the Kubeflow bundle
For the scripts to work, you need to supply the public address of your AWS instance in place of
screen git clone https://github.com/canonical/bundle-kubeflow.git cd bundle-kubeflow/ python3 scripts/cli.py microk8s setup --test-mode python3 scripts/cli.py deploy-to uk8s --public-address x.x.x.x.nip.io --bundle full --password=foobar
7. Create an Ingress resource
Again, substitute in the public IP address of the instance in place of
x.x.x.x in the code below:
echo "apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: istio-ingressgateway spec: rules: - host: x.x.x.x.nip.io http: paths: - path: / backend: serviceName: istio-ingressgateway servicePort: 80 " | microk8s kubectl apply -n kubeflow -f -
8. Run tests
To run the tests on the AWS instance
sudo apt install -y python3-pytest sudo ln -s $(which pytest-3) /usr/local/bin/pytest pip3 install --user kfp sh ./tests/run.sh -m full
If you want to run the tests (such as the selenium tests) from your laptop:
pytest -svv tests/ -m full -k selenium --headful --url http://(juju status --format yaml | yq e '.applications.demo.units["demo/0"]'.public-address -).nip.io/ --password=foobar
# Watch Juju status watch -tcn2 timeout 2s juju status --color # Watch MicroK8s pods watch -tn2 timeout 2s microk8s kubectl get pods -nkubeflow --sort-by=.metadata.name
Last updated 6 days ago.