jupyterhub
69 posts — page 4
-
Scale Kubernetes manually on Jetstream
We would like to modify the number of Openstack virtual machines available to Kubernetes. Ideally we would like to do this automatically based on the load on JupyterHub, that is the target. For now we will increase and decrease the size manually.
-
Use the distributed file format Zarr on Jetstream Swift object storage, 2019
This is an updated version of the 2018 edition Zarr
-
Deploy Pangeo on Kubernetes deployment on Jetstream created with Kubespray
The Pangeo collaboration for Big Data Geoscience maintains a helm chart with a prefigured JupyterHub deployment on Kubernetes which also supports launching private dask workers.
-
Deploy JupyterHub on a Supercomputer for a workshop or tutorial 2018 edition
I described how to deploy JupyterHub with each user session running on a different node of a Supercomputer in my paper for PEARC18, however things are moving fast in the space and I am employing a different strategy this year, in particular relying on the littlest JupyterHub project for the initial deployment.
-
Deploy JupyterHub on a Supercomputer for a workshop or tutorial 2018 edition
I described how to deploy JupyterHub with each user session running on a different node of a Supercomputer in my paper for PEARC18, however things are moving fast in the space and I am employing a different strategy this year, in particular relying on the littlest JupyterHub project for the initial deployment.
-
Deploy JupyterHub on Kubernetes deployment on Jetstream created with Kubespray 3/3
All of the following assumes you are logged in to the master node of the Kubernetes cluster deployed with kubespray and checked out the repository:
-
PEARC18 paper on deploying Jupyterhub at scale on XSEDE
Bob Sinkovits and I are presenting a paper at PEARC18 about: "Deploying Jupyter Notebooks at scale on XSEDE resources for Science Gateways and workshops"
-
Setup private dask clusters in Kubernetes alongside JupyterHub on Jetstream
In this post we will leverage software made available by the Pangeo community to allow each user of a Jupyterhub instance deployed on Jetstream on top of Kubernetes to launch a set of dask workers as containers running inside Kubernetes itself and use them for distributed computing.
-
Launch a shared dask cluster in Kubernetes alongside JupyterHub on Jetstream
Let's assume we have already a Kubernetes deployment and have installed JupyterHub, see for example my previous tutorial on Jetstream. Now that users can login and access a Jupyter Notebook, we would also like to provide them more computing power for their interactive data exploration.
-
Install custom Python environment on Jupyter Notebooks at NERSC
NERSC has provided a JupyterHub instance for quite some time to all NERSC users. It is currently running on a dedicated large-memory node on Cori, so now it can access also data on Cori $SCRATCH, not only /project and $HOME. See their documentation
-
ECSS Symposium about Jupyterhub deployments on XSEDE
Note: XSEDE has been replaced by ACCESS. ECSS Symposium, 19 December 2017, Web presentation to the XSEDE Extended Collaborative Support Services.
-
Deploy scalable Jupyterhub with Kubernetes on Jetstream
The best infrastructure available to deploy Jupyterhub at scale is Kubernetes. Kubernetes provides a fault-tolerant system to deploy, manage and scale containers. The Jupyter team released a recipe to deploy Jupyterhub on top of Kubernetes, Zero to Jupyterhub.
-
Deploy scalable Jupyterhub on Docker Swarm mode
Jupyterhub genrally requires roughly 500MB per user for light data processing and many GB for heavy data processing, therefore it is often necessary to deploy it across multiple machines to support many users.
-
Deployment of Jupyterhub with Globus Auth to spawn notebook on Comet in Singularity containers
Follow the instructions at to build images from the ubuntuanacondajupyterhub.def and centosanacondajupyterhub.def definition files, or use the containers I have already built on Comet:
-
Deploy Jupyterhub on a supercomputer with SSH authentication
The best way to deploy Jupyterhub with an interface to a Supercomputer is through the use of batchspawner. I have a sample deployment explained in an older blog post: