jetstream
66 posts — page 2
-
Deploy a 70B LLM to Jetstream
Deploying large language models on Jetstream is getting easier thanks to the official Jetstream LLM guide. Here I follow that walkthrough but scale the hardware and model so we can run something far more capable than the defaults.
-
Timing the unshelving of a Jetstream 70B LLM instance
Following the work documented in Deploy a 70B LLM to Jetstream, the Meta-Llama-3.1-70B-Instruct-GGUF deployment is now running on a g3.xl instance. The goal of this follow-up is to measure how long it takes to unshelve that virtual machine and bring the chat interface back online.
-
Deploy a NFS server to share data between JupyterHub users on Jetstream
This is an updated version of my 2023 tutorial on deploying a NFS server to share data between JupyterHub users on Jetstream.
-
Use Jetstream's DeepSeek R1 as a code assistant on JupyterAI
Thanks to Openrouter, there is now a way of using the Jetstream LLM inference system, in particular the powerful Deepseek R1 model, as a code and documentation assistant in JupyterLab via JupyterAI.
-
Deploy Kubernetes and JupyterHub on Jetstream with Magnum and Cluster API
This guide demonstrates how to deploy Kubernetes on Jetstream with Magnum and then install JupyterHub on top using zero-to-jupyterhub. Jetstream recently enabled the Cluster API on its OpenStack deployment as the backend for Magnum, making it faster and more straightforward to launch Kubernetes clusters.
-
Run Windows and WSL on Jetstream
Need access to a Windows machine? You can leverage Jetstream 2, spin up a Virtual Machine with Windows Server 2022 and access the Windows graphical desktop through your browser on any operating system.
-
Deploy an LLM ChatGPT-like service on Jetstream
In this tutorial we will deploy a LLM Chat-GPT like service on a GPU node on Jetstream. However the same instructions can be used to deploy any other model available on the Hugging Face model hub.
-
Kubernetes monitoring with Prometheus and Grafana
In a production Kubernetes deployment it is necessary to make it easier to monitor the status of the cluster effectively. Kubernetes provides Prometheus to gather data from the different components of Kubernetes and Grafana to access those data and provide real-time plotting and inspection capability.
-
Received DeSouza award for work on deploying software infrastructure on Jetstream
I am honored to have received the 2023 DeSouza award from Unidata, a NSF-funded center providing data, software and support for the field of Geoscience.
-
Gateways 2023 tutorial about Dask and JupyterHub on Kubernetes on Jetstream
Gateways 2023 tutorial about Dask and JupyterHub on Kubernetes on Jetstream
-
Deploy GitHub Authenticator in JupyterHub
Updated June 2024: added more options to config file Quick reference on how to deploy the Github Authenticator in JupyterHub, the main reference is the Zero to JupyterHub docs.
-
Setup HTTPS on Kubernetes with cert-manager
Update March 2024: the routing issue that force cert-manager pods to run on the control-plane are back, see this Github issue, so we had to add back the pinning of cert-manager services on one of the nodes in the control plane.
-
Update OpenStack credentials in Kubernetes
If you are deploying Kubernetes on top of Openstack, the Openstack External cloud provider stores the ID and Secret necessary to authenticate with the cloud infrastructure in a Secret.
-
Deploy a NFS server to share data between JupyterHub users on Jetstream
This tutorial is a minor update of . Also consider that a more robust and low-maintenance way of providing shared data volumes is to rely on Manila shares provided by Jetstream 2, see the tutorial
-
Deploy Dask Gateway with JupyterHub on Kubernetes
Tutorial OBSOLETE Please check the updated version of this tutorial. This tutorial follows the work by the Pangeo collaboration, the main difference is that I prefer to keep JupyterHub and the Dask infrastructure in 2 separate Helm recipes.