High Performance Computing & AI
Practical notes on Python, JupyterHub, Kubernetes and AI for science — from the San Diego Supercomputer Center.
-
Generate figures for PySM emissions
This notebook generates some figures of Galactic and Extra-Galactic emissions using PySM. Mostly for displaying purposes.
-
Run PyTorch with GPU support on Expanse
In this tutorial we will create a Python environment with PyTorch and see how to get Jupyterlab running on a GPU node. First of all we want to create an isolated Python environment, I generally favor micromamba, see the documentation on how to install it
-
Deploy Kubernetes on Jetstream 2 with GPU support
This work has been supported by Indiana University and is cross-posted on the Jetstream 2 official documentation website .
-
Soft scaling Kubernetes on Jetstream
Work and post contributed by Ana Espinoza, only tested and benchmarked by Andrea Zonca. Summary
-
Make a PDF from a series of screenshots
You might need to digitize some web content by taking screenshots and then bundling them up in a PDF. Here I am using Linux from the command line, leveraging convert from Imagemagick and pdftk.
-
Received DeSouza award for work on deploying software infrastructure on Jetstream
I am honored to have received the 2023 DeSouza award from Unidata, a NSF-funded center providing data, software and support for the field of Geoscience.
-
Buy US Treasury bonds on Fidelity
The Fidelity interface is complicated, here a description of the steps necessary to buy newly issued US Treasury bonds at the next available auction. Currently auctions are every Thursday.
-
Gateways 2023 tutorial about Dask and JupyterHub on Kubernetes on Jetstream
Gateways 2023 tutorial about Dask and JupyterHub on Kubernetes on Jetstream
-
Deploy Dask Gateway with JupyterHub on Kubernetes
In this tutorial we will install Dask Gateway, currently version 2023.9.0, on Kubernetes and configure JupyterHub so Jupyter Notebook users can launch private Dask cluster and connect to them.
-
Deploy GitHub Authenticator in JupyterHub
Updated June 2024: added more options to config file Quick reference on how to deploy the Github Authenticator in JupyterHub, the main reference is the Zero to JupyterHub docs.
-
Setup HTTPS on Kubernetes with cert-manager
Update March 2024: the routing issue that force cert-manager pods to run on the control-plane are back, see this Github issue, so we had to add back the pinning of cert-manager services on one of the nodes in the control plane.
-
Tutorial on Python for HPC at the SDSC Summer Institute
Each August the San Diego Supercomputer Center organizes a week-long summer school that teaches a large number of topics related to Supercomputing and AI to early career scientists.
-
Deploy Kubernetes on Jetstream 2 with Kubespray 2.21.0
::: {.callout-note} Obsolete Tutorial This tutorial is obsolete. Please refer to the Magnum tutorial to deploy Kubernetes instead. :::
-
Hiring a computational scientist at the San Diego Supercomputer Center
Come work in my group of High Performance Computing "consultants" at the San Diego Supercomputer Center, help scientists develop, scale and run Data Intensive computations on our Supercomputers.
-
Cosmology webinar Hyper Suprime-Cam
Webinar about Cosmology results, in particular tension in $S8$ by the Hyper Supreme Cam collaboration: Video on Youtube