High Performance Computing & AI
Practical notes on Python, JupyterHub, Kubernetes and AI for science — from the San Diego Supercomputer Center.
-
Learn Italian in San Diego
My wife Maura D'Andrea recently opened a new Italian language school in San Diego. All instructors are mother-tongue Italian, they teach to kids and adults.
-
Migrate from Google Docs to Overleaf and GitHub
Once a document on Google Docs becomes too big and complicated to make sure it is consistent, it is a good idea to migrate it to Latex. In particular we can support both Google Docs style editing via Overleaf and Pull Request reviewing via Github.
-
Correct a power spectrum by the instrument beam with healpy
If you are analyzing a map from an instrument with a specific beam width, you can correct the power spectrum by the smoothing factor caused by that beam and obtain a better approximation of the power spectrum of the orignal sky.
-
Backup Kubernetes volumes to OpenStorageNetwork object store
In my specific scenario, I have users running JupyterHub on top of Kubernetes on the Jetstream XSEDE Cloud resouce. Each user has a persistent volume as their home folder of a few GB.
-
Science Gateways Tech blog - Kubernetes and JupyterHub on Jetstream
This is cross-posted from the Science Gateways Community institute tech blog, thanks to Marlon Pierce and Katherine Lawrence for their feeback.
-
Stream video from object store on Jetstream
In this tutorial we will serve video stored in Jetstream's object store (like Amazon S3) as HTTP live streaming to a user' browser. This makes it difficult for the user to save the whole video as it is served in chunks.
-
Tutorial on how to rotate maps in healpy
import healpy as hp import numpy as np import os import astropy.units as u import matplotlib.pyplot as plt hp.disablewarnings() %matplotlib inline
-
Compute the Planck CMB temperature power spectrum with healpy anafast
import healpy as hp import numpy as np import os import astropy.units as u import matplotlib.pyplot as plt %matplotlib inline
-
Coordinate a large Latex document with multiple Overleaf projects and Github
In case we need to build a large document (hundreds of pages) and we have a large number of collaborators (more than 50), it is convenient to have each section of the document be a different Overleaf project. Also, we don't want to rely on Overleaf only, we prefer backup of everything to Github.
-
Deploy Kubernetes on Jetstream with Kubespray 2.15.0
Note: Jetstream 1 has been retired. For current Kubernetes deployments, see Jetstream 2 documentation. This is an update to previous tutorials, focused on deploying Kubernetes 1.19.7 (released in Jan 2021, based on 1.19.0 released in August 2020), compared to 1.17.6 of the previous version of the tutorial. Last executed in September 2021.
-
Autoscaling script for JupyterHub on top of Kubespray
I think the most reliable way of deploying Kubernetes on Openstack is Kubespray, see my latest tutorial about it focused on Jetstream.
-
CFITSIO file writing with checkpointing
I wrote a prototype implementation of writing FITS files in C checkpointing using the FITS flush function so that if the code segfaults, the file is not corrupted, see:
-
Paper review workflow with Overleaf, git and Google Docs
In this blog post I propose a workflow to use the collaborative features of Google Docs to review a paper being written in Overleaf (unless you have Overleaf Pro which has change tracking embedded).
-
Install HEALPix and PolSpice in a conda environment
Some notes on how to install HEALPix and PolSpice inside a conda environment, with some details about doing it at NERSC, but most of the tutorial is independent of that.
-
Example of the `healpy` query disc function
Example of the `healpy` query disc function