High Performance Computing & AI
Practical notes on Python, JupyterHub, Kubernetes and AI for science — from the San Diego Supercomputer Center.
-
Tutorial on how to rotate maps in healpy
import healpy as hp import numpy as np import os import astropy.units as u import matplotlib.pyplot as plt hp.disablewarnings() %matplotlib inline
-
Compute the Planck CMB temperature power spectrum with healpy anafast
import healpy as hp import numpy as np import os import astropy.units as u import matplotlib.pyplot as plt %matplotlib inline
-
Coordinate a large Latex document with multiple Overleaf projects and Github
In case we need to build a large document (hundreds of pages) and we have a large number of collaborators (more than 50), it is convenient to have each section of the document be a different Overleaf project. Also, we don't want to rely on Overleaf only, we prefer backup of everything to Github.
-
Autoscaling script for JupyterHub on top of Kubespray
I think the most reliable way of deploying Kubernetes on Openstack is Kubespray, see my latest tutorial about it focused on Jetstream.
-
Deploy Kubernetes on Jetstream with Kubespray 2.15.0
Note: Jetstream 1 has been retired. For current Kubernetes deployments, see Jetstream 2 documentation. This is an update to previous tutorials, focused on deploying Kubernetes 1.19.7 (released in Jan 2021, based on 1.19.0 released in August 2020), compared to 1.17.6 of the previous version of the tutorial. Last executed in September 2021.
-
CFITSIO file writing with checkpointing
I wrote a prototype implementation of writing FITS files in C checkpointing using the FITS flush function so that if the code segfaults, the file is not corrupted, see:
-
Paper review workflow with Overleaf, git and Google Docs
In this blog post I propose a workflow to use the collaborative features of Google Docs to review a paper being written in Overleaf (unless you have Overleaf Pro which has change tracking embedded).
-
Install HEALPix and PolSpice in a conda environment
Some notes on how to install HEALPix and PolSpice inside a conda environment, with some details about doing it at NERSC, but most of the tutorial is independent of that.
-
Example of the `healpy` query disc function
Example of the `healpy` query disc function
-
Read and process Planck CMB power spectra with `healpy`
In this tutorial we will load CMB power spectra generated by the cosmological parameters measured by Planck, we will create a realization of the Spherical Harmonics coefficients $a{\ell m}$, save it to disk, and use them to create maps at different resolutions.
-
Edit video recordings with ffmpeg
A few scripts to speedup, concatenate and possibly other operations using ffmpeg on linux (also works on Chromebooks) from the command line:
-
How to share Jupyter Notebooks
I tend to preserve and post online all the work I do for later reference and sharing. Jupyter Notebooks are a bit more complex than standard source code because they are natively in a format (JSON) that needs to be rendered to be readable.
-
Simple WCS with astropy modeling and gwcs
Some notes on how to convert a FITS WCS to a WCS from gwcs (Generalized WCS) to be used within the JWST pipeline python import astropy from astropy.io import fits import numpy as np from astropy import units as u
-
Gateways 2020 paper about Kubernetes and JupyterHub on Jetstream
My paper in collaboration with Richard Signell (USGS), Julien Chastang (UCAR), John Michael Lowe (IU/Jetstream), Jeremy Fischer (IU/Jetstream) and Robert Sinkovits (SDSC) was accepted at Gateways 2020 (October 12–23, 2020).
-
Unit conversion with broadband detectors looking at foregrounds
import healpy as hp import numpy as np import matplotlib.pyplot as plt from pysm3 import units as u import pysm3 as pysm %matplotlib inline