High Performance Computing & AI
Practical notes on Python, JupyterHub, Kubernetes and AI for science — from the San Diego Supercomputer Center.
-
Read and process Planck CMB power spectra with `healpy`
In this tutorial we will load CMB power spectra generated by the cosmological parameters measured by Planck, we will create a realization of the Spherical Harmonics coefficients $a{\ell m}$, save it to disk, and use them to create maps at different resolutions.
-
Edit video recordings with ffmpeg
A few scripts to speedup, concatenate and possibly other operations using ffmpeg on linux (also works on Chromebooks) from the command line:
-
How to share Jupyter Notebooks
I tend to preserve and post online all the work I do for later reference and sharing. Jupyter Notebooks are a bit more complex than standard source code because they are natively in a format (JSON) that needs to be rendered to be readable.
-
Simple WCS with astropy modeling and gwcs
Some notes on how to convert a FITS WCS to a WCS from gwcs (Generalized WCS) to be used within the JWST pipeline python import astropy from astropy.io import fits import numpy as np from astropy import units as u
-
Gateways 2020 paper about Kubernetes and JupyterHub on Jetstream
My paper in collaboration with Richard Signell (USGS), Julien Chastang (UCAR), John Michael Lowe (IU/Jetstream), Jeremy Fischer (IU/Jetstream) and Robert Sinkovits (SDSC) was accepted at Gateways 2020 (October 12–23, 2020).
-
Unit conversion with broadband detectors looking at foregrounds
import healpy as hp import numpy as np import matplotlib.pyplot as plt from pysm3 import units as u import pysm3 as pysm %matplotlib inline
-
Unit conversion with broadband detectors looking at the CMB
import healpy as hp import numpy as np import matplotlib.pyplot as plt from pysm3 import units as u import pysm3 as pysm %matplotlib inline
-
Migrate from Travis-CI and ReadTheDocs to GitHub Actions
For a number of years I have been concerned about the duplication of work having to maintain continuous integration environments both on Travis-CI to run the unit tests and on Readthedocs to build and host the documentation.
-
Redirect readthedocs documentation to another website
This is useful if you switch to hosting your own documentation, for example using Sphinx Multiversion on Github pages, tutorial coming soon.
-
Renew letsencrypt certificate with NGINX
If you have a default domain already configured with NGINX, the easiest way to renew manually a domain is to temporarily disable your custom domain and then use certbot-auto renew the certificate.
-
Investigate broken implementation of bandpass unit conversion in PySM 3
import pysm3 import numpy as np import healpy as hp from astropy.tests.helper import assertquantityallclose import pysm3.units as u
-
Deploy Dask Gateway with JupyterHub on Kubernetes
This tutorial is obsolete, please follow This tutorial follows the work by the Pangeo collaboration, the main difference is that I prefer to keep JupyterHub and the Dask infrastructure in 2 separate Helm recipes.
-
Dask array rounding
Dask array rounding float32
-
Deploy a NFS server to share data between JupyterHub users on Jetstream
Important: This tutorial is obsolete, please refer to the new version from February 2023 In this tutorial I'll show how to create a data volume on Jetstream and share it using a NFS server to all JupyterHub users. All JupyterHub users run as the jovyan user, therefore each folder in the shared filesystem can be either read-only, or writable by every user. The main concern is that a user could delete by mistake data of another user, however the users still have access to their own home folder.
-
Handle white noise with healpy 3 not-uniform and partial sky coverage
Simulate white noise maps and use hitmaps