High Performance Computing & AI
Practical notes on Python, JupyterHub, Kubernetes and AI for science — from the San Diego Supercomputer Center.
-
Load data from Globus in the browser and plot with pyscript
Globus supports a Javascript SDK which allows to authenticate against Globus Auth and then access all the Globus services, for example accessing protected data or setting up transfers between endpoints.
-
Ubuntu 22.04 minimal on Jetstream with GPU support
Just notes on how to create a Ubuntu 22.04 image with GPU support First create an image with Ubuntu Minimal from Horizon
-
News item on SDSC website about Advanced Simons Observatory and PySM
In collaboration with the SDSC Communications team I prepared a news item for the SDSC Website announcing my participation in the Advanced Simons Observatory project focusing on my work on supporting PySM.
-
Italian classes in San Diego
My wife Maura D'Andrea is Principal and Teacher of Italian school of San Diego. All instructors are mother-tongue Italian, they teach to kids and adults.
-
Setup a conda environment at NERSC
NERSC recommends to use the "Global Common" to store Conda environments, because it is optimized to store lots of small files. Consider that it is mounted read-only on computing nodes. The filesystem is organized by groups, so choose one of your groups:
-
2024-05-02-pysm-point-source-pixell
Testing the pixell simobjects functionality to create maps of point sources pre-smoothed with a gaussian beam. The purpose is to include this functionality in PySM to be able to generate on the fly maps of source starting from a catalog.
-
Flatcar Container Linux on Jetstream
Virtual Machine images provided by the Jetstream team are fully fledged and therefore quite large. Julien Chastang proposed the shift to a minimalist distribution to save disk space, resources and shorten deployment time, see the relevant issue discussion on the Jetstream Gitlab organization.
-
Automate sending polls via WhatsApp
Polls in WhatsApp are so convenient for the users, but so painful for group admins. Cannot modify them, cannot re-use or replicate them.
-
2-way synchronization of Overleaf and GitHub via a GitHub Action
Updated: 2025-02-20 In the past I wrote tutorials on how to handle Github and Overleaf, the latest for example is this. However the Github button in Overleaf is fragile, it often gets disconnected for unknown reasons, and it cannot be reconnected, it forces deletion of the Github repository. Moreover, it seems it is now a Premium feature. Finally, it needs to be triggered manually, and nobody remembers to do that.
-
Generate figures for PySM emissions
This notebook generates some figures of Galactic and Extra-Galactic emissions using PySM. Mostly for displaying purposes.
-
Run PyTorch with GPU support on Expanse
In this tutorial we will create a Python environment with PyTorch and see how to get Jupyterlab running on a GPU node. First of all we want to create an isolated Python environment, I generally favor micromamba, see the documentation on how to install it
-
Deploy Kubernetes on Jetstream 2 with GPU support
This work has been supported by Indiana University and is cross-posted on the Jetstream 2 official documentation website .
-
Soft scaling Kubernetes on Jetstream
Work and post contributed by Ana Espinoza, only tested and benchmarked by Andrea Zonca. Summary
-
Make a PDF from a series of screenshots
You might need to digitize some web content by taking screenshots and then bundling them up in a PDF. Here I am using Linux from the command line, leveraging convert from Imagemagick and pdftk.
-
Received DeSouza award for work on deploying software infrastructure on Jetstream
I am honored to have received the 2023 DeSouza award from Unidata, a NSF-funded center providing data, software and support for the field of Geoscience.