High Performance Computing & AI
Practical notes on Python, JupyterHub, Kubernetes and AI for science — from the San Diego Supercomputer Center.
-
New paper on arXiv about models of Galactic emission in the microwaves for CMB experiments
A new paper has been published on Arxiv discussing models of Galactic emission in the microwaves for Cosmic Microwave Background (CMB) experiments:
-
Use Jetstream's DeepSeek R1 as a code assistant on JupyterAI
Thanks to Openrouter, there is now a way of using the Jetstream LLM inference system, in particular the powerful Deepseek R1 model, as a code and documentation assistant in JupyterLab via JupyterAI.
-
Authenticate to GitHub in the browser with the device flow
Authenticating to GitHub in the browser using the device flow is essential for applications that cannot directly handle OAuth redirects.
-
Save Jupyterlite notebooks to GitHub
JupyterLite runs entirely inside the browser, which means it doesn't have access to the underlying operating system. Its storage is persistent across restarts of the machine but could be deleted if the browser cache is cleared.
-
Using GitHub Releases to keep track of large artifacts
GitHub Releases provide a way to attach large artifacts, such as binaries or large autogenerated files, without polluting the repository. A release points to a specific commit in a repository and can include files up to 2GB in size.
-
Italian and chess classes in San Diego
My wife Maura D'Andrea is Principal and Teacher of Italian school of San Diego. They offer an extensive Adults Italian classes program in-person in San Diego (Kearny Mesa), including Italian for Travelers specifically for tourists travelling to Italy.
-
Deploy Kubernetes and JupyterHub on Jetstream with Magnum and Cluster API
This guide demonstrates how to deploy Kubernetes on Jetstream with Magnum and then install JupyterHub on top using zero-to-jupyterhub. Jetstream recently enabled the Cluster API on its OpenStack deployment as the backend for Magnum, making it faster and more straightforward to launch Kubernetes clusters.
-
Proposal for Simons Observatory data products attribution
CC BY 4.0 only requiring attribution and not restricting commercial use or remixes. Attribution
-
Run Windows and WSL on Jetstream
Need access to a Windows machine? You can leverage Jetstream 2, spin up a Virtual Machine with Windows Server 2022 and access the Windows graphical desktop through your browser on any operating system.
-
Allow users to self-invite to a Google Calendar event
Google Calendar is the most popular calendar service, and it is widely used in academia, often Scientific Collaborations maintain one or multiple shared calendars to keep track of events, deadlines, and meetings.
-
Deploy an LLM ChatGPT-like service on Jetstream
In this tutorial we will deploy a LLM Chat-GPT like service on a GPU node on Jetstream. However the same instructions can be used to deploy any other model available on the Hugging Face model hub.
-
Kubernetes monitoring with Prometheus and Grafana
In a production Kubernetes deployment it is necessary to make it easier to monitor the status of the cluster effectively. Kubernetes provides Prometheus to gather data from the different components of Kubernetes and Grafana to access those data and provide real-time plotting and inspection capability.
-
Generate script to mount a Manila share
Jetstream 2 makes creating volumes that are shared across instances as easy as creating standard volumes. This is provided by the Manila service, which creates volumes based on the Ceph file system that can be mounted simultaneously on multiple Jetstream 2 Virtual Machines.
-
Configure ssh-agent on Linux
I keep Googling this again and again. First, ssh-agent does not need to run on the remote server, it is the local agent which is forwarded over ssh to the remote server.
-
Tutorial on Python for HPC at the SDSC Summer Institute 2024
Each August the San Diego Supercomputer Center organizes a week-long summer school that teaches a large number of topics related to Supercomputing and AI to early career scientists.