python
132 posts — page 4
-
Run PyTorch with GPU support on Expanse
In this tutorial we will create a Python environment with PyTorch and see how to get Jupyterlab running on a GPU node. First of all we want to create an isolated Python environment, I generally favor micromamba, see the documentation on how to install it
-
Gateways 2023 tutorial about Dask and JupyterHub on Kubernetes on Jetstream
Gateways 2023 tutorial about Dask and JupyterHub on Kubernetes on Jetstream
-
Deploy Dask Gateway with JupyterHub on Kubernetes
In this tutorial we will install Dask Gateway, currently version 2023.9.0, on Kubernetes and configure JupyterHub so Jupyter Notebook users can launch private Dask cluster and connect to them.
-
Tutorial on Python for HPC at the SDSC Summer Institute
Each August the San Diego Supercomputer Center organizes a week-long summer school that teaches a large number of topics related to Supercomputing and AI to early career scientists.
-
Showyourwork! Globus demo
Showyourwork! is a great project to make scientific articles reproducible. I participated to its first Hack Day, I focused on building a simple recipe in SnakeMake (the workflow engine behind Showyourwork!) to interface with data stored on Globus.
-
Setup a conda environment at NERSC
Obsolete: see the new version NERSC recommends to use the "Global Common" to store Conda environments, because it is optimized to store lots of small files. Consider that it is mounted read-only on computing nodes. The filesystem is organized by groups, so choose one of your groups:
-
Generate Click command line options dynamically from class arguments
In this application I need to use command line options to create class objects. In order not to repeat all class arguments in the Click configuration, I created a simple function to dynamically create Click options. See the source notebook of this page on Github
-
Closures in Numba
Closures in Numba
-
Science Gateway with Dask and Zarr
This material was presented on April 2022 at the MiniGateways 2022 conference organized by the wonderful Science Gateways Community Institute (SGCI).
-
Use the distributed file format Zarr on Jetstream 2 object storage
Zarr is a file format designed for cloud computing, see documentation. Zarr is also supported by dask, the parallel computing framework for Python, and the Dask team implemented storage backends for Google Cloud Storage and Amazon S3.
-
Deploy Dask Gateway with JupyterHub on Kubernetes
Tutorial obsolete, see the new version of the tutorial Updated 28 April 2022: switched to Dask Gateway 2022.4.0
-
Deploy Dask Gateway with JupyterHub on Kubernetes
Tutorial OBSOLETE Please check the updated version of this tutorial. This tutorial follows the work by the Pangeo collaboration, the main difference is that I prefer to keep JupyterHub and the Dask infrastructure in 2 separate Helm recipes.
-
Manage Globus groups with the Python SDK
Globus is the best tool to transfer quickly Terabytes data between Supercomputers because it automatically parallelizes the transfer to saturate the network (yeah I know I always simplify too much).
-
New paper about PySM 3
Update November 2021 Paper was published on JOSS:
-
healpy 1.15.0 released
Just released healpy 1.15.0, this is a large release with many important changes, please test and open issues at New viz function projview based on plain matplotlib, with lon/lat labels and more