python
135 posts — page 4
-
Load data from Globus in the browser and plot with pyscript
Globus supports a Javascript SDK which allows to authenticate against Globus Auth and then access all the Globus services, for example accessing protected data or setting up transfers between endpoints.
-
News item on SDSC website about Advanced Simons Observatory and PySM
In collaboration with the SDSC Communications team I prepared a news item for the SDSC Website announcing my participation in the Advanced Simons Observatory project focusing on my work on supporting PySM.
-
Setup a conda environment at NERSC
NERSC recommends to use the "Global Common" to store Conda environments, because it is optimized to store lots of small files. Consider that it is mounted read-only on computing nodes. The filesystem is organized by groups, so choose one of your groups:
-
Run PyTorch with GPU support on Expanse
In this tutorial we will create a Python environment with PyTorch and see how to get Jupyterlab running on a GPU node. First of all we want to create an isolated Python environment, I generally favor micromamba, see the documentation on how to install it
-
Gateways 2023 tutorial about Dask and JupyterHub on Kubernetes on Jetstream
Gateways 2023 tutorial about Dask and JupyterHub on Kubernetes on Jetstream
-
Deploy Dask Gateway with JupyterHub on Kubernetes
In this tutorial we will install Dask Gateway, currently version 2023.9.0, on Kubernetes and configure JupyterHub so Jupyter Notebook users can launch private Dask cluster and connect to them.
-
Tutorial on Python for HPC at the SDSC Summer Institute
Each August the San Diego Supercomputer Center organizes a week-long summer school that teaches a large number of topics related to Supercomputing and AI to early career scientists.
-
Showyourwork! Globus demo
Showyourwork! is a great project to make scientific articles reproducible. I participated to its first Hack Day, I focused on building a simple recipe in SnakeMake (the workflow engine behind Showyourwork!) to interface with data stored on Globus.
-
Setup a conda environment at NERSC
Obsolete: see the new version NERSC recommends to use the "Global Common" to store Conda environments, because it is optimized to store lots of small files. Consider that it is mounted read-only on computing nodes. The filesystem is organized by groups, so choose one of your groups:
-
Generate Click command line options dynamically from class arguments
In this application I need to use command line options to create class objects. In order not to repeat all class arguments in the Click configuration, I created a simple function to dynamically create Click options. See the source notebook of this page on Github
-
Closures in Numba
Closures in Numba
-
Science Gateway with Dask and Zarr
This material was presented on April 2022 at the MiniGateways 2022 conference organized by the wonderful Science Gateways Community Institute (SGCI).
-
Use the distributed file format Zarr on Jetstream 2 object storage
Zarr is a file format designed for cloud computing, see documentation. Zarr is also supported by dask, the parallel computing framework for Python, and the Dask team implemented storage backends for Google Cloud Storage and Amazon S3.
-
Deploy Dask Gateway with JupyterHub on Kubernetes
Tutorial obsolete, see the new version of the tutorial Updated 28 April 2022: switched to Dask Gateway 2022.4.0
-
Deploy Dask Gateway with JupyterHub on Kubernetes
Tutorial OBSOLETE Please check the updated version of this tutorial. This tutorial follows the work by the Pangeo collaboration, the main difference is that I prefer to keep JupyterHub and the Dask infrastructure in 2 separate Helm recipes.