nersc
9 posts
-
2024-05-28-ai-code-assistant-jupyter-nersc
If you are used to Github Copilot on VS Code or on Google Colaboratory as I am, coding without an AI assistant is so slow!
-
Setup a conda environment at NERSC
NERSC recommends to use the "Global Common" to store Conda environments, because it is optimized to store lots of small files. Consider that it is mounted read-only on computing nodes. The filesystem is organized by groups, so choose one of your groups:
-
Gzipping or not gzipping HEALPix maps
HEALPix maps are generally stored in FITS format, they can be Gzipped to save disk space. Reading gzipped maps is natively supported in healpy, however, it takes longer because maps are uncompressed on the fly.
-
Share data from NERSC publicly via web
NERSC allows to make data which is on their Community File System publicly available via a web interface. This is handled via "Project folders", so the requirement is to be part of a "project", for example most scientists working on Cosmic Microwave Background are part of the cmb project (and of the cmb Unix group at NERSC):
-
Setup a conda environment at NERSC
Obsolete: see the new version NERSC recommends to use the "Global Common" to store Conda environments, because it is optimized to store lots of small files. Consider that it is mounted read-only on computing nodes. The filesystem is organized by groups, so choose one of your groups:
-
Install HEALPix and PolSpice in a conda environment
Some notes on how to install HEALPix and PolSpice inside a conda environment, with some details about doing it at NERSC, but most of the tutorial is independent of that.
-
How to organize code and data for simulations at NERSC
I recently improved my strategy for organizing code and data for simulations run at NERSC, I'll write it here for reference.
-
Install custom Python environment on Jupyter Notebooks at NERSC
NERSC has provided a JupyterHub instance for quite some time to all NERSC users. It is currently running on a dedicated large-memory node on Cori, so now it can access also data on Cori $SCRATCH, not only /project and $HOME. See their documentation
-
IPython/Jupyter notebook setup on NERSC Edison
This tutorial explains the setup to run an IPython Notebook on a computing node on the supercomputer Edison at NERSC and forward its port encrypted with SSH to the browser on a local laptop. This setup is a bit more complicated than other supercomputers, i.e. see my tutorial for Comet for 2 reasons: