python
132 posts — page 6
-
How to organize code and data for simulations at NERSC
I recently improved my strategy for organizing code and data for simulations run at NERSC, I'll write it here for reference.
-
Install custom Python environment on Jupyter Notebooks at NERSC
NERSC has provided a JupyterHub instance for quite some time to all NERSC users. It is currently running on a dedicated large-memory node on Cori, so now it can access also data on Cori $SCRATCH, not only /project and $HOME. See their documentation
-
Store a conda environment inside a Notebook
Last August, during the Container Analysis Environments Workshop held at Urbana-Champaign, we had discussion about reproducibility in the Jupyter Notebooks. There came out the idea of storing all the details about the Python environment inside the Notebook, in the metadata.
-
Deploy Jupyterhub on a supercomputer with SSH authentication
The best way to deploy Jupyterhub with an interface to a Supercomputer is through the use of batchspawner. I have a sample deployment explained in an older blog post:
-
Sample deployment of Jupyterhub in HPC on SDSC Comet
I have deployed an experimental Jupyterhub service (ask me privately if you would like access) installed on a SDSC Cloud virtual machine that spawns single user Jupyter notebooks on Comet computing nodes using batchspawner and then proxies the Notebook back to the user using SSH-tunneling.
-
Customize your Python environment in Jupyterhub
Usecase: You have access to a Jupyterhub server and you would like to install some packages but cannot use pip install and modify the systemwide Python installation.
-
Jupyterhub Docker Spawner with GPU support
Docker Spawner allows users of Jupyterhub to run Jupyter Notebook inside isolated Docker Containers. Access to the host NVIDIA GPU was not allowed until NVIDIA release the NVIDIA-docker plugin.
-
Quick Jupyterhub deployment for workshops with pre-built image
This tutorial explains how to use a OpenStack image I already built to quickly deploy a Jupyterhub Virtual Machine that can provide a good initial setup for a workshop, providing students access to Python 2/3, Julia, R, file editor and terminal with bash.
-
Deploy Jupyterhub on a virtual machine for a workshop
This tutorial describes the steps to install a Jupyterhub instance on a single machine suitable for hosting a workshop, suitable for having people login with training accounts on Jupyter Notebooks running Python 2/3, R, Julia with also Terminal access on Docker containers. Details about the setup:
-
Use your own Python installation (kernel) in Jupyterhub
Updated February 2017 You have access to a Jupyterhub server but the Python installation provided does not satisfy your needs, how to use your own?
-
IPython/Jupyter notebook setup on NERSC Edison
This tutorial explains the setup to run an IPython Notebook on a computing node on the supercomputer Edison at NERSC and forward its port encrypted with SSH to the browser on a local laptop. This setup is a bit more complicated than other supercomputers, i.e. see my tutorial for Comet for 2 reasons:
-
IPython/Jupyter notebook setup on SDSC Comet
This tutorial explains the setup to run an IPython Notebook on a computing node on the supercomputer Comet at the San Diego Supercomputer Center and forward the port encrypted with SSH to the browser on a local laptop.
-
Run Jupyterhub on a supercomputer
The IPython (recently renamed Jupyter) Notebook is a powerful tool for analyzing and visualizing data in Python and other programming languages. A key feature is that a single document contains code, figures, text and equations. Everything is saved in a single .ipynb file that can be shared, executed and modified.
-
Accelerate groupby operation on pixels with Numba
Download the original IPython notebook Astrophysics background
-
Software Carpentry setup for Chromebook
In this post I'll provide instructions on how to install the main requirements of a Software Carpentry workshop on a Chromebook. Bash, git, IPython notebook and R.