High Performance Computing & AI
Practical notes on Python, JupyterHub, Kubernetes and AI for science — from the San Diego Supercomputer Center.
-
Jupyterhub Docker Spawner with GPU support
Docker Spawner allows users of Jupyterhub to run Jupyter Notebook inside isolated Docker Containers. Access to the host NVIDIA GPU was not allowed until NVIDIA release the NVIDIA-docker plugin.
-
Jupyterhub deployment on multiple nodes with Docker Swarm
This post is part of a series on deploying Jupyterhub on OpenStack tailored at workshops, in the previous posts I showed:
-
Quick Jupyterhub deployment for workshops with pre-built image
This tutorial explains how to use a OpenStack image I already built to quickly deploy a Jupyterhub Virtual Machine that can provide a good initial setup for a workshop, providing students access to Python 2/3, Julia, R, file editor and terminal with bash.
-
Deploy Jupyterhub on a virtual machine for a workshop
This tutorial describes the steps to install a Jupyterhub instance on a single machine suitable for hosting a workshop, suitable for having people login with training accounts on Jupyter Notebooks running Python 2/3, R, Julia with also Terminal access on Docker containers. Details about the setup:
-
Use your own Python installation (kernel) in Jupyterhub
Updated February 2017 You have access to a Jupyterhub server but the Python installation provided does not satisfy your needs, how to use your own?
-
IPython/Jupyter notebook setup on NERSC Edison
This tutorial explains the setup to run an IPython Notebook on a computing node on the supercomputer Edison at NERSC and forward its port encrypted with SSH to the browser on a local laptop. This setup is a bit more complicated than other supercomputers, i.e. see my tutorial for Comet for 2 reasons:
-
IPython/Jupyter notebook setup on SDSC Comet
This tutorial explains the setup to run an IPython Notebook on a computing node on the supercomputer Comet at the San Diego Supercomputer Center and forward the port encrypted with SSH to the browser on a local laptop.
-
Run Jupyterhub on a supercomputer
The IPython (recently renamed Jupyter) Notebook is a powerful tool for analyzing and visualizing data in Python and other programming languages. A key feature is that a single document contains code, figures, text and equations. Everything is saved in a single .ipynb file that can be shared, executed and modified.
-
Accelerate groupby operation on pixels with Numba
Download the original IPython notebook Astrophysics background
-
Software Carpentry setup for Chromebook
In this post I'll provide instructions on how to install the main requirements of a Software Carpentry workshop on a Chromebook. Bash, git, IPython notebook and R.
-
Zero based indexing
For Europeans zero based indexing feels reasonable if we think of floors in a house, the lowest floor is ground floor, then 1st floor and so on.
-
Write unit tests as cells of IPython notebooks
Plugin for py.test to write unit tests as cells in IPython notebooks: Homepage on Github:
-
How to perform code review for scientific software
Code review is the formal process where a programmer inspects in detail a piece of software developed by somebody else in order to improve code quality by catching bugs, improve readibility and usability. It is used extensively in industry, not much in academia.
-
Create a Github account for your research group with free private repositories
See the updated version at Github allows a research group to create their own webpage where they can host, share and develop their software using the git version control system and the powerful Github online issue-tracking interface.
-
Thoughts on a career as a computational scientist
Recently I've been asked what are the prospects of a wannabe computational scientist, both in terms of training and in terms of job opportunities.