High Performance Computing & AI
Practical notes on Python, JupyterHub, Kubernetes and AI for science — from the San Diego Supercomputer Center.
-
change permission recursively to folders only
change permission recursively to folders only
-
aptitude search 'and'
this is really something really annoying about aptitude, if you run: aptitude search linux headers it will make an 'or' search...to perform a 'and' search, which I need 99.9% of the time, you need quotation marks: aptitude search 'linux headers'
-
using numpy dtype with loadtxt
Let's say you want to read a text file like this: filename start end fdsafda.fits 23143214 23143214 safdsafafds.fits 21423 23423432 you can use dtype to create a custom array, which is very flexible as you can work by row or columns with strings and floats in the same array: dt=np.dtype({'names':['filename','start','end'],'formats':['S100',np.float,np.float]}) [I tried also using np.str instead of S100 without success, anyone knows why?] then give this as input to loadtxt to load the file and create the array.
-
Stop ipcluster from a script
Ipcluster is easy to start but not trivial to stop from a script, after having finished the processing, here's the solution: from IPython.kernel import client mec = client.MultiEngineClient() mec.kill(controller=True)
-
Correlation
Expectation value or first moment of a random variable is the probability weighted sum of the possible values (weighted mean). Expectation value of a 6-dice is 1+2+3+4+5+6 / 6 = 3.5 Covariance of 2 random variables is: COV(X,Y)=E[(X-E(X))(Y-E(Y))]=E(XY) - E(X)E(Y) i.e.
-
execute bash script remotely with ssh
a bash script launched remotely via ssh does not load the environment, if this is an issue it is necessary to specify --login when calling bash: ssh user@remoteserver.com 'bash --login lifeom/cronodproc' | mail your@email.com -s cronodproc
-
lock pin hold a package using apt on ubuntu
set hold: echo packagename hold | dpkg --set-selections check, should be hi : dpkg -l packagename unset hold: echo packagename install | dpkg --set-selections
-
load arrays from a text file with numpy
space separated text file with 5 arrays in columns: [sourcecode language="python"] ods,rings,gains,offsets,rparams = np.loadtxt(filename,unpack=True) [/sourcecode] quite impressive...
-
Latest Maxima and WxMaxima for Ubuntu Karmic
Note: The original third-party repository for Maxima packages is no longer available. On the maxima mailing lists they suggested to install the sbcl built, so I first installed sbcl from the Ubuntu repositories and then maxima and wxmaxima from the third-party repository.
-
number of files in a folder and subfolders
folders are not counted find . -type f | wc -l
-
forcefully unmount a disk partition
check which processes are accessing a partition: [sourcecode language="python"]lsof | grep '/opt'[/sourcecode] kill all the processes accessing the partition (check what you're killing, you could loose data): [sourcecode language="python"]fuser -km /mnt[/sourcecode] try to unmount now: [sourcecode language="python"]umo
-
netcat, quickly send binaries through network
just start nc in server mode on localhost: [sourcecode language='python'] nc -l -p 3333 [/sourcecode] send a string to localhost on port 3333: [sourcecode language='python'] echo "hello world" | nc localhost 3333 [/sourcecode] you'll see on server side appearing the string you sent.
-
Decibels, dB and dBm, in terms of Power and Amplitude
It's not difficult, just always having some doubts... Power $latex L{dB} = 10 log{10} \left( \dfrac{P1}{P0} \right) $ 10 dB increase for a factor 10 increase in the ratio 3 dB = doubling 40 dB = 10000 times Amplitude $latex L{dB} = 10 log{10} \left( \dfrac{A1^2}{A0^2} \right) = 20 log{10} \left( \dfrac{A1}{A0} \right)
-
Relation between Power density and temperature in an antenna
Considering an antenna placed inside a blackbody enclosure at temperature T, the power received per unit bandwidth is: $latex \omega = kT$ where k is Boltzmann constant.
-
Producing PDF from XML files
I need to produce formatted pdf from XML data input file. The more standard way looks like to use XSL stylesheets. Associating a XSL sheet to an XML file permits most browsers to render them directly as HMTL, this can be used for web publishing XML sheets.